Obstructive sleep apnea detection method and system based on multi-modal lightweight transformation

Abstract

The invention discloses an obstructive sleep apnea detection method based on multi-modal lightweight transformation, and the method comprises the following steps: data acquisition and preprocessing: obtaining an original audio signal, and carrying out the standardization processing and clinical baseline data preprocessing of the original audio signal; audio feature extraction based on MiniRocket: using random convolution kernel transformation to replace a traditional deep convolutional neural network, realizing efficient time sequence feature extraction of audio signals, and obtaining an audio feature vector; the method comprises the following steps of: extracting clinical semantic features based on BioBERT: enhancing the understanding of a pre-training language model on specific patient data by using general medical knowledge contained in the pre-training language model to obtain a text feature vector; and multi-modal feature fusion and classification judgment: fusing the obtained audio feature vector and the obtained text feature vector in a feature splicing mode, and then carrying out classification judgment through a ridge regression classifier (RidgeClassifier).

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of biomedical engineering, signal processing and artificial intelligence. More specifically, this invention relates to a method and system for detecting obstructive sleep apnea based on multimodal lightweight transformation. Background Technology

[0002] Obstructive sleep apnea-hypopnea syndrome (OSAHS) is a common sleep-related breathing disorder. Its core pathological mechanism is that the muscles of the upper airway (especially the pharynx) relax during sleep, leading to airway collapse or obstruction. This obstruction causes repeated stops (apnea) or significant reductions (hypopnea) of airflow, which in turn leads to intermittent hypoxemia and hypercapnia.

[0003] This repeated hypoxia-reoxygenation process can induce oxidative stress, systemic inflammation, and excessive activation of the sympathetic nervous system. Long-term untreated OSAHS is an independent risk factor for many chronic diseases. Cardiovascular system: It can lead to refractory hypertension, coronary atherosclerotic heart disease, nocturnal arrhythmia, heart failure, and even sudden cardiac death. Metabolic system: It is closely related to insulin resistance, type 2 diabetes, and metabolic syndrome. Nervous system: It leads to fragmented sleep structure, reduced deep sleep, causing daytime sleepiness, cognitive decline, and memory loss, which seriously affects the patient's quality of life and work efficiency. Public safety: OSAHS patients have a 2-7 times higher risk of traffic accidents than normal people, making them a hidden killer on public transportation.

[0004] Currently, polysomnography (PSG) is widely recognized as the "gold standard" for diagnosing OSAHS. Standard PSG monitoring needs to be performed in a specialized sleep laboratory, and it involves a wide range of indicators, including electroencephalography (EEG), electrooculography (EOG), electromyography (EMG), electrocardiography (ECG), nasal and oral airflow, chest and abdominal respiratory movements, pulse oxygen saturation (SpO2), body position, and snoring.

[0005] Despite its accurate diagnostic capabilities, PSG faces significant challenges in widespread clinical application and struggles to meet the ever-increasing screening demands: Limited medical resources: Building a standard sleep laboratory is costly and requires professionally trained technicians to be on duty all night and perform tedious data interpretation. Currently, there is a severe shortage of sleep centers in my country, resulting in long waiting times for patients to make appointments.

[0006] The monitoring environment is unnatural: patients need to wear numerous electrodes and sensors to fall asleep in an unfamiliar hospital environment. This sense of constraint and psychological pressure can easily lead to the "first night effect," resulting in decreased sleep quality, and the monitoring data may not accurately reflect the patient's daily sleep status.

[0007] High diagnostic costs: A single PSG monitoring test is expensive and time-consuming, making it unsuitable as a primary screening method for large populations and also detrimental to long-term patient follow-up and efficacy evaluation.

[0008] Given the limitations of PSG, non-contact detection methods based on audio (snoring, breathing sounds) have become a research hotspot due to their advantages such as simple acquisition equipment (only a microphone is needed), non-contact operation, and low cost. However, existing technologies still face significant bottlenecks in practical applications: Limitations of traditional acoustic feature extraction: Early research relied primarily on hand-crafted features. Researchers typically segmented audio signals to extract time-domain features (such as zero-crossing rate and short-time energy), frequency-domain features (such as spectral centroids and formants), and cepstral features (such as MFCCs). However, these features have limited expressive power. Hand-crafted features depend heavily on the researcher's prior knowledge, making it difficult to fully capture the complex nonlinear dynamic changes and subtle pathological features in breath sounds. Furthermore, they exhibit poor noise resistance: these features are extremely sensitive to environmental noise (such as air conditioning noise, fan noise, and partner snoring). In home environments, background noise is complex and variable, leading to a high false alarm rate for traditional methods.

[0009] The computational bottleneck of deep learning methods is a significant challenge. While Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have achieved breakthroughs in audio processing in recent years, the mainstream approach involves converting one-dimensional audio signals into two-dimensional spectrograms and processing them using image classification models such as VGG and ResNet. This process consumes enormous computational resources: deep models typically contain millions or even hundreds of millions of parameters, involving massive floating-point operations (FLOPs). For example, ResNet-50 requires approximately 4 GFLOPs of computation. This is a huge burden for battery-powered portable monitoring devices or smartphones with limited computing power, making long-term real-time monitoring difficult and causing severe overheating. Furthermore, deep models are highly dependent on training data: they are prone to overfitting and require massive amounts of labeled data for training. However, labeling medical data requires a significant investment of time by medical professionals, and obtaining large-scale, high-quality labeled data is extremely costly.

[0010] The superficial use of clinical information is a concern. Some studies attempt to combine patients' baseline clinical data (such as age, sex, and BMI). Current fusion methods mostly involve simple numerical concatenation, creating a semantic gap: this simplistic numerical concatenation ignores the medical semantics behind the clinical data. For example, "BMI=32" medically means "obesity," and obesity leads to OSAHS through pathological mechanisms such as "upper airway fat accumulation" and "pharyngeal narrowing." Simple numerical inputs cannot allow the model to utilize this general medical knowledge, resulting in limited model generalization ability. When the distribution of test data changes (e.g., different races, different age groups), the model's performance often drops significantly. Summary of the Invention

[0011] The purpose of this application is to address the problems of high computational resource consumption, low feature extraction efficiency, insufficient semantic mining of clinical information, and weak model generalization ability in existing methods.

[0012] A method for detecting obstructive sleep apnea based on multimodal lightweight transformation includes the following steps: Data acquisition and preprocessing: Acquire raw audio signals, perform standardization on the raw audio signals and preprocess clinical baseline data to obtain a dataset;

[0013] MiniRocket-based audio feature extraction: This method utilizes random convolutional kernel transformations to replace traditional deep convolutional neural networks, achieving efficient temporal feature extraction of audio signals to obtain audio feature vectors. ; BioBERT-based clinical semantic feature extraction: Utilizing general medical knowledge embedded in a pre-trained language model to enhance the model's understanding of specific patient data and obtain text feature vectors. ; Multimodal feature fusion and classification decision: The audio feature vector obtained above is... and text feature vectors Feature concatenation is used for fusion, and then a ridge regression classifier is used for classification.

[0014] Furthermore, the original audio signal is acquired from various acquisition environments through multiple acquisition devices, specifically: the acquisition devices include smartphones, professional voice recorders, and microphone arrays, and the acquisition environments include home bedrooms and hospital wards.

[0015] Furthermore, the standardization process specifically includes: Resampling: Converting the sampling rate of all acquired raw audio signals to a uniform rate. ; Fixed-length segmentation: This method divides a continuous, long-duration sleep audio stream into short segments of fixed length. Let the original audio signal be... The i-th segment after segmentation is The duration is set to Second; Parameter selection: The corresponding number of sampling points is One point; Z-score standardization: for fragments First, calculate its mean. and standard deviation ; Standardized signal for: in, This represents the total number of sample points for a single audio segment. This represents the original amplitude of the nth sampling point in the i-th segment. Indicates the minimum value. This represents the amplitude of the nth sampling point in the i-th audio segment after standardization. After processing, the mean of each audio segment is 0 and the standard deviation is 1, eliminating the influence of the amplitude dimension.

[0016] Furthermore, the preprocessing of the clinical baseline data specifically includes: First, baseline data of the patients is collected and structured, including: Numerical features: Age: in years; Height: The unit is meters (m); Weight: The unit is kilograms (kg); Body Mass Index (BMI): A key indicator for measuring obesity, calculated using the following formula: in, Body Mass Index (BMI) The patient's weight (Mass) is expressed in kilograms (kg). The patient's height is expressed in meters (m).

[0017] Classification features: Gender: Male / Female.

[0018] Past medical history includes: hypertension, diabetes mellitus, hyperlipidemia, and heart disease.

[0019] Furthermore, the MiniRocket-based audio feature extraction specifically involves: The construction of the random convolution kernel space: MiniRocket's convolution kernel parameters are randomly generated and then fixed, specifically as follows: The kernel length is fixed at 100%. The weights of the convolution kernel Controlled in the set In the middle, and satisfying the constraint that the sum of the weights is 0; Number of convolution kernels: set to indivual; Multi-scale dilated convolution mechanism: To capture features at different scales using a fixed-length convolution kernel, specifically: dilation... The sampling interval of the convolution kernel on the input signal is determined by the input signal. When it is a standard convolution, when At this time, the receptive field of the convolution kernel expands; The exponential growth strategy employs an exponential growth void ratio strategy; in, exist Uniformly distributed within the range; For length of The input signal, the range of void ratio covers from local details To global outline At all levels; Convolutional transform and positive value scaling (PPV) statistics, for input signal and a specific convolution kernel Convolution output: for ; MiniRocket does not directly use convolution output. Instead of using features (which would result in excessive dimensionality), we calculate the Proportion of Positive Values ​​(PPV), a non-linear pooling operation that counts the proportion of samples in the convolution output that are greater than a certain threshold (Bias).

[0020] in, This represents the output sequence of the convolution operation. Indicates the length of the convolution output sequence. This indicates an indicator function that takes the value 1 when the condition within the parentheses is met, and 0 otherwise. This represents the bias threshold, and j represents the index of the output sequence. PV represents the value of the j-th sampling point in the output sequence. Physically, PV reflects the degree to which the input signal matches the convolution kernel pattern (i.e., the frequency of "activation"). Data-driven bias MiniRocket uses a data-driven approach to determine the bias threshold. Specifically: First, a small number of samples are randomly selected from the training set. Then, the convolution outputs of these samples and all convolution kernels are calculated. Finally, the quantiles of the convolution outputs are calculated as the bias term. The final audio feature vector generation: Assuming 84 convolutional kernels were used, each kernel applied... The void ratio was determined, and for each combination, a void ratio was used. Each bias term, ultimately generating audio feature vectors The dimensions are: Furthermore, the BioBERT-based clinical semantic feature extraction specifically involves: Clinical text generation (PromptEngineering);

[0021] The principle of BioBERT semantic encoding is to use a pre-trained BioBERT model as a feature extractor, selecting BioBERT models pre-trained on large-scale biomedical literature databases such as PubMed; Furthermore, the BioBERT semantic encoding specifically includes: Tokenization: Using BioBERT's tokenizer to generate text sequences BioBERT uses the WordPiece algorithm to convert token sequences. in, It is a special classification marker located at the beginning of a sentence, and its corresponding vector is usually regarded as the semantic representation of the entire sentence. It is a separator used to indicate the end of a sentence; Transformer encoding and self-attention mechanism: The token sequence is input into the BioBERT model, which contains multiple layers (usually 12 layers) of TransformerEncoder, and each layer contains a multi-head self-attention mechanism. The core formula of the self-attention mechanism is: in, This represents the query matrix, i.e., the currently viewed keywords. This represents the key matrix, i.e., the terms being queried. The value matrix represents the content information of the words. This indicates the dimension of the attention head, used to scale the dot product result and prevent gradient vanishing.

[0022] Semantic feature extraction: After processing through multiple Transformer layers, the semantic features are extracted from the last hidden layer. The vector corresponding to the tag As the semantic representation of the entire sentence, i.e., the text feature vector : Furthermore, the formula for feature fusion is: Before splicing, usually... L2 normalization is performed to match its modulus with the audio features.

[0023] Furthermore, the ridge regression classifier makes classification decisions as follows: Ridge regression solves for the weight matrix by minimizing a loss function with an L2 regularization term. : in, This represents a label matrix (e.g., {-1, 1} for binary classification). This represents the fused feature matrix (i.e., composed of multiple...) (The matrix formed) This represents the classifier weight matrix to be solved. This represents the regularization coefficient, indicating that the optimization problem has an analytical solution. Furthermore, the obstructive sleep apnea detection system based on multimodal lightweight transformation includes: Audio acquisition module: used to acquire ambient sounds during the patient's sleep in real time; Storage module: Used to store the acquired audio data, pre-trained MiniRocket convolutional kernel parameters (approximately 84 sets), pre-trained BioBERT model parameters, and classifier weight matrix; Processor module: Responsible for executing preprocessing, feature extraction, and classification algorithms; Human-computer interaction module: including touch screen or buttons, for users to input baseline data such as height, weight, age, and medical history, and to display the final test results. Beneficial effects

[0024] 1. The accuracy improvement brought about by the multimodal complementarity of this invention does not simply superimpose data, but utilizes the complementarity of audio and text at the information level. The audio signal provides the patient's immediate physiological state during sleep (such as the acoustic manifestations of airway obstruction), while the clinical text provides the patient's long-term health background (such as obesity, chronic disease history). Experiments have shown that this fusion mode of "immediate state + long-term background" significantly improves the accuracy and F1 score of OSAHS detection, especially in distinguishing between mild and moderate to severe patients. 2. MiniRocket offers extreme efficiency. Compared to deep learning methods based on ResNet or VGG, MiniRocket does not require GPUs for intensive backpropagation training. Its feature extraction process only involves convolution and addition, with a computational complexity of O(n log n). (Linear complexity) Under the same hardware conditions, the inference speed of this method is 10-100 times faster than that of traditional CNN, making it possible to achieve real-time monitoring on low-power devices such as smartphones and embedded chips. 3. The semantic understanding capability brought by BioBERT: Traditional numerical normalization methods lose the semantic information of the data. This invention transforms "BMI=35" into "SevereObesity" and inputs it into BioBERT. The model can then use pre-trained knowledge to understand the high correlation between "SevereObesity" and sleep apnea. This semantic awareness capability enhances the model's generalization ability to different patient groups, and the model maintains good robustness even when the data distribution changes (e.g., data from different hospitals). 4. Enhanced interpretability: Since the final classifier is linear (RidgeClassifier), we can analyze the weight matrix... This can be used to assess the importance of different features. For example, it can be used to analyze which audio frequency bands or clinical keywords (such as "Hypertension") contribute most to the final decision, thus providing doctors with a basis for auxiliary diagnosis. Attached Figure Description

[0025] The invention will now be further described with reference to the accompanying drawings; Figure 1 This is an overall flowchart of the OSAHS detection method (MR-BioBERT-OSA-Net) based on multimodal feature fusion proposed in this invention; Figure 2 This is a detailed schematic diagram illustrating the audio preprocessing and MiniRocket feature extraction principles of the present invention. Figure 3 This is a schematic diagram illustrating the process of generating clinical data text and using BioBERT semantic encoding in this invention. Figure 4 This is a comparison chart of ROC curves for different modalities (audio only, text only, and multimodal fusion) on the test set in this embodiment of the invention. Figure 5 This is a scatter plot comparing the inference time and accuracy of the method of this invention with traditional deep learning methods (such as ResNet and LSTM). Figure 6 This is a framework diagram of the method system of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0028] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] This invention provides a method for detecting obstructive sleep apnea-hypopnea syndrome based on multimodal feature fusion. The method includes the following core steps: data acquisition and preprocessing, audio feature extraction based on MiniRocket, clinical semantic feature extraction based on BERT, and multimodal feature fusion and classification decision.

[0030] Example 1 like Figure 1 - Figure 5 A method for detecting obstructive sleep apnea based on multimodal lightweight transformation is disclosed, comprising: Audio signals are the core data source for OSAHS detection. Due to the diversity of acquisition environments (home bedrooms, hospital wards) and acquisition devices (smartphones, professional recorders, microphone arrays), raw audio signals often suffer from inconsistent sampling rates and large amplitude differences. To ensure the stability and effectiveness of subsequent feature extraction, this invention designs a rigorous standardized preprocessing workflow, specifically: acquiring the raw audio signal, performing standardization processing on the raw audio signal, and preprocessing clinical baseline data to obtain a dataset;

[0031] The raw audio signals were acquired from various acquisition environments using a variety of acquisition devices, including smartphones, professional voice recorders, and microphone arrays, and the acquisition environments included home bedrooms and hospital wards.

[0032] The standardization process specifically involves: Resampling: Converting the sampling rate of all acquired raw audio signals to a uniform rate. ; Theoretical Basis: According to the Nyquist-Shannon sampling theorem, in order to recover an analog signal without distortion, the sampling frequency must be greater than twice the highest frequency of the signal. The main energy of human breathing sounds and snoring is concentrated in the low-frequency range, typically between 20Hz and 2000Hz, with some high-frequency components extending to 7-8kHz. Choosing a 16kHz sampling rate (Nyquist frequency of 8kHz) is sufficient to cover all effective frequency bands of breathing sounds, while reducing the data volume by more than 60% compared to CD quality's 44.1kHz or 48kHz, significantly reducing storage and computational burden.

[0033] Fixed-length segmentation: This method divides a continuous, long-duration sleep audio stream into short segments of fixed length. Let the original audio signal be... The i-th segment after segmentation is The duration is set to Second; Parameter selection: The corresponding number of sampling points is Two seconds is sufficient to include a complete breathing cycle (inhalation or exhalation) or a typical snoring event, without being too long to include multiple mixed events, which helps to improve temporal resolution.

[0034] Z-score standardization: To eliminate the gain differences between different recording devices and the volume effects caused by different microphone-patient distances, each audio segment is independently Z-score standardized.

[0035] For fragments First, calculate its mean. and standard deviation ; Standardized signal for: in, This represents the total number of sample points for a single audio segment. This represents the original amplitude of the nth sampling point in the i-th segment. This indicates a minimum value (e.g., 1e-8). This represents the amplitude of the nth sample point in the i-th audio segment after standardization; it is used to prevent division by zero error when the standard deviation is zero. After processing, the mean of each audio segment is 0 and the standard deviation is 1, thus eliminating the influence of the amplitude dimension.

[0036] Clinical baseline data preprocessing specifically involves: First, baseline data of the patients is collected and structured, including: Numerical features: Age: in years; Height: The unit is meters (m); Weight: The unit is kilograms (kg); Body Mass Index (BMI): A key indicator for measuring obesity, calculated using the following formula: in, Body Mass Index (BMI) The patient's weight (Mass) is expressed in kilograms (kg). The patient's height is expressed in meters (m).

[0037] Classification features: Gender: Male / Female.

[0038] Past medical history: This includes a history of hypertension, diabetes mellitus, hyperlipidemia, and heart disease. These characteristics are usually recorded as binary variables (yes / no).

[0039] Based on MiniRocket audio feature extraction, this application innovatively introduces the MiniRocket (MiniRandomConvolutionalKernelTransform) algorithm to process audio temporal signals: it uses random convolutional kernel transformation to replace the traditional deep convolutional neural network, achieving efficient temporal feature extraction of audio signals to obtain audio feature vectors. Its core advantages lie in its extremely fast transformation speed (10-100 times faster than traditional CNN) and deterministic feature mapping; The specific steps for audio feature extraction based on MiniRocket are as follows: For constructing the random convolutional kernel space, MiniRocket uses a predefined, fixed set of convolutional kernels. Unlike traditional deep convolutional neural networks (CNNs) that iteratively update convolutional kernel weights through backpropagation, MiniRocket's convolutional kernel parameters are randomly generated and then fixed, which avoids expensive gradient calculations.

[0040] The kernel length is fixed at 100%. Shorter convolutional kernels are better at capturing local high-frequency changes. Weight Constraint: To further accelerate computation, the weights of the convolutional kernel... Strictly limited to sets In addition, it must satisfy the constraint that the sum of the weights is 0 (i.e. ; Physical meaning: A weight sum of 0 means that the convolution kernel is a high-pass filter, which can effectively remove the DC component and low-frequency drift in the signal, focusing on the relative changes of the signal rather than the absolute values.

[0041] Number of convolution kernels: set to These 84 convolutional kernels are a carefully selected subset of all possible combinations that satisfy the above constraints (length 9, weights -1 or 2, sum 0), ensuring feature diversity.

[0042] Multi-scale dilated convolution mechanism: Breath sound signals contain rich temporal information: they range from short, plosive sounds (such as the onset of snoring) to longer, continuous airflow sounds. To capture features at different scales using fixed-length convolutional kernels, MiniRocket introduces a dilated convolution mechanism.

[0043] Dilation The hole ratio determines the sampling interval of the convolution kernel on the input signal. When it is a standard convolution, when At this time, the receptive field of the convolution kernel expands; The exponential growth strategy employs an exponential growth void ratio strategy; in, exist Uniformly distributed within the range; For length of The input signal, the range of void ratio covers from local details To global outline Multi-resolution analysis was achieved at each level; Convolutional transform and positive value scaling (PPV) statistics, for input signal and a specific convolution kernel (with void ratio) Convolution output for: Since the convolution kernel weights are only Convolution operations can be transformed into simple addition and subtraction operations, eliminating the need for floating-point multiplication and greatly improving computational efficiency. MiniRocket does not directly use convolution output. Instead of using features (which would result in excessive dimensionality), we calculate the Proportion of Positive Values ​​(PPV), a non-linear pooling operation that counts the proportion of samples in the convolution output that are greater than a certain threshold (Bias).

[0044] in, This represents the output sequence of the convolution operation. Indicates the length of the convolution output sequence. This indicates an indicator function that takes the value 1 when the condition within the parentheses is met, and 0 otherwise. This represents the bias threshold, and j represents the index of the output sequence. PV represents the value of the j-th sampling point in the output sequence. Physically, PV reflects the degree to which the input signal matches the convolution kernel pattern (i.e., the frequency of "activation"). Data-driven bias Choosing the discriminative power of a feature is crucial; data-driven bias terms MiniRocket uses a data-driven approach to determine the bias threshold. Specifically, it is: instead of random initialization: First, a small number of samples are randomly selected from the training set. Then, the convolution outputs of these samples and all convolution kernels are calculated. Finally, the quantiles of the convolution outputs are calculated as the bias term. This application calculates multiple quantiles (e.g., 0.25, 0.5, 0.75, etc.) of the convolutional output for each convolutional kernel, thereby generating multiple features. This method ensures that the generated features have good discriminative power in terms of numerical distribution.

[0045] The final audio feature vector generation: Assuming 84 convolutional kernels were used, each kernel applied... The void ratio was determined, and for each combination, a void ratio was used. Each bias term generates the final audio feature vector. The dimensions are: This high-dimensional feature vector is sparse, and the calculation process involves only addition and simple comparison operations, which greatly improves computational efficiency. The feature vector is then fed into the fusion layer.

[0046] BioBERT-based Clinical Semantic Feature Extraction: This application utilizes Natural Language Processing (NLP) technology to transform structured clinical data into semantic vectors to mine implicit medical knowledge. The core idea of ​​this step is to leverage the general medical knowledge inherent in a pre-trained language model to enhance the model's understanding of specific patient data and obtain text feature vectors. ; The clinical semantic feature extraction based on BioBERT is as follows: Clinical text generation (PromptEngineering) converts data into natural language text and inputs it into the BioBERT model; To input numerical and categorical data into the BioBERT model, it is first necessary to convert them into natural language text. A rule-based text generation template (PromptTemplate) was designed. This approach, known as "prompt engineering," aims to unify heterogeneous data into a natural language format that the model can understand.

[0047] Rule definition and medical logic: BMI Description Generation Rule: BMI is an important indicator for diagnosing OSAHS and is positively correlated with the risk of airway collapse. We map BMI values ​​to medical obesity classifications so that the model can understand their severity; like "...whichindicatesunderweight." (too underweight); like "...whichisnormalweight." (normal weight); like "...which indicates overweight." like "...which indicates mild obesity." like "...which indicates moderate obesity." like "...which indicates severe obesity." Example of generated sentence: "The body mass index is 29.5, which indicates mildobesity."; Past medical history description generation rules: Convert Boolean-type medical history fields into declarative sentences to clarify the patient's comorbidities; If Hypertension=Yes "The patient has hypertension."; If Diabetes=Yes "The patient has diabetes mellitus."; If Hyperlipoidemia = Yes "The patient has hyperlipoidemia."; If HeartDisease=Yes "The patient has heart disease."; If a certain item is No, then no corresponding sentence is generated or a negative sentence is generated (in this embodiment, it is preferred to generate only an affirmative sentence to reduce redundancy); Age description generation rules: "The age is {Age}." Text splicing and patient profile construction: The short sentences generated above are spliced ​​together in a fixed order to form a complete patient clinical profile. Full example: "Thebodymassindexis34.2,whichindicatesmoderateobesity.Thepatienthashypertension.Thepatienthasdiabetesmellitus.Theageis58."; The BioBERT semantic encoding principle employs a pre-trained BioBERT model as a feature extractor. BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) is a deep learning model based on the Transformer encoder. This invention selects BioBERT pre-trained on large-scale biomedical literature databases such as PubMed because, compared to general-purpose BERT or ClinicalBERT, it has a more accurate semantic understanding ability for specialized medical terms (such as "Hypopnea" and "Obesity").

[0048] BioBERT semantic encoding specifically involves: Tokenization: Using BioBERT's tokenizer to generate text sequences BioBERT converts token sequences using the WordPiece algorithm, which effectively handles out-of-vocabulary (OOV) words.

[0049] in, It is a special classification marker located at the beginning of a sentence, and its corresponding vector is usually regarded as the semantic representation of the entire sentence. It is a separator used to indicate the end of a sentence; Transformer encoding and self-attention mechanism: The token sequence is input into the BioBERT model, which contains multiple layers (usually 12 layers) of TransformerEncoder, and each layer contains a multi-head self-attention mechanism. The core formula of the self-attention mechanism is: in, This represents the query matrix, i.e., the currently viewed keywords. This represents the key matrix, i.e., the terms being queried. The value matrix represents the content information of the words. This indicates the dimension of the attention head, used to scale the dot product result and prevent gradient vanishing.

[0050] Through this mechanism, the model can calculate the relevance (AttentionScore) of each word in a sentence to all other words, thereby capturing long-distance dependencies and contextual semantics between words. For example, the model can "notice" that "Obesity" and "Hypertension" often coexist, thus bridging the gap between them in the feature space.

[0051] Semantic feature extraction: After processing through 12 Transformer layers, the semantic features are extracted from the last hidden layer. The vector corresponding to the tag As the semantic representation of the entire sentence, i.e., the text feature vector : This vector not only contains various indicators of the patient, but also implies the deep semantic associations of words such as "obesity" and "hypertension" in the medical context, which is something that traditional numerical coding cannot achieve.

[0052] Multimodal feature fusion and classification decision: The audio feature vector obtained above is... (Approximately 10,000 dimensions) and text feature vectors (768 dimensions) The features are fused using concatenation, and then classified using a ridge regression classifier.

[0053] The formula for feature fusion is: Before splicing, usually... L2 normalization is performed to match its modulus with the audio features.

[0054] Because the fused features have a high dimensionality (>10,000 dimensions) and the features generated by MiniRocket are sparse, using a linear classifier usually achieves better generalization performance than a complex nonlinear classifier, and the training speed is faster. The Ridge Regression Classifier makes its classification decision as follows: Ridge regression solves for the weight matrix by minimizing a loss function with an L2 regularization term. : in, This represents a label matrix (e.g., {-1, 1} for binary classification). This represents the fused feature matrix (i.e., composed of multiple...) (The matrix formed) This represents the classifier weight matrix to be solved. This represents the regularization coefficient, used to control model complexity and prevent overfitting. The optimization problem has an analytical solution.

[0055] This makes the training process highly efficient, eliminating the need for multiple epochs of optimization required by neural networks. In practical applications, leave-one-out cross-validation (LOOCV) can be used to quickly determine the optimal result. value.

[0056] In practice, the specific steps for audio feature extraction are as follows: Step 1: Data Reading and Buffering The system reads audio data in a streaming manner through the audio interface. For real-time processing, a circular buffer with a length of 2 seconds is set up. When the buffer is full, subsequent processing is triggered.

[0057] Step 2: Signal Standardization. The system reads 32,000 samples from the buffer. First, it calculates the mean and standard deviation, then performs Z-score standardization to normalize the signal amplitude to a standard normal distribution, eliminating volume differences caused by varying recording distances.

[0058] Step 3: MiniRocket Transformation Loading convolution kernels: During system initialization, 84 pre-generated random convolution kernels and their corresponding dilatation rate configurations are loaded from the storage module.

[0059] Perform convolution: Perform dilated convolution operations on the standardized signal. Since the convolution kernel weights are only -1 and 2, the processor only needs to perform addition and bit shift operations, without the need for floating-point multipliers, which greatly improves speed.

[0060] PPV feature calculation: For the output of each convolution kernel, calculate the proportion that is greater than a preset bias term. The bias term is also pre-trained and stored.

[0061] Vector generation: Combine the PPV values ​​of all convolution kernels under different dilation rates and bias terms to form a sparse feature vector with a dimension of approximately 10,000.

[0062] The specific implementation steps for clinical semantic feature extraction are described in detail in this embodiment, outlining the logic for processing clinical data: User data acquisition: Users input the following information through the human-computer interaction interface: gender (male), age (45 years old), height (175cm), weight (85kg), whether they have high blood pressure (yes), and whether they have diabetes (no).

[0063] Step S302: The text generation processor generates descriptive text based on the built-in rule engine: Calculate BMI = 85 / (1.75 * 1.75) = 27.76.

[0064] Matching rule: 27.5 <= BMI < 32.5, classified as "mild obesity".

[0065] Sentence 1 generated: "The body mass index is 27.76, which indicates mild obesity." Sentence 2 generated: "The patient has hypertension." Sentence 3 generated: "The age is 45." Spliced ​​text: "The body mass index is 27.76, which indicates mild obesity. The patient has hypertension. The age is 45." BioBERT inference Token segmentation: The BioBERT tokenizer is called to convert the above text into a sequence of token IDs.

[0066] Encoding: Input the Token ID sequence into a lightweight BioBERT model (a compressed version using model distillation techniques can be used to adapt it to mobile devices).

[0067] Extraction: Obtain the 768-dimensional vector of the first position ([CLS]) of the model output layer.

[0068] This embodiment describes the model training and validation process, specifically how to train the final classifier weights and hyperparameters during the research and development phase.

[0069] Dataset preparation: Collect overnight PSG monitoring data and synchronized audio recordings from 500 patients. Each audio segment was labeled (normal / apnea / hypoventilation) by a professional physician.

[0070] Feature engineering: Perform the feature extraction steps in Examples 2 and 3 on all samples respectively to obtain the audio feature matrix. and text feature matrix .

[0071] Feature fusion: and The features are concatenated to obtain the fused feature matrix. .

[0072] Classifier training: A Ridge Regression classifier is used. To select the optimal regularization parameters... Leave-One-Out Cross-Validation (LOOCV) is used.

[0073] Traversal .

[0074] For each Calculate the accuracy on the validation set.

[0075] Choose the one with the highest accuracy The values ​​are then used to retrain the dataset on the full dataset to obtain the final weight matrix. .

[0076] Experimental results analysis: The method of the present invention (MR-BioBERT-OSA-Net) was compared with existing mainstream methods on an independent test set (100 patients).

[0077] Table 1: Performance Comparison of Different Methods

[0078] Table 2: Comparison of Reasoning Speed ​​and Resource Consumption

[0079] Results analysis: Accuracy Advantage: MR-BioBERT-OSA-Net achieves an accuracy of 92.1%, significantly outperforming traditional SVM methods and the unimodal ResNet method. This demonstrates that incorporating clinical semantic information can effectively correct the bias of relying solely on audio judgment.

[0080] Speed ​​advantage: In a CPU environment, MR-BioBERT-OSA-Net's inference speed is 8 times that of ResNet-50 (15ms vs 120ms), which fully meets the needs of real-time monitoring (processing once every 2 seconds, with a time consumption of much less than 2 seconds).

[0081] Lightweight: The model size is only 25MB, making it easy to deploy on mobile devices.

[0082] Example 2 like Figure 6 This paper discloses an obstructive sleep apnea detection system based on multimodal lightweight transformation. The system architecture (named MR-BioBERT-OSA-Net, Multi-Modal Fusion Network) is a two-stream network structure. Audio Stream: The input is the original audio waveform, which is preprocessed and then mapped to a high-dimensional sparse feature vector through the MiniRocket transform layer.

[0083] Text Stream: Input the patient's clinical baseline data, generate natural language text through rule templates, and then extract semantic feature vectors through the BioBERT encoder.

[0084] Fusion Layer: Concatenates the two feature vectors.

[0085] Classification layer: A ridge regression classifier or a fully connected layer is used to classify the fused features and output diagnostic results, specifically: Audio acquisition module: Employs a high-sensitivity condenser microphone or MEMS microphone array with a sampling rate of no less than 16kHz, used to acquire ambient sounds during the patient's sleep in real time.

[0086] Storage module: Used to store the acquired audio data, pre-trained MiniRocket convolutional kernel parameters (approximately 84 sets), pre-trained BioBERT model parameters, and classifier weight matrix.

[0087] Processor module: This can be a low-power ARM architecture processor (such as the Cortex-A series) or an embedded DSP. The processor is responsible for executing preprocessing, feature extraction, and classification algorithms. Due to the high efficiency of the MiniRocket algorithm, there is no need to equip it with an expensive GPU accelerator.

[0088] Human-computer interaction module: Includes a touchscreen or buttons for users to input baseline data such as height, weight, age, and medical history, and displays the final test results (e.g., "Normal," "Mild OSAHS Risk," "Medical Consultation Recommended"). The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0089] The above formulas are all derived from software simulation using a large amount of data, and are selected to be close to the actual values. The coefficients in the formulas are set by those skilled in the art based on the actual situation. The above are only preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or changes made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting obstructive sleep apnea based on multi-modal lightweight transformation, characterized in that, The method comprises the following steps: Data acquisition and preprocessing: obtaining original audio signals through acquisition devices, standardizing the original audio signals, and preprocessing clinical baseline data to obtain a data set; Based on MiniRocket audio feature extraction: using random convolution kernel transformation instead of traditional deep convolutional neural network, realizing efficient time sequence feature extraction of audio signal to obtain audio feature vector ; BioBERT-based clinical semantic feature extraction: Utilize the general medical knowledge contained in the pre-trained language model to enhance the model's understanding of specific patient data to obtain a text feature vector ; Multimodal feature fusion and classification decision: the audio feature vector and the text feature vector are fused by feature splicing, and then classified by a ridge regression classifier.

2. The method of claim 1, wherein, The original audio signals are obtained through various acquisition devices from various acquisition environments, specifically: the acquisition devices include smartphones, professional recording pens, and microphone arrays, and the acquisition environments include home bedrooms and hospital wards.

3. The multi-modal lightweight transform based obstructive sleep apnea detection method according to claim 2, wherein, The standardization processing specifically includes: Resampling: converting the sampling rate of all collected raw audio signals uniformly to ; Fixed-length segmentation: the continuous long sleep audio stream is cut into short segments with fixed length, and the original audio signal is , the i-th segment after cutting is , and the length is set to seconds; The corresponding sampling point number is points; Z-score standardization: for fragments , first compute their mean and standard deviation ; standardized signal is: wherein, denotes the total number of sample points of a single audio segment, denotes the original amplitude of the nth sample point in the ith segment, denotes the minimum value, denotes the amplitude of the nth sample point in the ith audio segment after normalization.

4. The multi-modal lightweight transform based obstructive sleep apnea detection method of claim 3, wherein, The clinical baseline data preprocessing specifically includes: First, collect the patient's baseline data and structure it, including numerical features and categorical features.

5. The multi-modal lightweight transform based obstructive sleep apnea detection method according to claim 4, wherein, The MiniRocket audio feature extraction specifically includes: Construction of random convolution kernel space: the convolution kernel parameters of MiniRocket are fixed after being randomly generated, specifically: The kernel length is fixed as The weights of the convolution kernel The convolution kernel is prepared in a set , and satisfies the constraint that the sum of the weights is 0, the number of convolution kernels: set to The multi-scale dilated convolution mechanism: To capture features of different scales with fixed-length convolution kernels, specifically: the dilation rate determines the sampling interval of the convolution kernel on the input signal, when is the standard convolution, when the receptive field of the convolution kernel is expanded; Exponential growth strategy: exponential growth of the hole rate strategy; wherein In are uniformly distributed within the range For input signals of length the range of voids covers various levels from local details to global contours ; Convolution transform and positive predictive value (PPV) statistics, for an input signal and a certain specific convolution kernel , the convolution output is ; MiniRocket does not directly use the convolution output Instead of using the features as such, it computes their positive value proportion PPV, which is a non-linear pooling operation that counts the proportion of samples in the convolution output that are larger than a certain threshold; wherein, represents an output sequence of a convolution operation, represents a length of the convolution output sequence, represents an Indicator Function, which takes a value of 1 when the condition in the bracket is satisfied, otherwise 0, represents a Bias Threshold, j represents an index of the output sequence, represents a value of the jth sample point in the output sequence; MiniRocket employs a data-driven approach to determine the bias term threshold , in particular: First, randomly select a small number of samples from the training set, then calculate the convolution output of these samples with all convolution kernels, and finally calculate the quantile of the convolution output as the bias term; Generation of audio feature vectors: Assume that 84 convolutional kernels are used, each of which applies a void fraction; and for each combination a bias term is used The final generated audio feature vector has a dimension of wherein, represents a preset number of hole rate transformation seeds, represents the number of quantile bias items extracted for each combination of convolution kernel and hole rate.

6. The multi-modal lightweight transform based obstructive sleep apnea detection method according to claim 5, wherein, The BioBERT-based clinical semantic feature extraction specifically includes: Clinical text generation and BioBERT semantic encoding: BioBERT semantic encoding uses a pre-trained BioBERT model as a feature extractor, and selects a BioBERT pre-trained on the PubMed large biomedical literature database.

7. The multi-modal lightweight transform based obstructive sleep apnea detection method according to claim 6, wherein, The BioBERT semantic encoding specifically includes: Tokenization: The generated text sequence is tokenized using the tokenizer of BioBERT converted into a Token sequence, BioBERT uses the WordPiece algorithm; wherein, is a special classification token, located at the beginning of a sentence, whose corresponding vector is usually considered as the semantic representation of the whole sentence, is a separator token, used to identify the end of a sentence; Transformer encoding and self-attention mechanism: input Token sequence into the BioBERT model, which contains multiple layers of TransformerEncoder, each layer containing a multi-head self-attention mechanism; The core formula of the self-attention mechanism is: wherein, denotes the query matrix, i.e. the word of current interest, denotes the key matrix, i.e. the word being queried, denotes the value matrix, i.e. the content information of the word, denotes the dimension of the attention head; Semantic feature extraction: After multi-layer Transformer processing, the last hidden layer is extracted The corresponding vector is marked As the semantic representation of the entire sentence, i.e. the text feature vector :

8. The multi-modal lightweight transform based obstructive sleep apnea detection method according to claim 7, wherein, The formula for feature fusion is: Before splicing, L2 normalization is usually performed on the image.

9. The multi-modal lightweight transform based obstructive sleep apnea detection method according to claim 8, wherein, The ridge regression classifier performs classification and decision specifically as follows: Ridge regression solves the weight matrix by minimizing a loss function with an L2 regularization term : wherein, denotes a label matrix, denotes a fused feature matrix, denotes a classifier weight matrix to be solved, denotes a regularization coefficient.

10. A system for obstructive sleep apnea detection based on multi-modal lightweight transformation, the system being adapted to the method of any one of claims 1 to 9, characterized in that, It includes: Audio acquisition module: used for real-time acquisition of environmental sounds during patient sleep; Storage module: used for storing acquired audio data, pre-trained MiniRocket convolution kernel parameters, pre-trained BioBERT model parameters, and classifier weight matrix; Processor module: responsible for executing preprocessing, feature extraction, and classification algorithms; Human-computer interaction module: including touch screen or keys, used for user input of height, weight, age, and medical history baseline data, and displaying the final detection results.

Images