A lung function parameter measurement system based on multi-modal feature processing, a data processing method and a terminal device

Abstract

The application discloses a lung function parameter measurement system based on multi-modal feature processing, a data processing method and a terminal device, and belongs to the technical field of audio signal processing and deep learning algorithm. In order to solve the defects that the existing respiratory acoustic analysis mode is single, the lung function index cannot be quantified, the cross-site generalization is poor, the measurement precision is low, and there is no standardized clinical grading, the application adopts a cough, forced exhalation and blowing dual-channel acoustic signal input, relies on two independent AST audio Transform encoder to realize feature decoupling extraction, combines multi-center site adversarial training to remove scene interference, fuses population characteristics, and outputs respiratory disease correlation probability, FEV1 and FVC core lung function parameters in parallel. The measurement precision and generalization ability of the application are excellent, the application can be deployed on a general intelligent terminal, is suitable for multiple scenes such as home self-measurement and primary screening, and has high landing application value.

Description

Technical Field

[0001] This invention belongs to the fields of audio signal processing and multimodal deep learning computer algorithm technology, specifically relating to a multimodal feature processing system for respiratory acoustic data, an automated lung function index calculation system, a data processing method, and a portable smart terminal device. Background Technology

[0002] Respiratory health screening and quantitative monitoring of lung function are important technical components of primary public health screening, home health monitoring, and follow-up of chronic respiratory diseases. Traditional clinical lung function testing relies on professional pulmonary function instruments, which require patients to cooperate in performing a standardized set of forceful exhalation and inhalation. The equipment is expensive, testing sites are limited, the testing process is complex, and it cannot achieve routine home screening, making it difficult to meet the needs of large-scale initial screening and long-term dynamic monitoring.

[0003] With the development of acoustic sensing technology and deep learning algorithms, intelligent analysis technology based on human respiratory acoustic signals has become an important research direction for non-invasive quantitative assessment of lung function. Current technologies generally use a single cough audio recording as model input, extracting cough acoustic features through deep learning models to achieve probabilistic screening and classification of respiratory diseases. However, this approach can only output qualitative disease-related probabilities, suffers from limitations such as a single input modality and significant technical functional limitations, and cannot quantitatively output core clinical lung function indicators. Overall, its practical applicability and reference value are limited.

[0004] Existing adversarial optimization mechanisms for similar acoustic analysis models only focus on differences in data acquisition hardware, performing adversarial corrections for microphone hardware offsets in different brands and models of mobile phones and wearable devices. This only addresses data distribution differences at the hardware level. However, in real-world multi-center clinical deployment scenarios, different hospitals and screening sites experience multi-dimensional non-target interference offsets due to environmental noise, acquisition distance, operator guidance methods, regional population physiological characteristics, and acquisition posture. Single hardware adversarial mechanisms cannot eliminate multi-site domain offsets, resulting in poor generalization ability and significant attenuation of measurement accuracy after cross-center deployment, failing to meet the needs of large-scale clinical implementation.

[0005] Existing technical solutions generally adopt a single-task, single-branch model structure, which couples the disease classification task and the parameter regression task in the same feature extraction link for training. The feature learning objectives, loss convergence direction and feature interest regions of the two types of tasks are significantly different. Coupled training will result in technical defects such as mutual interference between tasks, inaccurate feature extraction, unstable model convergence, and large parameter measurement errors.

[0006] Furthermore, existing publicly available technologies can only output discrete probability values ​​and raw acoustic feature vectors, lacking a standardized grading calculation system that conforms to clinical lung function evaluation standards. The output data lacks unified grade labels and quantitative comparison standards, has a low degree of data structuring, poor clinical readability, and cannot directly provide standardized reference for screening work.

[0007] The applicant's prior authorized patent CN120954690B discloses a multi-label respiratory disease classification system based on cough audio combined with symptom text and device adversarial training. This solution only uses a single-channel cough audio input, and the adversarial optimization goal is only to eliminate the differences in the acquisition hardware devices. It can only realize the output of respiratory disease association probability classification. It does not have the ability to extract the audio features of forceful blowing, does not have an independent branch for lung function parameter regression calculation, does not have a multi-center site domain adversarial optimization mechanism, and does not have an automated grading calculation function for clinical standard matching. It cannot solve the above-mentioned technical pain points that are common in the industry. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, this invention aims to provide a lung function parameter measurement system, data processing method, and terminal device based on multimodal feature processing. This invention employs dual independent input modalities for cough acoustic signals and forced expiratory blowing acoustic signals, coupled with a dual-branch independent AST audio Transformer feature extraction architecture. This decouples the disease association probability classification task from the lung function parameter regression measurement task. Furthermore, it adds a multi-center site adversarial optimization mechanism and an automatic grading calculation module conforming to international clinical standards, addressing the shortcomings of existing technologies from five dimensions: data source, feature extraction, task training, anti-interference, and data output.

[0009] To solve the above-mentioned technical problems and achieve the above-mentioned technical effects, the present invention is implemented through the following technical solution: A lung function parameter measurement system based on multimodal feature processing includes: a data preprocessing module, a bi-branch feature extraction module, a multimodal fusion module, a numerical prediction module, a site adversarial optimization module, a medical data hierarchical calculation module, and a data output module; The data preprocessing module is configured to acquire and process two channels of respiratory acoustic signals and structured population feature data. The two channels of respiratory acoustic signals include cough audio signals and forced exhalation audio signals. The data preprocessing module performs endpoint detection, resampling, amplitude normalization, Mel spectrum feature extraction, and temporal alignment processing on the cough audio signals and forced exhalation audio signals, respectively, to generate cough audio tensors and exhalation audio tensors. The module performs standardization, one-hot encoding, and derives BMI (Body Mass Index) features from the structured population feature data to generate population feature vectors. The dual-branch feature extraction module includes a first audio encoding branch and a second audio encoding branch that are independent of each other. The first audio encoding branch uses an AST audio Transformer encoder (audio spectrum Transformer encoder) to extract global acoustic features of cough from the cough audio tensor. The second audio encoding branch uses an independent AST audio Transformer encoder (audio spectrum Transformer encoder) to extract global acoustic features of blowing from the blowing audio tensor. The site adversarial optimization module is coupled to the outputs of the first audio coding branch and the second audio coding branch, respectively. The site adversarial optimization module is only enabled during the model training phase. Each set of site adversarial optimization modules includes a self-attention aggregation layer, a gradient inversion layer (GRL), and a site classification head. The gradient inversion layer is configured with forward propagation identity mapping and backpropagation gradient inversion to enable the corresponding AST audio Transformer encoder (audio spectrum Transformer encoder) to strip away the site features and generate site-independent acoustic representations through adversarial training. The gradient inversion layer is configured with a dynamic scheduling mechanism to control the adversarial weight λ to adaptively adjust with the training progress. λ represents the gradient scaling coefficient of the gradient inversion layer, and its value increases with the training progress. During backpropagation, the gradient generated by the site classification loss is scaled by λ, inverted, and then passed to the corresponding AST audio Transformer encoder. This enables the feature extraction network to maximize the site classification error while optimizing the target task loss, thereby reducing the information of the sites included in the acoustic features. The multimodal fusion module includes two independent fusion units, corresponding to the first audio coding branch and the second audio coding branch, respectively. Each fusion unit is configured to: map the acoustic features and population feature vectors output by the branch to the same feature dimension space and then concatenate them, and process them through the fusion MLP layer (multilayer perceptron layer) to generate branch fusion features. The numerical prediction module includes a parallel-configured correlation probability prediction submodule and a lung function parameter regression submodule. The correlation probability prediction submodule receives the branch fusion features output from the first audio encoding branch and outputs correlation probability values ​​for various respiratory diseases through a multi-layer MLP classification head. The lung function parameter regression submodule receives the branch fusion features output from the second audio encoding branch and outputs FEV1 and FVC values ​​through a regression head with ReLU activation (linear rectified activation function). Here, FEV1 represents forced expiratory volume in one second, and FVC represents forced vital capacity. The medical data grading and calculation module is configured to read the associated probability values, FEV1 calculation values, and FVC calculation values, calculate the predicted FVC value and the percentage of FVC to the predicted value based on the GLI-2012 standard formula (Global Lung Function Reference Values ​​Standard Formula) combined with population characteristics, and complete the indicator grading calculation by preset numerical matching matrix to generate standardized feature grading data. The data output module is configured to output respiratory disease association probability, FEV1 measurement value, FVC measurement value, FVC percentage of predicted value, and standardized feature classification data.

[0010] Furthermore, the steps of the data preprocessing module in processing the forced exhalation audio signal include: calibrating the effective exhalation start and end intervals of the exhalation audio through energy envelope detection, and unifying the time sequence alignment to a fixed long time sequence dimension; when the data preprocessing module processes the cough audio signal, it generates cough audio tensors for multiple effective cough segments, and after processing by the first audio encoding branch, it takes the average of the correlation probability values ​​of the multiple output segments as the final correlation probability.

[0011] Furthermore, the site classification head is a two-layer MLP structure. The input is the global acoustic features compressed by the self-attention aggregation layer, and the output is the classification probability of each collection site in the training set. During the training phase, the site classification loss adopts the cross-entropy loss, which is used to drive the site classification head to identify the collection site. The gradient inversion layer backpropagates and inverts the gradient, driving the AST audio Transformer encoder (audio spectrum Transformer encoder) to weaken the site-specific acoustic features.

[0012] Furthermore, the association probability prediction submodule uses the Sigmoid activation function to output multi-label association probabilities, and the training loss uses a binary cross-entropy loss with Logits; the lung function parameter regression submodule uses the SmoothL1 loss function to reduce the interference of abnormally collected samples on the accuracy of parameter calculation.

[0013] Furthermore, the preset numerical matching matrix within the medical data grading and calculation module contains three levels of calculation logic: Level 1: Based on a preset probability threshold, match the associated probability values ​​to generate disease association level identifiers; Level 2: Lung function impairment classification based on the numerical range of FVC as a percentage of the predicted value; Level 3: Based on the ratio of FEV1 measured value to FVC measured value, generate ventilation obstacle type feature identifier; The standardized feature classification data includes three types of structured numerical identifiers: disease association level, lung function impairment classification, and ventilation disorder type identifier.

[0014] Furthermore, the AST audio Transformer encoder (audio spectrum Transformer encoder) is initialized using AudioSet pre-trained weights, and the parameters of the first 6 layers of the network are frozen during training. After unfreezing, the 6 layers of the network are fine-tuned. The AST audio Transformer encoder (audio spectrum Transformer encoder) parameters of the first audio encoding branch and the second audio encoding branch in the dual-branch feature extraction module are independent of each other and are not shared.

[0015] Furthermore, a differentiated training strategy is adopted during the model training phase. The first audio encoding branch and the association probability prediction submodule use the first set of hyperparameter training configurations, while the second audio encoding branch and the lung function parameter regression submodule use the second set of hyperparameter training configurations. The learning rate, iteration rounds, and data augmentation strategies of the two sets of training configurations are independent of each other. The first set of hyperparameter configurations consists of a set of training parameters set for the first audio encoding branch and the association probability prediction submodule, including the learning rate, weight decay coefficient, batch size, iteration rounds, loss function weights, and data augmentation strategies, wherein the data augmentation strategies are as follows: The first set of hyperparameters includes at least one or more of temporal occlusion, frequency occlusion, random pruning, and noise perturbation, used to enhance the robustness of the association probability prediction task to different acoustic scenarios; the second set of hyperparameters is a set of training parameters set for the second audio coding branch and the lung function parameter regression submodule, including learning rate, weight decay coefficient, batch size, iteration rounds, regression loss weights, and data augmentation strategies, wherein the data augmentation strategies include at least one or more of loudness perturbation, duration stretching, spectral perturbation, and hybrid augmentation, used to improve the adaptability of the lung function parameter regression task to individual differences and changes in acquisition conditions.

[0016] This invention also provides a method for processing lung function parameter measurement data based on multimodal feature processing, executed by the aforementioned measurement system, comprising the following steps: S1. Data Acquisition and Preprocessing Steps: Acquire user cough audio, forced exhalation audio, and population structure features; Perform endpoint detection, resampling, normalization, Mel spectrum extraction, and temporal alignment on the two acoustic signals to generate audio tensors; Standardize and encode population features to generate population feature vectors; S2, Dual-branch feature extraction and site adversarial constraint steps: Input the cough audio tensor into the first audio coding branch to extract cough acoustic features, and input the blowing audio tensor into the second audio coding branch to extract blowing acoustic features; During the training phase, gradient inversion adversarial constraints are applied through the corresponding site adversarial optimization module to remove the acquisition site interference information in the acoustic features; S3. Multimodal feature fusion step: Map the two acoustic features and population feature vectors to a unified dimensional space respectively, and after concatenation, obtain the fused features of two independent branches through fusion MLP processing; S4. Parallel numerical prediction step: Input the first branch fusion feature into the association probability prediction submodule and output the association probability values ​​of multiple respiratory diseases; input the second branch fusion feature into the lung function parameter regression submodule and output the FEV1 and FVC values; where FEV1 represents the forced expiratory volume in one second and FVC represents the forced vital capacity. S5. Medical data grading calculation steps: Calculate the percentage of FVC to the predicted value based on the GLI-2012 formula, call the preset numerical matching matrix to perform grading calculations on the association probability, the percentage of FVC to the predicted value, and the FEV1 / FVC ratio, and generate standardized feature grading data; S6. Data Output Steps: Output all calculated values ​​and standardized feature classification data to generate a structured reference dataset.

[0017] Furthermore, the complete model training process includes: T1. Dataset Construction: Samples collected from multiple centers are divided into training set, validation set, and test set according to the patient dimension. Samples from each collection site within the training set are sampled evenly. The sample labels include the site identifier corresponding to the audio, the clinical gold standard lung function value, and the disease label annotated by the physician. T2, Branch Independent Pre-training: Initialize two sets of AST audio Transformer encoders (audio spectrum Transformer encoders) respectively, freeze the underlying parameters, and train the correlation probability prediction link of the first audio coding branch and the lung function parameter regression link of the second audio coding branch independently in sequence; during the training process, the site adversarial weights λ are dynamically adjusted according to the preset scheduling rules. T3. Model convergence verification: Early stopping is performed based on the validation set AUC (area under the curve) and FVC (mean absolute error) to calculate the MAE (mean absolute error) and save the optimal weight model. T4. Classification of the tiered operation module: The probability of the output of the validation set samples and the calculation parameters are used to traverse the grid search to determine the threshold of each tier in the numerical matching matrix, and the tiered operation logic is solidified.

[0018] A third aspect of the present invention provides an intelligent terminal device, comprising: a memory, a processor, a microphone acquisition unit, and a display output unit; The memory stores a computer-executable program, and when the processor executes the computer-executable program, it implements the above-mentioned lung function parameter measurement data processing method based on multimodal feature processing. The microphone acquisition unit is configured to acquire audio of the user coughing and forceful exhalation; the display output unit is configured to display the FEV1 measurement value, FVC measurement value, disease association probability, and standardized feature classification data output by the processor.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention adopts a dual-channel respiratory acoustic signal joint analysis mechanism, which is different from the traditional single cough audio solution. It adds a forced exhalation and blowing audio processing link, which is specifically adapted to the technical requirements of lung ventilation function parameter measurement. It can accurately output the core lung function quantitative indicators of FEV1 and FVC, which fills the technical gap of existing technologies that can only conduct qualitative screening but cannot conduct quantitative assessment, and greatly improves the clinical reference value and application scope of acoustic intelligent analysis technology.

[0020] 2. The site adversarial optimization mechanism of this invention differs from traditional adversarial training schemes that only target hardware devices. It takes multi-center collection sites as adversarial learning targets and uses a gradient inversion adaptive scheduling mechanism to remove non-target interference information such as environment, personnel, region, and collection process from acoustic features. It learns general and robust respiratory acoustic features, which greatly improves the generalization ability of the model across hospitals and sites. There is no need to readjust the model for new sites, making it suitable for large-scale public health screening scenarios.

[0021] 3. This invention introduces the GLI-2012 internationally recognized lung function reference standard and builds a three-level automated numerical matching and grading calculation system. It can automatically complete the structured labeling output of disease association level, degree of lung function impairment, and ventilation disorder type. It solves the problems of traditional model output data being scattered, lacking standards, having poor readability, and being unable to be directly adapted to screening business, and realizes the automated conversion of raw measurement data to standardized clinical reference data.

[0022] 4. This invention splits the disease multi-label classification task and the lung function parameter regression task into two completely independent audio coding branches and prediction heads. It adopts a differentiated training strategy and a dedicated data augmentation mechanism to avoid mutual interference between the training processes of the two types of tasks. It matches the feature learning rules of different tasks, significantly improves the accuracy of model feature extraction, probability prediction accuracy and parameter calculation accuracy. The model's convergence stability is significantly better than the traditional single-branch coupling architecture.

[0023] 5. This invention can be deployed on conventional smart terminals (such as mobile phones), and only relies on a general microphone to complete the acquisition of dual acoustic signals. It does not require professional lung function testing hardware, fixed testing sites, or special operation by medical personnel. It can be adapted to multiple scenarios such as home self-testing, primary screening at the grassroots level, follow-up of chronic diseases, and large-scale population surveys, which greatly reduces the technical threshold and equipment cost of quantitative lung function monitoring.

[0024] It should be noted that the FEV1, FVC calculation values, disease association probability, and grading labels output by this invention are only computer reference data for medical personnel to conduct health assessments and cannot be used as the sole basis for disease determination. A complete health assessment still requires professional physicians to make a comprehensive judgment based on imaging and clinical examinations. This invention does not protect any human disease diagnosis or treatment procedures.

[0025] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0026] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall module architecture of the measurement system of the present invention.

[0027] Figure 2 This is a schematic diagram of the complete reasoning process of the data processing method of the present invention.

[0028] Figure 3 This is a schematic diagram of the entire training process of the model of this invention. Detailed Implementation

[0029] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings to provide a clearer understanding of the invention's purpose, features, and advantages. It should be understood that the embodiments shown in the drawings are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the invention's technical solution.

[0030] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known apparatuses, structures, and techniques associated with this application may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0031] Unless the context requires otherwise, throughout the specification and claims, the word “comprising” and its variations, such as “including” and “having”, shall be understood to have an open, inclusive meaning, that is, to be interpreted as “including, but not limited to”.

[0032] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0033] The singular forms “a” and “the” used in this specification and the appended claims include plural references unless otherwise expressly stated herein. It should be noted that the term “or” is generally used to mean “and / or” unless otherwise expressly stated herein.

[0034] Furthermore, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. Example 1:

[0035] See Figure 1 As shown, the lung function parameter calculation system based on multimodal feature processing in this embodiment includes: 1. Data Preprocessing Module In accordance with the dual-channel acoustic acquisition standard, the terminal microphone acquires multiple valid cough audio segments and a single complete forceful exhalation audio segment from the user, while also recording the user's demographic data (age, gender, height, weight).

[0036] Cough audio preprocessing: VAD endpoint detection (voice activity endpoint detection) is used to remove silent segments and environmental noise interference, and the audio is uniformly resampled to the standard sampling frequency. Amplitude normalization is performed, Mel spectrum temporal features are extracted, the temporal dimension is standardized, and a standardized cough audio tensor is generated to match the model input requirements.

[0037] Forced exhalation audio preprocessing: The effective exhalation start and end intervals are accurately located using the energy envelope detection algorithm, the early inhalation noise and the late tail noise interference are eliminated, the time sequence length is unified, and a standardized exhalation audio tensor is generated.

[0038] Population feature preprocessing: Z-score standardization (standard deviation standardization) is performed on continuous physical characteristic data, one-hot encoding is performed on discrete gender features, and derived BMI (body mass index) features are calculated. These are integrated to generate population feature vectors with fixed dimensions and standardized distribution, meeting the requirements of multimodal fusion input.

[0039] 2. Dual-branch feature extraction module Following a decoupled design approach, the system is configured with two sets of completely independent and non-shared AST audio Transformer encoders (audio spectrum Transformer encoders), corresponding to cough audio feature extraction and blowing audio feature extraction, respectively. The encoders are initialized using industry-standard pre-trained weights. The underlying basic feature layer is frozen according to a fine-tuning strategy, and only the upper semantic feature layer is fine-tuned. The system learns disease-related features corresponding to cough acoustic signals and pulmonary function ventilation features corresponding to blowing acoustic signals, respectively, to avoid confusion between the two types of features and ensure the relevance and accuracy of feature extraction.

[0040] 3. Site Anti-Optimization Module Based on the multi-center generalization optimization scheme, each of the two encoding branches is equipped with an independent site adversarial optimization module. This module is only enabled during the model training phase and automatically disabled during the inference and prediction phase, without affecting normal measurement efficiency. Global feature compression is achieved through a self-attention aggregation layer, and adversarial training is implemented by combining it with a gradient reversal layer. The adversarial weights are dynamically adjusted in conjunction with preset adaptive scheduling rules, gradually stripping away interference information such as acquisition sites, environment, and operation procedures contained in the acoustic features. This allows the encoder to learn only the inherent acoustic features that are strongly correlated with the respiratory system state and lung function indicators, achieving stable model adaptation across sites and scenarios.

[0041] 4. Multimodal fusion module Based on the multimodal fusion design, two high-dimensional acoustic features and standardized population feature vectors are mapped to a unified feature dimension space. Through splicing and fusion and multilayer perceptron nonlinear transformation, the acoustic features and human physiological features are deeply integrated. This fully utilizes the constraint and correction effect of population signs on lung function parameters, makes up for the lack of robustness of single acoustic features, and improves the accuracy of subsequent probability prediction and parameter regression.

[0042] 5. Numerical Prediction Module Following the dual-task decoupling approach, parallel and independent correlation probability prediction submodules and lung function parameter regression submodules are set up.

[0043] The cough audio branch corresponds to the probability prediction submodule, which outputs the probability values ​​of association of multiple respiratory diseases based on fusion features, covering the screening needs of common respiratory diseases in clinical practice. The blowing audio branch corresponds to the lung function parameter regression submodule, which accurately outputs core quantitative indicators such as FEV1 (forced expiratory volume in one second) and FVC (forced vital capacity), realizing non-invasive and intelligent measurement of lung ventilation function.

[0044] During the training phase, a differential loss function is used. For classification tasks, a binary cross-entropy loss with Logits is used, while for regression tasks, a SmoothL1 loss function (smoothing absolute loss function) is used to effectively suppress the interference of abnormal samples and noisy samples on the model accuracy.

[0045] 6. Medical Data Hierarchical Calculation Module Based on the clinical standard adaptation plan, the standard formula of GLI-2012 Global Lung Function Reference Values ​​is retrieved, and the individual FVC standard predicted value and FVC%pred (FVC as a percentage of the predicted value) are calculated in combination with the user's demographic characteristics. Relying on the preset numerical matching matrix, three-level standardized grading calculations are completed: disease association level classification, lung function impairment degree classification, and ventilation disorder type identification classification. The discrete raw values ​​output by the model are converted into structured graded data that fits the clinical evaluation system, realizing the standardization, visualization, and readability of the measured data, and fully matching the standardized output design goals.

[0046] 7. Data Output Module The terminal integrates all calculation results, including disease association probability, FEV1 calculation value, FVC calculation value, FVC percentage of predicted value, and three-level standardized classification identifier, and generates a structured JSON (Lightweight Data Exchange Format) reference dataset for visualization, which can be viewed by users, referenced by medical staff, and stored and exported in the terminal backend. Example 2:

[0047] See Figure 2 As shown, the lung function parameter measurement data processing method based on multimodal feature processing in this embodiment adopts the system described in Embodiment 1, covering the entire chain of steps from raw data acquisition, dual-channel audio preprocessing, dual-branch feature extraction, site interference removal, multimodal feature fusion, parallel numerical prediction, standardized hierarchical calculation to final data output. The following is a complete experimental description using a 58-year-old male subject as an example: S1. Collect 3 cough audio segments, 1 blowing audio segment, and population characteristics (58 years old, male, 172cm, 68kg). Preprocess to generate two audio tensors and a 5-dimensional population vector. S2, the two AST encoders extract acoustic features respectively, and the site adversarial module is not enabled during the inference stage; S3. The two features are fused with population features to generate 256-dimensional branch fused features. S4, cough branch output P_ILD=0.78, blowing branch output FEV1=2.31L, FVC=2.68L; S5. The predicted FVC value calculated by GLI-2012 is 4.02L, and FVC%pred (FVC as a percentage of the predicted value) = 66.7%; Matching matrix analysis indicates: high correlation with ILD, moderate lung function impairment, and restrictive ventilation characteristics. S6. The terminal outputs all calculated values ​​and standardized hierarchical datasets for medical staff to use as reference data. Example 3:

[0048] See Figure 3 As shown, the complete model training process in this embodiment includes the following steps: 1. Dataset Construction: Samples were collected from multiple tertiary hospitals and divided into training / validation / test sets according to the patient disjointness rule. The training set was sampled evenly from each site, and the labels included site number, clinically measured FEV1 / FVC, and disease labels annotated by physicians. 2. Independent Branch Training: First, train the cough classification branch using the AdamW optimizer (adaptive moment estimator with weight decay), encoder with a learning rate of 1e-4, classification head with a learning rate of 5e-4, epoch 100, and early stopping for 15 epochs; then train the blowing regression branch independently, epoch 150, and early stopping for 20 epochs; add temporal stretching data augmentation to the blowing audio; during training, λ is dynamically adjusted according to the preset scheduling rules. 3. Convergence Validation: After the validation set AUC and FVC-MAE meet the criteria, the optimal model weights are saved; 4. Grading threshold calibration: Traverse the validation set output data grid to search for the optimal probability and FVC%pred (FVC percentage of the predicted value) grading threshold, and solidify it into the medical data grading calculation module. Example 4:

[0049] This embodiment discloses a mobile terminal, including a microphone, storage, a processor, and a display screen. The storage stores the computer program for this method, the processor executes the program to complete audio acquisition, feature extraction, parameter calculation, and hierarchical calculation, and the display screen visually outputs all the calculated reference data. This mobile terminal is fully compatible with... Figure 1 The system's overall modular architecture, as shown, meets the deployment requirements and can be fully implemented. Figure 2 The data processing and reasoning flow shown enables routine, non-invasive, and intelligent measurement of lung function parameters and standardized data output.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lung function parameter measurement system based on multimodal feature processing, characterized in that, include: The system includes a data preprocessing module, a dual-branch feature extraction module, a multimodal fusion module, a numerical prediction module, a site adversarial optimization module, a medical data hierarchical calculation module, and a data output module. The data preprocessing module is configured to acquire and process two respiratory acoustic signals and structured population feature data. The two respiratory acoustic signals include a cough audio signal and a forced exhalation blowing audio signal. The data preprocessing module performs endpoint detection, resampling, amplitude normalization, Mel spectrum feature extraction, and temporal alignment processing on the cough audio signal and the forced exhalation blowing audio signal, respectively, to generate a cough audio tensor and a blowing audio tensor. The structured demographic data is standardized, one-hot encoded, and BMI features are derived to generate a demographic feature vector. The dual-branch feature extraction module includes a first audio encoding branch and a second audio encoding branch that are independent of each other; the first audio encoding branch uses an AST audio Transformer encoder to extract global acoustic features of cough from the cough audio tensor; the second audio encoding branch uses an independent AST audio Transformer encoder to extract global acoustic features of blowing from the blowing audio tensor. The site adversarial optimization module is coupled to the output of the first audio coding branch and the second audio coding branch, respectively. The site adversarial optimization module is only enabled during the model training phase. Each set of site adversarial optimization modules includes a self-attention aggregation layer, a gradient inversion layer (GRL), and a site classification head. The gradient inversion layer is configured with forward propagation identity mapping and backpropagation gradient inversion to enable the corresponding AST audio Transformer encoder to strip away the features of the collected sites and generate site-independent acoustic representations through adversarial training. The gradient inversion layer is configured with a dynamic scheduling mechanism to control the adversarial weights λ to adaptively adjust with the training progress, where λ represents the gradient scaling coefficient of the gradient inversion layer. The multimodal fusion module includes two independent fusion units, corresponding to the first audio coding branch and the second audio coding branch, respectively; any fusion unit is configured to: map the acoustic features and population feature vectors output by the branch to the same feature dimension space and then concatenate them, and process them through the fusion MLP layer to generate branch fusion features; The numerical prediction module includes a parallel-configured correlation probability prediction submodule and a lung function parameter regression submodule. The correlation probability prediction submodule receives the branch fusion features output from the first audio encoding branch and outputs correlation probability values ​​for various respiratory diseases through a multi-layer MLP classification head. The lung function parameter regression submodule receives the branch fusion features output from the second audio encoding branch and outputs FEV1 and FVC values ​​through a regression head with ReLU activation. Here, FEV1 represents forced expiratory volume in one second, and FVC represents forced vital capacity. The medical data grading and calculation module is configured to read the associated probability value, FEV1 calculation value, and FVC calculation value, calculate the FVC predicted value and the percentage of FVC to the predicted value based on the GLI-2012 standard formula and population characteristics, and complete the indicator grading calculation by preset numerical matching matrix to generate standardized feature grading data. The data output module is configured to output respiratory disease association probability, FEV1 measurement value, FVC measurement value, FVC percentage of predicted value, and standardized feature classification data.

2. The lung function parameter measurement system according to claim 1, characterized in that, The steps of the data preprocessing module in processing the forced exhalation audio signal include: calibrating the effective exhalation start and end intervals of the exhalation audio through energy envelope detection, and unifying the time sequence to a fixed long time sequence dimension; when the data preprocessing module processes the cough audio signal, it generates cough audio tensors for multiple effective cough segments, and after processing by the first audio encoding branch, it takes the average of the correlation probability values ​​of the multiple output segments as the final correlation probability.

3. The lung function parameter measurement system according to claim 1, characterized in that, The site classification head is a two-layer MLP structure. The input is the global acoustic features compressed by the self-attention aggregation layer, and the output is the classification probability of each collected site in the training set. During the training phase, the site classification loss adopts cross-entropy loss, which drives the site classification head to identify the collected sites. The gradient inversion layer reverses the gradient through backpropagation, driving the AST audio Transformer encoder to weaken the site-specific acoustic features.

4. The lung function parameter measurement system according to claim 1, characterized in that, The association probability prediction submodule uses the Sigmoid activation function to output multi-label association probabilities, and the training loss uses a binary cross-entropy loss with Logits; the lung function parameter regression submodule uses the SmoothL1 loss function for training loss to reduce the interference of abnormally collected samples on the accuracy of parameter measurement.

5. The lung function parameter measurement system according to claim 1, characterized in that, The preset numerical matching matrix within the medical data grading and calculation module contains three levels of calculation logic: Level 1: Based on a preset probability threshold, match the associated probability values ​​to generate disease association level identifiers; Level 2: Lung function impairment classification based on the numerical range of FVC as a percentage of the predicted value; Level 3: Based on the ratio of FEV1 measured value to FVC measured value, generate a ventilation obstacle type feature identifier; The standardized feature classification data includes three types of structured numerical identifiers: disease association level, lung function impairment classification, and ventilation disorder type identifier.

6. The lung function parameter calculation system according to claim 1, characterized in that, The AST audio Transformer encoder is initialized using AudioSet pre-trained weights. During training, the parameters of the first 6 layers of the network are frozen, and after unfreezing, the 6 layers of the network are fine-tuned. The AST audio Transformer encoder parameters of the first audio encoding branch and the second audio encoding branch in the dual-branch feature extraction module are independent of each other and are not shared.

7. The lung function parameter measurement system according to claim 1, characterized in that, The model training phase employs a differentiated training strategy. The first audio encoding branch and the association probability prediction submodule use the first set of hyperparameter training configurations, while the second audio encoding branch and the lung function parameter regression submodule use the second set of hyperparameter training configurations. The learning rate, iteration rounds, and data augmentation strategies of the two sets of training configurations are independent of each other. The first set of hyperparameters is a set of training parameters set for the first audio coding branch and the association probability prediction submodule, including learning rate, weight decay coefficient, batch size, iteration rounds, loss function weights and data augmentation strategies. The data augmentation strategies include at least one or more of temporal occlusion, frequency occlusion, random pruning and noise perturbation, which are used to enhance the robustness of the association probability prediction task to different acoustic scenarios. The second set of hyperparameters is a set of training parameters set for the second audio coding branch and the lung function parameter regression submodule, including learning rate, weight decay coefficient, batch size, iteration rounds, regression loss weights, and data augmentation strategies. The data augmentation strategies include at least one or more of loudness perturbation, duration stretching, spectral perturbation, and hybrid augmentation, which are used to improve the adaptability of the lung function parameter regression task to individual differences and changes in acquisition conditions.

8. A method for processing lung function parameter measurement data based on multimodal feature processing, characterized in that, Performed by the lung function parameter measurement system according to any one of claims 1-7, the method includes the following steps: S1. Data Acquisition and Preprocessing Steps: Acquire user cough audio, forced exhalation audio, and population structure features; Perform endpoint detection, resampling, normalization, Mel spectrum extraction, and temporal alignment on the two acoustic signals to generate audio tensors; Standardize and encode population features to generate population feature vectors; S2, Dual-branch feature extraction and site adversarial constraint steps: Input the cough audio tensor into the first audio coding branch to extract cough acoustic features, and input the blowing audio tensor into the second audio coding branch to extract blowing acoustic features; During the model training phase, gradient inversion adversarial constraints are applied through the corresponding site adversarial optimization module to remove interference information from the acoustic features at the acquisition sites. S3. Multimodal feature fusion step: Map the two acoustic features and population feature vectors to a unified dimensional space respectively, and after concatenation, obtain the fused features of two independent branches through fusion MLP processing; S4. Parallel numerical prediction step: Input the first branch fusion feature into the association probability prediction submodule and output the association probability values ​​of multiple respiratory diseases; input the second branch fusion feature into the lung function parameter regression submodule and output the FEV1 and FVC calculation values; where FEV1 represents the forced expiratory volume in one second and FVC represents the forced vital capacity. S5. Medical data grading calculation steps: Calculate the percentage of FVC to the predicted value based on the GLI-2012 standard formula, call the preset numerical matching matrix to perform grading calculations on the association probability, the percentage of FVC to the predicted value, and the FEV1 / FVC ratio, and generate standardized feature grading data; S6. Data Output Steps: Output all calculated values ​​and standardized feature classification data to generate a structured reference dataset.

9. The method for processing lung function parameter measurement data according to claim 8, characterized in that, The complete training process for the model training phase includes: T1. Dataset Construction: Samples collected from multiple centers are divided into training set, validation set, and test set according to the patient dimension. Samples from each collection site within the training set are sampled evenly. The sample labels include the site identifier corresponding to the audio, the clinical gold standard lung function value, and the disease label annotated by the physician. T2, Branch Independent Pre-training: Initialize two sets of AST audio Transformer encoders respectively, freeze the underlying parameters, and train the correlation probability prediction link of the first audio coding branch and the lung function parameter regression link of the second audio coding branch independently in sequence; during the training process, the site adversarial weights λ are dynamically adjusted according to the preset scheduling rules. T3. Model convergence verification: Calculate the MAE index based on the validation set AUC index and FVC index respectively, execute early stopping, and save the optimal weight model; T4. Classification of the Calculation Module: The probability of the output of the validation set samples and the calculation parameters are used to traverse the grid search to determine the threshold of each level in the numerical matching matrix, and the classification calculation logic is solidified.

10. A smart terminal device, characterized in that, include: Memory, processor, microphone acquisition unit, display output unit; The memory stores a computer-executable program, and when the processor executes the computer-executable program, it implements the lung function parameter measurement data processing method based on multimodal feature processing as described in claim 9. The microphone acquisition unit is configured to acquire audio of the user coughing and audio of forceful exhalation. The display output unit is configured to display the FEV1 measurement value, FVC measurement value, disease association probability, and standardized feature grading data output by the processor.

Images