A Controllable Generative Enhancement Method and System for Breath Sound Categories Based on Discrete Tokens and Transformers

Abstract

This invention provides a controllable generative enhancement method for respiratory sound categories based on discrete tokens and Transformer, belonging to the interdisciplinary field of medical signal processing and artificial intelligence. The method employs a two-stage framework: In the first stage, a conditional vector quantization autoencoder is trained, injecting pathological category labels during encoding and introducing multi-scale category prototypes based on statistical analysis of similar samples during decoding and reconstruction, thereby constructing a discrete latent space with explicit semantic structure. In the second stage, an autoregressive Transformer model is trained to learn the category conditional distribution of token sequences within this space. In the enhancement stage, for the target minority categories, the Transformer generates new token sequences, which are then fused with the corresponding category's multi-scale prototypes and broadcast decoded to reconstruct high-fidelity, pathologically semantically consistent synthetic respiratory sound samples. Finally, the synthetic samples are added to the training set to balance the data distribution, effectively improving the downstream respiratory sound classification model, especially its performance and overall robustness in identifying a minority of abnormal categories.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of medical signal processing and artificial intelligence. Specifically, this invention relates to a method and system for controllable generative enhancement of respiratory sound categories based on discrete tokens and Transformer, which is used to solve the problems of sample scarcity, background noise interference and severe category imbalance in medical auscultation data. It is applicable to the automatic auxiliary diagnosis of lung pathology (such as wheezing, crackles, etc.). Background Technology

[0002] Breath sound classification plays a crucial role in clinical medicine for identifying pulmonary pathological conditions such as asthma, chronic obstructive pulmonary disease, and pneumonia. By analyzing abnormal sounds in auscultation recordings (such as continuous high-frequency wheezes and intermittent low-frequency crackles), physicians can non-invasively gain insights into lung conditions. With the development of machine learning algorithms and medical devices, neural network-based automated breath sound classification systems have gradually become a research hotspot, aiming to reduce reliance on manual auscultation by physicians. Existing mainstream methods include using convolutional neural networks (CNNs) to extract features such as Mel-spectrograms and Mel-frequency cepstral coefficients (MFCCs), or combining recurrent neural networks (RNNs) and attention mechanisms to capture long-short-term dependencies. Recent studies have also explored contrastive learning strategies to enhance intra-class compactness and inter-class separability.

[0003] Despite some progress, automated breath sound classification systems still face significant challenges in actual clinical deployment, mainly in the following aspects: 1) Data scarcity and class imbalance: Real-world auscultation datasets are typically small in size and exhibit severe imbalances between different disease categories. Insufficient sample sizes for a few pathological categories (such as rare abnormal breath sounds) makes it difficult for deep learning models to generalize, leading to overfitting to the majority class (such as normal breath sounds) and resulting in extremely low recall for rare categories. This poses a serious risk in early clinical detection.

[0004] 2) Severe noise interference: The clinical auscultation environment often contains complex background noise, which further increases the difficulty of extracting fine-grained pathological acoustic features.

[0005] 3) Limitations of Traditional Data Augmentation Methods: Existing audio augmentation methods mainly fall into two categories. The first category is based on perturbation-based standard operations, such as time stretching, pitch shifting, noise injection, or masking. While these methods are simple, they often disrupt subtle pathological cues in fine-grained medical acoustics. For example, changes in time or pitch may introduce unnatural variations that do not conform to pathological clinical characteristics, reducing the model's ability to recognize important diagnostic signals.

[0006] 4) Shortcomings of existing generative augmentation methods: The second category is augmentation methods based on generative models (such as VAEs and GANs). However, existing medical audio generators often struggle to generate high-fidelity samples under limited supervision and lack reliable class controllability. This leads to the generated synthetic samples potentially deviating from the true distribution of clinical data, introducing noise that cannot accurately reflect underlying disease patterns. Furthermore, generative models trained on small datasets typically fail to produce diverse, class-specific data, thus failing to effectively enrich minority class samples.

[0007] In summary, existing technologies lack an enhancement scheme that can generate high-fidelity samples, precisely control the generated categories, and not destroy pathological semantics. Summary of the Invention

[0008] The purpose of this invention is to provide a controllable generative enhancement method for respiratory sound categories based on discrete tokens and Transformer, so as to overcome the problems of poor generalization ability and low recognition rate of a few pathological categories in the prior art due to the scarcity of respiratory sound data, category imbalance and noise interference.

[0009] The technical solution of the present invention is as follows: A controllable generative enhancement method for breath sounds based on discrete tokens and Transformer is characterized by the following steps: The first training step is to train a conditional vector quantization autoencoder using a breath sound training set containing pathological category labels to construct a discrete latent space with semantic structure. During the encoding process, the embedding vectors of the pathological category labels are injected into continuous latent features, and during the decoding and reconstruction process, a multi-scale category prototype obtained based on the jump feature statistics of similar samples is introduced simultaneously. The second training step is to use the discrete latent space obtained in the first training step to encode the samples in the breathing sound training set into discrete token sequences, and train an autoregressive Transformer model to learn the distribution law of generating legal token sequences under given class labels. Controllable sample generation steps: For the target pathological category, a new discrete token sequence is generated using the autoregressive Transformer model trained in the second stage training step. This sequence is then fused and broadcast with the multi-scale category prototypes statistically obtained for the target pathological category in the first stage training step. The result is input into the decoder of the conditional vector quantization autoencoder trained in the first stage training step to reconstruct and generate synthetic breath sound samples with high fidelity and pathological semantic consistency. Data augmentation and training steps: The synthetic samples generated in the controllable sample generation step are added to the original training set to form an augmented training set for training the breath sound classification model.

[0010] Furthermore, the first stage of training specifically includes: Encoding and Conditional Injection Steps: The respiratory audio spectrogram is input into the encoder network to obtain continuous latent features. At the same time, the pathological category label is mapped to a conditional vector, and the vector is fused with the continuous latent features at each time step through projection. Vector Quantization Steps: The fused continuous latent features are discretized into discrete token indices using a learnable codebook, and the original features are replaced with the corresponding codebook vectors. Category prototype extraction steps: During training, for each category of samples, extract the multi-resolution jump features of the encoder's intermediate layer, and calculate the static prototype vector of each category at each resolution level through global average pooling and exponential moving average. Prototype-guided decoding and reconstruction steps: In the decoder, the codebook vector sequence from the vector quantization step, the corresponding jump features from the encoder, and the category prototype vectors broadcast to the corresponding spatial size are fused in a multi-level manner to reconstruct the original spectrogram.

[0011] Furthermore, the decoder is a U-Net structure, and at least one of its decoding layers contains a self-attention module for modeling the global temporal dependence of the spectrogram.

[0012] Furthermore, the encoder is a pre-trained convolutional neural network, and the output feature map of its last convolutional layer serves as the continuous latent feature.

[0013] Furthermore, the second phase of training specifically includes: Sequence data preparation steps: Encode the breath sound training set using the trained conditional vector quantization autoencoder to obtain discrete token sequences corresponding to all samples; Autoregressive modeling steps: Using the pathological category label and the starting token as conditions, a Transformer model with a pure decoder structure is used. The token of the previous time step is taken as input to predict the probability distribution of the token of the next time step. The model is trained by maximizing the joint conditional likelihood of the sequence.

[0014] Furthermore, the controllable sample generation step specifically includes: Token sequence generation steps: The target pathological category label is used as the starting condition and input into the trained autoregressive Transformer model to generate a brand new discrete token sequence through autoregressive sampling; Prototype fusion step: The generated discrete token sequence is converted into a codebook vector sequence through the codebook, and the sequence is fused with the multi-scale category prototype of the target pathological category at the feature level to form a semantically enhanced latent representation; Prototype broadcasting and decoding steps: The fused latent representation sequence and the multi-scale category prototype of the target pathological category after spatial broadcasting are input into the trained decoder to synthesize the final Mel spectrogram.

[0015] Furthermore, in the prototype fusion step, the fusion operation is achieved by concatenating the codebook vector with the concatenated multi-scale category prototype vector, and then performing linear projection.

[0016] Furthermore, in the first stage of training, the training loss function of the conditional vector quantization autoencoder is a combination loss, which includes at least: Reconstruction loss is used to constrain the pixel-level similarity between the synthesized spectrogram and the original spectrogram; Perceptual loss is calculated based on the feature space distance of the pre-trained network. To combat loss, discriminator networks are used to improve the distribution realism of synthetic data. The commitment loss is used to constrain the encoder output to be close to the selected codebook vector.

[0017] Furthermore, the multi-scale category prototype is dynamically updated during training. After each training batch, the jump features after global average pooling are smoothly accumulated by category and level using an exponential moving average method.

[0018] Second, the present invention also provides a training system for a respiratory sound classification model, characterized by including: The data preprocessing module is used to convert the raw breath sound audio into a log-Mel spectrogram and associate it with pathological category labels; The category-controlled generative enhancement module is used to execute the above method, generate synthetic breath sound samples of a specified pathological category, and output the enhancement training set; The classification model training module is used to receive the enhanced training set and train the respiratory sound classification model.

[0019] Third, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method described above when executing the program.

[0020] Fourth, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the method described above.

[0021] Compared to existing traditional methods and generative models such as Generative Adversarial Networks (GANs) or Variational Autoencoders (VAEs), the C2GA framework of this invention has the following significant advantages: 1) Improved Minority Classification Performance. In traditional audio data augmentation methods, due to class imbalance, especially the scarcity of minority class samples, classifiers often overfit to the majority class, resulting in low recall for the minority class. This invention effectively enhances the performance of minority class samples by generating high-fidelity Mel-ray spectra consistent with the class labels, thereby improving the classification accuracy and recall of the classifier in the minority class. By employing class-conditional generation (C2GA), the representativeness of minority class samples in the training set can be effectively increased, significantly improving the model's ability to identify the minority class. Especially when facing data imbalance, this invention can avoid over-reliance on majority class samples in the training set.

[0022] 2) Stable Performance Improvement. Under various experimental settings, C2GA consistently demonstrates stable performance improvement, especially under conditions of data scarcity, class imbalance, and strong noise interference. With different hyperparameter configurations (such as varying the proportion of synthetic data and loss weights), this invention maintains relatively stable performance improvements and exhibits good generalization ability across different datasets. Compared to traditional perturbation-based augmentation methods, C2GA's synthetic data effectively supplements the scarce portions of the training set, reduces the impact of data noise, and thus improves the model's performance on the test set.

[0023] 3) Improved classification robustness and adaptability to complex environments. In real clinical settings, breath sound data are often affected by noise and label interference, making traditional audio classification methods ineffective in identifying pathological features. This invention improves the model's robustness in environments with strong noise and label interference by generating high-quality, class-consistent synthetic data. In experiments, this invention maintains high classification accuracy on complex datasets containing noise and label interference, especially showing significant improvement in the classification of normal and minority classes (such as moist rales). C2GA enhances the model's adaptability to multiple classes, thereby improving its stability and reliability in practical applications.

[0024] 4) Enhanced high fidelity and class consistency of generated samples. This invention, by using class-conditional VQ-VAE and a Transformer autoregressive model, ensures that the generated Mel spectra possess both high fidelity and strict adherence to the clinical characteristics of each category. This generation strategy avoids the inconsistency between generated samples and real data distribution present in traditional generation models, thus ensuring the practical application value of the generated data. The synthetic data generated by C2GA effectively enriches the training set without introducing artifacts or samples that do not conform to clinicopathological characteristics, greatly enhancing the clinical applicability of the breath sound classification system. 5) Significantly reduces training time and computational overhead. Compared to traditional deep learning model training methods, C2GA effectively reduces the need for large amounts of labeled data by augmenting the model with synthetic data on limited labeled data. In experiments, training with generated synthetic data not only improved model accuracy but also significantly shortened training time. Especially in situations where data is scarce, this invention can supplement the training set by generating high-quality samples, thereby improving training efficiency and reducing computational overhead. This makes this invention highly suitable for clinical environments requiring rapid training and application.

[0025] 6) Scalability and Generalization. The method of this invention demonstrates good generalization ability on different types of respiratory sound datasets, and can adapt to different acoustic environments and noise conditions. Furthermore, the C2GA framework proposed in this invention has good scalability and can be extended to other types of medical audio data augmentation, such as classification tasks for heart sounds, gastrointestinal sounds, and other biomedical audio. Its class-conditional generation ideas and generation enhancement strategies are not limited to respiratory sound applications and have broad cross-domain application potential. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the breath sound category controllable generative enhancement method based on discrete tokens and Transformer of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the implementation methods of the Discrete Token and Transformer-based Controllable Generative Enhancement Method for Breath Sound Categories (C2GA) proposed in this invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art will understand that the described embodiments are only some, not all, of the embodiments of this invention.

[0028] By constructing a two-stage generative augmentation framework that deeply integrates category semantic information, high-fidelity and strong category consistency respiratory sound sample synthesis is achieved, thereby providing high-quality and balanced training data for downstream classification models.

[0029] Phase 1: Learning Categorical Conditional Discrete Representations The goal of this stage is to construct a discrete latent space that includes diagnostic semantics. The specific steps are as follows: Step 1.1 Acoustic Coding and Latent Feature Extraction First, for the input breath sound waveform, the system performs standard time-frequency preprocessing, converting it into a log-Mel spectrogram, denoted as... The spectrogram is then input into a pre-trained acoustic encoder. (For example, in PANNS-CNN14). The encoder's role is to map high-dimensional acoustic signals to a low-dimensional continuous feature space to capture local acoustic features. This process is described by Equation 1: (1) In the formula, The log-Mel spectrogram representing the input contains the time-frequency information of the original audio. The parameter is The pre-trained acoustic encoder neural network; It is the latent feature matrix output by the encoder; This represents the length of the encoded time dimension, which is usually less than the number of original time frames. This achieves compression of the time dimension; This represents the channel dimension of the latent features. This step ensures that subsequent processing is carried out in a highly abstract and semantically rich feature space.

[0030] Step 1.2 Category Condition Injection To include category-specific semantic information in the latent space, label-dependent embedding information is injected at each time step of the latent features. This step fuses the global category labels into the local feature representation through linear projection, enabling the subsequent quantization process to be aware of the sample's category attribute. This operation is described by Equation 2: (2) In the formula, Indicates at time Latent feature values ​​after injecting category conditions; It is the original latent feature at time The value; These are the category labels corresponding to the samples (such as "normal", "wheezing", etc.). It is to class label The embedded vector obtained by mapping, Total number of categories; It is a learnable linear projection matrix used to adjust the dimensions of the class vectors to match the hidden layer dimensions. Consistency. In this way, the global category context is explicitly injected into the local feature stream.

[0031] Meanwhile, the output of the intermediate activation layer of the encoder is cached as multi-resolution skip features. These features preserve multi-scale details and will be used to assist in reconstruction during the decoding stage.

[0032] Step 1.3 Vector Quantization To discretize continuous features, a learnable vocabulary (Codebook) is used. For the conditionalized features Every time step The system will search the vocabulary list for the closest Euclidean distance to the vocabulary vector index. The original features are then replaced with this vector. This process achieves information compression and symbolization, as described by Equation 3: (3) In the formula, It is a moment The index of the selected discrete element; It is the first in the vocabulary. One prototype vector; It refers to the size of the vocabulary; It is the quantized latent vector. In the formula... The Stop-Gradient operator blocks the flow of gradients to the vocabulary selection process during backpropagation, allowing gradients to flow directly from the decoder to the encoder output, thus achieving end-to-end training. This step generates a sequence of discrete units. It is a compact symbolic representation of the breath sound signal.

[0033] Step 1.4 Decoding and Reconstruction To recover the detailed acoustic feature matrix from the discrete latent representation, an enhanced U-shaped reconstruction network is employed. This decoder combines upsampled features and residual features from the encoder. And category conditions. Each layer of the decoder The calculation process is described by Equation 4: (4) In the formula, Indicates the first Hidden state features of the layer; It is an upsampling operation used to gradually restore spatial resolution; These are residual features from the corresponding level of the encoder, which fill in the spatial details lost during downsampling. Concatenation operations representing channel dimensions; It is the category projection matrix at this level, which ensures that the decoder utilizes global semantic information while preserving local texture details when reconstructing the acoustic feature spectrum.

[0034] To further capture long-range dependencies in the time-frequency graph, a self-attention mechanism is introduced at each decoding stage. Its calculation process is described by Equations 5 and 6: (5) in: (6) In the formula, These represent the three mapping matrices (corresponding to the original Query, Key, and Value) in the attention mechanism, which are derived from the features of the current layer. Through learnable projection weights It comes from transformation; This is the dimension of the attention head. The Softmax function is used to normalize the attention weights.

[0035] Ultimately, reconstruct the network. The overall output is the reconstructed acoustic feature matrix. This is expressed by Formula 7: (7) in This is the total number of skip connections. The output will be compared with the original input. Compare the results to calculate the loss.

[0036] Step 1.5 Optimization Target and Prototype Export The first stage of model training is achieved by minimizing a composite objective function that integrates reconstruction quality, perceptual similarity, adversarial realism, and quantization commitment. The specific objective function is defined by Equation 8: (8) In the formula, the first term It is the L1 reconstruction loss, which directly constrains pixel-level differences; It is a perceptual loss, which calculates distance based on intermediate acoustic features; It is the adversarial loss; the last term is the VQ commitment loss, with a coefficient Used to adjust its weights, this constraint controls the encoder output. As close as possible to the selected vocabulary vector .

[0037] After training, the system will analyze the residual features of all samples. Global average pooling (GAP) is applied, and exponential moving average (EMA) is used to accumulate by category, thereby deriving global class centers (ClassPrototypes) for each category. These centers will play a key guiding role in the second phase of the process.

[0038] Phase 2: Autoregressive Prior Module Based on Transformer Using the discrete codebook and category prototypes learned in the first stage, an autoregressive model is trained to generate labeled sequences that conform to a specific category distribution. The specific steps are as follows: Step 2.1 Sequence Modeling Initialization At the start of the generation process, a target generation category is given. The system first maps it to an initial unit, which serves as the starting point for sequence generation. This step is described by Equation 9: (9) In the formula, It is the first input vector of the sequence; It is a specialized embedding function that maps category labels to latent dimensions.

[0039] Step 2.2 Autoregressive Probability Prediction Subsequently, a Transformer model is used to model the probability distribution of the tag sequence in an autoregressive manner. The goal of the model is to maximize the likelihood probability of the current tag given historical tags and categories. This process is described by Equation 10: (10) In the formula, The parameter is The Transformer prior model; It is a complete unit sequence; Indicates time All previous historical units. By maximizing this joint probability, the Transformer learns how to generate a permutation of markers that conforms to specific pathological features, such as the continuity of wheezing.

[0040] Step 2.3 Prototype Fusion and Enhancement Generation During the inference (generation) phase, since there is no real input image, it is impossible to obtain real residual features. To address this issue and ensure the high fidelity of the generated image, this invention proposes a "Dual Injection Strategy." First, the generated discrete label embedding is fused with the category prototype derived in the first stage to enhance the semantic consistency of the latent features. This fusion process is described by Equation 11: (11) In the formula, It is the latent vector after fusion; It is a generating unit The corresponding vocabulary vector; It is the result of concatenating the central summaries of all levels of this category; It is a fused projection matrix.

[0041] Finally, using these fused features and the broadcast category centers, the final synthetic acoustic feature matrix is ​​generated through a reconstruction network. This process is described by Equation 12: (12) In the formula, It is the set of potential sequences after fusion; The operation broadcasts the global center vector spatially to match the size of the feature maps in each layer of the reconstructed network. This mechanism effectively replaces the true skip features, enabling the decoder to utilize learned class statistics to synthesize respiratory audio spectrograms with rich detail and correct pathological semantics.

[0042] Example 1: Implementing Category-Controlled Data Augmentation on the ICBHI Breath Sound Dataset This embodiment details how to apply the C2GA framework of the present invention to the publicly available ICBHI respiratory sound dataset to solve the serious class imbalance problem in the dataset (normal respiratory sound samples are the majority, while abnormal respiratory sound samples such as wheezing and crackling are scarce), and improve the performance of downstream classifiers through generative augmentation.

[0043] Step 1: Dataset Preparation and Preprocessing Obtain respiratory sound audio files and their corresponding four-category labels from the ICBHI dataset: Normal, Wheeze, Crackle, and Both.

[0044] Before training begins, we need to preprocess the collected lung auscultation audio. Specifically, we first convert the collected audio data into a format suitable for input into the deep learning model. Assume we have a... Annotated dataset of 1 sample ,in It is each audio sample, These are the corresponding pathological labels (such as normal, moist rales, etc.). This audio data needs to undergo feature extraction; typically, we use time-frequency analysis techniques to analyze each audio waveform. Convert to the corresponding acoustic feature matrix (log-Mel spectrum). ,in It is the number of time steps. This refers to the spectral channel dimension. To transform these spectrograms into more representative features, we use standard time-frequency analysis tools, setting appropriate time windows and step lengths, to calculate the Log-Mel spectrogram for each audio sample.

[0045] After feature extraction is complete, the various model components will be initialized: Feature Extraction (e.g., pre-trained PANNs-CNN14 models), vocabulary Reconstructing the network Category embedding and Transformer priors At the same time, initialize the category center set. These components lay the foundation for the subsequent model training and generation process.

[0046] Step 2: First Stage The goal is to train a variational autoencoder (VQ-VAE) to generate discrete representations conditioned on class and derive class centers. Following steps 1 to 18 of the algorithm, we first perform encoding and conditional injection. For each training batch... First, through the feature extractor Calculate the encoded hidden state: Then, inject category conditions into the encoded implicit state, and assign category labels. The embedded information is added to the latent representation to obtain the conditionalized hidden state: in, Category tags Embedded vector, It is the transformation matrix of category embedding.

[0047] Next, during the discretization process, we perform a step-by-step analysis at each time step. Find the nearest vocabulary vector : In this way, continuous latent representations are transformed into discrete unit sequences. .

[0048] After generating the discrete unit sequence, we feed it into the reconstruction network. The reconstruction error is calculated. The reconstructed network is determined using the formula... The discretized latent representation is reconstructed to generate a synthesized acoustic feature matrix. And perform loss calculation.

[0049] In this step, we also need to calculate the overall objective function. This includes reconstruction error, perceptual loss, adversarial loss, and commitment loss. Model parameters are updated using the backpropagation algorithm. and At the same time, the exponential moving average (EMA) technique is used to update the vocabulary. .

[0050] Step 3: Second Stage The goal is to generate valid labeled sequences using the Transformer model. First, for each saved batch... We will label the categories Mapped to start tag Initialize the latent representation: Then, an autoregressive method is used to train the Transformer model, minimizing the negative log-likelihood of the generated token sequence. The goal is to generate a token sequence K that matches the class label. This process is achieved by minimizing the objective function. accomplish.

[0051] During this process, the parameters of the Transformer model It will be updated to maximize the probability of generating a target category label sequence.

[0052] After completing the above two training phases, we proceed to the actual data augmentation and classifier training phase. In this phase, we first select the target category. This typically involves classes with fewer samples. Then, a second-stage Transformer model is used to initialize the labels. Initially, Top-p sampling autoregression is used to generate new labeled sequences. .

[0053] Next, we will combine the category prototypes exported in the first stage. The generated tags are combined with category prototypes, and a high-fidelity synthetic Mel spectrogram is generated through a fusion formula and a broadcast decoding formula. This synthetic data will be used as part of the training set to help the classifier better identify minority class samples.

[0054] During training, we set the proportion of synthetic data to be used. and loss weight It is usually set to 0.5, and the actual data is used. With the generated synthetic data Hybridization. In each training epoch, the hybrid data is fed into a classifier (e.g., ResNet18) and trained under supervision using a weighted loss function.

[0055] Experimental results show that this data augmentation method significantly improves the accuracy of the classifier on the validation set, typically by about 1-3%.

[0056] Through the steps described above, we demonstrate how to implement a framework for a controllable generative augmentation method for breath sounds based on discrete tokens and Transformers, utilizing preprocessing, categorical conditional discrete representation learning, and Transformer autoregressive learning. This effectively generates augmented data and improves the performance of the breath sound classification model. This process not only addresses the problems of data scarcity and class imbalance but also enhances the model's robustness and generalization ability by generating high-quality synthetic data.

[0057] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for controllable generative enhancement of breath sounds based on discrete tokens and Transformer, characterized in that, Includes the following steps: The first training step is to train a conditional vector quantization autoencoder using a breath sound training set containing pathological category labels to construct a discrete latent space with semantic structure. During the encoding process, the embedding vectors of the pathological category labels are injected into continuous latent features, and during the decoding and reconstruction process, a multi-scale category prototype obtained based on the jump feature statistics of similar samples is introduced simultaneously. The second training step is to use the discrete latent space obtained in the first training step to encode the samples in the breathing sound training set into discrete token sequences, and train an autoregressive Transformer model to learn the distribution law of generating legal token sequences under given class labels. Controllable sample generation steps: For the target pathological category, a new discrete token sequence is generated using the autoregressive Transformer model trained in the second stage training step. This sequence is then fused and broadcast with the multi-scale category prototypes statistically obtained for the target pathological category in the first stage training step. The result is input into the decoder of the conditional vector quantization autoencoder trained in the first stage training step to reconstruct and generate synthetic breath sound samples with high fidelity and pathological semantic consistency. Data augmentation and training steps: The synthetic samples generated in the controllable sample generation step are added to the original training set to form an augmented training set for training the breath sound classification model.

2. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 1, characterized in that, The first stage of training specifically includes: Encoding and Conditional Injection Steps: The respiratory audio spectrogram is input into the encoder network to obtain continuous latent features. At the same time, the pathological category label is mapped to a conditional vector, and the vector is fused with the continuous latent features at each time step through projection. Vector Quantization Steps: The fused continuous latent features are discretized into discrete token indices using a learnable codebook, and the original features are replaced with the corresponding codebook vectors. Category prototype extraction steps: During training, for each category of samples, extract the multi-resolution jump features of the encoder's intermediate layer, and calculate the static prototype vector of each category at each resolution level through global average pooling and exponential moving average. Prototype-guided decoding and reconstruction steps: In the decoder, the codebook vector sequence from the vector quantization step, the corresponding jump features from the encoder, and the category prototype vectors broadcast to the corresponding spatial size are fused in a multi-level manner to reconstruct the original spectrogram.

3. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 2, characterized in that, The decoder is a U-Net structure, and at least one of its decoding layers contains a self-attention module for modeling the global temporal dependence of the spectrogram.

4. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 2, characterized in that, The encoder is a pre-trained convolutional neural network, and the output feature map of its last convolutional layer serves as the continuous latent feature.

5. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 1, characterized in that, The second phase of training specifically includes: Sequence data preparation steps: Encode the breath sound training set using the trained conditional vector quantization autoencoder to obtain discrete token sequences corresponding to all samples; Autoregressive modeling steps: Using the pathological category label and the starting token as conditions, a Transformer model with a pure decoder structure is used. The token of the previous time step is taken as input to predict the probability distribution of the token of the next time step. The model is trained by maximizing the joint conditional likelihood of the sequence.

6. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 1, characterized in that, The controllable sample generation step specifically includes: Token sequence generation steps: The target pathological category label is used as the starting condition and input into the trained autoregressive Transformer model to generate a brand new discrete token sequence through autoregressive sampling; Prototype fusion step: The generated discrete token sequence is converted into a codebook vector sequence through the codebook, and the sequence is fused with the multi-scale category prototype of the target pathological category at the feature level to form a semantically enhanced latent representation; Prototype broadcasting and decoding steps: The fused latent representation sequence and the multi-scale category prototype of the target pathological category after spatial broadcasting are input into the trained decoder to synthesize the final Mel spectrogram.

7. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 6, characterized in that, In the prototype fusion step, the fusion operation is achieved by concatenating the codebook vector with the concatenated multi-scale category prototype vector and then performing linear projection.

8. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 1, characterized in that, In the first stage of training, the training loss function of the conditional vector quantization autoencoder is a combination loss, which includes at least: Reconstruction loss is used to constrain the pixel-level similarity between the synthesized spectrogram and the original spectrogram; Perceptual loss is calculated based on the feature space distance of the pre-trained network. To combat loss, discriminator networks are used to improve the distribution realism of synthetic data. The commitment loss is used to constrain the encoder output to be close to the selected codebook vector.

9. The breath sound category controllable generative enhancement method based on discrete tokens and Transformer according to claim 1, characterized in that, The multi-scale category prototype is dynamically updated during training. After each training batch, the jump features after global average pooling are smoothly accumulated by category and level using the exponential moving average method.

10. A training system for a respiratory sound classification model, characterized in that, include: The data preprocessing module is used to convert the raw breath sound audio into a log-Mel spectrogram and associate it with pathological category labels; A category-controllable generative enhancement module is used to perform the method as described in any one of claims 1 to 9, generate synthetic breath sound samples of a specified pathological category, and output an enhancement training set; The classification model training module is used to receive the enhanced training set and train the respiratory sound classification model.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 9.

Images