Breathing sound classification method and system based on low-rank adaptation

Abstract

The invention provides a breath sound classification method and system based on low-rank adaptation, and relates to the technical field of artificial intelligence and voice signal processing, and the method comprises the steps: obtaining and preprocessing breath sound data, carrying out the truncation of the preprocessed breath sound data, and carrying out the circulation filling of data shorter than a preset time length; sequentially carrying out framing, windowing and fast Fourier transform on the breath sound data with the fixed length, and adding a Mel filter to obtain a Mel frequency spectrum; injecting a trainable low-rank decomposition matrix into each encoder layer in the pre-trained AST model to obtain an updated attention weight; respectively carrying out Gaussian distribution initialization and zero matrix initialization on the initial low-rank decomposition matrix to correspondingly obtain a Gaussian distribution matrix and a zero matrix, injecting the Gaussian distribution matrix and the zero matrix for query, keys and values, and training a pre-trained AST model to obtain a trained AST model; and inputting the breath sound data with the fixed length into the trained AST model, and obtaining a classification result based on the evaluation index.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and speech signal processing, and in particular to a respiratory sound classification method and system based on low-rank adaptation. Background Art

[0002] Respiratory diseases are a huge threat to human health and life. Early diagnosis and treatment are of great significance to improving the quality of life of patients. Since breath sounds are closely related to the normal and abnormal status of the respiratory system, monitoring these sounds can prevent respiratory diseases in advance. Abnormal breath sounds, such as snoring, are detected in the chest cavity of patients with respiratory diseases and are usually early clinical indicators of diseases such as asthma.

[0003] In recent years, deep learning methods have been widely used by researchers for automatic breath sound classification. Many studies have been performed to classify breath sounds by using or improving different deep learning networks. Due to the lack of breath sound datasets, in order to improve the classification performance, researchers have used models pre-trained on large non-medical datasets to fine-tune the breath sound dataset. Most recent studies are based on transfer learning from AST or Vision Transformer, using models pre-trained on large-scale data to classify breath sounds. Although the full fine-tuning method is popular, it has some key disadvantages such as long training time, high memory consumption, and easy overfitting as the model size grows.

[0004] Therefore, it is urgent to provide a solution to improve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a respiratory sound classification method and system based on low-rank adaptation, so as to improve the problems of long training time, high memory consumption and easy overfitting in the prior art.

[0006] In a first aspect, the present invention provides a method for classifying respiratory sounds based on low-rank adaptation, which adopts the following technical solution: Acquire and preprocess the respiratory sound data collected by the stethoscope to obtain multiple segments of preprocessed respiratory sound data, and cut each segment of the preprocessed respiratory sound data by 15 seconds, perform cyclic padding on the data shorter than the preset time length, and obtain multiple fixed-length respiratory sound data; The fixed-length respiratory sound data is sequentially framed, windowed, and fast Fourier transformed, and a Mel filter is added to perform low-pass filtering to obtain a Mel spectrum, where the Mel spectrum is used to freeze the weight matrix of a preset pre-trained AST model without performing gradient update; Each encoder layer in the pre-trained AST model injects a trainable low-rank decomposition matrix to obtain updated attention weights; The initial low-rank decomposition matrix is ​​initialized with Gaussian distribution and zero matrix respectively, and Gaussian distribution matrix and zero matrix are obtained respectively. After injecting trainable Gaussian distribution matrix and zero matrix into query, key and value respectively, the pre-trained AST model is trained based on the improved forward propagation update and the updated attention weight to obtain the trained AST model; The fixed-length respiratory sound data is input into the trained AST model, and a classification result is obtained based on the evaluation index.

[0007] The beneficial effect of a respiratory sound classification method based on low-rank adaptation provided by the present invention is that by locally fine-tuning the pre-trained AST model, a low-rank matrix is ​​introduced to emphasize the features that are not fully represented in the pre-training stage, thereby improving the adaptability of the model to the respiratory sound classification task. The present invention can reduce the number of training parameters while maintaining high performance and improving training efficiency.

[0008] Optionally, the low-rank decomposition matrix includes a dimensionality reduction matrix and a dimensionality increase matrix, wherein the dimensionality reduction matrix is ​​initialized using a Gaussian distribution to reduce the feature dimension of the input respiratory sound data from 768 dimensions to 8 dimensions, and the dimensionality increase matrix is ​​initialized using a zero matrix to increase the feature dimension of the input respiratory sound data from 8 dimensions to 768 dimensions.

[0009] Optionally, the rank of the low-rank decomposition matrix is ​​smaller than the rank of the pre-trained AST model weight matrix, and the rank of the low-rank decomposition matrix has a unique optimal value for achieving an optimal balance between model complexity and classification accuracy.

[0010] Optionally, the mathematical expression of the improved forward propagation update is: ; ; ; ; in, is the feature dimension of the input breath sound, , , Represents a frozen pre-trained weight matrix, for which no gradient update is performed. represents the input respiratory sound features, , , represents the reduced dimension matrix in the low-rank decomposition matrix, , , represents the dimension-raising matrix in the low-rank decomposition matrix, Indicates Query of the multi-head attention mechanism of the layer, Indicates The key of the multi-head attention mechanism of the layer, for The transpose of Indicates The value of the multi-head attention mechanism of the layer, represents the activation function, Represents the attention value.

[0011] In a second aspect, the present invention further provides a respiratory sound classification system based on low-rank adaptation, comprising:

[0012] An efficient sound collection module is used to collect the respiratory sound data with high fidelity using a stethoscope to obtain the collected respiratory sound signal;

[0013] A breathing sound enhancement module, used to perform data enhancement processing on the collected breathing sound signal to remove environmental noise and non-target interference signals;

[0014] A feature extraction module, used for converting the collected respiratory sound signal into a spectrum diagram rich in diagnostic information according to Mel spectrum transformation;

[0015] The deep learning disease classification and recognition module is used to train the pre-trained AST model based on the spectrum graph, attention mechanism and low-rank matching, and obtain the trained AST model.

[0016] For the beneficial effects of the second aspect, reference may be made to the description of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of a respiratory sound classification method based on low-rank adaptation provided by the present invention;

[0018] Figure 2 It is an internal structure diagram of the pre-trained AST model provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0020] The embodiment of the present invention provides a method for classifying respiratory sounds based on low-rank adaptation, comprising: S1, obtaining and preprocessing the respiratory sound data collected by the stethoscope to obtain multiple segments of preprocessed respiratory sound data, and truncating each segment of the preprocessed respiratory sound data by 15 seconds, cyclically filling the data shorter than the preset time length, and obtaining multiple fixed-length respiratory sound data; S2, sequentially performing frame division, windowing, fast Fourier transformation, and adding a Mel filter to the fixed-length respiratory sound data for low-pass filtering to obtain a Mel spectrum, wherein the Mel spectrum is used to freeze the weight matrix of the preset pre-trained AST model without performing gradient update; S3. Inject the trainable low-rank decomposition matrix into each encoder layer in the pre-trained AST model to obtain the updated attention weights. S4. Initialize the initial low-rank decomposition matrix with Gaussian distribution and zero matrix respectively, obtain Gaussian distribution matrix and zero matrix respectively, and inject trainable Gaussian distribution matrix and zero matrix into query, key and value respectively, and then train the pre-trained AST model based on the improved forward propagation update and the updated attention weight to obtain the trained AST model; S5. Input the fixed-length respiratory sound data into the trained AST model, and obtain classification results based on evaluation indicators.

[0021] In some embodiments, when executing step S1, the respiratory sound data is accurately collected and recorded by using a precision electronic stethoscope to perform a detailed scan of multiple areas of the patient's anterior and posterior chest, wherein the multiple collection areas of the patient's anterior and posterior chest include: trachea, left front chest, right front chest and other positions.

[0022] Furthermore, the preprocessing includes: resampling, intensity normalization, fade-in and fade-out, audio enhancement and other operations, wherein, when resampling, the data collected at each point does not need to be equal in duration, the resampling frequency is 4K, and the use of linear fade-in and fade-out can reduce noise and distortion at the beginning and end of the frequency clip.

[0023] Furthermore, the time interval of the loop filling is 8 seconds to handle recordings of different lengths and ensure that the model can handle audio data of various lengths. The audio enhancement is implemented based on SpecAug data enhancement technology, including frequency masking and time masking, to increase data diversity and improve the generalization ability of the model.

[0024] In some embodiments, when executing step S2, the fixed-length respiratory sound data is sequentially framed, windowed, fast Fourier transformed, and a Mel filter is added to perform low-pass filtering to obtain a Mel spectrum. The mathematical expression of windowing is: ; in, represents the windowed breath sound data, Represents the respiratory sound data after frame division, Represents the window function.

[0025] In some embodiments, when executing step S2, the Mel spectrum is obtained by converting the audio waveform into a 128-dimensional logarithmic Mel filter bank characteristic spectrum, which is used to extract key features of the audio signal.

[0026] Specifically, the mathematical expression of the characteristic spectrum of the Mel filter bank is: ; in, Indicates frequency, Indicates a Mel value.

[0027] In some embodiments, when executing step S3, the low-rank decomposition matrix includes a reduced-dimensional matrix and an increased-dimensional matrix, wherein the reduced-dimensional matrix is ​​initialized using a Gaussian distribution, and the increased-dimensional matrix is ​​initialized using a zero matrix.

[0028] Specifically, the rank of the low-rank decomposition matrix is ​​smaller than the rank of the pre-trained AST model weight matrix, and the rank of the low-rank decomposition matrix has a unique optimal value for achieving the best balance between model complexity and classification accuracy.

[0029] In fact, the low-rank adaptation method is a technology used to optimize deep learning models. It reduces the number of parameters that need to be updated during fine-tuning of the model through low-rank decomposition, thereby reducing GPU memory usage and improving training efficiency.

[0030] In some embodiments, when executing step S4, during the training phase, experiments are conducted on an NVIDIA RTX4090 GPU using a Pytorch framework, with a weight decay of 0.1, a learning rate of 1×10^(-14), a batch size of 16, and a training cycle of 100.

[0031] In some embodiments, when executing step S4, the mathematical expression of the improved forward propagation update is: ; ; ; ; in, is the feature dimension of the input breath sound, , , Represents a frozen pre-trained weight matrix, for which no gradient update is performed. represents the input respiratory sound features, , , represents the reduced dimension matrix in the low-rank decomposition matrix, , , represents the dimension-raising matrix in the low-rank decomposition matrix, Indicates Query of the multi-head attention mechanism of the layer, Indicates The key of the multi-head attention mechanism of the layer, for The transpose of Indicates The value of the multi-head attention mechanism of the layer, represents the activation function, Represents the attention value.

[0032] In some embodiments, when executing step S5, the evaluation index includes: ICBHI score, sensitivity (SE), specificity (SP), wherein the ICBHI score reflects the average level of specificity and sensitivity. Specificity (SP) reflects the ratio of the number of correctly predicted normal breath sound samples to the number of all normal breath sound samples, and sensitivity (SE) reflects the ratio of the number of correctly predicted crackle, wheeze, and both (crackle and wheeze) samples to the number of all crackle, wheeze, and both (crackle and wheeze) samples.

[0033] Furthermore, the experiment uses the largest available respiratory sound dataset, ICBHI2017, to evaluate whether the performance requirements are met: whether the performance of the model has reached the expected goal. If the performance requirements are met, the model training is completed by outputting the results; if the performance requirements are not met, the parameters such as learning rate, number of training rounds, etc. are adjusted, and the training model step is returned to continue training until the performance requirements are met.

[0034] See also Figure 2 , the fixed-length breath sound data is embedded in the block position after Mel transformation and input into the Transformer encoder layer (number of layers ), and the output is obtained after linear projection.

[0035] The Transformer encoder layer includes an attention mechanism, which inputs fixed-length breathing sound data into the hidden state and converts the dimension reduction matrix in the low-rank decomposition matrix into and the dimension-increasing matrix Query with attention mechanism ,key , and value To update the weights , , .

[0036] The embodiment of the present invention further provides a respiratory sound classification system based on low-rank adaptation, comprising: An efficient sound collection module is used to collect the respiratory sound data with high fidelity using a stethoscope to obtain the collected respiratory sound signal; A breathing sound enhancement module, used to perform data enhancement processing on the collected breathing sound signal to remove environmental noise and non-target interference signals; A feature extraction module, used for converting the collected respiratory sound signal into a spectrum diagram rich in diagnostic information according to Mel spectrum transformation; The deep learning disease classification and recognition module is used to train the pre-trained AST model based on the spectrum graph, attention mechanism and low-rank matching, and obtain the trained AST model.

[0037] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. A respiratory sound classification method based on low-rank adaptation, comprising: Acquire and preprocess the respiratory sound data collected by the stethoscope to obtain multiple segments of preprocessed respiratory sound data, and cut each segment of the preprocessed respiratory sound data by 15 seconds, perform cyclic padding on the data shorter than the preset time length, and obtain multiple fixed-length respiratory sound data; The fixed-length respiratory sound data is sequentially framed, windowed, and fast Fourier transformed, and a Mel filter is added to perform low-pass filtering to obtain a Mel spectrum, where the Mel spectrum is used to freeze the weight matrix of a preset pre-trained AST model without performing gradient update; Each encoder layer in the pre-trained AST model injects a trainable low-rank decomposition matrix to obtain updated attention weights; The initial low-rank decomposition matrix is ​​initialized with Gaussian distribution and zero matrix respectively, and Gaussian distribution matrix and zero matrix are obtained respectively. After injecting trainable Gaussian distribution matrix and zero matrix into query, key and value respectively, the pre-trained AST model is trained based on the improved forward propagation update and the updated attention weight to obtain the trained AST model; The fixed-length respiratory sound data is input into the trained AST model, and a classification result is obtained based on the evaluation index.

2. A respiratory sound classification method based on low-rank adaptation according to claim 1, characterized in that: The low-rank decomposition matrix includes a dimensionality reduction matrix and a dimensionality increase matrix, wherein the dimensionality reduction matrix is ​​initialized using a Gaussian distribution to reduce the feature dimension of the input respiratory sound data from 768 dimensions to 8 dimensions, and the dimensionality increase matrix is ​​initialized using a zero matrix to increase the feature dimension of the input respiratory sound data from 8 dimensions to 768 dimensions.

3. A respiratory sound classification method based on low-rank adaptation according to claim 1, characterized in that: The rank of the low-rank decomposition matrix is ​​smaller than the rank of the pre-trained AST model weight matrix, and the rank of the low-rank decomposition matrix has a unique optimal value for achieving the best balance between model complexity and classification accuracy.

4. A method for classifying respiratory sounds based on low-rank adaptation according to claim 1, characterized in that: The mathematical expression of the improved forward propagation update is: ； ； ； ； in, is the feature dimension of the input breath sound, , , Represents a frozen pre-trained weight matrix, for which no gradient update is performed. represents the input respiratory sound features, , , represents the reduced dimension matrix in the low-rank decomposition matrix, , , represents the dimension-raising matrix in the low-rank decomposition matrix, Indicates Query of the multi-head attention mechanism of the layer, Indicates The key of the multi-head attention mechanism of the layer, for The transpose of Indicates The value of the multi-head attention mechanism of the layer, represents the activation function, Represents the attention value.

5. A respiratory sound classification system based on low-rank adaptation, comprising: An efficient sound collection module is used to collect the respiratory sound data with high fidelity using a stethoscope to obtain the collected respiratory sound signal; A breathing sound enhancement module, used to perform data enhancement processing on the collected breathing sound signal to remove environmental noise and non-target interference signals; A feature extraction module, used for converting the collected respiratory sound signal into a spectrum diagram rich in diagnostic information according to Mel spectrum transformation; The deep learning disease classification and recognition module is used to train the pre-trained AST model based on the spectrum graph, attention mechanism and low-rank matching, and obtain the trained AST model.

Images