Method and system for designing breathing sound classification and recognition accelerator based on FPGA (Field Programmable Gate Array)

Abstract

The invention discloses a method and a system for designing a breath sound classification recognition accelerator based on an FPGA (Field Programmable Gate Array), and relates to the technical field of breath sound recognition. The method specifically comprises the following steps that a breath sound data set is established, and the breath sound data set comprises multiple kinds of breath event sounds such as normal sounds, dry rale sounds, gasp sounds and gasp sounds; extracting breathing sound features based on the breathing sound data set to obtain a breathing sound feature map; taking the breath sound feature map as a learning sample, and training a breath sound classification recognition model based on a neural network; and using the trained breath sound classification recognition model to perform inversion on the breath sound data set to obtain the recognition accuracy of different types of breath sounds. Compared with a stethoscope, the method has the advantages that the human resource cost is reduced, and the difference caused by human subjective judgment is reduced. Compared with other hardware platforms such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), the system has the advantages of lower power consumption, higher response speed and higher portability.

Description

Technical Field

[0001] The present invention relates to the technical field of respiratory sound recognition, and in particular to a method and system for designing a respiratory sound classification and recognition accelerator based on FPGA. Background Art

[0002] According to the World Health Organization, respiratory diseases, such as pneumonia, asthma, bronchitis, lung cancer and chronic obstructive pulmonary disease, are one of the most common causes of death in the world, causing more than 3 million deaths worldwide each year. Therefore, early diagnosis is crucial to prevent the spread of respiratory diseases and ensure people's healthy lives. Currently, detecting lung sounds is a common method to determine whether there is lung disease. Auscultation is a professional technique in which experts use a stethoscope to detect abnormal lung sounds to identify possible lung diseases.

[0003] Auscultation technology provides a simple, low-cost and non-invasive method for diagnosing respiratory diseases. However, this technology has two major disadvantages. First, auscultation requires professional doctors to analyze lung sounds. In poor areas with scarce medical resources and shortage of professional doctors, once a sudden, large-scale outbreak of infectious respiratory diseases occurs, it will greatly aggravate the spread of the disease and increase the mortality rate. Second, even if the diagnosis is made by experienced doctors, the interpretation of lung sounds is somewhat subjective, which may lead to differences in diagnostic results between different doctors. Summary of the invention

[0004] The purpose of the present invention is to propose a design method and system for a respiratory sound classification and recognition accelerator based on FPGA, so as to overcome the problems in the existing stethoscope technology that lung sound diagnosis relies on professional physician analysis and the diagnosis results are easily affected by the subjectivity of the physician.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for designing a respiratory sound classification and recognition accelerator based on FPGA is provided, comprising the following steps:

[0006] Establishing a respiratory sound dataset, wherein the respiratory sound dataset includes multiple types of respiratory event sounds such as normal, dry rales, wheezing, and stridor;

[0007] Extracting respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature graph;

[0008] The respiratory sound feature map is used as a learning sample to train a respiratory sound classification and recognition model based on a neural network;

[0009] The trained respiratory sound classification and recognition model is used to invert the respiratory sound dataset to obtain the accuracy of different types of respiratory sound recognition.

[0010] In one embodiment, extracting respiratory sound features based on the respiratory sound dataset is specifically performed as follows:

[0011] Filter the respiratory sound data set to remove the noise generated by the external environment or the equipment when collecting the respiratory sound;

[0012] The filtered breath sounds are pre-emphasized to enhance the high-frequency energy of the breath sounds and remove random noise;

[0013] Then, the pre-emphasized breath sound is subjected to frame division and windowing processing to obtain a plurality of short-time breath sound data frames, and a smooth transition is made between these short-time data frames;

[0014] Then, the time domain signals of the multiple short-term breath sound data frames are converted into frequency domain signals, and then the linear spectrum energy of the breath sound is obtained after modulus squaring;

[0015] The linear spectrum energy of the respiratory sound is processed by a Mel filter to obtain a Mel spectrogram, and the Mel spectrogram is subjected to energy compression processing to compress the energy data of the Mel spectrogram into a set range.

[0016] In one embodiment, the energy data compression formula is as follows:

[0017]

[0018] Among them, num is the number of frames, s(m) is the value of the Mel spectrum output by the Mel filter, abs(s i ) is the absolute value of the energy of the ith Mel filter spectrum.

[0019] In one embodiment, a neural network-based respiratory sound classification and recognition model includes a sequentially connected input port sampling layer, an attention mechanism layer, a RELU activation layer, a first convolution layer, a RELU 6 activation layer, a first pooling layer, a second convolution layer, a RELU6 activation layer, a second pooling layer, a third convolution layer, a RELU6 activation layer, a fully connected layer 1, a fully connected layer 2 (output layer), an attention mechanism layer, and a softmax classifier;

[0020] In one embodiment, the input port sampling layer uses a respiratory sound feature map of size 128*32 as input to obtain a 32*32 respiratory sound feature map, which is used as the input of the first convolution layer after the attention mechanism layer and the RELU activation layer, and is operated with a convolution kernel of size 5*5*6 to obtain a 28*28*6 respiratory sound feature map, and then RELU 6 activation and 2*2 maximum value pooling are performed to obtain a 14*14*6 respiratory sound feature map as the input of the second convolution layer, and is operated with a convolution kernel of size 5*5*16 to obtain a 10*10*16 respiratory sound feature map, and then RELU 6 activation and 2*2 maximum value pooling are performed to obtain a 5*5*16 feature map as the input of the third convolution layer, and is operated with a convolution kernel of size 5*5*120 to obtain a 1*1*120 respiratory sound feature map, and then RELU 6 activation, the breathing sound feature map of the softmax classifier input is obtained through the fully connected layer and the output layer, and finally the classification type index with the maximum probability is obtained through the interaction between the softmax classifier and the attention mechanism layer.

[0021] In one embodiment, the respiratory sound classification and recognition model is accelerated on FPGA, and the specific structure of each layer includes:

[0022] The input port sampling layer includes: input data RAM IP core, row counter, column counter, output data RAM IP core;

[0023] The attention mechanism layer includes: multiplier, selector, comparator, parameter RAM IP core;

[0024] The convolution layer includes: input data RAM IP core, convolution kernel RAM IP core, weight parameter RAM IP core, bias parameter RAM IP core, multiplier IP core, shift_ram IP core, output data RAM IP core;

[0025] The RELU 6 activation layer includes: 0 / 6 comparators, counters, and output data RAM IP cores;

[0026] The pooling layer includes: row counter, column counter, comparator, selector, shift_ram IP core, FIFO IP core;

[0027] The softmax classifier includes: a counter and a comparator.

[0028] In one of the embodiments, the FPGA transmits a certain type of breathing sound recognition rate to a PC for display via UART serial port communication.

[0029] According to a second aspect of an embodiment of the present disclosure, a system for designing a respiratory sound classification and recognition accelerator based on FPGA is provided, comprising:

[0030] A data set acquisition module is used to establish a respiratory sound data set, wherein the respiratory sound data set includes multiple types of respiratory event sounds such as normal, dry rales, wheezing, and stridor;

[0031] A feature extraction module extracts respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature map;

[0032] The training module uses the respiratory sound feature map as a learning sample to train a respiratory sound classification and recognition model based on a neural network;

[0033] The recognition module uses the trained respiratory sound classification and recognition model to invert the respiratory sound data set to obtain the accuracy of different types of respiratory sound recognition.

[0034] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, the method for designing a respiratory sound classification and recognition accelerator based on FPGA is implemented.

[0035] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for designing a respiratory sound classification and recognition accelerator based on FPGA is implemented.

[0036] Compared with the prior art, the above technical solution adopted by the present invention has the following advantages: compared with the stethoscope, the present invention not only reduces the human resource cost, but also reduces the difference caused by human subjective judgment. Compared with other hardware platforms such as CPU and GPU, the present invention has lower power consumption, faster response speed and stronger portability.

[0037] In terms of respiratory sound recognition, it can not only recognize four types of respiratory sounds, including normal, dry rales, wheezing, and stridor, but also conduct targeted training on other types of respiratory sounds. By adjusting the convolution kernel parameters, weight parameters, and bias parameters of the respiratory sound classification and recognition model, it can adapt to respiratory diseases that are prone to outbreaks in different regions, thereby improving the accuracy and applicability of recognition.

[0038] In order to reduce FPGA resource consumption, the respiratory sound classification and recognition model samples the input data. At the same time, in order to avoid a significant decrease in recognition and classification accuracy, an attention mechanism is also incorporated. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0040] Figure 1 This is a schematic diagram of the framework of the design method of the respiratory sound classification and recognition accelerator based on FPGA;

[0041] Figure 2 It is a flow chart of breath sound feature extraction;

[0042] Figure 3 is a schematic diagram of the existing neural network LeNet-5 in the embodiment;

[0043] Figure 4 is a schematic diagram of the modified neural network LeNet-5 in the embodiment;

[0044] Figure 5 It is a schematic diagram of the principle of self-attention mechanism;

[0045] Figure 6 It is a schematic diagram of the RELU6 activation layer principle. DETAILED DESCRIPTION

[0046] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0047] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0049] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0050] Embodiment 1:

[0051] like Figure 1 As shown, this embodiment provides a method for designing a respiratory sound classification and recognition accelerator based on FPGA, comprising the following steps:

[0052] S1. Establishing a respiratory sound dataset, wherein the respiratory sound dataset includes multiple types of respiratory event sounds;

[0053] Specifically, multiple types of respiratory event sounds include but are not limited to normal, dry rales, wheezing, stridor, etc.

[0054] S2. Extracting respiratory sound features based on the respiratory sound dataset to obtain a respiratory sound feature map;

[0055] This step is run on the PC side, and the Python deep learning library PyTorch is used to extract features from a large number of respiratory sound data sets; Figure 2 As shown:

[0056] S21. Filter the respiratory sound data set to remove noise generated by the external environment or equipment when collecting respiratory sounds; preferably, a second-order 20-200 Hz Butterworth bandpass filter can be used for filtering;

[0057] S22. The filtered breathing sound is pre-emphasized to enhance the high-frequency energy of the breathing sound, compensate for the attenuation of the high frequency by the channel, make the spectrum of the signal flat, and remove random noise; the pre-emphasis processing method is:

[0058] p(n)=s(n)-α×s(n-1)

[0059] Where s(n) is the nth sampling point of the signal, α is the pre-emphasis coefficient, and p(n) is the pre-emphasized breath sound signal;

[0060] S23. Then the pre-emphasized breath sound is framed and windowed to obtain a plurality of short-term breath sound data frames, and a smooth transition is made between these short-term data frames;

[0061] Specifically, the number of sampling points in each frame is 1024. In order to ensure smooth transition between two adjacent frames, the Hamming window function is used to add a window to provide a stable signal for subsequent FFT. The Hamming window function is:

[0062]

[0063] Len is the frame length. For the Hamming window, a is usually 0.46. The windowed respiratory sound signal is:

[0064] h(n)=p(n)*W Hamming (n)

[0065] S24. Convert the time domain signals of the multiple short-term breath sound data frames into frequency domain signals, and then obtain the linear spectrum energy of the breath sound after modulus squaring;

[0066] Since the Fourier transform factor is symmetric, the process uses discrete Fourier transform to take advantage of the symmetry:

[0067]

[0068] Among them, n is the time variable, k is the frequency variable, and N is the signal sequence length;

[0069] The linear spectrum energy of the breath sound signal is obtained by taking the modulus square of the FFT result. Since the sampling points of each frame are 1024 for Fourier transform, and the Fourier transform is symmetrical, the first 513-dimensional vector is selected for modulus square, and the linear spectrum energy is obtained as follows:

[0070] E(k)=(Re(H(k))) 2 +(Im(H(k))) 2

[0071] Where Re(h(K)) represents the real part of the signal after FFT transformation, and Im(H(k)) represents the imaginary part;

[0072] S25. The linear spectrum energy of the respiratory sound is processed by a Mel filter to obtain a Mel spectrogram, and the Mel spectrogram is subjected to energy compression processing to compress the energy data of the Mel spectrogram into a set range.

[0073] It should be noted that the number of Mel filters can be set to 32, and the matrix product of the frame energy vector and the filter bank frequency response is:

[0074]

[0075] The energy data of the Mel spectrogram output by the above Mel filter is mapped to between -6 and 6 to adapt to the input of the respiratory sound classification and recognition model. The energy compression formula is as follows:

[0076]

[0077] Among them, num is the number of frames, and s(m) is the value of the Mel spectrum output by the Mel filter.

[0078] S3. Using the respiratory sound feature map as a learning sample, training a respiratory sound classification and recognition model based on a neural network;

[0079] Specifically, the PC sends the extracted respiratory sound feature map to the FPGA, and uses the FPGA instead of the CPU or GPU to accelerate the respiratory sound classification and recognition model. The PC and FPGA are connected using the uart serial communication protocol.

[0080] like Figure 3 As shown in the figure, the attention mechanism is added to the neural network LeNet-5 to process the input data. The input data is first multiplied by the sigmoid function, and then the result is multiplied by its own value, and finally processed by the relu activation function. The values ​​of the input data are a and b, and a>b. The first part is through the sigmoid function: get Then multiply it with the original input data to get a and These two values ​​are compared to Since we assume a>b, so e a (e b +1)>e b (e a+1), the proportion of data processed by the sigmoid function is greater than the proportion a / b of the original data. At the same time, the numerical size of the data after processing by the sigmoid function will also be smaller than the original numerical size, which can reduce the influence of some local non-essential features, and set some non-important features to zero after the relu activation operation. The entire neural network output structure also makes this type of structure. The sigmoid function is added between the fully connected layer and the softmax selector to multiply it to increase the proportion of the recognized object. According to the selection principle of the softmax selector, the greater the proportion of the recognized object in all recognized objects, the higher the recognition accuracy. Based on the above theory, the classic neural network model LeNet-5 is modified to obtain the neural network-based respiratory sound classification and recognition model adopted by the present invention; Figure 4 shown.

[0081] The respiratory sound classification and recognition model includes a sequentially connected input port sampling layer, an attention mechanism layer, a RELU activation layer, a first convolution layer, a RELU 6 activation layer, a first pooling layer, a second convolution layer, a RELU6 activation layer, a second pooling layer, a third convolution layer, a RELU6 activation layer, a fully connected layer 1, a fully connected layer 2, i.e., an output layer, an attention mechanism layer, and a softmax classifier;

[0082] The input port sampling layer includes an input data RAM IP core, a row counter, a column counter, and an output data RAM IP core. The input data RAM IP core stores the breathing sound data transmitted from the PC to the FPGA, and gives a flag signal to start reading and sampling the stored data; in order to reduce the resource consumption of the FPGA, the target data 128*32 needs to be sampled into 32*32 to meet the input size of the convolution layer, and the write enable signal of each RAM is controlled by the row counter and the column counter. Since the original data has 128 rows, in order to write the original data into the RAM, the method of each 4 points is adopted during sampling to reduce the amount of data. When the row counter takes a multiple of 4, it only needs to check whether its lowest two bits (the rightmost 2 bits) are both 0 to determine whether to perform a write operation. Finally, the result of data storage is that the 32 RAMs store the sampled data of the corresponding rows in the original data respectively, thereby converting the original 128 rows and 32 columns of the original data into 32 rows and 32 columns of matrix data.

[0083] The attention mechanism includes a multiplier, a selector, a comparator, and a parameter RAM IP core. It weakens the significant features of the non-breathing sound features and increases the significant proportion of the special features. The attention mechanism is mainly processed by the sigmod function and then multiplied by it. The sigmod function formula is:

[0084]

[0085] It is difficult to implement exponential function calculations on FPGA, so the sigmoid function is fitted into multiple univariate functions for calculation, such as Figure 5 shown.

[0086] The convolution layer includes an input data RAM IP core, a convolution kernel RAM IP core, a weight parameter RAM IP core, a bias parameter RAM IP core, a multiplier IP core, a shift_ram IP core, and an output data RAM IP core. First, the data processed by the attention mechanism is stored in the input data RAM IP core. Subsequently, these data are shifted and delayed by the shift_ram IP core to realize the data beat processing. At the same time, the weight parameter is also stored in the weight parameter RAM IP core, and is also shifted and delayed to ensure that each column of breath sound data can meet the convolution kernel in the same clock cycle, so as to smoothly perform the convolution operation. In the clock cycle of the convolution operation, a 16-bit signed multiplier IP core works in conjunction with a 16-bit adder IP core. The multiplier is responsible for calculating the product of two 16-bit fixed-point quantization parameters (each parameter contains 12 decimals), and outputs a 32-bit signed product result, which contains 24 decimals. To process this product result, arithmetic truncation is performed to retain only the highest bit and necessary integer bits to reduce the data width. Next, these truncated results are accumulated and added to the value read from the weight parameter RAM IP core to obtain the final convolution operation result. This result is stored in the output data RAM IP core for subsequent processing. It is worth noting that since multiplication operations in FPGAs usually use two's complement representation, the data stored in the weight parameter RAM IP core and the weight parameter RAM IP core are in the form of two's complement. In this way, the entire convolution operation process can be executed efficiently and accurately on the FPGA.

[0087] The RELU 6 activation layer is embedded in the convolution layer to directly activate the processed convolution result, including 0 / 6 comparators, counters, and output data RAM IP cores. Figure 6 As shown in the figure, each result of the convolutional layer will be filtered out by the 0 / 6 comparator to remove the influence of subtle features, and the counter will increase by 1 each time a number is filtered. A relu_end signal will be sent out through the value of the counter to determine whether all the results have been filtered out. At the same time, this signal will also react to the counter to clear it to 0.

[0088] The pooling layer includes a row counter, a column counter, a comparator, a selector, a shift_ram IP core, and a FIFO IP core. Starting from the row count of 0, the following process is followed when performing the maximum pooling operation: when the row count reaches 0, for the 0th row in the same 2x2 matrix, two adjacent numbers are selected, and the shift_ram IP core is used for delay processing to ensure that the two numbers can appear in the same clock cycle. Subsequently, the two numbers are compared and the larger number is stored in the FIFO (first in, first out) IP core. At this time, if the column counter is 1 and the row count is 0, I perform a logical AND operation on the two signals, and the result is used as the write enable signal of the FIFO to write the larger number into the FIFO. When the row count reaches 1, starting from the column counter of 0, the number previously stored in the FIFO is directly read and compared with the currently input data. Since the enable signal of the FIFO is one clock cycle faster than the data read out, there is no need to perform additional row delay processing to ensure that the two numbers to be compared appear in the same clock cycle. By comparison, the result of the maximum pooling is obtained. At this time, if the column counter is 0 and the row count is 1, the two signals are logically ANDed, and the result is used as the read enable signal of the FIFO. Similarly, the read and write enable signals of the FIFO can be controlled according to the parity of the row counter and the column counter. Specifically, when the column counter is odd and the row count is even, I perform a logical AND operation on the two signals, and the result is used as the write enable signal of the FIFO; and when the column counter is even and the row count is odd, the same logical AND operation is performed, but the result is used as the read enable signal of the FIFO. To determine the parity of the row counter and the column counter, just check whether their lowest bit is 1. If the lowest bit is 1, it means that the counter is odd; if the lowest bit is 0, it means that the counter is even.

[0089] The workflow of the fully connected layer is managed and completed through a state machine. This state machine contains multiple states, namely: start state, read weight parameter state, read bias parameter state, load pooling layer output data state, multiplication and addition calculation state, data storage state, and check whether the full connection is completed.

[0090] S4. Using the trained respiratory sound classification and recognition model, the respiratory sound dataset is inverted to obtain the accuracy of different types of respiratory sound recognition.

[0091] Specifically, the FPGA side then performs recognition and classification through the trained respiratory sound classification and recognition model, and transmits the recognition accuracy to the PC side.

[0092] Embodiment 2:

[0093] This embodiment provides a FPGA-based respiratory sound classification and recognition accelerator design system, including:

[0094] A data set acquisition module is used to establish a respiratory sound data set, wherein the respiratory sound data set includes multiple types of respiratory event sounds;

[0095] A feature extraction module extracts respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature map;

[0096] The training module uses the respiratory sound feature map as a learning sample to train a respiratory sound classification and recognition model based on a neural network;

[0097] The recognition module uses the trained respiratory sound classification and recognition model to invert the respiratory sound data set to obtain the accuracy of different types of respiratory sound recognition.

[0098] Embodiment three:

[0099] An electronic device includes a memory, a processor, and a computer program stored and running on the memory, wherein the processor implements the above-mentioned FPGA-based respiratory sound classification and recognition accelerator design method when executing the program, including:

[0100] Establishing a respiratory sound data set, the respiratory sound data set includes multiple types of respiratory event sounds; extracting respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature map;

[0101] The respiratory sound feature map is used as a learning sample to train a respiratory sound classification and recognition model based on a neural network;

[0102] The trained respiratory sound classification and recognition model is used to invert the respiratory sound dataset to obtain the accuracy of different types of respiratory sound recognition.

[0103] Embodiment 4:

[0104] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for designing a respiratory sound classification and recognition accelerator based on FPGA, including:

[0105] Establishing a respiratory sound data set, the respiratory sound data set includes multiple types of respiratory event sounds; extracting respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature map;

[0106] The respiratory sound feature map is used as a learning sample to train a respiratory sound classification and recognition model based on a neural network;

[0107] The trained respiratory sound classification and recognition model is used to invert the respiratory sound dataset to obtain the accuracy of different types of respiratory sound recognition.

[0108] Those skilled in the art should understand that the modules or steps of the present disclosure can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present disclosure is not limited to any specific combination of hardware and software.

[0109] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0110] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A design method for a respiratory sound classification and recognition accelerator based on FPGA, characterized in that: The following steps are involved: Establishing a respiratory sound dataset, wherein the respiratory sound dataset includes multiple types of respiratory event sounds; Extracting respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature graph; The respiratory sound feature map is used as a learning sample to train a respiratory sound classification and recognition model based on a neural network; The trained respiratory sound classification and recognition model is used to invert the respiratory sound dataset to obtain the accuracy of different types of respiratory sound recognition.

2. According to the FPGA-based respiratory sound classification and recognition accelerator design method of claim 1, it is characterized in that: Breathing sound feature extraction is performed based on the breathing sound data set, specifically: Filter the respiratory sound data set to remove the noise generated by the external environment or the equipment when collecting the respiratory sound; The filtered breath sounds are pre-emphasized to enhance the high-frequency energy of the breath sounds and remove random noise; Then, the pre-emphasized breath sound is subjected to frame division and windowing processing to obtain a plurality of short-time breath sound data frames, and a smooth transition is made between these short-time data frames; Then, the time domain signals of the multiple short-term breath sound data frames are converted into frequency domain signals, and then the linear spectrum energy of the breath sound is obtained after modulus squaring; The linear spectrum energy of the respiratory sound is processed by a Mel filter to obtain a Mel spectrogram, and the Mel spectrogram is subjected to energy compression processing to compress the energy data of the Mel spectrogram into a set range.

3. According to the FPGA-based respiratory sound classification and recognition accelerator design method of claim 2, it is characterized in that: The energy data compression formula is as follows: Among them, num is the number of frames, s(m) is the value of the Mel spectrum output by the Mel filter, abs(s i ) is the absolute value of the energy of the ith Mel filter spectrum.

4. According to the FPGA-based respiratory sound classification and recognition accelerator design method of claim 1, it is characterized in that: The respiratory sound classification and recognition model based on neural network includes a sequentially connected input port sampling layer, an attention mechanism layer, a RELU activation layer, a first convolution layer, a RELU 6 activation layer, a first pooling layer, a second convolution layer, a RELU6 activation layer, a second pooling layer, a third convolution layer, a RELU6 activation layer, a fully connected layer 1, a fully connected layer 2, i.e., an output layer, an attention mechanism layer, and a softmax classifier.

5. According to the FPGA-based respiratory sound classification and recognition accelerator design method of claim 4, it is characterized in that: The input port sampling layer takes the respiratory sound feature map of size 128*32 as input to obtain a 32*32 respiratory sound feature map. After the attention mechanism layer and the RELU activation layer, it is used as the input of the first convolution layer, and the convolution kernel of size 5*5*6 is used to obtain a 28*28*6 respiratory sound feature map. Then RELU 6 activation and 2*2 maximum value pooling are performed to obtain a 14*14*6 respiratory sound feature map. As the input of the second convolution layer, the convolution kernel of size 5*5*16 is used to obtain a 10*10*16 respiratory sound feature map. Then RELU 6 activation and 2*2 maximum value pooling are performed to obtain a 5*5*16 feature map as the input of the third convolution layer. The convolution kernel of size 5*5*120 is used to obtain a 1*1*120 respiratory sound feature map. Then RELU 6 activation, the breathing sound feature map of the softmax classifier input is obtained through the fully connected layer and the output layer, and finally the classification type index with the maximum probability is obtained through the interaction between the softmax classifier and the attention mechanism layer.

6. The method for designing a respiratory sound classification and recognition accelerator based on FPGA according to claim 4, characterized in that: The respiratory sound classification and recognition model is accelerated on FPGA, and the specific structure of each layer includes: The input port sampling layer includes: input data RAM IP core, row counter, column counter, output data RAM IP core; The attention mechanism layer includes: multiplier, selector, comparator, parameter RAM IP core; The convolution layer includes: input data RAM IP core, convolution kernel RAM IP core, weight parameter RAM IP core, bias parameter RAM IP core, multiplier IP core, shift_ram IP core, output data RAM IP core; The RELU 6 activation layer includes: 0 / 6 comparators, counters, and output data RAM IP cores; The pooling layer includes: row counter, column counter, comparator, selector, shift_ram IP core, FIFO IP core; The softmax classifier includes: a counter and a comparator.

7. The method for designing a respiratory sound classification and recognition accelerator based on FPGA according to claim 6, characterized in that: The FPGA transmits a certain type of breathing sound recognition rate to the PC for display via UART serial port communication.

8. A respiratory sound classification and recognition accelerator design system based on FPGA, characterized in that: include: A data set acquisition module is used to establish a respiratory sound data set, wherein the respiratory sound data set includes multiple types of respiratory event sounds; A feature extraction module extracts respiratory sound features based on the respiratory sound data set to obtain a respiratory sound feature map; The training module uses the respiratory sound feature map as a learning sample to train a respiratory sound classification and recognition model based on a neural network; The recognition module uses the trained respiratory sound classification and recognition model to invert the respiratory sound data set to obtain the accuracy of different types of respiratory sound recognition.

9. An electronic device, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, the method for designing a respiratory sound classification and recognition accelerator based on FPGA according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for designing a respiratory sound classification and recognition accelerator based on FPGA according to claims 1-7 is implemented.

Images