Double-model fusion heart sound signal quality evaluation method, device, equipment and medium

Abstract

The invention provides a double-model fusion heart sound signal quality evaluation method and device, equipment and a medium, and relates to the technical field of computers. The double-model fused heart sound signal quality evaluation method comprises the following steps: acquiring a to-be-detected heart sound signal and the positions of a first heart sound and a second heart sound in the to-be-detected heart sound signal; based on the positions of the first heart sound and the second heart sound in the heart sound signal to be detected, extracting signal quality features from the heart sound signal to be detected; obtaining a first quality evaluation result based on the heart sound signal to be detected and a preset first quality evaluation model; taking the signal quality characteristics as input data of a preset second quality evaluation model to obtain a second quality evaluation result; wherein the first quality evaluation model and the second quality evaluation model are different in structure; and obtaining a heart sound signal quality evaluation result based on the first quality evaluation result and the second quality evaluation result.

Description

Technical Field

[0001] The present application relates to the technical field of computers, and in particular to a method, device, equipment and medium for evaluating the quality of heart sound signals by dual-model fusion. Background Art

[0002] Heart sound signals are collected by heart sound sensors and other equipment, but in the collection process, interference and noise caused by human or environmental factors will inevitably be introduced. As a result, when analyzing based on the heart sound signal, the accuracy of the analysis results will be reduced. Therefore, in order to ensure the accuracy of the heart sound signal analysis results, it is necessary to evaluate the quality of the heart sound signal. The existing heart sound signal quality evaluation is often performed manually, or a neural network model is used to evaluate the heart sound signal. However, the former has a low evaluation efficiency and it is difficult to make a detailed division of quality. The stability of the evaluation results of the neural network model for different heart sound signals is difficult to guarantee. Summary of the invention

[0003] The present application provides a dual-model fusion heart sound signal quality assessment method, device, equipment and medium to improve the stability of evaluation results of different heart sound signals.

[0004] In a first aspect, the present application provides a method for evaluating the quality of heart sound signals by fusion of dual models, including: obtaining a heart sound signal to be detected and the positions of a first heart sound and a second heart sound in the heart sound signal to be detected; extracting a signal quality feature from the heart sound signal to be detected based on the positions of the first heart sound and the second heart sound in the heart sound signal to be detected; obtaining a first quality evaluation result based on the heart sound signal to be detected and a preset first quality evaluation model; obtaining a second quality evaluation result by using the signal quality feature as input data of a preset second quality evaluation model; wherein the structures of the first quality evaluation model and the second quality evaluation model are different; and obtaining a heart sound signal quality evaluation result based on the first quality evaluation result and the second quality evaluation result.

[0005] In the embodiment of the present application, the quality of the heart sound signal to be detected is evaluated respectively by a first quality evaluation model and a second quality evaluation model with different structures, and then a final heart sound signal quality evaluation result is obtained based on the two quality evaluation results. Since the first quality evaluation model and the second quality evaluation model have different structures and the input data of the two models are also different, the two models can perform quality evaluation from two different angles. Therefore, the final heart sound signal quality evaluation result is obtained by combining the quality evaluation results of the two models. Even if there is a large deviation between the evaluation result of one model and the actual quality, the final heart sound signal quality evaluation result can be corrected by the evaluation result of the other model, thereby reducing the deviation between the final heart sound signal quality evaluation result and the actual quality, thereby improving the stability of the heart sound signal quality evaluation result.

[0006] In combination with the technical solution provided in the first aspect above, in some possible implementations, a heart sound signal quality assessment result is obtained based on the first quality assessment result and the second quality assessment result, including: determining whether the deviation between the first quality assessment result and the second quality assessment result is within a preset allowable deviation range; if the deviation between the first quality assessment result and the second quality assessment result is within the allowable deviation range, obtaining the heart sound signal quality assessment result based on the fusion of the first quality assessment result and the second quality assessment result; if the deviation between the first quality assessment result and the second quality assessment result is not within the allowable deviation range, determining a third quality assessment result based on a specified feature in the pre-specified signal quality features; obtaining the heart sound signal quality assessment result based on the fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result.

[0007] In the embodiment of the present application, when the deviation between the first quality assessment result and the second quality assessment result is not within the allowable deviation range, it indicates that there is a large difference in the assessment results of the two models. Therefore, a third quality assessment result can be introduced to correct the heart sound signal quality assessment result. Since the third quality assessment result is obtained through the specified features in the signal quality features, the third quality assessment result can also reflect the quality of the heart sound signal to be detected, so that the heart sound signal quality assessment result obtained by fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result can be more accurate than the case where only the first quality assessment result and the second quality assessment result are considered.

[0008] In combination with the technical solution provided in the first aspect above, in some possible implementations, a heart sound signal quality assessment result is obtained based on the fusion of the first quality assessment result and the second quality assessment result, including: calculating the product of the first confidence corresponding to the first quality assessment model and the first quality assessment result to obtain a first intermediate result; calculating the product of the second confidence corresponding to the second quality assessment model and the second quality assessment result to obtain a second intermediate result; wherein the sum of the first confidence and the second confidence is 1, and the first confidence and the second confidence are both positive numbers greater than 0; and taking the sum of the first intermediate result and the second intermediate result as the heart sound signal quality assessment result.

[0009] In the embodiment of the present application, since the accuracy of the heart sound signal quality assessment by different models may be different, therefore, by configuring the confidence level for the first quality assessment model and the second quality assessment model respectively, and then processing the results of the two models respectively based on the first confidence level and the second confidence level, the output result of the model with a higher confidence level can have a greater impact on the heart sound signal quality assessment result, thereby improving the accuracy of the heart sound signal quality assessment result.

[0010] In combination with the technical solution provided in the first aspect above, in some possible implementations, the value of the third quality assessment result is determined based on a standard value corresponding to the designated feature and the value of the designated feature.

[0011] In the embodiment of the present application, the closer the value of the specified feature is to the standard value corresponding to the specified feature, the better the quality of the heart sound signal to be detected. Therefore, the larger the third quality assessment result obtained based on the specified feature, the greater its impact on the final heart sound signal quality assessment result.

[0012] In combination with the technical solution provided in the first aspect above, in some possible implementations, the designated feature is the ratio of the number of first heart sounds to the number of peaks detected with distance, and based on the designated features in the pre-specified signal quality features, a third quality assessment result is determined, including: determining a first ratio of the number of first heart sounds included in the heart sound signal to be detected to the number of peaks detected with distance; obtaining the third quality assessment result based on the first ratio, wherein the third quality assessment result is at least 0 and at most 1.

[0013] In combination with the technical solution provided in the first aspect above, in some possible implementations, the heart sound signal quality assessment result is obtained based on the fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result, including: calculating the absolute value of the difference between the first quality assessment result and the second quality assessment result; calculating the product of the absolute value and the third quality assessment result; summing the product with the minimum value of the first quality assessment result and the second quality assessment result to obtain the heart sound signal quality assessment result.

[0014] In the embodiment of the present application, when the first quality assessment result and the second quality assessment result differ greatly, the probability that the result corresponding to the actual quality of the heart sound signal to be detected is between the first quality assessment result and the second quality assessment result is greater. Therefore, the absolute value of the difference between the first quality assessment result and the second quality assessment result is processed by the third quality assessment result, so as to evaluate the position of the result corresponding to the actual quality of the heart sound signal to be detected between the first quality assessment result and the second quality assessment result by the third quality assessment result. Since the third quality assessment result is also obtained based on the signal quality feature, the actual quality of the heart sound signal to be detected is further evaluated by the third quality assessment result, thereby making the heart sound signal quality assessment result more accurate.

[0015] In combination with the technical solution provided in the first aspect above, in some possible implementations, the first quality assessment model includes a first convolution layer, a first normalization layer, a first activation function layer, a first maximum pooling layer, a plurality of sequentially connected depthwise separable convolution blocks, a global average pooling layer, and a fully connected layer; wherein the depthwise separable convolution block is used to extract local features of the input data of each channel of itself, and to fuse the extracted local features.

[0016] In an embodiment of the present application, the local features of the input data of each channel of the deep separable convolution block are extracted, and the independence of each channel can be retained. The extracted local features are then fused. This can significantly reduce the amount of calculation and the number of parameters while maintaining the expressiveness of the model, thereby achieving a lightweight model.

[0017] In combination with the technical solution provided in the first aspect above, in some possible implementations, each of the depthwise separable convolution blocks includes a first branch, a second branch, an adder and a second activation function layer, the adder is respectively connected to the output end of the first branch and the output end of the second branch, the input ends of the first branch and the second branch are both connected to the first maximum pooling layer, the output ends of the first branch and the second branch are both connected to the adder, and the output end of the adder is connected to the input end of the second activation function layer; wherein the first branch is used to extract local features of the input data of each channel of itself, and fuse the extracted local features; the second branch is used to retain the input of the depthwise separable convolution block.

[0018] In the embodiment of the present application, the output results of the first branch and the second branch are processed by an adder, and the second branch retains the input of the depthwise separable convolution block, thereby alleviating the degradation of the first quality assessment model.

[0019] In combination with the technical solution provided in the first aspect above, in some possible implementations, the first branch includes multiple convolution blocks, each of which includes a depth convolution layer, a second batch normalization layer, a point-by-point convolution layer, a third batch normalization layer, and a third activation function layer.

[0020] In the embodiment of the present application, the deep convolution layer can extract the local features of the input data of each channel of itself, and can retain the independence of each channel. The point-by-point convolution layer can fuse the local features extracted by the deep convolution layer through the point multiplication operation across channels.

[0021] In combination with the technical solution provided in the first aspect above, in some possible implementations, the second branch includes a second convolutional layer and a fourth batch normalization layer connected in sequence.

[0022] In combination with the technical solution provided in the first aspect above, in some possible implementations, the signal quality characteristics include at least one of the interval between the first heart sound and the second heart sound, the number of the first heart sound and the second heart sound, the duration of the first heart sound and the second heart sound, the number of peaks with distance detection, the ratio of the number of first heart sounds to the number of peaks with distance detection, spectral power, the number of sampling points exceeding a preset threshold, the average value of peak intervals, signal length, and the average value of the peak intervals of the first heart sound.

[0023] In a second aspect, the present application provides a dual-model fused heart sound signal quality assessment device, including: an acquisition module and a processing module, the acquisition module being used to acquire a heart sound signal to be detected and the positions of a first heart sound and a second heart sound in the heart sound signal to be detected; the processing module being used to extract a signal quality feature from the heart sound signal to be detected based on the positions of the first heart sound and the second heart sound in the heart sound signal to be detected; obtaining a first quality assessment result based on the heart sound signal to be detected and a preset first quality assessment model; obtaining a second quality assessment result by using the signal quality feature as input data of a preset second quality assessment model; and obtaining a heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result.

[0024] In a third aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory and the processor being connected; the memory being used to store programs; and the processor being used to call the programs stored in the memory to execute the method described in the first aspect above and / or in combination with any possible implementation of the first aspect above.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer, the computer program executes the method described in the first aspect and / or in combination with any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A schematic diagram of a process flow of a heart sound signal quality assessment method using dual-model fusion shown in an embodiment of the present application;

[0028] Figure 2 A structural block diagram of a quality assessment model shown in an embodiment of the present application;

[0029] Figure 3 A structural block diagram of a depth-separable convolution block shown in an embodiment of the present application;

[0030] Figure 4 This is a structural block diagram of a heart sound signal quality assessment device using dual-model fusion as shown in an embodiment of the present application;

[0031] Figure 5A structural block diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it is not necessary to further define and explain it in the subsequent drawings. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0034] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0035] See also Figure 1 , Figure 1 This is a flow chart of a dual-model fusion heart sound signal quality assessment method shown in an embodiment of the present application. Figure 1 Describe the steps it involves.

[0036] S100: Acquire a heart sound signal to be detected and positions of a first heart sound and a second heart sound in the heart sound signal to be detected.

[0037] The heart sound signal to be detected and the positions of the first heart sound and the second heart sound in the heart sound signal to be detected may be acquired in advance and stored in a local storage medium, and may be directly called when needed. Alternatively, they may be acquired in real time when needed.

[0038] In one implementation, a device for collecting heart sound signals may be used to collect heart sound signals at a position close to the user's chest or other position where heart sound can be collected, to obtain a heart sound signal to be detected.

[0039] The device for collecting heart sound signals may be, for example, a heart sound sensor or the like, and its specific type is not limited here.

[0040] Optionally, in order to reduce the influence of other sounds on the heart sound signal, when collecting the heart sound signal, it is necessary to be in a quiet environment to reduce the influence of other sounds on the collected heart sound signal.

[0041] In one implementation, the method of obtaining the heart sound signal to be detected may also be: first obtain an initial heart sound signal to be detected. Then remove noise from the initial heart sound signal to be detected to obtain the heart sound signal to be detected. The initial heart sound signal may be collected by a device for collecting heart sound signals close to the heart of the user.

[0042] Optionally, the noise in the initial heart sound signal to be detected may be removed by using one or more filters such as a low-pass filter and a high-pass filter.

[0043] After obtaining the heart sound signal to be detected, the method of determining the positions of the first heart sound and the second heart sound from the heart sound signal to be detected can be any existing method of determining the positions of the first heart sound and the second heart sound in the heart sound signal, and its specific method is not limited here.

[0044] The positioning and extraction method of the first and second heart sounds mainly takes into account the local maximum of heart sounds and the periodicity of heartbeats. On the preprocessed PCG (Phonocardiogram, heart sound signal) signal segment, the maximum points with reasonable intervals are first located as candidate points for the first and second heart sounds. Then, according to the periodicity of the first and second heart sounds, the peak points of the first and second heart sounds with reasonable intervals and amplitudes are selected to obtain the positions of the first and second heart sounds respectively.

[0045] S200: Extracting a signal quality feature from the heart sound signal to be detected based on positions of the first heart sound and the second heart sound in the heart sound signal to be detected.

[0046] Extracting signal quality features is to extract computational features that can characterize the quality of the heart sound signal from the heart sound signal to be detected, which is used to evaluate the quality of the real-time heart rate signal of the heart sound signal to be detected. In order to fully extract the information of the heart sound signal to be detected and the degree of interference, avoid the influence of individual waveform differences and differences in acquisition equipment, and improve the performance of quality assessment, the signal quality features respectively select waveform features, statistical features, nonlinear features, dynamic templates and other features of the heart sound signal.

[0047] In one implementation, the signal quality characteristics include at least one of the interval between the first heart sound and the second heart sound, the number of the first heart sound and the second heart sound, the duration of the first heart sound and the second heart sound, the number of peaks with distance detection, the ratio of the number of first heart sounds to the number of peaks with distance detection, spectral power, the number of sampling points exceeding a preset threshold, the average value of the peak interval, the signal length, and the average value of the peak interval of the first heart sound.

[0048] The interval between the first heart sound and the second heart sound specifically refers to the interval time between the first heart sound and the second heart sound in the same cardiac cycle. Due to the limitation of physiological function, the length of the cardiac cycle cannot be too large or too small, that is, the interval between the first heart sound and the second heart sound cannot be too long or too short (compared with the interval from the first heart sound to the second heart sound under normal circumstances), that is, the interval between the first heart sound and the second heart sound has a maximum value and a minimum value. Therefore, if the interval between the first heart sound and the second heart sound is too short (less than the minimum value), there may be a situation where the noise is mistakenly identified as the first heart sound or the second heart sound; if the interval between the first heart sound and the second heart sound is too long (greater than the maximum value), there may be a situation where the first heart sound or the second heart sound is lost. Therefore, the interval between the first heart sound and the second heart sound can reflect the quality of the heart sound signal to a certain extent.

[0049] The number of first heart sounds and second heart sounds specifically refers to the number of first heart sounds and the number of second heart sounds in the entire heart sound signal to be detected. As mentioned above, the length of the cardiac cycle cannot be too large or too small. Therefore, when the length of the heart sound signal to be detected is known, the range of the number of cardiac cycles it includes can be roughly inferred. And there is only one first heart sound and one second heart sound in each cardiac cycle. Therefore, the difference between the number of first heart sounds and the number of second heart sounds will be very small (usually consistent), and the number of first heart sounds and the number of second heart sounds cannot be too much or too little within a fixed time. If the number of first heart sounds and the number of second heart sounds are too many, there may be a situation where the noise is mistakenly identified as the first heart sound or the second heart sound; if the number of first heart sounds and the second heart sound is too few, there may be a situation where the first heart sound or the second heart sound is lost. Therefore, the number of first heart sounds and second heart sounds can reflect the quality of the heart sound signal to a certain extent.

[0050] The duration of the first heart sound and the second heart sound, that is, the duration of the first heart sound and the duration of the second heart sound. The duration of the first heart sound and the second heart sound is affected by physiological factors and cannot exceed the normal range. Therefore, if the duration of the first heart sound and the second heart sound is too short, the noise may be mistakenly identified as the first heart sound or the second heart sound. If the duration of the first heart sound and the second heart sound is too long, it may be interfered by noise. Therefore, the duration of the first heart sound and the second heart sound can reflect the quality of the heart sound signal to a certain extent.

[0051] The number of peaks with distance detection specifically refers to the sum of a first number and a second number, the first number being the number of first heart sounds whose distance from a first heart sound to the next first heart sound is greater than a first preset distance threshold (the first preset distance threshold is the shortest distance between two first heart sounds), and the second number being the number of second heart sounds whose distance from a second heart sound to the next second heart sound is greater than a second preset distance threshold (the second preset distance threshold is the shortest distance between two second heart sounds). If the distance from a first heart sound to the next first heart sound is less than the first preset distance threshold, it indicates that noise may be mistakenly identified as the first heart sound, and if the distance from a second heart sound to the next second heart sound is less than the second preset distance threshold, it indicates that noise may be mistakenly identified as the second heart sound. Therefore, the number of peaks with distance detection can also reflect the quality of the heart sound signal to a certain extent.

[0052] The ratio of the number of first heart sounds to the number of peak values ​​with distance detection. Since the number of peak values ​​with distance detection refers to the sum of the first number and the second number, the closer the ratio of the number of first heart sounds to the number of peak values ​​with distance detection is to 0.5, the better the quality of the heart sound signal to be detected; otherwise, the worse the quality.

[0053] Spectral power is the power spectrum calculated after converting the heart sound signal to be detected into a frequency domain signal. Since the peak values ​​of the first heart sound and the second heart sound are relatively high, the power on the corresponding power spectrum is also relatively high. Therefore, the spectral power can also be used to further determine whether the recognition of the first heart sound and the second heart sound is correct, thereby reflecting the quality of the heart sound signal to be detected.

[0054] The number of sampling points exceeding a preset threshold, wherein the preset threshold is a value higher than the second heart sound but lower than the first heart sound. Therefore, ideally, the sampling points exceeding the preset threshold should all be sampling points of the first heart sound. Therefore, based on the number of sampling points exceeding the preset threshold, the noise with a higher peak value can be determined, thereby reflecting the quality of the heart sound signal to be detected.

[0055] The average value of the peak interval, because in the same heart sound signal, the interval between the first heart sound and the second heart sound and the interval between the second heart sound and the first heart sound of the next cardiac cycle are relatively uniform. Therefore, the average value of the peak interval under ideal conditions can be estimated based on the cardiac cycle. If the average value of the peak interval is larger than the interval estimated based on the cardiac cycle, it may be that there is too much noise causing the average value of the peak interval to be too small; or it may be that more first and second heart sounds are lost, causing the average value of the peak interval to be too large. Therefore, the average value of the peak interval can also reflect the quality of the heart sound signal to be detected.

[0056] The signal length specifically refers to the signal length of the first heart sound. Since the duration of noise is usually short, if the signal length of the first heart sound is short, it may be the noise that is mistakenly identified as the first heart sound. If the signal length of the first heart sound is long, it may be the first heart sound interfered by noise. Therefore, the signal length can also reflect the quality of the heart sound signal to be detected.

[0057] The average value of the peak interval of the first heart sound specifically refers to the average interval between the first heart sound and the first heart sound of the next cardiac cycle. If the distance between the first heart sounds is too large relative to the cardiac cycle, it is possible that many first heart sounds are lost. If the distance between the first heart sounds is too small relative to the cardiac cycle, it is possible that noise is mistakenly identified as the first heart sound. Therefore, the average value of the peak interval of the first heart sound can also reflect the quality of the heart sound signal to be detected.

[0058] Since the interval between the first and second heart sounds, the number of the first and second heart sounds, the duration of the first and second heart sounds, the number of peaks with distance detection, the ratio of the number of first heart sounds to the number of peaks with distance detection, the spectral power, the number of sampling points exceeding the preset threshold, the average value of the peak interval, the signal length, and the average value of the peak interval of the first heart sound can all reflect the quality of the heart sound signal to a certain extent, the final heart sound signal quality assessment result obtained based on such signal quality characteristics can be more accurate.

[0059] S300: Obtaining a first quality assessment result based on the heart sound signal to be detected and a preset first quality assessment model.

[0060] In one implementation, based on the heart sound signal to be detected and a preset first quality assessment model, a specific method for obtaining a first quality assessment result may be: directly using the heart sound signal to be detected as input data of the first quality assessment model to obtain a first quality assessment result output by the first quality assessment model.

[0061] Alternatively, based on the heart sound signal to be detected and a preset first quality assessment model, a specific method for obtaining the first quality assessment result may also be: cutting the heart sound signal to be detected to obtain an input heart sound signal of a preset length, and then using the input heart sound signal as input data for the first quality assessment model to obtain a first quality assessment result output by the first quality assessment model.

[0062] The preset length may be set according to actual requirements of the first quality assessment model, and its specific length is not limited here.

[0063] In one implementation, the first quality assessment model may be any type of neural network model, and its specific type is not limited herein.

[0064] In one implementation, the first quality assessment model includes a first convolutional layer, a first batch of normalization layers, a first activation function layer, a first maximum pooling layer, a plurality of sequentially connected depthwise separable convolutional blocks, a global average pooling layer, and a fully connected layer; wherein the depthwise separable convolutional block is used to extract local features of the input data of each channel of itself, and fuse the extracted local features.

[0065] Among them, the specific structures and working principles of the first convolutional layer, batch normalization layer, first activation function layer, first maximum pooling layer, global average pooling layer, and fully connected layer are well known to those skilled in the art and will not be repeated here for the sake of brief description.

[0066] The specific number of depth-wise separable convolution blocks can be set according to actual needs, for example, it can be 1, 2, 3, 4, 5, 6, 7, 8, 9, etc. The specific number of depth-wise separable convolution blocks includes but is not limited to the examples given here.

[0067] Optionally, each depth-wise separable convolution block may include multiple convolution blocks, each convolution block includes a depth-wise convolution layer, a second batch normalization layer, a point-by-point convolution layer, a third batch normalization layer, and a third activation function layer connected in sequence.

[0068] Optionally, each depth-separable convolution block includes a first branch, a second branch, an adder, and a second activation function layer.

[0069] The adder is connected to the output end of the first branch and the output end of the second branch respectively, the input ends of the first branch and the second branch are both connected to the first maximum pooling layer, the output ends of the first branch and the second branch are both connected to the adder, and the output end of the adder is connected to the input end of the second activation function layer. Among them, the first branch is used to extract the local features of the input data of each channel of itself and fuse the extracted local features; the second branch is used to retain the input of the depth-separable convolution block to alleviate the degradation of the first quality assessment model.

[0070] Optionally, the first branch may include multiple convolution blocks, each convolution block includes a depth convolution layer, a second batch normalization layer, a point-by-point convolution layer, a third batch normalization layer, and a third activation function layer connected in sequence.

[0071] Among them, the number of convolution blocks included in the first branch can be set according to actual needs, for example, it can be 1, 2, 3, 4, 5, 6, 7, 8, 9, etc. The specific number of convolution blocks includes but is not limited to the examples given here.

[0072] Optionally, the second branch includes a second convolutional layer and a fourth batch normalization layer connected in sequence.

[0073] To understand the first quality assessment model, please refer to Figure 2 , Figure 2 This is a specific implementation of a first quality assessment model shown in an embodiment of the present application.

[0074] like Figure 2 As shown, the first quality assessment model includes a first convolutional layer, a first batch of normalization layers, a first activation function layer, a first maximum pooling layer, three sequentially connected depthwise separable convolutional blocks, a global average pooling layer, and a fully connected layer.

[0075] like Figure 3 As shown, each depth-separable convolution block includes a first branch, a second branch, an adder, and a second activation function layer. The first branch includes two convolution blocks, each of which includes a depth-wise convolution layer, a second batch normalization layer, a point-by-point convolution layer, a third batch normalization layer, and a third activation function layer connected in sequence. The second branch includes a second convolution layer and a fourth batch normalization layer connected in sequence.

[0076] in, Figure 2 The first quality assessment model shown, Figure 3 The structure of each layer and the connection method between the structures included in the depth-separable convolution block shown have been clearly described in the previous text. For the sake of brief description, they will not be repeated here.

[0077] When the 1×1×F heart sound signal is input as Figure 2 In the first quality assessment model shown in FIG. 1 , first, a 1×3 Conv layer (first convolution layer) is used to perform a convolution operation on the signal. Subsequently, after the normalization processing of the BN layer (the first batch of normalization layers) and the nonlinear processing of the ReLU activation function (the first activation function layer), a C×1×F multi-channel time domain feature is obtained, where C represents the number of channels and F represents the length of the input heart sound signal. The output data is input into a 1×9 maximum pooling layer (the first maximum pooling layer) to reduce the time dimension of the feature, reduce the amount of calculation, and emphasize noise.

[0078] Then, the compressed data is input into the depthwise separable convolution block. In the first branch, the multi-channel time domain features (data output by the maximum pooling layer) are first input into the depthwise convolution layer, that is, the number of channels is kept unchanged, and each channel is convolved by a 1×9 Conv layer (depthwise convolution layer), and then normalized by a BN layer (second batch normalization layer); then the features are input into the pointwise convolution layer, and a 1×1 Conv layer (that is, pointwise convolution layer) is used to reduce the amount of calculation while using a 1x1 convolution kernel to combine features from different input channels, and then normalized by a BN layer (third batch normalization layer), and nonlinearly processed by a ReLU activation function (third activation function layer). The above operation is repeated twice in the first branch. The second branch is used to retain the full-precision input of the depthwise separable convolution block, and add it to the output of the first branch, and then the result of the addition is used as the output of a depthwise separable convolution block.

[0079] The operation of the depthwise separable convolution block is repeated three times in sequence to achieve three depthwise separable convolution blocks, followed by a global average pooling layer and a classification layer, namely the FC layer (fully connected layer), to obtain the final output result.

[0080] The examples given here are only for ease of understanding, and specific implementations of the first quality assessment model include but are not limited to the examples given here.

[0081] S400: Using the signal quality feature as input data of a preset second quality assessment model to obtain a second quality assessment result.

[0082] The structures of the first quality assessment model and the second quality assessment model are different.

[0083] In one implementation, the second quality assessment model may be any type of neural network model, and its specific type is not limited herein.

[0084] The difference in structure between the first quality assessment model and the second quality assessment model may refer to different data processing procedures of the two models, or may refer to different at least some parameters in the two models.

[0085] For example, the first quality assessment model and the second quality assessment model may both be the above Figure 2 The examples here are only for ease of understanding and should not be used as limitations on this application.

[0086] In one implementation, the first quality assessment model may be the aforementioned model including the first convolution layer, the first batch of normalization layers, the first activation function layer, the first maximum pooling layer, a plurality of sequentially connected depth-separable convolution blocks, the global average pooling layer, and the fully connected layer. The second quality assessment model may be an XGB (Extreme Gradient Boosting) model.

[0087] The execution order of the above S300 and S400 can be set according to actual needs. For example, S300 can be executed first and then S400, or S400 can be executed first and then S300, or S300 and S400 can be executed in parallel. Figure 1 The order shown is to be regarded as a limitation of the application.

[0088] In one implementation, in order to divide the quality of the heart sound signal more finely, the output results of the first quality assessment model and the second quality assessment model may be mapped to score labels within a certain range according to the quality.

[0089] For example, it can be an interval of 0-100 points. In this case, the output results of the first quality assessment model and the second quality assessment model are also scores within the range of 0-100. The higher the score, the better the quality of the heart sound signal. The score annotation has a wider annotation interval. Compared with the simple signal quality category annotation, this method has the advantages of a wider range, more detailed distinction, and richer levels.

[0090] In addition to the above-mentioned 0-100 points, other score intervals may also be used, for example, 0-10, 0-20, 0-30, 10-50, etc., and the specific numerical range is not limited here.

[0091] S500: Obtain a heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result.

[0092] In one implementation, the first quality assessment result and the second quality assessment result may be weighted and summed to obtain a final heart sound signal quality assessment result.

[0093] The first quality assessment model corresponds to a first weight value, the second quality assessment model corresponds to a second weight value, and the sum of the first weight value and the second weight value is 1.

[0094] The first weight value and the second weight value may be set according to actual conditions, for example, according to the accuracy ratio of the first quality assessment model and the second quality assessment model, and the higher the accuracy of the model, the greater the corresponding weight value.

[0095] Alternatively, the weight values ​​of the first quality assessment model and the second quality assessment model may be set to be the same, that is, both are 0.5. In this case, the heart sound signal quality assessment result is the average of the first quality assessment result and the second quality assessment result.

[0096] For example, the calculation method of the heart sound signal quality assessment result can be: Score = k*A+(1-k)*B, where Score is the heart sound signal quality assessment result, k is the first weight value, (1-k) is the second weight value, A is the first quality assessment result, and B is the second quality assessment result.

[0097] In one implementation, based on the first quality assessment result and the second quality assessment result, a heart sound signal quality assessment result may be obtained by first determining whether a deviation between the first quality assessment result and the second quality assessment result is within a preset allowable deviation range.

[0098] If the deviation between the first quality assessment result and the second quality assessment result is within the allowable deviation range, the heart sound signal quality assessment result is obtained based on the fusion of the first quality assessment result and the second quality assessment result.

[0099] If the deviation between the first quality assessment result and the second quality assessment result is within the allowable deviation range, a third quality assessment result is determined based on a specified feature in the pre-specified signal quality features; and a heart sound signal quality assessment result is obtained by fusing the first quality assessment result, the second quality assessment result, and the third quality assessment result.

[0100] The deviation between the first quality assessment result and the second quality assessment result may be the value of the first quality assessment result minus the second quality assessment result. Alternatively, the deviation between the first quality assessment result and the second quality assessment result may be the absolute value of the value of the first quality assessment result minus the second quality assessment result, that is, the difference between the larger value of the quality assessment result and the smaller value of the second quality assessment result.

[0101] The allowable deviation range can be set according to actual conditions, for example, it can be 0~t (or -t~t, the deviation between the first quality assessment result and the second quality assessment result is: the value of the first quality assessment result minus the value of the second quality assessment result, so in this case the deviation may be a negative number, therefore, the allowable deviation range should include a negative number range), t is a positive number. The specific value of t can be set according to actual needs, and its specific value is not limited here.

[0102] Optionally, the specific method of obtaining the heart sound signal quality assessment result based on the fusion of the first quality assessment result and the second quality assessment result may be: calculating the product of the first confidence corresponding to the first quality assessment model and the first quality assessment result to obtain a first intermediate result. Calculating the product of the second confidence corresponding to the second quality assessment model and the second quality assessment result to obtain a second intermediate result; wherein the sum of the first confidence and the second confidence is 1, and both the first confidence and the second confidence are positive numbers greater than 0. The sum of the first intermediate result and the second intermediate result is used as the heart sound signal quality assessment result.

[0103] Since different models may have different accuracy in evaluating the quality of heart sound signals, the first quality evaluation model and the second quality evaluation model are configured with confidence respectively, and the results of the two models are processed respectively based on the first confidence and the second confidence, so that the output result of the model with higher confidence can have a greater impact on the quality evaluation result of the heart sound signal, thereby improving the accuracy of the quality evaluation result of the heart sound signal.

[0104] Optionally, the confidence of the model may be determined by an indicator such as root mean square error, which may be used to judge the accuracy of the model.

[0105] Taking the determination of confidence based on the root mean square error (the smaller the root mean square error, the higher the accuracy of the model) as an example, if the root mean square error of the first quality assessment model is x and the root mean square error of the second quality assessment model is y, then the confidence of the first quality assessment model is y / (x+y), and the confidence of the second quality assessment model can be x / (x+y). The examples here are only for ease of understanding and should not be used as limitations on this application.

[0106] Optionally, the closer the value of the specified feature is to the standard value corresponding to the specified feature, the greater the third quality assessment result is. Different types of features have different corresponding standard values.

[0107] Since the values ​​of some of the above-mentioned quality assessment features are related to the length of the heart sound signal, the standard value can be set to the standard value when the heart sound signal is a unit length. Accordingly, in actual application, the specific value of the standard value actually used is determined according to the ratio of the actual length of the heart sound signal to be detected to the unit length.

[0108] The closer the value of the specified feature is to the standard value corresponding to the specified feature, the better the quality of the heart sound signal to be detected. Therefore, the larger the third quality assessment result obtained based on the specified feature, the greater its impact on the final heart sound signal quality assessment result.

[0109] Optionally, a maximum value and a minimum value may be preset for each feature, and when the feature value is within its corresponding standard value range, the third quality evaluation result corresponding to the feature value is determined to be the maximum value. When the feature value is its corresponding maximum value or minimum value, the third quality evaluation result corresponding to the feature value is determined to be the minimum value.

[0110] Among them, the maximum value and the minimum value need to be set according to the type of feature. For example, if the feature is the ratio of the number of first heart sounds to the number of peak values ​​with distance detection, the standard value can be set to 0.5, the maximum value to 1, and the minimum value to 0. When the ratio of the number of first heart sounds to the number of peak values ​​with distance detection is 0.5, the third quality assessment result is determined to be the maximum value 1; when the ratio of the number of first heart sounds to the number of peak values ​​with distance detection is 0 or 1, the third quality assessment result is determined to be the minimum value 0. And the ratio of the number of first heart sounds to the number of peak values ​​with distance detection in the range of 0 to 0.5 and 0.5 to 1 is mapped to between 0 and 1. It can be a linear mapping or a nonlinear mapping, and its specific mapping method is not limited here.

[0111] Optionally, for a feature that does not have a clear maximum value and minimum value, the feature can be mapped to a fixed interval, so that the two endpoint values ​​of the interval are respectively used as the maximum value and minimum value corresponding to the feature.

[0112] For example, if the signal quality feature includes waveform length, since the waveform length has no clear maximum and minimum values, the arctangent function can be used to map the waveform length of the existing normal heart sound into the range of 0 to 1.

[0113] Specifically, the function arctan(kx+b) / π+0.5 is used to map the existing waveform length obtained by statistics to a range of 0 to 1. The values ​​of k and b can be adjusted according to the statistical value range of the waveform length in the existing data, and x is the existing waveform length.

[0114] In addition to the above-mentioned inverse tangent function, other methods may be used to map the feature into a specified range.

[0115] When the specified features include multiple types of features, the initial evaluation result corresponding to each feature may be calculated first. Then all the initial evaluation results are averaged to obtain the final third evaluation result. The initial evaluation result corresponding to each feature is the same as the method of calculating the third quality evaluation result corresponding to a single feature, which is not repeated here for the sake of simplicity.

[0116] Alternatively, a weight value may be preset for each feature, and then all initial evaluation results are weighted averaged to obtain a final third evaluation result.

[0117] In one implementation, the third quality assessment result may be used as a correction coefficient. In this case, the third quality assessment result is a value in the range of 0 to 1, that is, the maximum value of the third assessment result is 1 and the minimum value is 0.

[0118] For example, the designated feature may be the ratio of the number of first heart sounds to the number of peaks with distance detection. In this case, based on the designated feature in the pre-designated signal quality feature, the specific method of determining the third quality evaluation result may be: first determine the first ratio of the number of first heart sounds included in the heart sound signal to be detected to the number of peaks with distance detection. Then, the third quality evaluation result is obtained based on the first ratio, wherein the closer the first ratio is to 0.5, the larger the third quality evaluation result; the closer the first ratio is to 1 or 0, the smaller the third quality evaluation result; the minimum value of the third quality evaluation result is 0 and the maximum value is 1.

[0119] Specifically, the third quality evaluation result may be C=(1-2*|p-1 / 2|), where C is the third quality evaluation result and p is the first ratio. In addition to the example here, the third quality evaluation result may also be calculated in other ways.

[0120] Optionally, a specific method of obtaining the heart sound signal quality assessment result based on the fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result may be: first, the absolute value of the difference between the first quality assessment result and the second quality assessment result is calculated. Then, the product of the absolute value and the third quality assessment result is calculated. Finally, the product is summed with the minimum value of the first quality assessment result and the second quality assessment result to obtain the heart sound signal quality assessment result.

[0121] In the case where the first quality assessment result and the second quality assessment result differ greatly, the probability that the result corresponding to the actual quality of the heart sound signal to be detected is between the first quality assessment result and the second quality assessment result is greater. Therefore, the absolute value of the difference between the first quality assessment result and the second quality assessment result is processed by the third quality assessment result, so as to evaluate the position of the result corresponding to the actual quality of the heart sound signal to be detected between the first quality assessment result and the second quality assessment result by the third quality assessment result. Since the third quality assessment result is also obtained based on the signal quality feature, the actual quality of the heart sound signal to be detected is further evaluated by the third quality assessment result, thereby making the heart sound signal quality assessment result more accurate.

[0122] For example, the heart sound signal quality assessment result can be: Score = min(A, B) + |AB|*C, where Score is the heart sound signal quality assessment result, A is the first quality assessment result, B is the second quality assessment result, C is the third quality assessment result, and min(A, B) means taking the minimum value of A and B.

[0123] Alternatively, the specific method of obtaining the heart sound signal quality assessment result based on the fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result may also be: first, the absolute value of the difference between the first quality assessment result and the second quality assessment result is calculated. Then, the product of the absolute value and the third quality assessment result is calculated. Finally, the maximum value of the first quality assessment result and the second quality assessment result is subtracted from the product to obtain the heart sound signal quality assessment result.

[0124] In one implementation, the third quality assessment result may be one of the dimensions used as the weighted sum. In this case, the numerical range of the third quality assessment result is the same as the numerical range of the first quality assessment result, that is, the maximum value of the third assessment result is the same as the maximum value of the first quality assessment result, and the minimum value of the third assessment result is the same as the minimum value of the first quality assessment result.

[0125] For example, the designated feature may be the ratio of the number of first heart sounds to the number of peaks with distance detection. In this case, based on the designated feature in the pre-designated signal quality feature, the specific method of determining the third quality assessment result may be: first determine the first ratio of the number of first heart sounds included in the heart sound signal to be detected to the number of peaks with distance detection. Then, the third quality assessment result is obtained based on the first ratio, wherein the closer the first ratio is to 0.5, the larger the third quality assessment result; the closer the first ratio is to 1 or 0, the smaller the third quality assessment result; the minimum value of the third quality assessment result is 0 and the maximum value is 100 (the value range of the first quality assessment result is set to 0 to 100).

[0126] Specifically, the third quality evaluation result may be C=100*(1-2*|p-1 / 2|), where C is the third quality evaluation result and p is the first ratio. In addition to the example here, the third quality evaluation result may also be calculated in other ways.

[0127] Under this implementation, the specific method of obtaining the heart sound signal quality assessment result based on the fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result can also be: calculating the average of the first quality assessment result, the second quality assessment result, and the third quality assessment result to obtain the heart sound signal quality assessment result.

[0128] In one implementation, the training method of the first quality assessment model and the second quality assessment model can be: first, a training data set is obtained, the training data set includes multiple groups of signal quality features, and each group of signal quality features is annotated with a standard heart sound signal quality score. Then, based on the training data set, the initial first quality assessment model and the initial second quality assessment model are trained respectively to obtain a trained first quality assessment model and a trained second quality assessment model.

[0129] Optionally, during training, multiple initial models may be prepared in advance, and after the multiple initial models are trained using the training data set, the confidence of each trained initial model is tested based on the test data set. The two models with the highest confidence are then used as the first quality assessment model and the second quality assessment model, respectively.

[0130] In one implementation, a specific method of obtaining a training data set may be to use a special heart sound signal acquisition device to collect the user's heart sound signal. In this process, in order to obtain a heart sound signal that can be better used to evaluate the signal quality, signals of a predetermined length (such as 45 seconds, etc.) may be collected from different groups of people at different time periods. Then, the positions of the first heart sound and the second heart sound in each segment of the heart sound signal are determined, and then the signal quality features are extracted from the heart sound signal to be detected based on the positions of the first heart sound and the second heart sound in the heart sound signal to be detected. At the same time, a quality score is given to each segment of the heart sound signal. Among them, each segment of the heart sound signal includes at least one complete cardiac cycle.

[0131] If the collected heart sound signal is long, a fixed-length window may be used to cut the heart sound signal, and feature extraction and scoring may be performed on each segment of the cut heart sound signal based on the aforementioned method to obtain a piece of training data.

[0132] The basis for scoring the quality of each heart sound signal is to mark the waveform as a score within a preset segmentation interval (eg, 0 to 100) according to the degree of waveform deformation, noise level, and whether the first heart sound and the second heart sound are clearly visible.

[0133] The specific rules are as follows: low-score segments are of poor quality, no obvious first heart sound can be observed, the overall waveform is messy, and it looks like a noise signal overall, and no useful information can be obtained. Compared with signals of poor quality, if the first heart sound can be seen but the second heart sound cannot be seen clearly, a relatively higher score will be evaluated, and there may be baseline drift and serious noise. If the first and second heart sounds exist, but the noise in the interval segment of the first heart sound is close to the peak position of the heart sound, a relatively higher score can be evaluated. If the first and second heart sounds of the signal can be seen, the outline is obvious, and the noise is far from the peak position of the heart sound, a higher score will be given. When the waveform of the heart sound signal is complete, without serious deformation, the first and second heart sounds can be clearly observed, and the noise is very small, it can be evaluated as a higher score (not exceeding the maximum value of the preset segment interval).

[0134] In one implementation manner, in addition to the first quality assessment model and the second quality assessment model described above, in actual use, more quality assessment models may be used to process signal quality features to obtain multiple quality assessment results.

[0135] Correspondingly, the specific method of obtaining the heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result can also be: obtaining the heart sound signal quality assessment result based on the quality assessment results of all model outputs (including the first quality assessment result and the second quality assessment result).

[0136] Based on the quality evaluation results output by all models, the heart sound signal quality evaluation result can be obtained by performing weighted average processing on all quality evaluation results to obtain the heart sound signal quality evaluation result.

[0137] Among them, the weight value corresponding to each model can be determined according to the confidence of each model. The higher the confidence (the higher the accuracy of the model), the greater the weight value corresponding to the model.

[0138] Alternatively, the maximum value and the minimum value in all the quality evaluation results may be removed, and then weighted average processing may be performed to obtain the heart sound signal quality evaluation result.

[0139] Optionally, the method for obtaining the heart sound signal quality assessment result based on the quality assessment results output by all models can also be: selecting two quality assessment results that are closest to the median (or average) of the quality assessment results from all the quality assessment results, and then obtaining the final heart sound signal quality assessment result based on the two quality assessment results.

[0140] Among them, the method of obtaining the final heart sound signal quality assessment result based on the two quality assessment results is the same as the specific method of obtaining the heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result. For the sake of brief description, it will not be repeated here.

[0141] Based on the same technical concept, the present application also provides a quality assessment model, which includes a first convolutional layer, a first batch of normalization layers, a first activation function layer, a first maximum pooling layer, multiple depth-separable convolution blocks connected in sequence, a global average pooling layer, and a fully connected layer; wherein the depth-separable convolution block is used to extract local features of the input data of each channel of itself, and fuse the extracted local features.

[0142] Among them, the specific structure and implementation method of the quality assessment model have been clearly described in the previous article. For the sake of brief description, they will not be repeated here.

[0143] Based on the same technical concept, the present application also provides a dual-model fusion heart sound signal quality assessment device, such as Figure 4 As shown, the dual-model fusion heart sound signal quality assessment device 100 includes an acquisition module 110 and a processing module 120 .

[0144] The acquisition module 110 is used to acquire the heart sound signal to be detected and the positions of the first heart sound and the second heart sound in the heart sound signal to be detected.

[0145] The processing module 120 is used to extract signal quality features from the heart sound signal to be detected based on the positions of the first heart sound and the second heart sound in the heart sound signal to be detected; obtain a first quality assessment result based on the heart sound signal to be detected and a preset first quality assessment model; use the signal quality features as input data of a preset second quality assessment model to obtain a second quality assessment result; and obtain a heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result.

[0146] The processing module 120 is specifically used to determine whether the deviation between the first quality assessment result and the second quality assessment result is within a preset allowable deviation range; if the deviation between the first quality assessment result and the second quality assessment result is within the allowable deviation range, a heart sound signal quality assessment result is obtained based on the fusion of the first quality assessment result and the second quality assessment result; if the deviation between the first quality assessment result and the second quality assessment result is not within the allowable deviation range, a third quality assessment result is determined based on a specified feature in the pre-specified signal quality features; and a heart sound signal quality assessment result is obtained based on the fusion of the first quality assessment result, the second quality assessment result and the third quality assessment result.

[0147] Processing module 120 is specifically used to calculate the product of a first confidence level corresponding to the first quality assessment model and the first quality assessment result to obtain a first intermediate result; calculate the product of a second confidence level corresponding to the second quality assessment model and the second quality assessment result to obtain a second intermediate result; wherein the sum of the first confidence level and the second confidence level is 1, and the first confidence level and the second confidence level are both positive numbers greater than 0; and use the sum of the first intermediate result and the second intermediate result as the heart sound signal quality assessment result.

[0148] In one implementation, the closer the value of the designated feature is to the standard value corresponding to the designated feature, the greater the third quality assessment result is.

[0149] The designated feature is the ratio of the number of first heart sounds to the number of peaks detected with distance, and the processing module 120 is specifically used to determine a first ratio of the number of first heart sounds included in the heart sound signal to be detected to the number of peaks detected with distance; based on the first ratio, the third quality assessment result is obtained, wherein the closer the first ratio is to 0.5, the larger the third quality assessment result; the closer the first ratio is to 1 or 0, the smaller the third quality assessment result; the minimum value of the third quality assessment result is 0 and the maximum value is 1.

[0150] The processing module 120 is specifically used to calculate the absolute value of the difference between the first quality assessment result and the second quality assessment result; calculate the product of the absolute value and the third quality assessment result; sum the product with the minimum value of the first quality assessment result and the second quality assessment result to obtain the heart sound signal quality assessment result.

[0151] In one implementation, the first quality assessment model includes a first convolutional layer, a first batch of normalization layers, a first activation function layer, a first maximum pooling layer, a plurality of sequentially connected depthwise separable convolutional blocks, a global average pooling layer, and a fully connected layer; wherein the depthwise separable convolutional block is used to extract local features of the input data of each channel of itself, and fuse the extracted local features.

[0152] In one implementation manner, each of the depthwise separable convolution blocks includes a first branch, a second branch, an adder and a second activation function layer, the adder is connected to the output end of the first branch and the output end of the second branch, respectively, the input ends of the first branch and the second branch are both connected to the first maximum pooling layer, the output ends of the first branch and the second branch are both connected to the adder, and the output end of the adder is connected to the input end of the second activation function layer; wherein the first branch is used to extract local features of the input data of each channel of itself and fuse the extracted local features; the second branch is used to retain the input of the depthwise separable convolution block to alleviate the degradation of the first quality assessment model.

[0153] In one implementation, the first branch includes multiple convolution blocks, each of which includes a depth convolution layer, a second batch normalization layer, a point-by-point convolution layer, a third batch normalization layer, and a third activation function layer connected in sequence.

[0154] In one implementation, the second branch includes a second convolutional layer and a fourth batch normalization layer connected in sequence.

[0155] In one embodiment, the signal quality characteristics include at least one of the interval between the first heart sound and the second heart sound, the number of the first heart sound and the second heart sound, the duration of the first heart sound and the second heart sound, the number of peaks with distance detection, the ratio of the number of first heart sounds to the number of peaks with distance detection, spectral power, the number of sampling points exceeding a preset threshold, the average value of peak intervals, signal length, and the average value of the peak intervals of the first heart sound.

[0156] The dual-model fusion heart sound signal quality assessment device 100 provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned dual-model fusion heart sound signal quality assessment method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned dual-model fusion heart sound signal quality assessment method embodiment.

[0157] See also Figure 5 , which is an electronic device 200 provided in an embodiment of the present application. The electronic device 200 includes: a processor 210 and a memory 220.

[0158] The memory 220 and the processor 210 are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The memory 220 is used to store computer programs, such as storing Figure 4The software function module shown in the figure is the heart sound signal quality assessment device 100 with dual model fusion. The heart sound signal quality assessment device 100 with dual model fusion includes at least one software function module that can be stored in the memory 220 in the form of software or firmware or fixed in the operating system (OS) of the electronic device 200. The processor 210 is used to execute the executable module stored in the memory 220, such as the software function module or computer program included in the heart sound signal quality assessment device 100 with dual model fusion. At this time, the processor 210 is used to obtain the heart sound signal to be detected and the positions of the first heart sound and the second heart sound in the heart sound signal to be detected; extract the signal quality feature from the heart sound signal to be detected based on the positions of the first heart sound and the second heart sound in the heart sound signal to be detected; obtain a first quality assessment result based on the heart sound signal to be detected and a preset first quality assessment model; use the signal quality feature as input data of a preset second quality assessment model to obtain a second quality assessment result; wherein the structures of the first quality assessment model and the second quality assessment model are different; and obtain a heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result.

[0159] Among them, the memory 220 can be, but is not limited to, RAM (Random Access Memory), ROM (Read Only Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0160] The processor 210 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it may also be a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. A general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0161] The electronic device 200 mentioned above includes but is not limited to a personal computer, a server, etc.

[0162] The embodiment of the present application also provides a computer-readable storage medium (hereinafter referred to as storage medium), on which a computer program is stored. When the computer program is run by a computer such as the electronic device 200 described above, the heart sound signal quality assessment method of dual-model fusion shown above is executed. The computer-readable storage medium includes: a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0163] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A dual-model fusion heart sound signal quality assessment method, characterized in that: include: Acquire a heart sound signal to be detected and positions of a first heart sound and a second heart sound in the heart sound signal to be detected; extracting a signal quality feature from the heart sound signal to be detected based on positions of the first heart sound and the second heart sound in the heart sound signal to be detected; Obtaining a first quality assessment result based on the heart sound signal to be detected and a preset first quality assessment model; Using the signal quality feature as input data of a preset second quality assessment model to obtain a second quality assessment result; wherein the structures of the first quality assessment model and the second quality assessment model are different; A heart sound signal quality assessment result is obtained based on the first quality assessment result and the second quality assessment result.

2. The method according to claim 1, characterized in that Obtaining a heart sound signal quality assessment result based on the first quality assessment result and the second quality assessment result, including: Determining whether a deviation between the first quality assessment result and the second quality assessment result is within a preset deviation allowable range; If the deviation between the first quality assessment result and the second quality assessment result is within the allowable deviation range, obtaining a heart sound signal quality assessment result based on the fusion of the first quality assessment result and the second quality assessment result; If the deviation between the first quality assessment result and the second quality assessment result is within the allowable deviation range, a third quality assessment result is determined based on a specified feature among the pre-specified signal quality features; and a heart sound signal quality assessment result is obtained by fusing the first quality assessment result, the second quality assessment result, and the third quality assessment result.

3. The method according to claim 2, characterized in that The heart sound signal quality assessment result is obtained based on the fusion of the first quality assessment result and the second quality assessment result, including: Calculating the product of a first confidence level corresponding to the first quality assessment model and the first quality assessment result to obtain a first intermediate result; Calculate the product of a second confidence level corresponding to the second quality assessment model and the second quality assessment result to obtain a second intermediate result; wherein the sum of the first confidence level and the second confidence level is 1, and the first confidence level and the second confidence level are both positive numbers greater than 0; The sum of the first intermediate result and the second intermediate result is used as the heart sound signal quality assessment result.

4. The method according to claim 2, characterized in that: The value of the third quality assessment result is determined according to the standard value corresponding to the designated feature and the value of the designated feature.

5. The method according to claim 2, characterized in that: The specified feature is a ratio of the number of first heart sounds to the number of peaks with distance detection, and based on the specified feature in the pre-specified signal quality features, determining a third quality assessment result includes: Determine a first ratio of the number of first heart sounds included in the heart sound signal to be detected to the number of peaks with distance detection; The third quality assessment result is obtained based on the first ratio, wherein the third quality assessment result has a minimum of 0 and a maximum of 1.

6. The method according to claim 5, characterized in that The obtaining of the heart sound signal quality assessment result based on the fusion of the first quality assessment result, the second quality assessment result, and the third quality assessment result includes: Calculating the absolute value of the difference between the first quality assessment result and the second quality assessment result; Calculating a product of the absolute value and the third quality assessment result; The product is summed with a minimum value of the first quality assessment result and the second quality assessment result to obtain the heart sound signal quality assessment result.

7. The method according to claim 1, characterized in that The first quality assessment model includes a first convolution layer, a first batch of normalization layers, a first activation function layer, a first maximum pooling layer, multiple depth-separable convolution blocks, a global average pooling layer, and a fully connected layer; wherein the depth-separable convolution block is used to extract local features of the input data of each channel of itself, and fuse the extracted local features.

8. The method according to claim 7, characterized in that Each of the depth-separable convolution blocks includes a first branch, a second branch, an adder, and a second activation function layer, the adder is connected to the output end of the first branch and the output end of the second branch respectively, the input ends of the first branch and the second branch are both connected to the first maximum pooling layer, the output ends of the first branch and the second branch are both connected to the adder, and the output end of the adder is connected to the input end of the second activation function layer; wherein the first branch is used to extract local features of the input data of each channel of itself and fuse the extracted local features; the second branch is used to retain the input of the depth-separable convolution block to alleviate the degradation of the first quality assessment model.

9. The method according to claim 8, characterized in that The first branch includes a plurality of convolution blocks, each of which includes a depthwise convolution layer, a second batch normalization layer, a point-by-point convolution layer, a third batch normalization layer, and a third activation function layer.

10. The method according to claim 8, characterized in that The second branch includes a second convolutional layer and a fourth batch normalization layer which are connected in sequence.

11. The method according to any one of claims 1 to 10, characterized in that: The signal quality characteristics include at least one of the interval between the first heart sound and the second heart sound, the number of the first heart sound and the second heart sound, the duration of the first heart sound and the second heart sound, the number of peaks with distance detection, the ratio of the number of first heart sounds to the number of peaks with distance detection, spectral power, the number of sampling points exceeding a preset threshold, the average value of peak intervals, signal length, and the average value of the peak intervals of the first heart sound.

12. A dual-model fusion heart sound signal quality assessment device, characterized in that: include: An acquisition module, used for acquiring a heart sound signal to be detected and positions of a first heart sound and a second heart sound in the heart sound signal to be detected; a processing module, configured to extract a signal quality feature from the heart sound signal to be detected based on positions of the first heart sound and the second heart sound in the heart sound signal to be detected; Based on the heart sound signal to be detected and a preset first quality assessment model, a first quality assessment result is obtained; the signal quality feature is used as input data of a preset second quality assessment model to obtain a second quality assessment result; based on the first quality assessment result and the second quality assessment result, a heart sound signal quality assessment result is obtained.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are connected; The memory is used to store programs; The processor is used to call the program stored in the memory to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 11 is executed.

Images