A heart sound signal processing method, device, electronic device and storage medium

Abstract

The present invention relates to the field of signal processing technology, and in particular to a method, device, electronic device and storage medium for processing heart sound signals, the method comprising: determining the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability of each preset heart sound stage under the target duration; segmenting the heart sound signal to be processed according to the preset heart sound cycle and the target heart sound stage of each heart sound signal time segment to obtain heart sound signal period segments; inputting the heart sound feature image corresponding to the heart sound signal period segment into a heart sound type recognition model to obtain the target heart sound type corresponding to the heart sound signal period segment; the target heart sound type is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image. The present application can increase the interpretability of the processing result of the heart sound signal and improve the processing accuracy.

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to a method, device, electronic equipment and storage medium for processing heart sound signals. Background Art

[0002] Heart sound signals refer to the sounds produced by the heart during its beating. Heart sound signal processing technology based on machine learning and deep learning has developed rapidly.

[0003] However, the decision-making process of the heart sound signal processing method based on machine learning and deep learning is often "black box", and it is difficult to explain why the final processing result of the heart sound signal is obtained. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and storage medium for processing heart sound signals, which can increase the interpretability of the processing results of heart sound signals and improve the processing accuracy.

[0005] In a first aspect, an embodiment of the present application provides a method for processing a heart sound signal, the method comprising:

[0006] Obtaining a heart sound signal time segment at each moment in the heart sound signal to be processed, and determining a first probability that each heart sound signal time segment is in each preset heart sound stage;

[0007] Determine the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability that each preset heart sound stage is at each target duration;

[0008] According to a preset heart sound cycle and a target heart sound stage where each heart sound signal time segment is located, the heart sound signal to be processed is segmented to obtain heart sound signal cycle segments;

[0009] For any of the heart sound signal periodic segments, the heart sound feature image corresponding to the heart sound signal periodic segment is input into a pre-trained heart sound type recognition model to obtain a target heart sound type corresponding to the heart sound signal periodic segment; the heart sound feature image is used to display the nonlinear phase coupling strength of each audio component group of the heart sound signal periodic segment through color; the target heart sound type corresponding to the heart sound signal periodic segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image.

[0010] In a possible implementation, determining the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability that each preset heart sound stage is at each target duration includes:

[0011] For any of the heart sound signal time segments, the first probability that the heart sound signal time segment is in each preset heart sound stage is corrected according to the second probability that each preset heart sound stage is in each target duration, so as to obtain a third probability that the heart sound signal time segment is in each preset heart sound stage;

[0012] According to the third probability that all heart sound signal time segments are in each preset heart sound stage, the target heart sound stage in which each heart sound signal time segment is located is determined.

[0013] In a possible implementation, the first probability that the heart sound signal time segment is in each preset heart sound stage is corrected according to the second probability of each preset heart sound stage at each target duration to obtain a third probability that the heart sound signal time segment is in each preset heart sound stage, including:

[0014] Substitute the second probability of each preset heart sound stage at each target duration and the first probability of the heart sound signal time segment being in each preset heart sound stage into the following formula to obtain the third probability of the heart sound signal time segment being in each preset heart sound stage:

[0015] ;

[0016] in, is the first probability that the heart sound signal time segment at time t is in the i-th preset heart sound stage, is the i-th preset heart sound stage, is the second probability of the i-th preset heart sound stage under the j-th target duration, is the probability that the heart sound signal time segment at the preset initial moment is in the i-th preset heart sound stage.

[0017] In a possible implementation, the target duration is determined by the following steps:

[0018] Obtaining the historical duration of the historical heart sound signal in each preset heart sound stage;

[0019] For each preset heart sound stage, calculate the standard deviation and average of all historical durations of the preset heart sound stage; determine the duration range of the preset heart sound stage based on the standard deviation and average; and determine the target duration of the preset heart sound stage from the duration range.

[0020] In a possible implementation, the second probability of any preset heart sound stage at any target duration is calculated by the following formula:

[0021] ;

[0022] in, is the second probability of the preset heart sound stage under the target duration, is the average of all historical durations of the preset heart sound stage, is the variance of all historical durations of the preset heart sound stage, is the standard deviation of all historical durations of the preset heart sound stage, Duration of the stated goal.

[0023] In a possible implementation, determining the target heart sound type corresponding to the heart sound signal periodic segment and the target local image block in the heart sound characteristic image through the following steps includes:

[0024] Segmenting the heart sound characteristic image according to a preset image size to obtain a local image block;

[0025] For each local image block, generating an embedding vector corresponding to the local image block by using the local image block and the corresponding position information in the heart sound feature image;

[0026] Inputting the embedding vectors corresponding to all local image blocks into the encoder of the heart sound type recognition model to determine the image features of the heart sound feature image and the self-attention weights corresponding to each local image block by using a multi-head self-attention mechanism;

[0027] A local image block whose self-attention weight is greater than a preset weight is determined as a target local image block, and the image features of the heart sound feature image are input into a decoder of the heart sound type recognition model to obtain a target heart sound type corresponding to the periodic segment of the heart sound signal.

[0028] In a possible implementation, the method further includes:

[0029] For any periodic segment of a heart sound signal, the similarity between the heart sound characteristic image corresponding to the periodic segment of the heart sound signal and a preset heart sound characteristic image corresponding to the target heart sound type in a pre-established image library is determined as a degree parameter of the heart sound type corresponding to the periodic segment of the heart sound signal being the target heart sound type.

[0030] In a second aspect, an embodiment of the present application further provides a device for processing a heart sound signal, the device for processing a heart sound signal comprising:

[0031] An acquisition module, used for acquiring a heart sound signal time segment at each moment in the heart sound signal to be processed, and determining a first probability that each heart sound signal time segment is in each preset heart sound stage;

[0032] a determination module, configured to determine a target heart sound stage at which each heart sound signal time segment is located based on a first probability that each heart sound signal time segment is at each preset heart sound stage and a second probability that each preset heart sound stage is at each target duration;

[0033] A segmentation module, configured to segment the heart sound signal to be processed according to a preset heart sound cycle and a target heart sound stage at which each heart sound signal time segment is located, to obtain heart sound signal cycle segments;

[0034] An input module is used to input the heart sound feature image corresponding to any of the heart sound signal periodic segments into a pre-trained heart sound type recognition model to obtain a target heart sound type corresponding to the heart sound signal periodic segment; the heart sound feature image is used to display the nonlinear phase coupling strength of each audio component group of the heart sound signal periodic segment through color; the target heart sound type corresponding to the heart sound signal periodic segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image.

[0035] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for processing cardiac sound signals as described in any one of the first aspects.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for processing heart sound signals as described in any one of the first aspects are executed.

[0037] The embodiment of the present application provides a method, device, electronic device and storage medium for processing heart sound signals, the method comprising: determining the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability of each preset heart sound stage under each target duration; segmenting the heart sound signal to be processed according to the preset heart sound cycle and the target heart sound stage of each heart sound signal time segment to obtain the heart sound signal period segment; inputting the heart sound feature image corresponding to the heart sound signal period segment into a pre-trained heart sound type recognition model to obtain the target heart sound type corresponding to the heart sound signal period segment; the target heart sound type corresponding to the heart sound signal period segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image. The present application can increase the interpretability of the processing result of the heart sound signal and improve the processing accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flowchart of a method for processing a heart sound signal provided in an embodiment of the present application is shown;

[0040] Figure 2 A schematic diagram of a process for determining a target heart sound stage of a heart sound signal time segment provided in an embodiment of the present application is shown;

[0041] Figure 3 A schematic diagram showing the structure of a heart sound signal processing device provided in an embodiment of the present application is shown;

[0042] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0044] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0045] In order to enable those skilled in the art to use the contents of this application, the following implementation is provided in conjunction with a specific application scenario, "the field of signal processing technology". For those skilled in the art, the general principles defined herein may be applied to other embodiments and application scenarios without departing from the spirit and scope of this application. Although this application is mainly described around the "field of signal processing technology", it should be understood that this is only an exemplary embodiment.

[0046] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0047] A method for processing heart sound signals provided in an embodiment of the present application is described in detail below.

[0048] Reference Figure 1 As shown, it is a flowchart of a method for processing a heart sound signal provided in an embodiment of the present application. The exemplary steps of the embodiment of the present application are described below:

[0049] S101, obtaining a heart sound signal time segment at each moment in a heart sound signal to be processed, and determining a first probability that each heart sound signal time segment is in each preset heart sound stage.

[0050] In the implementation mode of the present application, a second-order Butterworth filter is used to remove noise below 25 Hz and above 400 Hz in the heart sound signal to be processed to eliminate environmental and high-frequency interference; according to the Nyquist sampling theory, the heart sound signal to be processed after the noise is removed is resampled to 1000 Hz to reduce data redundancy and retain key features; the amplitude of the resampled heart sound signal to be processed is normalized so that the normalized eigenvalue is within the range of [-1, 1] to enhance the comparability between different signals; the normalized heart sound signal to be processed is decomposed into multiple preset scales using wavelet transform to further remove random noise and high-frequency interference, while retaining the effective physiological characteristics of the heart sound, providing a clearer signal basis for subsequent analysis; the decomposed heart sound signal to be processed is determined as the final heart sound signal to be processed. Specifically, the heart sound signal to be processed is substituted into the preset logistic regression model to obtain the first probability that each heart sound signal time segment is in each preset heart sound stage. The main purpose of the logistic regression model is to predict the probability of an event occurring, that is, it is a probability prediction model that uses the Sigmoid function (or logistic function) to convert the output of the linear regression model into a probability value (that is, the first probability that each heart sound signal time segment is in each preset heart sound stage).

[0051] Among them, there are four preset heart sound stages in a preset heart sound cycle, namely S1 (first heart sound), systole, S2 (second heart sound) and diastole. S1: marks the beginning of the heart's systole, usually caused by the closure of the ventricles, that is, the closure of the mitral valve and tricuspid valve. Systole: The ventricles contract and pump blood into the arteries. S2: marks the beginning of the heart's diastole, usually caused by the closure of the aortic valve and pulmonary valve. Diastole: The ventricles relax, the atria contract, and blood flows back to the ventricles.

[0052] Example 1: The heart sound signal to be processed is a two-minute signal, and one minute is regarded as a moment. Therefore, the heart sound signal to be processed includes a heart sound signal time segment at the first moment (0 to 1 minute) and a heart sound signal time segment at the second moment (1 to 2 minutes). Determine the first probability that the heart sound signal time segment at the first moment is in S1, the first probability that the heart sound signal time segment at the first moment is in the systolic period, the first probability that the heart sound signal time segment at the first moment is in S2, the first probability that the heart sound signal time segment at the first moment is in the diastolic period, the first probability that the heart sound signal time segment at the second moment is in S1, the first probability that the heart sound signal time segment at the second moment is in the systolic period, the first probability that the heart sound signal time segment at the second moment is in S2, and the first probability that the heart sound signal time segment at the second moment is in the diastolic period.

[0053] S102: Determine a target heart sound stage for each heart sound signal time segment based on a first probability that each heart sound signal time segment is in each preset heart sound stage and a second probability that each preset heart sound stage is at each target duration.

[0054] Reference Figure 2 As shown, it is a schematic diagram of a process of determining a target heart sound stage of a heart sound signal time segment provided in an embodiment of the present application:

[0055] S201. For any heart sound signal time segment, the first probability that the heart sound signal time segment is in each preset heart sound stage is corrected according to the second probability that the heart sound signal time segment is in each preset heart sound stage at each target duration, so as to obtain a third probability that the heart sound signal time segment is in each preset heart sound stage.

[0056] In the implementation manner of the present application, different preset heart sound stages may correspond to different target durations; the target duration of each preset heart sound stage is determined based on the standard deviation of all historical durations of each preset heart sound stage. The number of third probabilities that the heart sound signal time segment is in each preset heart sound stage is the same as the number of target durations of each preset heart sound stage.

[0057] For example, assuming that each preset heart sound stage has two target durations, then for the heart sound signal time segment at the first moment, the first probability that the heart sound signal time segment at the first moment is in S1 is corrected according to the second probability of S1 at the first target duration; the first probability that the heart sound signal time segment at the first moment is in S1 is corrected according to the second probability of S1 at the second target duration; the first probability that the heart sound signal time segment at the first moment is in the systole is corrected according to the second probability of the systole at the first target duration; the heart sound signal at the first moment is corrected according to the second probability of the systole at the second target duration The first probability that the signal time segment is in the systolic period is corrected; the first probability that the heart sound signal time segment at the first moment is in S2 is corrected according to the second probability of S2 at the first target duration; the first probability that the heart sound signal time segment at the first moment is in S2 is corrected according to the second probability of S2 at the second target duration; the first probability that the heart sound signal time segment at the first moment is in diastole is corrected according to the second probability of diastole at the first target duration; the first probability that the heart sound signal time segment at the first moment is in diastole is corrected according to the second probability of diastole at the second target duration. The correction process for the heart sound signal time segment at the second moment is the same as that for the heart sound signal time segment at the first moment.

[0058] Specifically, the target duration is determined by the following steps: obtaining the historical duration of the historical heart sound signal in each preset heart sound stage; for each preset heart sound stage, calculating the standard deviation and the average value of all historical durations of the preset heart sound stage; determining the duration range of the preset heart sound stage according to the standard deviation and the average value; and determining the target duration of the preset heart sound stage from the duration range.

[0059] In the implementation mode of the present application, for each preset heart sound stage, the standard deviation of all historical durations of the preset heart sound stage is multiplied by a preset multiple, and the value is added to the average value to obtain the maximum value in the duration range of the preset heart sound stage; the average value is subtracted from the value of the standard deviation multiplied by a preset multiple (such as 3 times) to obtain the minimum value in the duration range of the preset heart sound stage. A preset number of target durations can be determined from the duration range, and the longest preset number of target durations can also be determined from the duration range; the target duration can be within the duration range.

[0060] Among them, the historical heart sound signals include heart sound signals of multiple heart positions such as the aortic valve, mitral valve, pulmonary valve and tricuspid valve.

[0061] In this way, it can be ensured that the duration of each preset heart sound stage conforms to physiological laws and is reasonably limited.

[0062] Specifically, the second probability of any preset heart sound stage at any target duration is calculated by the following formula:

[0063] ;

[0064] in, is the second probability of the preset heart sound stage under the target duration, is the average of all historical durations of the preset heart sound stage, is the variance of all historical durations of the preset heart sound stage, is the standard deviation of all historical durations of the preset heart sound stage, Duration for this goal.

[0065] Specifically, the second probability of each preset heart sound stage at each target duration and the first probability that the heart sound signal time segment is in each preset heart sound stage are substituted into the following formula to obtain the third probability that the heart sound signal time segment is in each preset heart sound stage:

[0066] ;

[0067] in, is the first probability that the heart sound signal time segment at time t is in the i-th preset heart sound stage, is the i-th preset heart sound stage, is the second probability of the i-th preset heart sound stage under the j-th target duration, is the probability that the heart sound signal time segment at the preset initial moment is in the i-th preset heart sound stage.

[0068] S202: Determine a target heart sound stage in which each heart sound signal time segment is located according to a third probability that all heart sound signal time segments are in each preset heart sound stage.

[0069] In the implementation manner of the present application, the target heart sound stage is a preset heart sound stage in a preset heart sound cycle.

[0070] Step 1: Calculate the fourth probability that the time segment of the heart sound signal at the initial moment is in each preset heart sound stage based on the third probability that the time segment of the heart sound signal at the initial moment is in each preset heart sound stage and the preset probability that the time segment of the heart sound signal at the initial moment is in each preset heart sound stage.

[0071] In the implementation manner of the present application, the fourth probability that the heart sound signal time segment at the initial moment is in each preset heart sound stage is calculated according to the following formula:

[0072] ;

[0073] in, For the initial moment The heart sound signal time segment is in the i-th preset heart sound phase The fourth probability, is the probability that the heart sound signal time segment at the preset initial moment is in the i-th preset heart sound stage, For the initial moment The heart sound signal time segment is in the i-th preset heart sound phase The third probability.

[0074] Step 2: For the heart sound signal time segment at any time t except the heart sound signal time segment at the initial time, according to the fourth probability that the heart sound signal time segment at time t-1 is in each preset heart sound stage and the third probability that the heart sound signal time segment is in each preset heart sound stage, determine the fourth probability that the heart sound signal time segment at time t is in each preset heart sound stage, and the initial heart sound stage of the heart sound signal time segment at time t-1 when the heart sound signal time segment at time t is in each preset heart sound stage. The initial heart sound stage is a preset heart sound stage in the preset heart sound cycle.

[0075] In the implementation manner of the present application, the fourth probability that the heart sound signal time segment at time t is in each preset heart sound stage, and the initial heart sound stage of the heart sound signal time segment at time t-1 when the heart sound signal time segment at time t is in each preset heart sound stage are determined by the following formula:

[0076] ;

[0077] in, for The heart sound signal time segment at the moment is in the jth preset heart sound stage The fourth probability, for The heart sound signal time segment at time is in the i-th preset heart sound stage The fourth probability, is the preset heart sound phase from the i-th preset Transfer to the jth preset heart sound stage The transition probability, The time segment of the heart sound signal at time t is in the jth preset heart sound stage The third probability of Determined as The heart sound signal time segment at the moment is in the jth preset heart sound stage The initial heart sound stage of the heart sound signal time segment at time t-1.

[0078] Step three, among all preset heart sound stages, the preset heart sound stage with the fourth largest probability when the heart sound signal time segment at the last moment is in the last moment is determined as the target heart sound stage for the heart sound signal time segment at the last moment; for any heart sound signal time segment except the heart sound signal time segment at the last moment, when the heart sound signal time segment at time t+1 is in the target heart sound stage, the initial heart sound stage of the heart sound signal time segment at time t is determined as the target heart sound stage for the heart sound signal time segment at time t.

[0079] S103 . Segment the heart sound signal to be processed according to the preset heart sound cycle and the target heart sound stage where each heart sound signal time segment is located to obtain heart sound signal cycle segments.

[0080] In the implementation manner of the present application, the heart sound signals in each heart sound signal cycle segment are signals within the same preset heart sound cycle.

[0081] S104. For any periodic segment of the heart sound signal, input the heart sound feature image corresponding to the periodic segment of the heart sound signal into a pre-trained heart sound type recognition model to obtain a target heart sound type corresponding to the periodic segment of the heart sound signal.

[0082] In the implementation mode of the present application, the heart sound characteristic image is used to display the nonlinear phase coupling strength of each audio component group of the heart sound signal periodic segment by color; the target heart sound type corresponding to the heart sound signal periodic segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound characteristic image. For each heart sound signal periodic segment, there is a corresponding target heart sound type. In this way, the heart sound type (such as abnormal, normal, etc.) of each time period in the heart sound signal to be processed can be determined. The nonlinear phase coupling strength of the audio component group in the target local image block is also given, which improves the credibility of the processing results and improves the interpretability.

[0083] Specifically, the heart sound characteristic image corresponding to the heart sound signal period segment is determined by the following steps:

[0084] Step 1: Substitute the complex amplitude of the heart sound signal periodic segment on each audio component into the following formula to obtain the heart sound signal periodic feature:

[0085] ;

[0086] in, is the i-th audio component in the periodic characteristics of the heart sound signal With the jth audio component The nonlinear phase coupling strength between is the heart sound signal periodic segment in the i-th audio component The complex magnitude (i.e., Fourier coefficient) of is the jth audio component of the heart sound signal periodic segment The complex magnitude (i.e., Fourier coefficient) of The heart sound signal periodic segment in the audio component + The complex conjugate of the complex magnitude on , For all time windows The expected value of (used to eliminate the influence of random noise on the calculation of nonlinear phase coupling strength).

[0087] Step 2: Normalize the characteristic value of each characteristic point in the heart sound signal periodic feature to a preset color value range (i.e., 0 to 255) using the following formula:

[0088] ;

[0089] in, is the normalized eigenvalue of the ith feature point in the periodic feature of the heart sound signal, is the periodic characteristic of the heart sound signal The eigenvalue of the i-th feature point before normalization, is the periodic characteristic of the heart sound signal The largest eigenvalue in is the periodic characteristic of the heart sound signal The smallest eigenvalue in .

[0090] Here, the larger the eigenvalue of the feature point after normalization, the higher the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the feature point. Therefore, through this step, the color type and color depth of the feature point in the heart sound feature image can be more intuitively represented by the corresponding eigenvalue, which facilitates the subsequent mapping of the eigenvalue of the feature point to the color value and reduces the complexity of the subsequent mapping.

[0091] Step 3: Map the normalized feature values ​​of each feature point in the periodic feature of the heart sound signal to the color values ​​of the three channels to obtain a heart sound feature image.

[0092] In an embodiment of the present application, for each feature point in the periodic feature of the heart sound signal, if the normalized feature value of the feature point in the periodic feature of the heart sound signal is the maximum value in the color value range (i.e., 255), it means that the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the feature point is the highest, then the normalized feature value of the feature point is determined as the color value of the first color channel (e.g., red channel) corresponding to the feature point; the value obtained by subtracting the normalized feature value of the feature point from the maximum value in the color value range is determined as the color value of the second color channel (e.g., blue channel) corresponding to the feature point; and the color value of the third color channel (e.g., green channel) corresponding to the feature point is determined as 0.

[0093] Taking the first channel as the red channel, the second channel as the blue channel, and the third channel as the green channel as an example, the redder the pixel color, the higher the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel; the bluer the pixel color, the lower the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel; each depth of each color represents a nonlinear phase coupling intensity, and the distribution area of ​​the audio component group corresponding to each nonlinear phase coupling intensity can be intuitively reflected from the heart sound feature image; and the heart sound feature image can also intuitively reflect that the nonlinear phase coupling intensity of the audio component group distributed in the lower left area is higher than the nonlinear phase coupling intensity of the audio component groups in other areas.

[0094] Here, through the above steps, the periodic characteristics of the heart sound signal can be visualized through a heat map (heart sound characteristic image). In the heart sound characteristic image, the larger the color value of the first color channel, the higher the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel point; the larger the color value of the second color channel, the lower the nonlinear phase coupling intensity between the audio components in the audio component group corresponding to the pixel point.

[0095] Specifically, the target heart sound type corresponding to the heart sound signal periodic segment and the target local image block in the heart sound feature image are determined by the following steps, including:

[0096] Step 1: segment the heart sound feature image according to a preset image size to obtain a local image block.

[0097] Step 2: For each local image block, the local image block and the corresponding position information in the heart sound feature image are used to generate an embedding vector corresponding to the local image block.

[0098] In an implementation manner of the present application, an embedding vector corresponding to a local image block is generated by a linear transformation; an encoding of the local image block and the corresponding position information in the heart sound characteristic image is added to the embedding vector to retain the spatial position information of the heart sound characteristic image and ensure that the model can recognize the structural association between the local image blocks in the heart sound characteristic image.

[0099] Step 3: Input the embedding vectors corresponding to all local image blocks into the encoder of the heart sound type recognition model to determine the image features of the heart sound feature image and the self-attention weights corresponding to each local image block using a multi-head self-attention mechanism.

[0100] In the embodiment of the present application, the heart sound type recognition model is constructed using the deep learning model architecture Transformer. The multi-head self-attention mechanism in the encoder can determine the self-attention weight corresponding to each local image block, and capture the image features of the heart sound feature image based on the self-attention weight corresponding to each local image block.

[0101] Step 4: Determine the local image block whose self-attention weight is greater than the preset weight as the target local image block, input the image features of the heart sound feature image into the decoder of the heart sound type recognition model, and obtain the target heart sound type corresponding to the periodic segment of the heart sound signal.

[0102] In the implementation manner of the present application, the mathematical definition of the classification process of the heart sound type recognition model is as follows:

[0103]

[0104] in, is the classification label embedding output by the last layer of the encoder, , is a trainable parameter, is the probability of each preset heart sound type corresponding to the heart sound signal period segment. The preset heart sound type with the largest probability is the target heart sound type.

[0105] Here, evaluation indicators such as accuracy, sensitivity, specificity and ROC-AUC of the heart sound type recognition model can be evaluated to evaluate the detection performance of the heart sound type recognition model.

[0106] The method also includes: for any periodic segment of a heart sound signal, determining the similarity between a heart sound characteristic image corresponding to the periodic segment of the heart sound signal and a preset heart sound characteristic image corresponding to the target heart sound type in a pre-established image library as a degree parameter of the heart sound type corresponding to the periodic segment of the heart sound signal being the target heart sound type.

[0107] The method also includes: determining the heart sound type of the heart sound characteristic image as a score for each preset heart sound type by comparing the similarity between the heart sound characteristic image and the preset heart sound characteristic image corresponding to each preset heart sound type in the image library; and sorting all preset heart sound types according to the scores to obtain a ranking of the heart sound types corresponding to the heart sound characteristic image.

[0108] In the implementation manner of the present application, the credibility of the model recognition result can be increased by the above ranking of heart sound types.

[0109] The embodiment of the present application provides a method for processing a heart sound signal, the method comprising: determining the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability of each preset heart sound stage under each target duration; segmenting the heart sound signal to be processed according to the preset heart sound cycle and the target heart sound stage of each heart sound signal time segment to obtain a heart sound signal period segment; inputting the heart sound feature image corresponding to the heart sound signal period segment into a pre-trained heart sound type recognition model to obtain the target heart sound type corresponding to the heart sound signal period segment; the target heart sound type corresponding to the heart sound signal period segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image. The present application can increase the interpretability of the processing result of the heart sound signal and improve the processing accuracy.

[0110] Based on the same inventive concept, an embodiment of the present application also provides a device for processing heart sound signals corresponding to the method for processing heart sound signals. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the method for processing heart sound signals in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0111] Reference Figure 3 FIG. 1 is a schematic diagram of a heart sound signal processing device provided in an embodiment of the present application, wherein the heart sound signal processing device includes:

[0112] An acquisition module 301 is used to acquire a heart sound signal time segment at each moment in the heart sound signal to be processed, and determine a first probability that each heart sound signal time segment is in each preset heart sound stage;

[0113] A determination module 302, configured to determine a target heart sound stage at which each heart sound signal time segment is located based on a first probability that each heart sound signal time segment is at each preset heart sound stage and a second probability that each preset heart sound stage is at each target duration;

[0114] A segmentation module 303 is used to segment the heart sound signal to be processed according to a preset heart sound cycle and a target heart sound stage where each heart sound signal time segment is located, so as to obtain heart sound signal cycle segments;

[0115] The input module 304 is used to input the heart sound feature image corresponding to any of the heart sound signal periodic segments into a pre-trained heart sound type recognition model to obtain a target heart sound type corresponding to the heart sound signal periodic segment; the heart sound feature image is used to display the nonlinear phase coupling strength of each audio component group of the heart sound signal periodic segment through color; the target heart sound type corresponding to the heart sound signal periodic segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image.

[0116] In a possible implementation, the determination module 302 is specifically used to, for any of the heart sound signal time segments, correct the first probability that the heart sound signal time segment is in each preset heart sound stage according to the second probability that each preset heart sound stage is in each target duration, to obtain a third probability that the heart sound signal time segment is in each preset heart sound stage; and determine the target heart sound stage in which each heart sound signal time segment is located according to the third probability that all heart sound signal time segments are in each preset heart sound stage.

[0117] In a possible implementation, the determination module 302 is specifically configured to substitute the second probability of each preset heart sound stage at each target duration and the first probability of the heart sound signal time segment being in each preset heart sound stage into the following formula to obtain a third probability of the heart sound signal time segment being in each preset heart sound stage:

[0118] ;

[0119] in, is the first probability that the heart sound signal time segment at time t is in the i-th preset heart sound stage, is the i-th preset heart sound stage, is the second probability of the i-th preset heart sound stage under the j-th target duration, is the probability that the heart sound signal time segment at the preset initial moment is in the i-th preset heart sound stage.

[0120] In a possible implementation, the determination module 302 is specifically used to obtain the historical duration of the historical heart sound signal in each preset heart sound stage; for each preset heart sound stage, calculate the standard deviation and the average value of all historical durations of the preset heart sound stage; determine the duration range of the preset heart sound stage according to the standard deviation and the average value; and determine the target duration of the preset heart sound stage from the duration range.

[0121] In a possible implementation manner, the determination module 302 is specifically configured to calculate the second probability of any preset heart sound stage at any target duration by using the following formula:

[0122] ;

[0123] in, is the second probability of the preset heart sound stage under the target duration, is the average of all historical durations of the preset heart sound stage, is the variance of all historical durations of the preset heart sound stage, is the standard deviation of all historical durations of the preset heart sound stage, Duration of the stated goal.

[0124] In a possible implementation, the input module 304 is specifically configured to determine the target heart sound type corresponding to the heart sound signal periodic segment and the target local image block in the heart sound feature image through the following steps:

[0125] Segmenting the heart sound characteristic image according to a preset image size to obtain a local image block;

[0126] For each local image block, generating an embedding vector corresponding to the local image block by using the local image block and the corresponding position information in the heart sound feature image;

[0127] Inputting the embedding vectors corresponding to all local image blocks into the encoder of the heart sound type recognition model to determine the image features of the heart sound feature image and the self-attention weights corresponding to each local image block by using a multi-head self-attention mechanism;

[0128] A local image block whose self-attention weight is greater than a preset weight is determined as a target local image block, and the image features of the heart sound feature image are input into a decoder of the heart sound type recognition model to obtain a target heart sound type corresponding to the periodic segment of the heart sound signal.

[0129] In a possible implementation, the determination module 302 is further used to determine, for any periodic segment of the heart sound signal, the similarity between the heart sound characteristic image corresponding to the periodic segment of the heart sound signal and a preset heart sound characteristic image corresponding to the target heart sound type in a pre-established image library, as a degree parameter of the heart sound type corresponding to the periodic segment of the heart sound signal being the target heart sound type.

[0130] The embodiment of the present application provides a processing device for heart sound signals, the device is specifically used to determine the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability of each preset heart sound stage under each target duration; segment the heart sound signal to be processed according to the preset heart sound cycle and the target heart sound stage of each heart sound signal time segment to obtain the heart sound signal period segment; input the heart sound feature image corresponding to the heart sound signal period segment into a pre-trained heart sound type recognition model to obtain the target heart sound type corresponding to the heart sound signal period segment; the target heart sound type corresponding to the heart sound signal period segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound feature image. The present application can increase the interpretability of the processing result of the heart sound signal and improve the processing accuracy.

[0131] like Figure 4 As shown, an electronic device 400 provided in an embodiment of the present application includes: a processor 401, a memory 402 and a bus, wherein the memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device is running, the processor 401 communicates with the memory 402 through the bus, and the processor 401 executes the machine-readable instructions to perform the steps of the method for processing heart sound signals as described above.

[0132] Specifically, the memory 402 and the processor 401 can be general-purpose memories and processors, which are not specifically limited here. When the processor 401 runs the computer program stored in the memory 402, the method for processing the heart sound signal can be executed.

[0133] Corresponding to the above-mentioned method for processing heart sound signals, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the above-mentioned method for processing heart sound signals are executed.

[0134] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0135] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0137] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the information processing method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0138] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing heart sound signals, characterized in that: The method for processing the heart sound signal comprises: Obtaining a heart sound signal time segment at each moment in the heart sound signal to be processed, and determining a first probability that each heart sound signal time segment is in each preset heart sound stage; Determine the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability that each preset heart sound stage is at each target duration; According to a preset heart sound cycle and a target heart sound stage where each heart sound signal time segment is located, the heart sound signal to be processed is segmented to obtain heart sound signal cycle segments; For any of the heart sound signal periodic segments, the heart sound characteristic image corresponding to the heart sound signal periodic segment is input into a pre-trained heart sound type recognition model to obtain a target heart sound type corresponding to the heart sound signal periodic segment; the heart sound characteristic image is used to display the nonlinear phase coupling strength of each audio component group of the heart sound signal periodic segment by color; the target heart sound type corresponding to the heart sound signal periodic segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound characteristic image; The nonlinear phase coupling strength of each audio component group is calculated by the following formula: ； in, is the i-th audio component With the jth audio component The nonlinear phase coupling strength between is the heart sound signal periodic segment in the i-th audio component The Fourier coefficients on is the jth audio component of the heart sound signal periodic segment The Fourier coefficients on The heart sound signal periodic segment in the audio component + The complex conjugate of the complex magnitude on , For all time windows expected value.

2. The method for processing heart sound signals according to claim 1, characterized in that: The determining the target heart sound stage of each heart sound signal time segment based on the first probability that each heart sound signal time segment is in each preset heart sound stage and the second probability that each preset heart sound stage is at each target duration includes: For any of the heart sound signal time segments, the first probability that the heart sound signal time segment is in each preset heart sound stage is corrected according to the second probability that each preset heart sound stage is in each target duration, so as to obtain a third probability that the heart sound signal time segment is in each preset heart sound stage; According to the third probability that all heart sound signal time segments are in each preset heart sound stage, the target heart sound stage in which each heart sound signal time segment is located is determined.

3. The method for processing heart sound signals according to claim 2, characterized in that: The step of correcting the first probability that the heart sound signal time segment is in each preset heart sound stage according to the second probability that the heart sound signal time segment is in each preset heart sound stage at each target duration to obtain a third probability that the heart sound signal time segment is in each preset heart sound stage includes: Substitute the second probability of each preset heart sound stage at each target duration and the first probability of the heart sound signal time segment being in each preset heart sound stage into the following formula to obtain the third probability of the heart sound signal time segment being in each preset heart sound stage: ； in, is the first probability that the heart sound signal time segment at time t is in the i-th preset heart sound stage, is the i-th preset heart sound stage, is the second probability of the i-th preset heart sound stage under the j-th target duration, is the probability that the heart sound signal time segment at the preset initial moment is in the i-th preset heart sound stage.

4. The method for processing heart sound signals according to any one of claims 1 to 3, characterized in that: Determine the target duration by following these steps: Obtaining the historical duration of the historical heart sound signal in each preset heart sound stage; For each preset heart sound stage, calculate the standard deviation and average of all historical durations of the preset heart sound stage; determine the duration range of the preset heart sound stage based on the standard deviation and average; and determine the target duration of the preset heart sound stage from the duration range.

5. The method for processing heart sound signals according to claim 4, characterized in that: The second probability of any preset heart sound stage at any target duration is calculated by the following formula: ； in, is the second probability of the preset heart sound stage under the target duration, is the average of all historical durations of the preset heart sound stage, is the variance of all historical durations of the preset heart sound stage, is the standard deviation of all historical durations of the preset heart sound stage, Duration of the stated goal.

6. The method for processing heart sound signals according to claim 1, characterized in that: The target heart sound type corresponding to the heart sound signal periodic segment and the target local image block in the heart sound feature image are determined by the following steps, including: Segmenting the heart sound characteristic image according to a preset image size to obtain a local image block; For each local image block, generating an embedding vector corresponding to the local image block by using the local image block and the corresponding position information in the heart sound feature image; Inputting the embedding vectors corresponding to all local image blocks into the encoder of the heart sound type recognition model to determine the image features of the heart sound feature image and the self-attention weights corresponding to each local image block by using a multi-head self-attention mechanism; A local image block whose self-attention weight is greater than a preset weight is determined as a target local image block, and the image features of the heart sound feature image are input into a decoder of the heart sound type recognition model to obtain a target heart sound type corresponding to the periodic segment of the heart sound signal.

7. The method for processing heart sound signals according to claim 1, characterized in that: The method further comprises: For any periodic segment of a heart sound signal, the similarity between the heart sound characteristic image corresponding to the periodic segment of the heart sound signal and a preset heart sound characteristic image corresponding to the target heart sound type in a pre-established image library is determined as a degree parameter of the heart sound type corresponding to the periodic segment of the heart sound signal being the target heart sound type.

8. A heart sound signal processing device, characterized in that: The heart sound signal processing device comprises: An acquisition module, used for acquiring a heart sound signal time segment at each moment in the heart sound signal to be processed, and determining a first probability that each heart sound signal time segment is in each preset heart sound stage; a determination module, configured to determine a target heart sound stage at which each heart sound signal time segment is located based on a first probability that each heart sound signal time segment is at each preset heart sound stage and a second probability that each preset heart sound stage is at each target duration; A segmentation module, configured to segment the heart sound signal to be processed according to a preset heart sound cycle and a target heart sound stage at which each heart sound signal time segment is located, to obtain heart sound signal cycle segments; An input module is used for inputting the heart sound characteristic image corresponding to any of the heart sound signal periodic segments into a pre-trained heart sound type recognition model to obtain a target heart sound type corresponding to the heart sound signal periodic segment; the heart sound characteristic image is used to display the nonlinear phase coupling strength of each audio component group of the heart sound signal periodic segment by color; the target heart sound type corresponding to the heart sound signal periodic segment is determined by the heart sound type recognition model through the nonlinear phase coupling strength of the audio component group in the target local image block in the heart sound characteristic image; The nonlinear phase coupling strength of each audio component group is calculated by the following formula: ； in, is the i-th audio component With the jth audio component The nonlinear phase coupling strength between is the heart sound signal periodic segment in the i-th audio component The Fourier coefficients on is the jth audio component of the heart sound signal periodic segment The Fourier coefficients on The heart sound signal periodic segment in the audio component + The complex conjugate of the complex magnitude on , For all time windows expected value.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for processing heart sound signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for processing heart sound signals according to any one of claims 1 to 7 are executed.

Images