Heart function auxiliary analysis system based on neural network

Abstract

The invention provides a heart function auxiliary analysis system based on a neural network, and belongs to the technical field of heart assistance, and the system comprises a preprocessing module which is used for preprocessing a heart ultrasound image; the waveform recognition module is used for performing waveform detection and multi-scale feature recognition on a waveform curve in the preprocessed ultrasonic image, positioning a peak value and a valley value of a waveform, and recognizing a diastolic frame and a systolic frame in a cardiac ultrasonic period; the three-dimensional reconstruction module is used for performing three-dimensional reconstruction on the heart region by using a deep learning technology; the calculation module is used for calculating the volume and ejection fraction of the left ventricle according to the reconstructed heart area; and the auxiliary reminding module is used for carrying out abnormal value verification and elimination on the ejection fractions obtained in the plurality of periods through an abnormal value elimination algorithm to obtain an accurate value, and outputting the cardiac function information of the patient for auxiliary reminding. The cardiac function condition of a patient can be accurately reflected, an accurate basis is provided for auxiliary reminding, and effective support is provided for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of cardiac assist technology, and in particular to a cardiac function assist analysis system based on neural network. Background Art

[0002] Heart function is essential for maintaining normal body tissues. Heart failure is manifested as dyspnea, fatigue, exercise intolerance and increased risk of death. Impaired heart function is described as cardiomyopathy or heart failure, which is an increasingly serious global health problem. There are many methods for quantifying heart function and diagnosing dysfunction. Among them, left ventricular ejection fraction is the most commonly used and important indicator of contractile function. The evaluation of left ventricular contractile function is crucial for diagnosing cardiovascular disease, judging the condition, guiding treatment and prognosis. Accurate calculation of left ventricular ejection fraction can also provide doctors with correct diagnostic basis.

[0003] There are differences in people's assessment of ejection fraction, partly due to the irregularity of heart rate and the ambiguity of cardiac ultrasound. In the process of calculating left ventricular ejection fraction, imaging generally recommends the use of multiple continuous cardiac cycles for evaluation. If heart rate variability is found, it is inaccurate to use only one cycle of ultrasound for calculation. At the same time, due to the limitations of imaging principles during the scanning process, there may be noise artifacts and other factors that affect the image quality in cardiac ultrasound images, making the boundaries of heart tissue unclear, making it difficult for doctors to mark the left ventricular endocardium. Therefore, the calculation of clinical left ventricular ejection fraction is often highly empirical, usually resulting in large errors, affecting the accuracy of the results.

[0004] In recent years, with the rapid development of artificial intelligence technology, computer-aided diagnosis technology has also made significant progress. Computer-aided diagnosis technology has reduced the errors and burden of doctors' diagnosis in many aspects. It can effectively improve the detection efficiency and accuracy and reduce the impact of human errors by automatically identifying and segmenting the left ventricular endocardial contour and calculating the ejection fraction value through deep learning methods. However, the existing video-based left ventricular ejection fraction prediction method, deep learning methods, directly predict the left ventricular ejection fraction from the segmented video, which does not conform to the clinical calculation workflow and lacks interpretability. At the same time, these deep learning models rely on manually annotating static images of systole and diastole, rather than automatically detecting systole and systole. The existing models still have large errors compared with human evaluation of cardiac function.

[0005] Researchers have proposed a method for processing ejection fraction data based on cardiac ultrasound video. The method involves segmenting the left ventricle for each frame of the image, then filtering the image sequence for the end-systolic and end-diastolic frame images, calculating the left ventricular volume to generate the corresponding left ventricular volume parameters, and generating the end-systolic and end-diastolic volumes to obtain the ejection fraction. However, this method has some defects. First, the screening of the end-diastolic and end-systolic phases depends on the segmentation effect of the model. If there are images with poor segmentation effects, it will affect the screening of key frames and thus the calculation of the indicators. Second, the existing models usually only use a single cycle in the cardiac ultrasound video to calculate the ejection fraction indicator. Clinical diagnosis needs to consider the ejection fraction of multiple cycles of the case to obtain an accurate value. If the case mutates, the assessment of the ejection fraction in a single cycle is inaccurate.

[0006] In summary, various existing technologies are currently unable to provide good auxiliary evaluation for the analysis of cardiac function.

[0007] Therefore, the present invention proposes a cardiac function auxiliary analysis system based on neural network. Summary of the invention

[0008] The present invention provides a cardiac function auxiliary analysis system based on a neural network, which is used to effectively avoid human errors by performing waveform detection on cardiac ultrasound images and combining deep learning technology for regional reconstruction, and to obtain the final value by making abnormal judgments on the ejection fraction of multiple cycles. It can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

[0009] The present invention provides a cardiac function auxiliary analysis system based on a neural network, comprising:

[0010] A preprocessing module, used for acquiring cardiac ultrasound images of different patients from different viewing angles scanned by multiple ultrasound instruments, and preprocessing the cardiac ultrasound images;

[0011] A waveform recognition module is used to perform waveform detection and multi-scale feature recognition on the waveform curve in the preprocessed ultrasound image, locate the peak and valley of the waveform, and identify the diastolic frame and the systolic frame in the cardiac ultrasound cycle;

[0012] A three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the diastolic frame and the systolic frame of the heart region using deep learning technology to reconstruct the heart region;

[0013] A calculation module, used for calculating the volume and ejection fraction of the left ventricle according to the reconstructed heart area;

[0014] The auxiliary reminder module is used to verify and eliminate abnormal values ​​of the ejection fraction obtained in multiple cycles through an abnormal value elimination algorithm to obtain an accurate value, and output the patient's heart function information for auxiliary reminders.

[0015] Preferably, the preprocessing includes: information masking, format conversion, data cleaning, data enhancement and electrocardiogram signal extraction on the cardiac ultrasound image.

[0016] Preferably, the waveform recognition module includes:

[0017] A filtering unit, used for identifying the electrocardiogram signal in the preprocessed ultrasound image based on a bandpass filter, filtering out high-frequency noise and low-frequency drift, and obtaining a key heartbeat signal;

[0018] A double differentiation unit, used for performing double differentiation on the key heartbeat signal, amplifying the slope information of the R wave peak, and then using an adaptive smoothing technique to process the differentiated signal to obtain a smoothed signal;

[0019] A nonlinear transformation unit, used for performing nonlinear transformation on the smooth signal to obtain an enhanced signal;

[0020] A multi-scale analysis unit, for integrating the enhanced signal using a multi-scale moving integration method with different window sizes to extract the slope and width information of the R wave peak to obtain heartbeat information, wherein the heartbeat information is the peak and valley values ​​of the located waveform;

[0021] The heartbeat segmentation unit is used to perform heartbeat segmentation on the enhanced signal based on the dynamic threshold setting of the random forest model and in combination with the heartbeat information to obtain the diastolic frame and the systolic frame in the cardiac ultrasound cycle.

[0022] Preferably, the three-dimensional reconstruction module includes:

[0023] An image processing unit, used for performing image normalization processing, denoising processing and image enhancement processing on the image of the identified diastolic frame and the systolic frame to obtain an initial image;

[0024] A scale processing unit, used to use the ResNet network structure as the backbone network and introduce a dilated convolution layer to perform scale processing on the initial image to obtain multi-scale feature representation and semantic information, wherein an efficient channel attention mechanism is introduced in each dilated convolution layer;

[0025] A decoding unit, used to use depth-separable convolution to decode the multi-scale feature representation and semantic information, and combine convolution kernels of different sizes to extract multi-scale features, and then encode and fuse the spatial relationship of high-scale and low-scale features through linear mapping to obtain a fused feature map, wherein an efficient channel attention mechanism is also introduced in the decoding process;

[0026] A network analysis unit, configured to input the fused feature map into an average pooling layer output plane parameter, perform a pooling convolution operation on the fused feature map to obtain a predicted segmentation map, and predict a depth map of the fused feature map by combining the transverse and longitudinal feature maps of the two-chamber and four-chamber heart chambers;

[0027] The reconstruction unit is used to perform depth prediction on the pixels of the diastolic frame and the systolic frame based on the plane parameters, the predicted segmentation map and the depth map, fit the plane of the heart area and restore each plane structure to achieve three-dimensional reconstruction of the heart area.

[0028] Preferably, the calculation module includes:

[0029] A model acquisition unit, used for acquiring a left ventricle surface model based on three-dimensional reconstruction;

[0030] A slicing unit, used for dividing the left ventricle surface model into a plurality of slices along the long axis direction, wherein each slice represents a cross section of a corresponding surface of the left ventricle;

[0031] A volume calculation unit, used for calculating the volume of the left ventricle surface model by using an integral method;

[0032] The ejection fraction calculation unit is used to obtain the left ventricular end-systolic volume and the left ventricular end-diastolic volume from all volumes, and calculate the left ventricular ejection fraction.

[0033] Preferably, the auxiliary reminder module includes:

[0034] A feature construction unit is used to obtain the diastolic frame and the systolic frame in each set cardiac cycle, and perform feature extraction according to preset feature indicators to construct time series features;

[0035] A range definition unit, used for performing statistical analysis on the time series characteristics and setting a critical value to define a normal range;

[0036] A verification unit, used for comparing the ejection fraction under each set cardiac cycle with the normal range, extracting abnormal values ​​and performing verification;

[0037] The auxiliary reminder unit is used to obtain the final value based on the verification result, and to perform average processing on all the final values ​​to obtain the accurate value, to match it with the score-auxiliary comparison table, and to output the auxiliary reminder.

[0038] Preferably, the reconstruction unit comprises:

[0039] A coding subunit, configured to globally encode the predicted segmentation map and the depth map to obtain a corresponding first global sequence set and a second global sequence set;

[0040] a loss determination subunit, configured to determine a loss value of each pixel according to the first global sequence set, the second global sequence set, and in combination with the plane parameters;

[0041] A first locking subunit, used to lock a pixel point whose loss value is greater than a preset value and regard it as a first pixel point;

[0042] A capturing subunit, used to capture the difference between the first pixel and the surrounding pixels and the direction of the difference;

[0043] A first calculation subunit is used to determine a correction direction corresponding to a first pixel point according to the difference degree and the difference direction, and perform weighted fusion based on a pixel difference between the first pixel point and each neighboring pixel point under the first pixel point to obtain a first value;

[0044] A second locking subunit is used to lock a pixel point whose loss value is not less than a preset value and regard it as a second pixel point;

[0045] A variance determination subunit, configured to calculate a pixel difference between a second pixel and each of the neighboring pixels to obtain a first variance, and at the same time, calculate a second variance between all the neighboring pixels of the second pixel;

[0046] a level determination subunit, configured to determine an association level based on the first variance, the second variance, the pixel value of the second pixel, and the pixel values ​​of all neighboring pixels of the second pixel;

[0047] A second calculation subunit is used to match the current disparity matching the associated level from the level-disparity comparison table to obtain a second value corresponding to a second pixel point;

[0048] The structure reconstruction subunit is used to map all the first values ​​and the second values ​​to corresponding position points in the heart area to obtain various planar structures.

[0049] Preferably, the verification unit comprises:

[0050] A position determination subunit is used to determine whether the abnormal value is within the allowable error range. If not, the abnormal value is determined to have passed the verification and is removed;

[0051] Otherwise, the ejection fraction under each set cardiac cycle is continuously captured, and a score curve is drawn, and the first position of the corresponding abnormal value in the score curve is locked;

[0052] A conditional judgment subunit is used to determine whether the number of ejection fractions n1 on the left curve of the first position and the number of ejection fractions n2 on the right curve meet the conditional judgment subunit. as well as Then obtain the first score average value ave1 of the left curve and the second score average value ave2 of the right curve, wherein N0 represents the total number of ejection fractions existing on the score curve;

[0053] The expansion subunit is used to expand the normal range according to |ave1-ave2| to obtain a new range. If the corresponding abnormal value is within the corresponding new range, it is determined that the abnormal value verification has not passed, and according to The corresponding outliers are first adjusted and retained, where (F1, F2) represents the new range; min z,xl Indicates the minimum value of the absolute value of the difference between all two adjacent ejection fractions in the left curve; min y,xl It represents the minimum value among the absolute values ​​of the difference between all two adjacent ejection fractions in the right curve;

[0054] If the corresponding abnormal value is not within the corresponding new range, the abnormal value is determined to have passed the verification and is removed;

[0055] Extension subunit, used if satisfied or The short side curve is locked and the short side ratio of the short side curve is The difference is obtained by randomly selecting points that satisfy the corresponding difference from the long side curve, and extending the curve at the idle end of the short side curve to obtain the extended curve;

[0056] The adjustment subunit is used to determine the absolute value of the difference between the average value ave3 of the extension curve and the corresponding abnormal value. If the absolute value of the difference is within the set error range, it is determined that the abnormal value verification has failed, and the abnormal value verification is performed according to Performing a second adjustment and retaining the corresponding abnormal values, wherein (F01, F02) represents a normal range; Ny represents the number of extension points in the extension curve; represents the variance based on ave3, F01, and F02;

[0057] If the absolute value of the difference is not within the set error range, the abnormal value is determined to have passed the verification and is removed. Compared with the prior art, the beneficial effects of the present application are as follows:

[0058] By performing waveform detection on cardiac ultrasound images and combining deep learning technology for regional reconstruction, human errors can be effectively avoided, and abnormal ejection fractions of multiple cycles can be judged to obtain the final value. This can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

[0059] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0062] Figure 1 is a structural diagram of a cardiac function auxiliary analysis system based on a neural network in an embodiment of the present invention;

[0063] Figure 2 A structural diagram of a waveform recognition module in an embodiment of the present invention;

[0064] Figure 3 4 is a structural diagram of a 3D reconstruction module in an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0066] The present invention provides a cardiac function auxiliary analysis system based on a neural network, such as Figure 1 As shown, including:

[0067] A preprocessing module, used for acquiring cardiac ultrasound images of different patients from different viewing angles scanned by multiple ultrasound instruments, and preprocessing the cardiac ultrasound images;

[0068] A waveform recognition module is used to perform waveform detection and multi-scale feature recognition on the waveform curve in the preprocessed ultrasound image, locate the peak and valley of the waveform, and identify the diastolic frame and the systolic frame in the cardiac ultrasound cycle;

[0069] A three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the diastolic frame and the systolic frame of the heart region using deep learning technology to reconstruct the heart region;

[0070] A calculation module, used for calculating the volume and ejection fraction of the left ventricle according to the reconstructed heart area;

[0071] The auxiliary reminder module is used to verify and eliminate abnormal values ​​of the ejection fraction obtained in multiple cycles through an abnormal value elimination algorithm to obtain an accurate value, and output the patient's heart function information for auxiliary reminders.

[0072] Preferably, the preprocessing includes: information masking, format conversion, data cleaning, data enhancement and electrocardiogram signal extraction on the cardiac ultrasound image.

[0073] In this embodiment, cardiac ultrasound images are collected from healthy people and different patients, and the images include double-chamber and four-chamber images.

[0074] In this embodiment, the left ventricular endocardial contour is annotated to produce a cardiac ultrasound data set for model training. At the same time, a public cardiac ultrasound image data set can be used for pre-training to make the model more generalizable.

[0075] In this embodiment, the continuous ECG signal in the ultrasound image usually contains multiple heartbeat cycles. Before the ECG signal is subjected to waveform detection, the continuous ECG signal needs to be segmented into multiple heartbeat cycles to facilitate the subsequent waveform detection process. In the present invention, the heartbeat signal is first segmented. Then, the end-systole and end-diastole in the cardiac ultrasound cycle are detected using a waveform detection method (such as waveform detection based on template matching, waveform detection based on neural network, etc.).

[0076] In this embodiment, the backbone network used in the 3D reconstruction process may be ResNet, GoogleNet, U-Net, etc., and the network may be optimized by dilated convolution, residual connection, pyramid pooling, etc. to improve the learning ability of the model.

[0077] In this embodiment, the outlier elimination algorithm is to set a normal range for the ejection fraction to determine whether the score is within the range. If not, it is regarded as an outlier.

[0078] In this embodiment, the accurate value may be the average value of all the values ​​that have not been eliminated.

[0079] In this embodiment, information masking: performing information masking on the acquired cardiac ultrasound image to remove sensitive information and irrelevant information in the image, such as patient information and timestamp, to improve the generalization ability of the model.

[0080] In this embodiment, format conversion: ultrasound images in different formats are converted into AVI format to facilitate subsequent model training and testing.

[0081] In this embodiment, data cleaning: the collected cardiac ultrasound images may contain noise and artifacts, which may affect the model recognition effect. This part of the data is cleaned to improve the data quality.

[0082] In this embodiment, data enhancement: operations such as rotation, translation, and scaling are performed on the data to increase the amount and diversity of the data and improve the robustness of the model.

[0083] In this embodiment, ECG signal extraction: extracting the pressure curve of cardiac ultrasound at a certain spatial sampling interval, and constructing multi-channel ECG sequence data according to the sampling frequency and image width.

[0084] The beneficial effects of the above technical solution are: by performing waveform detection on cardiac ultrasound images and combining deep learning technology for regional reconstruction, human errors can be effectively avoided, and abnormal ejection fractions of multiple cycles can be judged to obtain the final value, which can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

[0085] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the waveform recognition module is as follows: Figure 2 As shown, including:

[0086] A filtering unit, used for identifying the electrocardiogram signal in the preprocessed ultrasound image based on a bandpass filter, filtering out high-frequency noise and low-frequency drift, and obtaining a key heartbeat signal;

[0087] A double differentiation unit, used for performing double differentiation on the key heartbeat signal, amplifying the slope information of the R wave peak, and then using an adaptive smoothing technique to process the differentiated signal to obtain a smoothed signal;

[0088] A nonlinear transformation unit, used for performing nonlinear transformation on the smooth signal to obtain an enhanced signal;

[0089] A multi-scale analysis unit, for integrating the enhanced signal using a multi-scale moving integration method with different window sizes to extract the slope and width information of the R wave peak to obtain heartbeat information, wherein the heartbeat information is the peak and valley values ​​of the located waveform;

[0090] The heartbeat segmentation unit is used to perform heartbeat segmentation on the enhanced signal based on the dynamic threshold setting of the random forest model and in combination with the heartbeat information to obtain the diastolic frame and the systolic frame in the cardiac ultrasound cycle.

[0091] In this embodiment, the heartbeat signal is segmented using an improved Pan-Tompkins algorithm. The Pan-Tompkins algorithm is a classic R wave peak detection algorithm, which is widely used in the processing of electrocardiogram (ECG) signals. In the present invention, the bandpass filter and the moving integration method are improved to improve the accuracy and robustness of heartbeat segmentation.

[0092] In this embodiment, the bandpass filter analyzes the spectral characteristics of the signal through short-time Fourier transform (STFT), and dynamically adjusts the cutoff frequency of the filter accordingly. This adaptive strategy can intelligently adjust the filter parameters according to the instantaneous characteristics of the ECG signal to identify and retain the key heartbeat information in the ECG signal, and effectively suppress high-frequency noise and low-frequency drift.

[0093] In this embodiment, the adaptive smoothing technology refers to the use of an adaptive filtering window based on the instantaneous frequency characteristics of the signal. The window can dynamically adjust the size and shape according to the instantaneous frequency of the signal, thereby maintaining the smoothness of the signal.

[0094] In this embodiment, a multi-scale moving integration method is used to integrate the signal using different window sizes to extract the slope and width information of the R wave. The integration results of different scales are fused to obtain more accurate heartbeat information. The multi-scale integration formula is as follows:

[0095]

[0096] Among them, N k is the window size of the kth scale.

[0097] In this embodiment, in the threshold processing stage, a dynamic threshold setting method based on a random forest model is used, and the random forest model training data set contains normal and abnormal ECG signals to improve the generalization ability of the model. The method dynamically adjusts the threshold according to historical data and real-time signal characteristics. Specifically, the R wave peak characteristics in the historical data are analyzed to determine the direction of threshold adjustment, and the threshold is adjusted in combination with the characteristics of the current signal, such as the slope and amplitude change of the signal. When the characteristic change of the signal exceeds the dynamic threshold, the R wave peak position is obtained, and the heartbeat position and window size are confirmed according to the sampling rate to achieve accurate segmentation of the heartbeat.

[0098] In this embodiment, the template is dynamically generated based on the patient's electrocardiogram characteristics, such as heart rate, QRS wave width, QRS wave amplitude, ST segment height, etc. The template generation takes into account individual differences, such as heart rate changes and electrocardiogram waveform variations.

[0099] In this embodiment, an adaptive template matching algorithm is used. The matching parameters include the time window width and the template displacement. These parameters are adjusted in real time according to the signal characteristics to adapt to the changes of different ECG waveforms. The matching score is calculated by performing template matching in the image. The matching score function is as follows:

[0100]

[0101] Where I is the image, T is the template, x, y are the starting coordinates of the search in the image, and i, j are the pixel indexes in the template, which are used to traverse every pixel in the template.

[0102] In this embodiment, signal features (such as shape, energy, frequency, etc.) are extracted at different scales, and multi-scale feature fusion is performed to improve the accuracy of waveform detection. Through the matching process, the peak and valley values ​​of the cardiac signal, that is, the diastolic frame and the systolic frame in the cardiac cycle, are identified.

[0103] The beneficial effects of the above technical solution are: based on the improved Pan-Tompkins heart beat segmentation and multi-scale template matching waveform detection method, the traditional Pan-Tompkins heart beat detection algorithm is optimized to improve the accuracy and robustness of heart beat segmentation. The multi-scale analysis technology combined with the template matching method can adapt to the changes in ECG waveforms under different patients and different conditions. The end diastole and end systole in the heart cycle are automatically identified through peak detection, saving the time of manually intercepting key frames.

[0104] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the three-dimensional reconstruction module is as follows: Figure 3 As shown, including:

[0105] An image processing unit, used for performing image normalization processing, denoising processing and image enhancement processing on the image of the identified diastolic frame and the systolic frame to obtain an initial image;

[0106] A scale processing unit, used to use the ResNet network structure as the backbone network and introduce a dilated convolution layer to perform scale processing on the initial image to obtain multi-scale feature representation and semantic information, wherein an efficient channel attention mechanism is introduced in each dilated convolution layer;

[0107] A decoding unit, used to use depth-separable convolution to decode the multi-scale feature representation and semantic information, and combine convolution kernels of different sizes to extract multi-scale features, and then encode and fuse the spatial relationship of high-scale and low-scale features through linear mapping to obtain a fused feature map, wherein an efficient channel attention mechanism is also introduced in the decoding process;

[0108] A network analysis unit, configured to input the fused feature map into an average pooling layer output plane parameter, perform a pooling convolution operation on the fused feature map to obtain a predicted segmentation map, and predict a depth map of the fused feature map by combining the transverse and longitudinal feature maps of the two-chamber and four-chamber heart chambers;

[0109] The reconstruction unit is used to perform depth prediction on the pixels of the diastolic frame and the systolic frame based on the plane parameters, the predicted segmentation map and the depth map, fit the plane of the heart area and restore each plane structure to achieve three-dimensional reconstruction of the heart area.

[0110] In this embodiment, image normalization: scaling the image pixel values ​​to a specific range to facilitate network training. Denoising: applying a bilateral filter to remove random noise in the image while retaining edge information. Image enhancement: using histogram equalization technology to enhance image contrast and highlight the details of the cardiac structure. Deep separable convolution is introduced to perform preliminary feature extraction on the image, reducing the amount of calculation and retaining key information. Deep separable convolution decomposes the convolution operation into deep convolution and point-by-point convolution to extract spatial features and channel features respectively. .

[0111] In this embodiment, the ResNet network structure is used as the backbone network, and a dilated convolution layer is introduced therein. The expansion rate of the dilated convolution layer gradually increases at different levels to form a multi-scale dilated convolution module to extract features of different scales. An efficient channel attention mechanism (ECA) module is added after each dilated convolution layer to effectively capture the dependencies between channels through a lightweight adaptive average pooling operation and one-dimensional convolution without requiring a large number of parameters. The ECA module can adaptively adjust the weight of each channel to enhance the network's feature representation of the cardiac structure. A pyramid pooling layer is introduced at the end of the encoder to perform pooling operations of different scales to obtain multi-scale feature representations and semantic information.

[0112] In this embodiment, depth-separable convolution is used to decode features, and convolution kernels of different sizes are combined to extract multi-scale features. The ECA module is also introduced in the decoder to ensure that effective attention between channels can be maintained during feature upsampling. Convolution kernels of different sizes are used to decode features. Small-sized convolution kernels (such as 3x3) are used to extract low-scale features, and large-sized convolution kernels (such as 7x7) are used to extract high-scale features. Through linear mapping, the spatial relationship between high and low scale features is encoded and fused to obtain a fused feature map.

[0113] In this embodiment, the output part of the deep learning network includes three branches. One branch outputs plane parameters including plane normal information and offset after the fused feature map passes through the average pooling layer. Another branch predicts the segmentation map after the fused feature map passes through pooling, convolution and other operations. The third branch combines the horizontal and vertical feature maps of the two-chamber and four-chamber to predict the depth map, inputs the horizontal two-chamber feature map, and jointly predicts the depth map based on the vertical four-chamber and horizontal two-chamber feature maps.

[0114] The beneficial effects of the above technical solution are: by reconstructing the heart area through three-dimensional reconstruction of single-plane cardiac ultrasound images, the fusion of void convolution and attention mechanism can obtain a larger receptive field and improve the recognition and reconstruction capabilities of important cardiac structures.

[0115] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the calculation module comprises:

[0116] A model acquisition unit, used for acquiring a left ventricle surface model based on three-dimensional reconstruction;

[0117] A slicing unit, used for dividing the left ventricle surface model into a plurality of slices along the long axis direction, wherein each slice represents a cross section of a corresponding surface of the left ventricle;

[0118] A volume calculation unit, used for calculating the volume of the left ventricle surface model by using an integral method;

[0119] The ejection fraction calculation unit is used to obtain the left ventricular end-systolic volume and the left ventricular end-diastolic volume from all volumes, and calculate the left ventricular ejection fraction.

[0120] In this embodiment, the volume of the left ventricle surface model is calculated as follows:

[0121]

[0122] Among them, A i is the area of ​​a single slice, and Δz represents the distance between slices.

[0123] In this embodiment, the left ventricular ejection fraction (LVEF) is calculated as follows:

[0124]

[0125] Among them, left ventricular end-systolic volume (ESV) and left ventricular end-diastolic volume (EDV).

[0126] The beneficial effect of the above technical solution is that the left ventricular ejection fraction index calculated by the integral method provides reliable data support for clinical diagnosis.

[0127] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the auxiliary reminder module comprises:

[0128] A feature construction unit is used to obtain the diastolic frame and the systolic frame in each set cardiac cycle, and perform feature extraction according to preset feature indicators to construct time series features;

[0129] A range definition unit, used for performing statistical analysis on the time series characteristics and setting a critical value to define a normal range;

[0130] A verification unit, used for comparing the ejection fraction under each set cardiac cycle with the normal range, extracting abnormal values ​​and performing verification;

[0131] The auxiliary reminder unit is used to obtain the final value based on the verification result, and to perform average processing on all the final values ​​to obtain the accurate value, to match it with the score-auxiliary comparison table, and to output the auxiliary reminder.

[0132] In this embodiment, the time series characteristics include, for example, heart rate variability, cycle length, time interval between ventricular contraction and relaxation, etc., and the preset characteristic indicators are related to the heart rate variability, cycle length, time interval between ventricular contraction and relaxation, in order to obtain specific results under the indicator.

[0133] In this embodiment, the time feature sequence is input into the feature analysis model to obtain two critical values, F01 and F02, as the normal range. The feature analysis model is obtained by training the neural network model with different time feature sequences and the analysis results of the time feature sequences by cardiologists (the boundary range of ejection fraction) as samples. Therefore, two boundary values ​​can be directly obtained, and then the normal range (F01, F02) can be obtained.

[0134] In this embodiment, the abnormal value refers to a value of the ejection fraction that is not within the normal range.

[0135] In this embodiment, the purpose of verification is to determine whether the outlier needs to be removed.

[0136] In this embodiment, the score-auxiliary comparison table contains different accurate values ​​and auxiliary information of the cardiac function conditions corresponding to the values, which are preset.

[0137] The beneficial effects of the above technical solution are: performing time feature comparison on the ejection fraction identified in each cardiac cycle to identify and eliminate abnormal values, and verifying the identified abnormal values, thereby improving the accuracy and reliability of ejection fraction calculation and ensuring the effectiveness of post-assisted reminders.

[0138] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the reconstruction unit comprises:

[0139] A coding subunit, configured to globally encode the predicted segmentation map and the depth map to obtain a corresponding first global sequence set and a second global sequence set;

[0140] a loss determination subunit, configured to determine a loss value of each pixel according to the first global sequence set, the second global sequence set, and in combination with the plane parameters;

[0141] A first locking subunit, used to lock a pixel point whose loss value is greater than a preset value and regard it as a first pixel point;

[0142] A capturing subunit, used to capture the difference between the first pixel and the surrounding pixels and the direction of the difference;

[0143] A first calculation subunit is used to determine a correction direction corresponding to a first pixel point according to the difference degree and the difference direction, and perform weighted fusion based on a pixel difference between the first pixel point and each neighboring pixel point under the first pixel point to obtain a first value;

[0144] A second locking subunit is used to lock a pixel whose loss value is not less than a preset value and regard it as a second pixel;

[0145] A variance determination subunit, configured to calculate a pixel difference between a second pixel and each of the neighboring pixels to obtain a first variance, and at the same time, calculate a second variance between all the neighboring pixels of the second pixel;

[0146] a level determination subunit, configured to determine an association level based on the first variance, the second variance, the pixel value of the second pixel, and the pixel values ​​of all neighboring pixels of the second pixel;

[0147] A second calculation subunit is used to match the current disparity matching the associated level from the level-disparity comparison table to obtain a second value corresponding to a second pixel point;

[0148] The structure reconstruction subunit is used to map all the first values ​​and the second values ​​to corresponding position points in the heart area to obtain various planar structures.

[0149] In this embodiment, global encoding refers to encoding the grayscale value of each pixel in the corresponding image, thereby obtaining a sequence set based on the predicted segmentation map and the depth map.

[0150] In this embodiment, the loss value = ((the grayscale value of the same pixel based on the first global sequence set + the grayscale value based on the second global sequence set) / 2-standard value) + offset in the plane parameter × cos (the angle between the normal vector of the plane parameter and the horizontal plane), wherein the standard value is the pixel value under ideal conditions, and the ideal condition refers to the image being undistorted, noise-free, etc.

[0151] In this embodiment, the preset value is set in advance and is generally 6.

[0152] In this embodiment, the difference refers to the absolute value of the image gradient between each first pixel point and the corresponding neighboring pixel points.

[0153] In this embodiment, the difference direction refers to whether the first pixel point and the corresponding neighboring pixel point have a difference in the horizontal direction or in the vertical direction.

[0154] For example, the difference is: a1, and the difference direction is: vertical;

[0155] For example, the difference is a2, and the difference direction is: horizontal;

[0156] At this time, the correction direction is: max(a1,a2) determines the corresponding correction direction.

[0157] In this embodiment, the pixel difference refers to the difference in grayscale values.

[0158] In this embodiment, the first value=the pixel value of the corresponding pixel+G(correction direction, pixel difference with each neighboring pixel, weighting coefficient of each neighboring pixel).

[0159] G (correction direction, pixel difference with each neighboring pixel, weighting coefficient of each neighboring pixel) is a fine-tuning function, and the corresponding fine-tuning amount can be matched from the function database. The function database includes fine-tuning amounts based on different correction directions, different pixel difference combinations, and different weighting coefficient combinations, which can be used directly. Among them, the weighting coefficient = pixel difference of the corresponding neighboring pixel / sum of the pixel differences of all neighboring pixels.

[0160] In this embodiment, the first variance=the variance of all pixel differences between the second pixel point and each of the neighboring pixels.

[0161] In this embodiment, the second variance=the variance of all neighboring pixels of the second pixel.

[0162] In this embodiment, if the difference between the average value of pixel values ​​of all neighboring pixels of the second pixel and the pixel value of the second pixel is less than the sum of the first variance and the second variance, the association level is considered to be 0.

[0163] Otherwise, the association level is considered to be 1.

[0164] In this embodiment, the level-disparity comparison table contains disparities matching corresponding levels and is preset.

[0165] In this embodiment, the second value=the gray value corresponding to the second pixel+the parallax.

[0166] The beneficial effect of the above technical solution is: grayscale value encoding is performed on the two images to determine the loss value, and then the loss value in different situations is discussed by classification to adjust the value of the corresponding pixel point, thereby ensuring the reliability of the pixel point value and providing a data basis for subsequent reconstruction.

[0167] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the verification unit comprises:

[0168] A position determination subunit is used to determine whether the abnormal value is within the allowable error range. If not, the abnormal value is determined to have passed the verification and is removed;

[0169] Otherwise, the ejection fraction under each set cardiac cycle is continuously captured, and a score curve is drawn, and the first position of the corresponding abnormal value in the score curve is locked;

[0170] A conditional judgment subunit is used to determine whether the number of ejection fractions n1 on the left curve of the first position and the number of ejection fractions n2 on the right curve meet the conditional judgment subunit. as well as Then obtain the first score average value ave1 of the left curve and the second score average value ave2 of the right curve, wherein N0 represents the total number of ejection fractions existing on the score curve;

[0171] The expansion subunit is used to expand the normal range according to |ave1-ave2| to obtain a new range. If the corresponding abnormal value is within the corresponding new range, it is determined that the abnormal value verification has not passed, and according to The corresponding outliers are first adjusted and retained, where (F1, F2) represents the new range; min z,xl Indicates the minimum value of the absolute value of the difference between all two adjacent ejection fractions in the left curve; min y,xl It represents the minimum value among the absolute values ​​of the difference between all two adjacent ejection fractions in the right curve;

[0172] If the corresponding abnormal value is not within the corresponding new range, the abnormal value is determined to have passed the verification and is removed;

[0173] Extension subunit, used if satisfied or The short side curve is locked and the short side ratio of the short side curve is The difference is obtained by randomly selecting points that satisfy the corresponding difference from the long side curve, and extending the curve at the idle end of the short side curve to obtain the extended curve;

[0174] The adjustment subunit is used to determine the absolute value of the difference between the average value ave3 of the extension curve and the corresponding abnormal value. If the absolute value of the difference is within the set error range, it is determined that the abnormal value verification has failed, and the abnormal value verification is performed according to Performing a second adjustment and retaining the corresponding abnormal values, wherein (F01, F02) represents the normal range; Ny represents the number of extension points in the extension curve; represents the variance based on ave3, F01, and F02;

[0175] If the absolute value of the difference is not within the set error range, it is determined that the abnormal value has passed the verification and is eliminated.

[0176] In this embodiment, for example, the outlier is located at point 2. At this time, the left curve includes point 1, and the right curve includes points 3, 4, 5, 6, 7, 8, 9 and 10. At this time, the left curve is the short-side curve and the right curve is the long-side curve.

[0177] In this embodiment, the short side ratio is The difference is: At this time, the number of randomly selected points is: Then round up. At this time, 2 points are randomly selected, that is, Ny=2, and attached to the left side of the short side curve to achieve extension.

[0178] In this embodiment, the average value ave3 of the extension curve is the average value of the scores of all points on the extension curve.

[0179] In this embodiment, the error range is preset to (0, u01).

[0180] In this embodiment, the second adjustment of the outlier value is: Among them, when the difference between the average value ave3 and the corresponding outlier value is less than 0, A2 takes the value of 1, otherwise, A2 takes the value of -1.

[0181] In this embodiment, the allowable error range refers to the outer range of the left boundary and the outer range of the right boundary of the normal range, for example, (F01-u01, F01), (F02, F02+u01), where u01 is the allowable error.

[0182] In this embodiment, continuous capture refers to the scores at the same specified period time under different set periods, which is convenient for subsequent comparative analysis, that is, to ensure the uniformity of the data basis.

[0183] In this embodiment, the first position refers to the position where the abnormal value appears on the curve.

[0184] In this embodiment, for example, the score curve exists: point 1, point 2, point 3, point 4, point 5, point 6, point 7, point 8, point 9 and point 10, among which the outlier is located at point 3. At this time, the left curve includes point 1 and point 2, and the right curve includes point 4, point 5, point 6, point 7, point 8, point 9 and point 10, that is, n1=2, n2=7, N0=10.

[0185] In this embodiment, ave1 is the average score of point 1 and point 2, and ave2 is the average score of point 4 to point 10.

[0186] In this embodiment, the new range is (F01-|ave1-ave2|, F02+|ave1-ave2|).

[0187] In this embodiment, the first adjustment of the outlier is: If the outlier is greater than the average of ave1 and ave2, then A1 is -1, otherwise, A1 is 1.

[0188] The beneficial effect of the above technical solution is: by drawing a curve of the continuously obtained ejection fraction, the left and right sides of the curve are analyzed, thereby providing an auxiliary theoretical basis for the adjustment of abnormal values, effectively realizing a reasonable judgment on whether to retain abnormal values, and providing a reliable basis for obtaining accurate values.

[0189] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A cardiac function auxiliary analysis system based on neural network, characterized in that: include: A preprocessing module, used for acquiring cardiac ultrasound images of different patients from different viewing angles scanned by multiple ultrasound instruments, and preprocessing the cardiac ultrasound images; A waveform recognition module is used to perform waveform detection and multi-scale feature recognition on the waveform curve in the preprocessed ultrasound image, locate the peak and valley of the waveform, and identify the diastolic frame and the systolic frame in the cardiac ultrasound cycle; A three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the diastolic frame and the systolic frame of the heart region using deep learning technology to reconstruct the heart region; A calculation module, used for calculating the volume and ejection fraction of the left ventricle according to the reconstructed heart area; The auxiliary reminder module is used to verify and eliminate abnormal values ​​of the ejection fraction obtained in multiple cycles through an abnormal value elimination algorithm to obtain an accurate value, and output the patient's heart function information for auxiliary reminders.

2. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The preprocessing includes: information masking, format conversion, data cleaning, data enhancement and electrocardiogram signal extraction for cardiac ultrasound images.

3. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The waveform recognition module comprises: A filtering unit, used for identifying the electrocardiogram signal in the preprocessed ultrasound image based on a bandpass filter, filtering out high-frequency noise and low-frequency drift, and obtaining a key heartbeat signal; A double differentiation unit, used for performing double differentiation on the key heartbeat signal, amplifying the slope information of the R wave peak, and then using an adaptive smoothing technique to process the differentiated signal to obtain a smoothed signal; A nonlinear transformation unit, used for performing nonlinear transformation on the smooth signal to obtain an enhanced signal; A multi-scale analysis unit, for integrating the enhanced signal using a multi-scale moving integration method with different window sizes to extract the slope and width information of the R wave peak to obtain heartbeat information, wherein the heartbeat information is the peak and valley values ​​of the located waveform; The heartbeat segmentation unit is used to perform heartbeat segmentation on the enhanced signal based on the dynamic threshold setting of the random forest model and in combination with the heartbeat information to obtain the diastolic frame and the systolic frame in the cardiac ultrasound cycle.

4. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The three-dimensional reconstruction module comprises: An image processing unit, used for performing image normalization processing, denoising processing and image enhancement processing on the image of the identified diastolic frame and the systolic frame to obtain an initial image; A scale processing unit, used to use the ResNet network structure as the backbone network and introduce a dilated convolution layer to perform scale processing on the initial image to obtain multi-scale feature representation and semantic information, wherein an efficient channel attention mechanism is introduced in each dilated convolution layer; A decoding unit, used to use depth-separable convolution to decode the multi-scale feature representation and semantic information, and combine convolution kernels of different sizes to extract multi-scale features, and then encode and fuse the spatial relationship of high-scale and low-scale features through linear mapping to obtain a fused feature map, wherein an efficient channel attention mechanism is also introduced in the decoding process; A network analysis unit, configured to input the fused feature map into an average pooling layer output plane parameter, perform a pooling convolution operation on the fused feature map to obtain a predicted segmentation map, and predict a depth map of the fused feature map by combining the transverse and longitudinal feature maps of the two-chamber and four-chamber heart chambers; The reconstruction unit is used to perform depth prediction on the pixels of the diastolic frame and the systolic frame based on the plane parameters, the predicted segmentation map and the depth map, fit the plane of the heart area and restore each plane structure to achieve three-dimensional reconstruction of the heart area.

5. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The computing module comprises: A model acquisition unit, used for acquiring a left ventricle surface model based on three-dimensional reconstruction; A slicing unit, used for dividing the left ventricle surface model into a plurality of slices along the long axis direction, wherein each slice represents a cross section of a corresponding surface of the left ventricle; A volume calculation unit, used for calculating the volume of the left ventricle surface model by using an integral method; The ejection fraction calculation unit is used to obtain the left ventricular end-systolic volume and the left ventricular end-diastolic volume from all volumes, and calculate the left ventricular ejection fraction.

6. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The auxiliary reminder module includes: A feature construction unit is used to obtain the diastolic frame and the systolic frame in each set cardiac cycle, and perform feature extraction according to preset feature indicators to construct time series features; A range definition unit, used for performing statistical analysis on the time series characteristics and setting a critical value to define a normal range; A verification unit, used for comparing the ejection fraction under each set cardiac cycle with the normal range, extracting abnormal values ​​and performing verification; The auxiliary reminder unit is used to obtain the final value based on the verification result, and perform average processing on all the final values ​​to obtain the accurate value, so as to match it with the score-auxiliary comparison table and output the auxiliary reminder.

7. The neural network-based cardiac function auxiliary analysis system according to claim 4, characterized in that: The reconstruction unit comprises: A coding subunit, configured to globally encode the predicted segmentation map and the depth map to obtain a corresponding first global sequence set and a second global sequence set; a loss determination subunit, configured to determine a loss value of each pixel according to the first global sequence set, the second global sequence set, and in combination with the plane parameters; A first locking subunit, used to lock a pixel point whose loss value is greater than a preset value and regard it as a first pixel point; A capturing subunit, used to capture the difference between the first pixel and the surrounding pixels and the direction of the difference; A first calculation subunit is used to determine a correction direction corresponding to a first pixel point according to the difference degree and the difference direction, and perform weighted fusion based on a pixel difference between the first pixel point and each neighboring pixel point under the first pixel point to obtain a first value; A second locking subunit is used to lock a pixel whose loss value is not less than a preset value and regard it as a second pixel; A variance determination subunit, configured to calculate a pixel difference between a second pixel and each of the neighboring pixels to obtain a first variance, and at the same time, calculate a second variance between all the neighboring pixels of the second pixel; a level determination subunit, configured to determine an association level based on the first variance, the second variance, the pixel value of the second pixel, and the pixel values ​​of all neighboring pixels of the second pixel; A second calculation subunit is used to match the current disparity matching the associated level from the level-disparity comparison table to obtain a second value corresponding to a second pixel point; The structure reconstruction subunit is used to map all the first values ​​and the second values ​​to corresponding position points in the heart area to obtain various planar structures.

8. The neural network-based cardiac function auxiliary analysis system according to claim 6, characterized in that: The verification unit comprises: A position determination subunit is used to determine whether the abnormal value is within the allowable error range. If not, the abnormal value is determined to have passed the verification and is removed; Otherwise, the ejection fraction under each set cardiac cycle is continuously captured, and a score curve is drawn, and the first position of the corresponding abnormal value in the score curve is locked; A conditional judgment subunit is used to determine whether the number of ejection fractions n1 on the left curve of the first position and the number of ejection fractions n2 on the right curve meet the conditional judgment subunit. as well as Then obtain the first score average value ave1 of the left curve and the second score average value ave2 of the right curve, wherein N0 represents the total number of ejection fractions existing on the score curve; The expansion subunit is used to expand the normal range according to |ave1-ave2| to obtain a new range. If the corresponding abnormal value is within the corresponding new range, it is determined that the abnormal value verification has not passed, and according to The corresponding outliers are first adjusted and retained, where (F1, F2) represents the new range; min z,xl Indicates the minimum value of the absolute value of the difference between all two adjacent ejection fractions in the left curve; min y,xl It represents the minimum value among the absolute values ​​of the difference between all two adjacent ejection fractions in the right curve; If the corresponding abnormal value is not within the corresponding new range, the abnormal value is determined to have passed the verification and is removed; Extension subunit, used if satisfied or The short side curve is locked and the short side ratio of the short side curve is The difference is obtained by randomly selecting points that satisfy the corresponding difference from the long side curve, and extending the curve at the idle end of the short side curve to obtain the extended curve; The adjustment subunit is used to determine the absolute value of the difference between the average value ave3 of the extension curve and the corresponding abnormal value. If the absolute value of the difference is within the set error range, it is determined that the abnormal value verification has failed, and the abnormal value verification is performed according to Performing a second adjustment and retaining the corresponding abnormal values, wherein (F01, F02) represents the normal range; Ny represents the number of extension points in the extension curve; represents the variance based on ave3, F01, and F02; If the absolute value of the difference is not within the set error range, it is determined that the abnormal value has passed the verification and is eliminated.

Images