A Cardiac Function Auxiliary Analysis System Based on Neural Network

Abstract

The present invention provides a neural network-based cardiac function auxiliary analysis system, belonging to the field of cardiac auxiliary technology, comprising: a preprocessing module for preprocessing cardiac ultrasound images; a waveform recognition module for performing waveform detection and multi-scale feature recognition on the waveform curve in the preprocessed ultrasound image, locating the peaks and valleys of the waveform, and identifying the diastolic and systolic frames in the cardiac ultrasound cycle; a three-dimensional reconstruction module for three-dimensionally reconstructing the cardiac region using deep learning technology; a calculation module for calculating the volume and ejection fraction of the left ventricle based on the reconstructed cardiac region; and an auxiliary reminder module for verifying and eliminating outliers in the ejection fraction obtained over multiple cycles using an outlier rejection algorithm to obtain an accurate value, and outputting the patient's cardiac function information for auxiliary reminders. This system can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

Description

Technical Field

[0001] The present invention relates to the field of cardiac assist technology, and in particular to a cardiac function assist analysis system based on a neural network. Background Art

[0002] Heart function is crucial for maintaining normal body tissue function. Cardiac insufficiency manifests as dyspnea, fatigue, exercise intolerance, and an increased risk of death. Impaired heart function, described as cardiomyopathy or heart failure, is a growing global health problem. Currently, there are multiple methods for quantifying cardiac function and diagnosing dysfunction. Among them, left ventricular ejection fraction is the most commonly used and important indicator of systolic function. The evaluation of left ventricular systolic function is crucial for diagnosing cardiovascular disease, assessing the condition, guiding treatment, and ensuring prognosis. Accurate calculation of left ventricular ejection fraction can also provide doctors with a correct diagnostic basis.

[0003] There are differences in people's assessment of ejection fraction, partly due to the irregularity of heart rate and the ambiguity of cardiac ultrasound. In the process of calculating left ventricular ejection fraction, imaging generally recommends the use of multiple continuous cardiac cycles for evaluation. If heart rate variations are found, it is inaccurate to use only one cycle of ultrasound for calculation. At the same time, due to the limitations of imaging principles during the scanning process, there may be noise artifacts and other factors that affect the image quality in cardiac ultrasound images, making the boundaries of cardiac tissue unclear, making it difficult for doctors to mark the left ventricular endothelium. Therefore, the calculation of clinical left ventricular ejection fraction is often highly empirical, which usually results in large errors and affects the accuracy of the results.

[0004] In recent years, with the rapid development of artificial intelligence (AI), computer-assisted diagnosis (CAD) has made significant progress. This technology has reduced the diagnostic errors and burden on physicians in many ways. Deep learning methods, which automatically identify and segment the left ventricular endocardial contour and calculate ejection fraction values, can effectively improve detection efficiency and accuracy and reduce the impact of human error. However, existing video-based methods for predicting left ventricular ejection fraction (LVEF) use deep learning methods that directly predict LVEF from segmented videos, which does not conform to clinical computational workflows and lacks interpretability. Furthermore, these deep learning models rely on manual labeling of static images of systole and diastole, rather than automatically detecting systole and diastole. Consequently, existing models exhibit significant errors compared to human assessments of cardiac function.

[0005] Researchers have proposed a method for processing ejection fraction data based on cardiac ultrasound videos. This method segments the left ventricle for each frame of the image, then filters the image sequence for the end-systolic and end-diastolic frames, calculates the left ventricular volume, generates the corresponding left ventricular volume parameters, and generates the end-systolic and end-diastolic volumes, thereby obtaining the ejection fraction. However, this method has some flaws. First, the screening of the end-diastolic and end-systolic phases depends on the model segmentation effect. If there are images with poor segmentation, it will affect the screening of key frames and thus the calculation of the indicators. Second, existing models usually only use a single cycle in the cardiac ultrasound video to calculate the ejection fraction indicator. Clinical diagnosis requires considering the ejection fraction of the case in multiple cycles to obtain an accurate value. If the case mutates, the assessment of the ejection fraction in a single cycle will be inaccurate.

[0006] In summary, various existing technologies are currently unable to provide a good auxiliary evaluation for the analysis of cardiac function.

[0007] Therefore, the present invention proposes a neural network-based cardiac function auxiliary analysis system. Summary of the Invention

[0008] The present invention provides a neural network-based cardiac function auxiliary analysis system, which is used to effectively avoid human errors by performing waveform detection on cardiac ultrasound images and combining deep learning technology for regional reconstruction, and to obtain the final value by abnormal judgment of ejection fraction in multiple cycles. It can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

[0009] The present invention provides a cardiac function auxiliary analysis system based on a neural network, comprising:

[0010] a preprocessing module, configured to obtain cardiac ultrasound images of different patients from different perspectives scanned by multiple ultrasound instruments, and to preprocess the cardiac ultrasound images;

[0011] The waveform recognition module is used to perform waveform detection and multi-scale feature recognition on the waveform curve in the pre-processed ultrasound image, locate the peak and valley of the waveform, and identify the diastolic frame and systolic frame in the cardiac ultrasound cycle;

[0012] A three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the diastolic frames and systolic frames of the heart region using deep learning technology to reconstruct the heart region;

[0013] a calculation module, used for calculating the volume and ejection fraction of the left ventricle based on the reconstructed heart area;

[0014] The auxiliary reminder module is used to verify and eliminate outliers in the ejection fraction obtained in multiple cycles through an outlier elimination algorithm to obtain an accurate value, and output the patient's cardiac function information for auxiliary reminders.

[0015] Preferably, the preprocessing includes: performing information masking, format conversion, data cleaning, data enhancement and electrocardiogram signal extraction on the cardiac ultrasound image.

[0016] Preferably, the waveform recognition module includes:

[0017] A filtering unit is used to identify the electrocardiogram signal in the preprocessed ultrasound image based on a bandpass filter, filter out high-frequency noise and low-frequency drift, and obtain a key heartbeat signal;

[0018] A double differentiation unit, configured to perform double differentiation on the key heartbeat signal, amplify the slope information of the R wave peak, and then process the differentiated signal using an adaptive smoothing technique to obtain a smoothed signal;

[0019] a nonlinear transformation unit, configured to perform nonlinear transformation on the smoothed signal to obtain an enhanced signal;

[0020] a multi-scale analysis unit, configured to integrate the enhanced signal using a multi-scale moving integration method with different window sizes to extract the slope and width information of the R wave peak to obtain heartbeat information, wherein the heartbeat information is the peak and valley values of the located waveform;

[0021] The heartbeat segmentation unit is used to perform heartbeat segmentation on the enhanced signal based on the dynamic threshold setting of the random forest model and in combination with the heartbeat information to obtain the diastolic frame and the systolic frame in the cardiac ultrasound cycle.

[0022] Preferably, the three-dimensional reconstruction module includes:

[0023] An image processing unit is used to perform image normalization processing, denoising processing, and image enhancement processing on the image of the identified diastolic frame and the systolic frame to obtain an initial image;

[0024] A scale processing unit is used to use the ResNet network structure as the backbone network and introduce a dilated convolution layer to perform scale processing on the initial image to obtain multi-scale feature representation and semantic information, wherein an efficient channel attention mechanism is introduced in each dilated convolution layer;

[0025] A decoding unit is used to decode the multi-scale feature representation and semantic information using depthwise separable convolution, and to extract multi-scale features by combining convolution kernels of different sizes. Then, through linear mapping, the spatial relationship between high-scale and low-scale features is encoded and fused to obtain a fused feature map. In this process, an efficient channel attention mechanism is also introduced;

[0026] a network analysis unit, configured to input the fused feature map into an average pooling layer output plane parameter, perform a pooling convolution operation on the fused feature map to obtain a predicted segmentation map, and predict a depth map of the fused feature map by combining the transverse and longitudinal feature maps of the two-chamber and four-chamber heart chambers;

[0027] The reconstruction unit is used to perform depth prediction on the pixels of the diastolic frame and the systolic frame based on the plane parameters, the predicted segmentation map and the depth map, fit the plane of the heart area and restore the plane structure to achieve three-dimensional reconstruction of the heart area.

[0028] Preferably, the calculation module includes:

[0029] A model acquisition unit, used for acquiring a left ventricular surface model based on three-dimensional reconstruction;

[0030] a slicing unit, configured to divide the left ventricle surface model into a plurality of slices along the long axis direction, wherein each slice represents a cross section of a corresponding surface of the left ventricle;

[0031] a volume calculation unit, configured to calculate the volume of the left ventricular surface model using an integral method;

[0032] The ejection fraction calculation unit is used to obtain the left ventricular end-systolic volume and the left ventricular end-diastolic volume from all volumes to calculate the left ventricular ejection fraction.

[0033] Preferably, the auxiliary reminder module includes:

[0034] A feature construction unit is used to obtain the diastolic frame and the systolic frame in each set cardiac cycle, and perform feature extraction according to preset feature indicators to construct time series features;

[0035] a range definition unit, configured to perform statistical analysis on the time series characteristics and set a critical value to define a normal range;

[0036] a verification unit, configured to compare the ejection fraction under each set cardiac cycle with the normal range, extract abnormal values and perform verification;

[0037] The auxiliary reminder unit is used to obtain the final value based on the verification result, and perform average processing on all the final values to obtain the accurate value, match it with the score-auxiliary comparison table, and output the auxiliary reminder.

[0038] Preferably, the reconstruction unit includes:

[0039] an encoding subunit, configured to globally encode the predicted segmentation map and the depth map to obtain a corresponding first global sequence set and a second global sequence set;

[0040] a loss determination subunit, configured to determine a loss value for each pixel based on the first global sequence set, the second global sequence set, and the plane parameters;

[0041] A first locking subunit, configured to lock a pixel whose loss value is greater than a preset value and regard it as a first pixel;

[0042] a capturing subunit, configured to capture the difference between the first pixel and surrounding pixels and the direction of the difference;

[0043] a first calculation subunit, configured to determine a correction direction corresponding to a first pixel point according to the difference degree and the difference direction, and perform weighted fusion based on a pixel difference between the first pixel point and each of the neighboring pixels below the first pixel point to obtain a first value;

[0044] A second locking subunit is used to lock a pixel whose loss value is not less than a preset value and regard it as a second pixel;

[0045] a variance determination subunit, configured to calculate a pixel difference between the second pixel and each of the neighboring pixels to obtain a first variance, and simultaneously calculate a second variance between all the neighboring pixels of the second pixel;

[0046] a level determination subunit, configured to determine an association level based on the first variance, the second variance, the pixel value of the second pixel, and the pixel values of all neighboring pixels of the second pixel;

[0047] a second calculation subunit, configured to match a current disparity matching the associated level from a level-disparity comparison table to obtain a second value corresponding to a second pixel point;

[0048] The structure reconstruction subunit is used to map all the first values and the second values to corresponding position points in the heart area to obtain various planar structures.

[0049] Preferably, the verification unit includes:

[0050] A position judgment subunit is used to judge whether the abnormal value is within the allowable error range. If not, the abnormal value is judged to have passed the verification and is eliminated;

[0051] Otherwise, the ejection fraction in each set cardiac cycle is continuously captured, and a score curve is drawn, and the first position of the corresponding abnormal value in the score curve is locked;

[0052] A conditional judgment subunit is configured to determine whether the ejection fraction number n1 on the left curve and the ejection fraction number n2 on the right curve of the first position meet the following conditions: as well as Then obtain the first score average value ave1 of the left curve and the second score average value ave2 of the right curve, wherein N0 represents the total number of ejection fractions existing on the score curve;

[0053] The expansion subunit is used to expand the normal range according to |ave1-ave2| to obtain a new range. If the corresponding abnormal value is within the corresponding new range, it is determined that the abnormal value verification has failed, and the abnormal value verification is performed according to The corresponding outliers are first adjusted and retained, where (F1, F2) represents the new range; min z,xl Indicates the minimum value of the absolute value of the difference between all two adjacent ejection fractions in the left curve; min y,xl It represents the minimum value among the absolute values of the difference between all two adjacent ejection fractions in the right curve;

[0054] If the corresponding outlier value is not within the corresponding new range, the outlier value is determined to have passed the verification and is removed;

[0055] Extension subunit, used if satisfied or The short side curve is locked and the short side ratio of the short side curve is The difference between the two is obtained by randomly selecting points that satisfy the corresponding difference from the long side curve, and extending the curve at the idle end of the short side curve to obtain the extended curve;

[0056] The adjustment subunit is used to determine the absolute value of the difference between the average value ave3 of the extension curve and the corresponding abnormal value. If the absolute value of the difference is within the set error range, it is determined that the abnormal value verification has failed, and the abnormal value verification is carried out according to Perform a second adjustment and retain the corresponding abnormal values, wherein (F01, F02) represents the normal range; Ny represents the number of extension points in the extension curve; represents the variance based on ave3, F01, and F02;

[0057] If the absolute value of the difference is not within the set error range, the abnormal value is determined to have passed the verification and is removed. Compared with the prior art, the beneficial effects of this application are as follows:

[0058] By performing waveform detection on cardiac ultrasound images and combining deep learning technology for regional reconstruction, human errors can be effectively avoided, and abnormal ejection fractions of multiple cycles can be judged to obtain the final value. This can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

[0059] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0062] Figure 1 1 is a structural diagram of a neural network-based cardiac function auxiliary analysis system according to an embodiment of the present invention;

[0063] Figure 2 This is a structural diagram of a waveform recognition module in an embodiment of the present invention;

[0064] Figure 3 3D reconstruction module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0066] The present invention provides a cardiac function auxiliary analysis system based on neural network, such as Figure 1 Shown, including:

[0067] a preprocessing module, configured to obtain cardiac ultrasound images of different patients from different perspectives scanned by multiple ultrasound instruments, and to preprocess the cardiac ultrasound images;

[0068] The waveform recognition module is used to perform waveform detection and multi-scale feature recognition on the waveform curve in the pre-processed ultrasound image, locate the peak and valley of the waveform, and identify the diastolic frame and systolic frame in the cardiac ultrasound cycle;

[0069] A three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the diastolic frames and systolic frames of the heart region using deep learning technology to reconstruct the heart region;

[0070] a calculation module, used for calculating the volume and ejection fraction of the left ventricle based on the reconstructed heart area;

[0071] The auxiliary reminder module is used to verify and eliminate outliers in the ejection fraction obtained in multiple cycles through an outlier elimination algorithm to obtain an accurate value, and output the patient's cardiac function information for auxiliary reminders.

[0072] Preferably, the preprocessing includes: performing information masking, format conversion, data cleaning, data enhancement and electrocardiogram signal extraction on the cardiac ultrasound image.

[0073] In this embodiment, cardiac ultrasound images are collected from healthy people and different patients, and the images include double-chamber and four-chamber images.

[0074] In this embodiment, the left ventricular endocardial contour is annotated to create a cardiac ultrasound dataset for model training. Public cardiac ultrasound image datasets can also be used for pre-training to improve the generalization of the model.

[0075] In this embodiment, the continuous ECG signal in the ultrasound image typically includes multiple cardiac cycles. Before waveform detection is performed on the ECG signal, the continuous ECG signal needs to be segmented into multiple cardiac cycles to facilitate the subsequent waveform detection process. In the present invention, the cardiac signal is first segmented. Subsequently, waveform detection methods (such as template matching-based waveform detection and neural network-based waveform detection) are used to detect the end-systole and end-diastole of the cardiac ultrasound cycle.

[0076] In this embodiment, the backbone network used in the 3D reconstruction process can be ResNet, GoogleNet, U-Net, etc., and the network optimization can be void convolution, residual connection, pyramid pooling, etc. to improve the learning ability of the model.

[0077] In this embodiment, the outlier elimination algorithm is used to set a normal range for the ejection fraction and determine whether the fraction is within the range. If not, it is considered an outlier.

[0078] In this embodiment, the accurate value may be the average value of all non-eliminated values.

[0079] In this embodiment, information masking is performed on the collected cardiac ultrasound image to remove sensitive information and irrelevant information in the image, such as patient information and timestamps, to improve the generalization ability of the model.

[0080] In this embodiment, format conversion: ultrasound images in different formats are converted into AVI format to facilitate subsequent model training and testing.

[0081] In this embodiment, data cleaning: the collected cardiac ultrasound images may contain noise and artifacts, which may affect the model recognition effect. This part of the data is cleaned to improve the data quality.

[0082] In this embodiment, data enhancement: operations such as rotation, translation, and scaling are performed on the data to increase the amount and diversity of the data and improve the robustness of the model.

[0083] In this embodiment, the ECG signal extraction is as follows: the pressure curve of the cardiac ultrasound is extracted at a certain spatial sampling interval, and multi-channel ECG sequence data is constructed according to the sampling frequency and image width.

[0084] The beneficial effects of the above technical solution are: by performing waveform detection on cardiac ultrasound images and combining deep learning technology for regional reconstruction, human errors can be effectively avoided, and abnormal ejection fractions of multiple cycles can be judged to obtain the final value, which can accurately reflect the patient's cardiac function status, provide a precise basis for auxiliary reminders, and provide effective support for clinical diagnosis.

[0085] The present invention provides a cardiac function auxiliary analysis system based on neural network, wherein the waveform recognition module is as follows: Figure 2 Shown, including:

[0086] A filtering unit is used to identify the electrocardiogram signal in the preprocessed ultrasound image based on a bandpass filter, filter out high-frequency noise and low-frequency drift, and obtain a key heartbeat signal;

[0087] A double differentiation unit, configured to perform double differentiation on the key heartbeat signal, amplify the slope information of the R wave peak, and then process the differentiated signal using an adaptive smoothing technique to obtain a smoothed signal;

[0088] a nonlinear transformation unit, configured to perform nonlinear transformation on the smoothed signal to obtain an enhanced signal;

[0089] a multi-scale analysis unit, configured to integrate the enhanced signal using a multi-scale moving integration method with different window sizes to extract the slope and width information of the R wave peak to obtain heartbeat information, wherein the heartbeat information is the peak and valley values of the located waveform;

[0090] The heartbeat segmentation unit is used to perform heartbeat segmentation on the enhanced signal based on the dynamic threshold setting of the random forest model and in combination with the heartbeat information to obtain the diastolic frame and the systolic frame in the cardiac ultrasound cycle.

[0091] In this embodiment, a modified Pan-Tompkins algorithm is used to segment heartbeat signals. The Pan-Tompkins algorithm is a classic R-wave peak detection algorithm widely used in electrocardiogram (ECG) signal processing. This invention improves the bandpass filter and moving integration method to enhance the accuracy and robustness of heartbeat segmentation.

[0092] In this embodiment, the bandpass filter analyzes the signal's spectral characteristics using a short-time Fourier transform (STFT) and dynamically adjusts the filter's cutoff frequency accordingly. This adaptive strategy intelligently adjusts filter parameters based on the instantaneous characteristics of the ECG signal, identifying and retaining critical heartbeat information while effectively suppressing high-frequency noise and low-frequency drift.

[0093] In this embodiment, the adaptive smoothing technology refers to the use of an adaptive filtering window based on the instantaneous frequency characteristics of the signal. The window can dynamically adjust its size and shape according to the instantaneous frequency of the signal, thereby maintaining signal smoothness.

[0094] In this embodiment, a multi-scale moving integration method is used to integrate the signal using different window sizes to extract the slope and width information of the R wave. The integration results of different scales are fused to obtain more accurate heartbeat information. The multi-scale integration formula is as follows:

[0095]

[0096] Among them, N k is the window size of the kth scale.

[0097] In this embodiment, a dynamic threshold setting method based on a random forest model is employed during the threshold processing phase. The training dataset for the random forest model contains both normal and abnormal ECG signals to improve the model's generalization capabilities. This method dynamically adjusts the threshold based on historical data and real-time signal characteristics. Specifically, the R-wave peak characteristics in the historical data are analyzed to determine the direction of threshold adjustment. The threshold is then adjusted based on current signal characteristics, such as slope and amplitude changes. When the signal characteristic change exceeds the dynamic threshold, the R-wave peak position is determined. The heartbeat location and window size are then determined based on the sampling rate to achieve accurate heartbeat segmentation.

[0098] In this embodiment, a template is dynamically generated based on the patient's electrocardiogram characteristics, such as heart rate, QRS wave width, QRS wave amplitude, ST segment height, etc. Template generation takes into account individual differences, such as heart rate changes and electrocardiogram waveform variations.

[0099] In this embodiment, an adaptive template matching algorithm is used. The matching parameters include the time window width and the template displacement. These parameters are adjusted in real time according to the signal characteristics to adapt to the changes in different ECG waveforms. The matching score is calculated by performing template matching in the image. The matching score function is as follows:

[0100]

[0101] Where I is the image, T is the template, x, y are the starting coordinates of the search in the image, and i, j are the pixel indices in the template, which are used to traverse every pixel in the template.

[0102] In this embodiment, signal features (such as shape, energy, and frequency) are extracted at different scales and multi-scale feature fusion is performed to improve the accuracy of waveform detection. Through the matching process, the peaks and valleys of the cardiac signal are identified, that is, the diastolic and systolic frames within the cardiac cycle.

[0103] The beneficial effects of the above technical solution are as follows: Based on an improved Pan-Tompkins beat segmentation and waveform detection method using multi-scale template matching, the traditional Pan-Tompkins beat detection algorithm is optimized to improve the accuracy and robustness of beat segmentation. The use of multi-scale analysis technology combined with template matching can adapt to ECG waveform variations in different patients and under different conditions. Peak detection automatically identifies the end-diastole and end-systole phases of the cardiac cycle, saving the time required to manually capture key frames.

[0104] The present invention provides a cardiac function auxiliary analysis system based on neural network, wherein the three-dimensional reconstruction module is as follows: Figure 3 Shown, including:

[0105] An image processing unit is used to perform image normalization processing, denoising processing, and image enhancement processing on the image of the identified diastolic frame and the systolic frame to obtain an initial image;

[0106] A scale processing unit is used to use the ResNet network structure as the backbone network and introduce a dilated convolution layer to perform scale processing on the initial image to obtain multi-scale feature representation and semantic information, wherein an efficient channel attention mechanism is introduced in each dilated convolution layer;

[0107] A decoding unit is used to decode the multi-scale feature representation and semantic information using depthwise separable convolution, and to extract multi-scale features by combining convolution kernels of different sizes. Then, through linear mapping, the spatial relationship between high-scale and low-scale features is encoded and fused to obtain a fused feature map. In this process, an efficient channel attention mechanism is also introduced;

[0108] a network analysis unit, configured to input the fused feature map into an average pooling layer output plane parameter, perform a pooling convolution operation on the fused feature map to obtain a predicted segmentation map, and predict a depth map of the fused feature map by combining the transverse and longitudinal feature maps of the two-chamber and four-chamber heart chambers;

[0109] The reconstruction unit is used to perform depth prediction on the pixels of the diastolic frame and the systolic frame based on the plane parameters, the predicted segmentation map and the depth map, fit the plane of the heart area and restore the plane structure to achieve three-dimensional reconstruction of the heart area.

[0110] In this embodiment, image normalization involves scaling image pixel values to a specific range to facilitate network training. Denoising involves applying a bilateral filter to remove random noise from the image while preserving edge information. Image enhancement involves using histogram equalization to enhance image contrast and highlight cardiac structural details. Depthwise separable convolution is introduced to perform preliminary feature extraction on the image, reducing computational effort while preserving key information. Depthwise separable convolution decomposes the convolution operation into depthwise convolution and pointwise convolution, respectively extracting spatial and channel-wise features.

[0111] In this embodiment, the ResNet network structure is used as the backbone network, and a dilated convolution layer is introduced into it. The expansion rate of the dilated convolution layer gradually increases at different levels to form a multi-scale dilated convolution module to extract features of different scales. An efficient channel attention mechanism (ECA) module is added after each dilated convolution layer to effectively capture the dependencies between channels through a lightweight adaptive average pooling operation and one-dimensional convolution without the need for a large number of parameters. The ECA module can adaptively adjust the weight of each channel and enhance the network's feature representation of the cardiac structure. A pyramid pooling layer is introduced at the end of the encoder to perform pooling operations of different scales to obtain multi-scale feature representation and semantic information.

[0112] In this embodiment, depth-wise separable convolution is used to decode features, and convolution kernels of different sizes are combined to extract multi-scale features. The ECA module is also introduced in the decoder to ensure that effective attention between channels can be maintained during feature upsampling. Convolution kernels of different sizes are used to decode features. Small-sized convolution kernels (such as 3x3) are used to extract low-scale features, and large-sized convolution kernels (such as 7x7) are used to extract high-scale features. Through linear mapping, the spatial relationship between high and low scale features is encoded and fused to obtain a fused feature map.

[0113] In this embodiment, the output part of the deep learning network includes three branches. One branch outputs plane parameters including plane normal information and offset after the fused feature map passes through the average pooling layer. Another branch predicts the segmentation map by performing pooling, convolution and other operations on the fused feature map. The third branch combines the horizontal and vertical feature maps of the two-chamber and four-chamber to predict the depth map, inputs the horizontal two-chamber feature map, and jointly predicts the depth map based on the vertical four-chamber and horizontal two-chamber feature maps.

[0114] The beneficial effects of the above technical solution are: by reconstructing the heart area through three-dimensional reconstruction of single-plane cardiac ultrasound images, the fusion of void convolution and attention mechanism can obtain a larger receptive field and improve the recognition and reconstruction capabilities of important cardiac structures.

[0115] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the calculation module comprises:

[0116] A model acquisition unit, used for acquiring a left ventricular surface model based on three-dimensional reconstruction;

[0117] a slicing unit, configured to divide the left ventricle surface model into a plurality of slices along the long axis direction, wherein each slice represents a cross section of a corresponding surface of the left ventricle;

[0118] a volume calculation unit, configured to calculate the volume of the left ventricular surface model using an integral method;

[0119] The ejection fraction calculation unit is used to obtain the left ventricular end-systolic volume and the left ventricular end-diastolic volume from all volumes to calculate the left ventricular ejection fraction.

[0120] In this embodiment, the volume of the left ventricle surface model is calculated as follows:

[0121]

[0122] Among them, A i is the area of a single slice, and Δz represents the distance between slices.

[0123] In this embodiment, the left ventricular ejection fraction (LVEF) is calculated as follows:

[0124]

[0125] Among them, left ventricular end-systolic volume (ESV) and left ventricular end-diastolic volume (EDV).

[0126] The beneficial effect of the above technical solution is that the left ventricular ejection fraction index calculated by the integral method provides reliable data support for clinical diagnosis.

[0127] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the auxiliary reminder module comprises:

[0128] A feature construction unit is used to obtain the diastolic frame and the systolic frame in each set cardiac cycle, and perform feature extraction according to preset feature indicators to construct time series features;

[0129] a range definition unit, configured to perform statistical analysis on the time series characteristics and set a critical value to define a normal range;

[0130] a verification unit, configured to compare the ejection fraction under each set cardiac cycle with the normal range, extract abnormal values and perform verification;

[0131] The auxiliary reminder unit is used to obtain the final value based on the verification result, and perform average processing on all the final values to obtain the accurate value, match it with the score-auxiliary comparison table, and output the auxiliary reminder.

[0132] In this embodiment, the time series features include, for example, heart rate variability, cycle length, time interval between ventricular contraction and relaxation, and the preset feature indicators are related to the heart rate variability, cycle length, time interval between ventricular contraction and relaxation, in order to obtain specific results under the indicators.

[0133] In this embodiment, the time feature sequence is input into the feature analysis model to obtain two critical values, F01 and F02, as the normal range. The feature analysis model is obtained by training the neural network model using different time feature sequences and the analysis results of the time feature sequences by cardiology experts (the boundary range of the ejection fraction) as samples. Therefore, the two boundary values can be directly obtained, and then the normal range (F01, F02) can be obtained.

[0134] In this embodiment, an abnormal value refers to a value where the ejection fraction is not within the normal range.

[0135] In this embodiment, the purpose of verification is to determine whether the outlier needs to be removed.

[0136] In this embodiment, the score-auxiliary comparison table contains different accurate values and auxiliary information of the cardiac function conditions corresponding to the values, which are preset.

[0137] The beneficial effects of the above technical solution are: performing time feature comparison on the ejection fraction identified in each cardiac cycle to identify and eliminate outliers, and verifying the identified outliers, thereby improving the accuracy and reliability of ejection fraction calculation and ensuring the effectiveness of post-assisted reminders.

[0138] The present invention provides a cardiac function auxiliary analysis system based on a neural network, wherein the reconstruction unit comprises:

[0139] an encoding subunit, configured to globally encode the predicted segmentation map and the depth map to obtain a corresponding first global sequence set and a second global sequence set;

[0140] a loss determination subunit, configured to determine a loss value for each pixel based on the first global sequence set, the second global sequence set, and the plane parameters;

[0141] A first locking subunit, configured to lock a pixel whose loss value is greater than a preset value and regard it as a first pixel;

[0142] a capturing subunit, configured to capture the difference between the first pixel and surrounding pixels and the direction of the difference;

[0143] a first calculation subunit, configured to determine a correction direction corresponding to a first pixel point according to the difference degree and the difference direction, and perform weighted fusion based on a pixel difference between the first pixel point and each of the neighboring pixels below the first pixel point to obtain a first value;

[0144] A second locking subunit is used to lock a pixel whose loss value is not less than a preset value and regard it as a second pixel;

[0145] a variance determination subunit, configured to calculate a pixel difference between the second pixel and each of the neighboring pixels to obtain a first variance, and simultaneously calculate a second variance between all the neighboring pixels of the second pixel;

[0146] a level determination subunit, configured to determine an association level based on the first variance, the second variance, the pixel value of the second pixel, and the pixel values of all neighboring pixels of the second pixel;

[0147] a second calculation subunit, configured to match a current disparity matching the associated level from a level-disparity comparison table to obtain a second value corresponding to a second pixel point;

[0148] The structure reconstruction subunit is used to map all the first values and the second values to corresponding position points in the heart area to obtain various planar structures.

[0149] In this embodiment, global encoding refers to encoding the grayscale value of each pixel in the corresponding image, thereby obtaining a sequence set based on the predicted segmentation map and the depth map.

[0150] In this embodiment, the loss value = ((the grayscale value of the same pixel based on the first global sequence set + the grayscale value based on the second global sequence set) / 2-standard value) + the offset in the plane parameter × cos (the angle between the normal vector of the plane parameter and the horizontal plane), where the standard value is the pixel value under ideal conditions, and the ideal condition refers to the image being undistorted, noise-free, etc.

[0151] In this embodiment, the preset value is set in advance and is generally 6.

[0152] In this embodiment, the difference refers to the absolute value of the image gradient between each first pixel point and the corresponding neighboring pixel points.

[0153] In this embodiment, the difference direction refers to whether there is a difference between the first pixel point and the corresponding neighboring pixel points in the horizontal direction or in the vertical direction.

[0154] For example, the difference is: a1, the difference direction is: vertical;

[0155] For example, the difference is a2, and the difference direction is: horizontal;

[0156] At this time, the correction direction is: max(a1,a2) determines the corresponding correction direction.

[0157] In this embodiment, the pixel difference refers to the difference in grayscale values.

[0158] In this embodiment, the first value=the pixel value of the corresponding pixel+G(correction direction, pixel difference with each neighboring pixel, weighting coefficient of each neighboring pixel).

[0159] G (correction direction, pixel difference with each neighboring pixel point, weighting coefficient of each neighboring pixel point) is a fine-tuning function, and the corresponding fine-tuning amount can be matched from the function database. The function database contains fine-tuning amounts based on different correction directions, different pixel difference combinations, and different weighting coefficient combinations, which can be used directly. Among them, the weighting coefficient = pixel difference of the corresponding neighboring pixel point / sum of the pixel differences of all neighboring pixels.

[0160] In this embodiment, the first variance=the variance of all pixel differences between the second pixel point and each of the neighboring pixels.

[0161] In this embodiment, the second variance=the variance of all neighboring pixels of the second pixel.

[0162] In this embodiment, if the difference between the average value of pixel values of all neighboring pixels of the second pixel and the pixel value of the second pixel is less than the sum of the first variance and the second variance, the correlation level is considered to be 0.

[0163] Otherwise, the association level is considered to be 1.

[0164] In this embodiment, the level-disparity comparison table contains disparities matching corresponding levels and is preset.

[0165] In this embodiment, the second value=the grayscale value corresponding to the second pixel+the parallax.

[0166] The beneficial effect of the above technical solution is: grayscale value encoding of the two images is performed to determine the loss value, and then the loss value under different situations is discussed through classification to adjust the value of the corresponding pixel point, thereby ensuring the reliability of the pixel point value and providing a data basis for subsequent reconstruction.

[0167] The present invention provides a neural network-based cardiac function auxiliary analysis system, wherein the verification unit comprises:

[0168] A position judgment subunit is used to judge whether the abnormal value is within the allowable error range. If not, the abnormal value is judged to have passed the verification and is eliminated;

[0169] Otherwise, the ejection fraction in each set cardiac cycle is continuously captured, and a score curve is drawn, and the first position of the corresponding abnormal value in the score curve is locked;

[0170] A conditional judgment subunit is configured to determine whether the ejection fraction number n1 on the left curve and the ejection fraction number n2 on the right curve of the first position meet the following conditions: as well as Then obtain the first score average value ave1 of the left curve and the second score average value ave2 of the right curve, wherein N0 represents the total number of ejection fractions existing on the score curve;

[0171] The expansion subunit is used to expand the normal range according to |ave1-ave2| to obtain a new range. If the corresponding abnormal value is within the corresponding new range, it is determined that the abnormal value verification has failed, and the abnormal value verification is performed according to The corresponding outliers are first adjusted and retained, where (F1, F2) represents the new range; min z,xl Indicates the minimum value of the absolute value of the difference between all two adjacent ejection fractions in the left curve; min y,xl It represents the minimum value among the absolute values of the difference between all two adjacent ejection fractions in the right curve;

[0172] If the corresponding outlier value is not within the corresponding new range, the outlier value is determined to have passed the verification and is removed;

[0173] Extension subunit, used if satisfied or The short side curve is locked and the short side ratio of the short side curve is The difference between the two is obtained by randomly selecting points that satisfy the corresponding difference from the long side curve, and extending the curve at the idle end of the short side curve to obtain the extended curve;

[0174] The adjustment subunit is used to determine the absolute value of the difference between the average value ave3 of the extension curve and the corresponding abnormal value. If the absolute value of the difference is within the set error range, it is determined that the abnormal value verification has failed, and the abnormal value verification is carried out according to Perform a second adjustment and retain the corresponding abnormal values, wherein (F01, F02) represents the normal range; Ny represents the number of extension points in the extension curve; represents the variance based on ave3, F01, and F02;

[0175] If the absolute value of the difference is not within the set error range, it is determined that the abnormal value has passed the verification and is eliminated.

[0176] In this embodiment, for example, the outlier is located at point 2. At this time, the left curve includes point 1, and the right curve includes points 3, 4, 5, 6, 7, 8, 9, and 10. At this time, the left curve is the short-side curve, and the right curve is the long-side curve.

[0177] In this embodiment, the short side ratio is The difference is: At this time, the number of randomly selected points is: Then round up. At this time, randomly select 2 points, that is, Ny=2, and add them to the left side of the short side curve to achieve extension.

[0178] In this embodiment, the average value ave3 of the extension curve is the average value of the scores of all points on the extension curve.

[0179] In this embodiment, the error range is preset to (0, u01).

[0180] In this embodiment, the second adjustment of the outlier value is: Among them, when the difference between the average value ave3 and the corresponding outlier is less than 0, A2 takes the value of 1, otherwise, A2 takes the value of -1.

[0181] In this embodiment, the allowable error range refers to the outer range of the left boundary and the outer range of the right boundary of the normal range, for example, (F01-u01, F01), (F02, F02+u01), where u01 is the allowable error.

[0182] In this embodiment, continuous capture refers to the scores at the same designated period time under different set periods, which facilitates subsequent comparative analysis, that is, ensures the uniformity of the data basis.

[0183] In this embodiment, the first position refers to the position where the abnormal value appears on the curve.

[0184] In this embodiment, for example, the score curve exists: point 1, point 2, point 3, point 4, point 5, point 6, point 7, point 8, point 9 and point 10, among which the outlier is located at point 3. At this time, the left curve includes point 1 and point 2, and the right curve includes point 4, point 5, point 6, point 7, point 8, point 9 and point 10, that is, n1=2, n2=7, N0=10.

[0185] In this embodiment, ave1 is the average score of points 1 and 2, and ave2 is the average score of points 4 to 10.

[0186] In this embodiment, the new range is (F01-|ave1-ave2|, F02+|ave1-ave2|).

[0187] In this embodiment, the first adjustment of the outlier value is: If the outlier is greater than the average of ave1 and ave2, then A1 is -1; otherwise, A1 is 1.

[0188] The beneficial effect of the above technical solution is: by plotting a curve of the continuously obtained ejection fraction, the left and right sides of the curve are analyzed, thereby providing an auxiliary theoretical basis for adjusting the outliers, effectively realizing a reasonable judgment on whether to retain the outliers, and providing a reliable basis for obtaining accurate values.

[0189] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A cardiac function auxiliary analysis system based on neural network, characterized in that: include: a preprocessing module, configured to obtain cardiac ultrasound images of different patients from different perspectives scanned by multiple ultrasound instruments, and to preprocess the cardiac ultrasound images; The waveform recognition module is used to perform waveform detection and multi-scale feature recognition on the waveform curve in the pre-processed ultrasound image, locate the peak and valley of the waveform, and identify the diastolic frame and systolic frame in the cardiac ultrasound cycle; A three-dimensional reconstruction module is used to perform three-dimensional reconstruction of the diastolic frames and systolic frames of the heart region using deep learning technology to reconstruct the heart region; a calculation module, used for calculating the volume and ejection fraction of the left ventricle based on the reconstructed heart area; The auxiliary reminder module is used to verify and eliminate outliers in the ejection fraction obtained in multiple cycles through an outlier elimination algorithm to obtain an accurate value, and output the patient's cardiac function information for auxiliary reminders; Wherein, the auxiliary reminder module includes: A feature construction unit is used to obtain the diastolic frame and the systolic frame in each set cardiac cycle, and perform feature extraction according to preset feature indicators to construct time series features; a range definition unit, configured to perform statistical analysis on the time series characteristics and set a critical value to define a normal range; a verification unit, configured to compare the ejection fraction under each set cardiac cycle with the normal range, extract abnormal values and perform verification; An auxiliary reminder unit is used to obtain the final value based on the verification result, and average all the final values to obtain the accurate value, match it with the score-auxiliary comparison table, and output an auxiliary reminder; Wherein, the verification unit includes: A position judgment subunit is used to judge whether the abnormal value is within the allowable error range. If not, the abnormal value is judged to have passed the verification and is eliminated; Otherwise, the ejection fraction in each set cardiac cycle is continuously captured, and a score curve is drawn, and the first position of the corresponding abnormal value in the score curve is locked; A conditional judgment subunit is configured to determine whether the ejection fraction number n1 on the left curve and the ejection fraction number n2 on the right curve of the first position meet the following conditions: as well as , then obtain the first fraction average value ave1 of the left curve and the second fraction average value ave2 of the right curve, wherein N0 represents the total number of ejection fractions existing on the fraction curve; Extension subunit, used to follow The normal range is expanded to obtain a new range. If the corresponding abnormal value is within the corresponding new range, it is determined that the abnormal value verification has failed, and the abnormal value verification is carried out according to The corresponding outliers are first adjusted and retained, wherein, Indicates a new range; It represents the minimum value among the absolute values of the difference between all two adjacent ejection fractions in the left curve; It represents the minimum value among the absolute values of the difference between all two adjacent ejection fractions in the right curve; If the corresponding outlier value is not within the corresponding new range, the outlier value is determined to have passed the verification and is removed; Extension subunit, used if satisfied , the short side curve is locked and the short side ratio of the short side curve is The difference between the two is obtained by randomly selecting points that satisfy the corresponding difference from the long side curve, and extending the curve at the idle end of the short side curve to obtain the extended curve; The adjustment subunit is used to determine the absolute value of the difference between the average value ave3 of the extension curve and the corresponding abnormal value. If the absolute value of the difference is within the set error range, it is determined that the abnormal value verification has failed, and the abnormal value verification is carried out according to The corresponding outliers are adjusted and retained for the second time, wherein: Indicates normal range; represents the number of extension points in the extension curve; Indicates based on variance; If the absolute value of the difference is not within the set error range, it is determined that the abnormal value has passed the verification and is eliminated.

2. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The preprocessing includes: performing information masking, format conversion, data cleaning, data enhancement and electrocardiogram signal extraction on the cardiac ultrasound image.

3. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The waveform recognition module includes: A filtering unit is used to identify the electrocardiogram signal in the preprocessed ultrasound image based on a bandpass filter, filter out high-frequency noise and low-frequency drift, and obtain a key heartbeat signal; A double differentiation unit, configured to perform double differentiation on the key heartbeat signal, amplify the slope information of the R wave peak, and then process the differentiated signal using an adaptive smoothing technique to obtain a smoothed signal; a nonlinear transformation unit, configured to perform nonlinear transformation on the smoothed signal to obtain an enhanced signal; a multi-scale analysis unit, configured to integrate the enhanced signal using a multi-scale moving integration method with different window sizes to extract the slope and width information of the R wave peak to obtain heartbeat information, wherein the heartbeat information is the peak and valley values of the located waveform; The heartbeat segmentation unit is used to perform heartbeat segmentation on the enhanced signal based on the dynamic threshold setting of the random forest model and in combination with the heartbeat information to obtain the diastolic frame and the systolic frame in the cardiac ultrasound cycle.

4. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The three-dimensional reconstruction module includes: An image processing unit is used to perform image normalization processing, denoising processing, and image enhancement processing on the image of the identified diastolic frame and the systolic frame to obtain an initial image; A scale processing unit is used to use the ResNet network structure as the backbone network and introduce a dilated convolution layer to perform scale processing on the initial image to obtain multi-scale feature representation and semantic information, wherein an efficient channel attention mechanism is introduced in each dilated convolution layer; A decoding unit is used to decode the multi-scale feature representation and semantic information using depthwise separable convolution, and to extract multi-scale features by combining convolution kernels of different sizes. Then, through linear mapping, the spatial relationship between high-scale and low-scale features is encoded and fused to obtain a fused feature map. In this process, an efficient channel attention mechanism is also introduced; a network analysis unit, configured to input the fused feature map into an average pooling layer output plane parameter, perform a pooling convolution operation on the fused feature map to obtain a predicted segmentation map, and predict a depth map of the fused feature map by combining the transverse and longitudinal feature maps of the two-chamber and four-chamber heart chambers; The reconstruction unit is used to perform depth prediction on the pixels of the diastolic frame and the systolic frame based on the plane parameters, the predicted segmentation map and the depth map, fit the plane of the heart area and restore the plane structure to achieve three-dimensional reconstruction of the heart area.

5. The neural network-based cardiac function auxiliary analysis system according to claim 1, characterized in that: The computing module includes: A model acquisition unit, used for acquiring a left ventricular surface model based on three-dimensional reconstruction; a slicing unit, configured to divide the left ventricle surface model into a plurality of slices along the long axis direction, wherein each slice represents a cross section of a corresponding surface of the left ventricle; a volume calculation unit, configured to calculate the volume of the left ventricular surface model using an integral method; The ejection fraction calculation unit is used to obtain the left ventricular end-systolic volume and the left ventricular end-diastolic volume from all volumes to calculate the left ventricular ejection fraction.

6. The neural network-based cardiac function auxiliary analysis system according to claim 4, characterized in that: The reconstruction unit comprises: an encoding subunit, configured to globally encode the predicted segmentation map and the depth map to obtain a corresponding first global sequence set and a second global sequence set; a loss determination subunit, configured to determine a loss value for each pixel based on the first global sequence set, the second global sequence set, and the plane parameters; A first locking subunit, configured to lock a pixel whose loss value is greater than a preset value and regard it as a first pixel; a capturing subunit, configured to capture the difference between the first pixel and surrounding pixels and the direction of the difference; a first calculation subunit, configured to determine a correction direction corresponding to a first pixel point according to the difference degree and the difference direction, and perform weighted fusion based on a pixel difference between the first pixel point and each of the neighboring pixels below the first pixel point to obtain a first value; A second locking subunit is used to lock a pixel whose loss value is not less than a preset value and regard it as a second pixel; a variance determination subunit, configured to calculate a pixel difference between the second pixel and each of the neighboring pixels to obtain a first variance, and simultaneously calculate a second variance between all the neighboring pixels of the second pixel; a level determination subunit, configured to determine an association level based on the first variance, the second variance, the pixel value of the second pixel, and the pixel values of all neighboring pixels of the second pixel; a second calculation subunit, configured to match a current disparity matching the associated level from a level-disparity comparison table to obtain a second value corresponding to a second pixel point; The structure reconstruction subunit is used to map all the first values and the second values to corresponding position points in the heart area to obtain various planar structures.

Images