Cardiac ultrasonic intelligent recognition system and method for myocardial ablation

Abstract

The invention discloses a cardiac ultrasound intelligent recognition system for myocardial ablation, which relates to the technical field of ultrasound medicine, and comprises a cardiac echocardiogram acquisition module, a neural network construction module, a neural network training module, a neural network recognition module and an image processing module, and is used for assisting in setting ablation parameters in an ablation process. Comprise output power, time, electrode position and the like. Wherein the cardiac echocardiogram acquisition module comprises an ultrasonic diagnosis device and an ablation instrument, compared with a traditional separated device, the cardiac echocardiogram intelligent identification system is higher in integration degree and more convenient to use, in addition, the invention further discloses a cardiac ultrasound intelligent identification method for myocardial ablation, an echocardiogram two-dimensional dynamic image of a detected object is acquired, and the detection accuracy is improved. The two-dimensional dynamic image of the echocardiogram of the detected object is input into the convolutional neural network model, the features of each section of the ultrasound image are automatically recognized and positioned, and the problem that the score deviation is large due to too strong subjective factors can be avoided by adopting the method.

Description

Technical Field

[0001] The present invention relates to the field of ultrasonic medical technology, and more particularly to a cardiac ultrasonic intelligent recognition system and method for myocardial ablation. Background Art

[0002] With the rapid development of computer technology and digital image processing technology, digital image processing technology is increasingly used in the field of automatic recognition of medical images. Its principle is to use image processing techniques such as segmentation, reconstruction, registration, and recognition on medical images acquired in different ways, so as to automatically recognize and acquire some specific medical images, providing auxiliary reference for clinicians in image recognition in clinical operations, which has very important practical significance.

[0003] Ultrasound medicine is highly practical, and traditional clinicians have many problems in their daily diagnosis and treatment activities: 1. Due to the complex structure of the heart, there are errors in the measurement of various structures of the ventricular wall by clinicians, resulting in deviations in clinical diagnosis between different doctors. 2. The number of outpatients in hospitals is large, the measurement of heart-related data is cumbersome, the work efficiency is low, and there is huge pressure on ultrasound patient examinations. 3. For the treatment of hypertrophic cardiomyopathy, ultrasound-guided percutaneous intramyocardial ventricular septal radiofrequency ablation is a promising new minimally invasive procedure developed in recent years, but this treatment method requires precise puncture of the ventricular septum on a beating heart, and the operation is extremely difficult; at the same time, due to the obvious heterogeneity of the ventricular septum in patients with myocardial hypertrophy and the complex spatial structure of the ablation thermal field, the endocardial structure needs to be clearly judged during the operation to avoid damage to the cardiac conduction system. For the above reasons, the use of digital image processing technology to realize the automatic recognition of the ventricular chambers and wall structures of the heart is of great significance to improve the consistency of clinicians in diagnosing heart diseases, improve the work efficiency of clinicians, and reduce the risks of intramyocardial ventricular septal radiofrequency ablation surgery. It is an urgent problem to be solved in the field of cardiac ultrasound.

[0004] Therefore, it is necessary to propose a cardiac ultrasound intelligent recognition system and method for myocardial ablation to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a cardiac ultrasound intelligent recognition system and method for myocardial ablation, which can help clinicians quickly identify standard sections and cardiac structures during the process of learning cardiac ultrasound, evaluate the doctors' skill operations, and point out any deficiencies and defects.

[0006] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0007] A cardiac ultrasound intelligent recognition system for myocardial ablation, comprising a cardiac ultrasound cardiogram acquisition module, for acquiring a two-dimensional dynamic image of an ultrasound cardiogram of a subject to be detected;

[0008] Neural network building module, used to build convolutional neural network models;

[0009] Neural network training module, which is used to train convolutional neural network models;

[0010] The neural network recognition module uses the trained convolutional neural network model to identify the ultrasound Figure 2 3D dynamic images are processed and identified, and the end-diastolic frame images of each standard section are automatically screened out;

[0011] The image processing module processes each frame of the image, automatically identifies, locates, and marks the mitral valve, tricuspid valve, aortic valve, and left and right atria and chambers on the image, and measures the inner diameters of the left and right atria and chambers and the thickness of each chamber wall on a standard section;

[0012] The image quality evaluation module evaluates and analyzes the input two-dimensional dynamic image of the echocardiogram, and makes a quantitative score based on whether it contains the end-diastolic frame image of the standard section required for cardiac ultrasound examination, the clarity of the image, and the fit of the measurement data.

[0013] Furthermore, the neural network training module specifically includes the following units:

[0014] A training sample acquisition unit, used for acquiring training samples;

[0015] A neural network training unit, used for inputting the training set in the training sample into the initial convolutional neural network to obtain an initial convolutional neural network model;

[0016] An output result acquisition unit, used for inputting the verification set in the training sample into the initial convolutional neural network model for processing to obtain an output result;

[0017] A result comparison unit, used to compare the output result with the verification set image label to obtain a comparison result;

[0018] A parameter optimization unit is used to optimize the parameters of the initial convolutional neural network model using the obtained comparison results to obtain a trained convolutional neural network model.

[0019] Furthermore, the neural network building module specifically includes the following units:

[0020] The convolution layer construction unit applies a preset convolution kernel according to the received echocardiographic data, performs convolution calculation according to the convolution step size, and obtains a convolution result;

[0021] The pooling layer construction unit will perform aggregate statistical processing based on the received convolution results to obtain the pooling results;

[0022] The fully connected layer construction unit combines the pooling results of the received local information to obtain the global information.

[0023] Furthermore, the image processing module specifically includes the following units:

[0024] An image preprocessing unit, used for preprocessing the ultrasound cardiogram obtained from the object to be detected to obtain a preprocessed image;

[0025] An image recognition unit, used for recognizing and extracting corresponding cardiac standard section frames from the preprocessed image according to the image features of each cardiac standard section;

[0026] An image structure positioning unit, used to determine the positions of the mitral valve, tricuspid valve, aorta, left and right atrium and ventricle cavities of the heart structures in the preprocessed image according to the image features of each standard section of the heart, and to calibrate the corresponding structures;

[0027] The image structure measurement unit is used to measure the inner diameters of the left and right atria and ventricles and the thickness of the ventricular septum of each structure of the heart in the preprocessed image according to the image features of each standard section of the heart.

[0028] Furthermore, the cardiac ultrasound cardiogram acquisition module includes an ultrasound diagnostic device and an ablation device, the lower end of the ultrasound diagnostic device is fixedly connected to a plurality of vertical poles, the upper ends of the vertical poles are fixedly connected to a circular track, and the ablation device is arranged on a mounting mechanism;

[0029] The mounting mechanism comprises a placement plate for mounting the ablation device, a first driving assembly for driving the placement plate to rotate, and a second driving assembly for driving the first driving assembly to slide on a circular track.

[0030] Furthermore, the second driving component includes an arc-shaped slider, a mounting plate, an insertion plate plugged into the side of the arc-shaped slider, and a second driving module for driving the arc-shaped slider to slide, the lower end of the arc-shaped slider is provided with a slot that can be plugged into the annular track, the side of the annular track is provided with a slide groove for the insertion plate to be plugged into, the upper half of the arc-shaped slider is provided with a through hole, the mounting plate is slidably arranged in the through hole along the up and down directions, and the driving module is arranged at the lower end of the mounting plate located in the annular track.

[0031] Furthermore, a fixing rod is fixedly connected to the lower end of the mounting plate outside the annular track, a through groove for inserting the fixing rod is opened at the end of the plug plate away from the slide groove, the lower end of the fixing rod is an inverted right-angled trapezoid, and a tension spring is fixedly connected to the upper end of the mounting plate, and the upper end of the tension spring is fixedly connected to the upper end of the arc-shaped slider.

[0032] Furthermore, the second driving module includes a first motor fixedly connected to the lower end of the mounting plate, a driving gear fixedly connected to the output shaft of the first motor, and an inner gear ring fixedly connected to the inner side of the annular track, wherein the inner gear ring is meshedly connected to the driving gear;

[0033] When the fixing rod is temporarily fixed to the mounting plate, the inner gear ring is meshed and connected with the driving gear; when the fixing rod is not temporarily fixed, the inner gear ring is not meshed with the driving gear.

[0034] Furthermore, the first driving assembly includes a circular plate, a rectangular groove provided on the circular plate, a telescopic rod fixed in the rectangular groove, a second motor fixed in the arc-shaped slider, an inner polygonal sleeve rotatably connected to the middle of the circular plate, and a sector gear fixedly connected to the outer side surface of the inner polygonal sleeve;

[0035] The placement plate is provided with a circular hole for matching the circular plate installation, and the placement plate at the upper end of the circular hole is provided with an annular tooth groove for meshing with the sector gear;

[0036] The upper end of the mounting plate is fixedly connected with an insertion rod which can be inserted into the rectangular groove, and the insertion rod is provided with a limiting groove for the telescopic rod to be inserted;

[0037] The output shaft of the second motor is fixedly connected with a polygonal shaft for fitting with the inner polygonal sleeve;

[0038] The upper end of the arc-shaped sliding block is provided with a mounting groove for mounting the second motor;

[0039] A lower ring plate is fixedly connected to the placement plate at the lower end of the circular hole, and an upper ring plate is fixedly connected to the upper end of the outer surface of the circular plate, and the upper ring plate is used to limit the lower ring plate on the arc-shaped slider;

[0040] A bandage for installing the ablation device is fixed on the placement board.

[0041] A cardiac ultrasound intelligent recognition method for myocardial ablation, comprising the above-mentioned cardiac ultrasound intelligent recognition system, comprises the following steps:

[0042] Step S1, moving the ultrasonic diagnostic device to the treatment position, using the ultrasonic diagnostic device to determine the position of the lesion, and then controlling the first motor and the second motor to adjust the position of the ablation device, and planning the path of the ablation needle or catheter;

[0043] Step S2, disinfecting and locally anesthetizing the puncture site;

[0044] Step S3: Under ultrasound guidance, the ablation needle or catheter is accurately inserted into the lesion area, the ablation instrument is started, the ablation parameters are set according to the treatment plan, and the ablation process begins. During the entire ablation process, the ultrasound diagnostic device is continuously used for monitoring to ensure that the ablation needle or catheter is positioned correctly, and the change of tissue temperature is monitored at the same time;

[0045] Step S4: Obtain the ultrasonic heart rate of the test object. Figure 2 dimensional dynamic image;

[0046] Step S5: The ultrasonic cardiogram of the test object Figure 2 The 3D dynamic image is input into the convolutional neural network model to automatically identify and locate the features of each ultrasound image, capture the end-diastolic image frames of each standard section, and mark the mitral valve, tricuspid valve, aortic valve, left and right atrioventricular cavities, measure the thickness of each atrioventricular cavity and ventricular wall and automatically mark them.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. The present invention is based on ultrasonic cardiology Figure 2 The 3D dynamic video of the cross-section can automatically identify and extract the corresponding cardiac ultrasound standard cross-section image frames through the convolutional neural network deep learning technology, identify and locate the heart valves and the structure of the heart's atrioventricular cavity, and automatically measure the inner diameter of the atrioventricular cavity and the thickness of the wall, thereby assisting in the setting of ablation parameters during the ablation process, including output power, time, electrode position, etc., avoiding the technical problem of large scoring deviations due to excessive human subjective factors.

[0049] 2. The present invention integrates the ultrasonic diagnostic device and the ablation device into one, and can automatically adjust the position of the ablation device. Compared with the traditional separate ablation device and ultrasonic diagnostic device, the operation is more convenient during the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is the image recognition flow chart of the present invention;

[0051] Figure 2 is a flow chart of the identification method of the present invention;

[0052] Figure 3 It is a schematic diagram of the structure of the present invention;

[0053] Figure 4 It is a structural schematic diagram of the convolutional neural network model in the present invention;

[0054] Figure 5 It is a training flow chart of the convolutional neural network model in the present invention;

[0055] Figure 6 is a system block diagram of the present invention;

[0056] Figure 7 is an application block diagram of the system of the present invention;

[0057] Figure 8 is a device block diagram of the system of the present invention;

[0058] Fig. 9 It is a three-dimensional schematic diagram of the structure of the cardiac echocardiogram acquisition module in the present invention;

[0059] Fig.10 It is a three-dimensional schematic diagram of the ultrasonic diagnostic device in the present invention;

[0060] Fig.11 It is a three-dimensional schematic diagram of the installation mechanism in the present invention;

[0061] Fig.12 It is a three-dimensional cross-sectional schematic diagram of the installation mechanism in the present invention;

[0062] Fig.13 It is a schematic diagram of the axial three-dimensional separation of the installation mechanism in the present invention;

[0063] Fig.14 For the present invention Fig.12 The enlarged schematic diagram at A in the middle;

[0064] Fig.15 It is a three-dimensional schematic diagram of the connection structure at the arc-shaped slider in the present invention.

[0065] Figure numerals: 1. ultrasonic diagnostic device; 2. vertical pole; 3. annular track; 4. slide groove; 5. inner gear ring; 6. arc-shaped slider; 7. slot; 8. plug plate; 9. through hole; 10. mounting plate; 11. tension spring; 12. first motor; 13. driving gear; 14. mounting groove; 15. second motor; 16. through groove; 17. polygonal shaft; 18. circular plate; 19. rectangular groove; 20. plug rod; 21. telescopic rod; 22. inner polygonal sleeve; 23. fan gear; 24. placement plate; 25. annular gear groove; 26. bandage; 27. ablation device; 28. upper ring plate; 29. ​​lower ring plate; 30. limit groove; 31. fixing rod. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] See also Figure 1-15, a cardiac ultrasound intelligent recognition system for myocardial ablation, comprising a cardiac ultrasound cardiogram acquisition module, for acquiring a two-dimensional dynamic image of an ultrasound cardiogram of a subject to be detected;

[0068] Neural network building module, used to build convolutional neural network models;

[0069] The neural network building blocks specifically include the following units:

[0070] The convolution layer construction unit applies a preset convolution kernel according to the received echocardiographic data, performs convolution calculation according to the convolution step size, and obtains a convolution result;

[0071] The pooling layer construction unit will perform aggregate statistical processing based on the received convolution results to obtain the pooling results;

[0072] The fully connected layer builds a unit that combines the pooling results of the received local information to obtain global information;

[0073] Construct an initial convolutional neural network model, which includes an input layer, a hidden layer, and an output layer.

[0074] The hidden layer includes at least three convolutional layers, at least three pooling layers, and at least two fully connected layers.

[0075] The initial convolutional neural network model further includes:

[0076] The convolution layer applies a preset number of convolution kernels according to the image data received from the input layer, performs convolution operations at a certain step size, and obtains the convolution result.

[0077] The pooling layer performs aggregation statistical calculations based on the received convolution results to obtain the pooling results.

[0078] Neural network training module, which is used to train convolutional neural network models;

[0079] The neural network training module specifically includes the following units:

[0080] A training sample acquisition unit, used for acquiring training samples;

[0081] A neural network training unit, used for inputting a training set in a training sample into an initial convolutional neural network to obtain an initial convolutional neural network model;

[0082] An output result acquisition unit is used to input the verification set in the training sample into the initial convolutional neural network model for processing to obtain an output result;

[0083] A result comparison unit is used to compare the output result with the image label of the verification set to obtain a comparison result;

[0084] The parameter optimization unit is used to optimize the parameters of the initial convolutional neural network model using the obtained comparison results to obtain a trained convolutional neural network model.

[0085] Training method of convolutional neural network model

[0086] Dynamic echocardiograms were obtained as training samples.

[0087] The acquired dynamic echocardiography training sample images were preprocessed, and the region of interest, namely ROI (region of interest), was manually marked. The training samples were divided into a training set and a validation set according to a 4:1 ratio.

[0088] The acquired dynamic echocardiogram training set is input into the initial convolutional neural network model for image processing to obtain the output results.

[0089] Compare the output result with the sample image to obtain the comparison result.

[0090] According to the comparison results, the parameters in the initial convolutional neural network model are optimized to obtain the trained convolutional neural network model.

[0091] The acquired dynamic echocardiogram verification set is input into the trained convolutional neural network model to verify the effectiveness of the trained convolutional neural network model.

[0092] Neural network recognition module uses the trained convolutional neural network model to identify echocardiograms. Figure 2 3D dynamic images are processed and identified, and the end-diastolic frame images of each standard section are automatically screened out;

[0093] The image processing module processes each frame of the image, automatically identifies, locates, and marks the mitral valve, tricuspid valve, aortic valve, and left and right atria and chambers on the image, and measures the inner diameters of the left and right atria and chambers and the thickness of each chamber wall on a standard section;

[0094] Image processing module, specifically including the following units:

[0095] An image preprocessing unit, used for preprocessing the ultrasound cardiogram obtained from the object to be detected to obtain a preprocessed image;

[0096] An image recognition unit, used for recognizing and extracting corresponding cardiac standard section frames from the preprocessed image according to the image features of each cardiac standard section;

[0097] An image structure positioning unit is used to determine the positions of the mitral valve, tricuspid valve, aorta, left and right atrium and ventricle cavities of various cardiac structures in the preprocessed image according to the image features of each standard section of the heart, and to calibrate the corresponding structures;

[0098] An image structure measurement unit, used to measure the inner diameters of the left and right atria and ventricles and the thickness of the ventricular septum of each structure of the heart in the preprocessed image according to the image features of each standard section of the heart;

[0099] The convolutional neural network model performs image preprocessing on the echocardiogram to obtain a preprocessed image.

[0100] The end-diastolic image frame images of each standard section of the heart in the preprocessed image are determined according to the preprocessed image.

[0101] The positions of the mitral valve, tricuspid valve, aortic valve, and left and right atria and chambers are automatically identified and calibrated based on the end-diastolic image frames of each standard section of the heart.

[0102] According to the structural characteristics of the mitral valve, tricuspid valve, aortic valve, left and right atria and chambers, the inner diameter and wall thickness of each atria and chamber are automatically measured to determine the quantitative evaluation index

[0103] The image quality evaluation module evaluates and analyzes the input two-dimensional dynamic image of the echocardiogram, and makes a quantitative score based on whether it contains the end-diastolic frame image of the standard section required for cardiac ultrasound examination, the clarity of the image, and the fit of the measurement data.

[0104] Specific, combined Figure 9-15 As shown, the cardiac ultrasound cardiogram acquisition module includes an ultrasound diagnostic device 1 and an ablation device 27. The ultrasound diagnostic device 1 is an ultrasound diagnostic device. The lower end of the ultrasound diagnostic device 1 is fixedly connected to a plurality of vertical poles 2. The upper end of the vertical poles 2 is fixedly connected to a circular track 3. The ablation device 27 is arranged on the mounting mechanism. The circular track 3 is convenient for the operator to push the entire ultrasound diagnostic device 1 in multiple directions when using it, and can also provide a guide for the rotation of the ablation device 27.

[0105] The mounting mechanism includes a placement plate 24 for mounting the ablation device 27, a first drive assembly for driving the placement plate 24 to rotate, and a second drive assembly for driving the first drive assembly to slide on the circular track 3, wherein the upper end surface of the placement plate 24 is higher than the upper end surface of the first drive assembly, thereby reducing interference between the ablation device 27 and the first drive assembly.

[0106] Specific, combined Figure 12-15As shown, the second driving component includes an arc-shaped slider 6, a mounting plate 10, an insert plate 8 plugged into the side of the arc-shaped slider 6, and a second driving module for driving the arc-shaped slider 6 to slide. A slot 7 that can be plugged into the annular track 3 is provided at the lower end of the arc-shaped slider 6. After being inserted into the annular track 3, the annular track 3 will limit the rotation of the arc-shaped slider 6 along the vertical plane, so that the arc-shaped slider 6 only has the freedom in the up and down directions. A slide groove 4 for the insert plate 8 to be plugged into the side of the annular track 3 is provided. After the insert plate 8 is plugged into the slide groove 4, the freedom of the arc-shaped slider 6 in the up and down directions can be limited. A through hole 9 is provided in the upper half of the arc-shaped slider 6. The mounting plate 10 is slidably set in the through hole 9 along the up and down directions. The mounting plate 10 is U-shaped as a whole, and the middle part of the mounting plate 10 faces the annular track 3, so that the driving module is set at the lower end of the mounting plate 10 located in the annular track 3.

[0107] Specifically, a fixing rod 31 is fixedly connected to the lower end of the mounting plate 10 located outside the annular track 3, and a through groove 16 for the fixing rod 31 to be inserted is provided at the end of the plug plate 8 away from the slide groove 4. The lower end of the fixing rod 31 is an inverted right-angled trapezoid. After the fixing rod 31 is inserted into the through groove 16, the position of the plug plate 8 is restricted so that the plug plate 8 can always be inserted into the slide groove 4. A tension spring 11 is fixedly connected to the upper end of the mounting plate 10, and the upper end of the tension spring 11 is fixedly connected to the upper end of the arc-shaped slider 6, thereby facilitating the resetting of the mounting plate 10 and facilitating the disassembly of the arc-shaped slider 6 as a whole.

[0108] Specifically, the second driving module includes a first motor 12 fixedly connected to the lower end of the mounting plate 10, a driving gear 13 fixedly connected to the output shaft of the first motor 12, and an inner gear ring 5 fixedly connected to the inner side of the annular track 3. The inner gear ring 5 and the driving gear 13 are meshed and connected to achieve driving of the arc-shaped slider 6, so that it slides along the annular track 3, thereby driving the ablation device 27 to move;

[0109] When installing the arc-shaped slider 6, adjust the angle of the driving gear 13. When the fixing rod 31 is temporarily fixed to the mounting plate 10, the inner gear ring 5 and the driving gear 13 are meshed and connected. When the fixing rod 31 is not inserted into the through groove 16, the inner gear ring 5 and the driving gear 13 are not meshed.

[0110] Specifically, the first driving assembly includes a circular plate 18, a rectangular groove 19 provided on the circular plate 18, a telescopic rod 21 fixed in the rectangular groove 19, a second motor 15 fixed in the arc-shaped slider 6, an inner polygonal sleeve 22 rotatably connected to the middle of the circular plate 18, and a sector gear 23 fixedly connected to the outer side surface of the inner polygonal sleeve 22;

[0111] The placement plate 24 is provided with a circular hole for matching the circular plate 18. The placement plate 24 at the upper end of the circular hole is provided with an annular tooth groove 25 for engaging the sector gear 23. When the sector gear 23 and the annular tooth groove 25 are engaged, the sector gear 23 can also limit the upward movement of the annular tooth groove 25.

[0112] The upper end of the mounting plate 10 is fixedly connected with an insertion rod 20 that can be inserted into the rectangular groove 19, and a limiting groove 30 for the telescopic rod 21 to be inserted is opened in the insertion rod 20;

[0113] The output shaft of the second motor 15 is fixedly connected to a polygonal shaft 17 for fitting with the inner polygonal sleeve 22;

[0114] The upper end of the arc-shaped slider 6 is provided with a mounting groove 14 for mounting the second motor 15;

[0115] A lower ring plate 29 is fixedly connected to the placement plate 24 at the lower end of the circular hole, and an upper ring plate 28 is fixedly connected to the upper end of the outer surface of the circular plate 18. The upper ring plate 28 is used to limit the lower ring plate 29 on the arc-shaped slider 6. Through the arrangement of the lower ring plate 29 and the sector gear 23, the placement plate 24 can be stabilized on the upper end of the arc-shaped slider 6 to ensure the stability of the placement plate 24 when rotating;

[0116] A bandage 26 for mounting the ablator 27 is fixed on the placement plate 24 . The bandage 26 may also be replaced with other fixing methods as long as the ablator 27 is fixed on the placement plate 24 .

[0117] Installation method: When installing the ablation device 27, first install the arc-shaped slider 6 on the annular track 3. After installation, first adjust the angle of the driving gear 13 so that when the mounting plate 10 is pulled down, the driving gear 13 can mesh with the inner gear ring 5, and at the same time push the plug plate 8 to insert into the slide groove 4. Then, as the mounting plate 10 is pulled down, the lower end of the fixing rod 31 is inserted into the through groove 16 to complete the fixing of the arc-shaped slider 6.

[0118] Then place the placement plate 24 on the arc-shaped slide block 6;

[0119] When installing the circular plate 18, first align the insertion rod 20 and insert it into the rectangular groove 19, align the inner polygonal sleeve 22 with the polygonal shaft 17 and insert it, and engage the fan gear 23 with the annular tooth groove 25, then lengthen the telescopic rod 21 and insert it into the limit groove 30 to complete the installation of the circular plate 18, and then install the ablation device 27 on the placement plate 24.

[0120] How to use: Fig. 9As shown, at this time, the position of the ablation instrument 27 is the storage position. When it is needed, first control the start of the second motor 15 to drive the placement plate 24 to rotate, so that the ablation instrument 27 can be away from the ultrasonic diagnostic device 1, and then control the start of the first motor 12 to adjust the position of the arc-shaped slider 6, and then adjust the position of the ablation instrument 27, so as to facilitate the use of medical staff.

[0121] A cardiac ultrasound intelligent recognition method for myocardial ablation comprises the following steps:

[0122] Step S1, moving the ultrasonic diagnostic device 1 to the treatment position, using the ultrasonic diagnostic device 1 to determine the position of the lesion, and then controlling the first motor 12 and the second motor 15 to adjust the position of the ablation device 27, and planning the path of the ablation needle or catheter;

[0123] Step S2, disinfecting and locally anesthetizing the puncture site;

[0124] Step S3: Under ultrasound guidance, the ablation needle or catheter is accurately inserted into the lesion area, the ablation device 27 is started, the ablation parameters are set according to the treatment plan, and the ablation process begins. During the entire ablation process, the ultrasound diagnostic device 1 is continuously used for monitoring to ensure that the ablation needle or catheter is positioned correctly, and the change of tissue temperature is monitored at the same time;

[0125] Step S4: Obtain the ultrasonic heart rate of the test object. Figure 2 dimensional dynamic image;

[0126] Each segment of echocardiographic image information of the detected object includes at least one of the following standard sections: standard section of left ventricular long axis, standard section of four-chamber heart, standard section of five-chamber heart, standard section of two-chamber heart, standard section of aorta short axis, apical section of left ventricular short axis, middle section of left ventricular short axis and basal section of left ventricular short axis.

[0127] Each standard section in each echocardiogram of the tested subject contains at least one complete cardiac cycle.

[0128] The output features of the detected object at least include an end-diastolic image frame of a standard cardiac section and image data of the anatomical structure corresponding to each feature;

[0129] Step S5: The ultrasonic cardiogram of the object to be tested Figure 2 The convolutional neural network model automatically identifies and locates the features of each ultrasound image, captures the end-diastolic image frames of each standard section, and marks the mitral valve, tricuspid valve, aortic valve, left and right atrium and ventricle cavities, measures the thickness of each atrium and ventricle cavity and automatically marks them. Figure 2 The output features are the end-diastolic image frames of each standard section of the heart and the image data of each characteristic anatomical structure.

[0130] The test object is the heart of a normal person. The ultrasound probe of the ultrasound diagnostic instrument is used to collect the ultrasound image of the heart. The image is a two-dimensional dynamic echocardiogram, which contains 8 standard heart sections, and each section contains at least one complete cardiac cycle.

[0131] The subject's ultrasound cardiology Figure 2 The 3D dynamic image is input into the convolutional neural network model to measure the thickness of each chamber and wall for automatic marking. The input feature of the convolutional neural network model is the echocardiogram. Figure 2 The output features are the end-diastolic image frames of each standard section of the heart and the image data of each characteristic anatomical structure.

[0132] The trained convolutional neural network model is used to locate the precise position of the endocardium in each section of the heart. Since the heart morphology in each frame of the acquired dynamic cardiogram varies, the grayscale values ​​of the cardiac muscle layer, endocardium and cardiac cavity are different, and the trained convolutional neural network model can quickly, efficiently and accurately locate the endocardium from the cardiac ultrasound image.

[0133] Echocardiograms were obtained as training samples.

[0134] The training sample images are preprocessed and divided into training set and validation set.

[0135] Image preprocessing mainly includes data processing, data enhancement and data entry. Data preprocessing: Use fuzzy enhancement image processing methods to improve the shortcomings of unclear echocardiographic images and insufficient contrast, and improve the image quality of the data set. Data enhancement: Perform data enhancement methods such as flipping and rotating the image data set, set the brightness, contrast, saturation and hue of the image, and standardize the data to improve the generalization ability of the model. The data set is divided into a training set and a validation set in a ratio of 9:1.

[0136] The training set samples are input into the initial convolutional neural network for processing to obtain the output results.

[0137] Importing the echocardiogram dataset: Build a UNet algorithm model based on the PyTorch framework with image segmentation capabilities. Import the training set, i.e., echocardiogram images and black-and-white binary images (manual labels), into the reading directory specified by the deep learning model.

[0138] Figure 4 As shown in Figure 1, the trained convolutional neural network model includes an input layer, a hidden layer, and an output layer. The input layer inputs sample images that have been labeled, and the hidden layer includes a convolution layer, a pooling layer, a fully connected layer, etc. The convolution layer and the pooling layer mainly use convolution calculations on the image to enhance the features of the image. The pooling layer is used to reduce noise.

[0139] Compare the output result with the sample image to obtain a comparison result.

[0140] According to the comparison results, the parameters in the initial convolutional neural network model are optimized to obtain the trained convolutional neural network model.

[0141] Optimize the parameters in the initial convolutional neural network model, including using the loss function and optimization function to back propagate along the neural network to update and optimize the convolutional neural network parameters when the output result is wrong.

[0142] Figure 5 As shown, the neural network model is started for training. The features of the training set cardiac ultrasound images are extracted, and the ventricular septum is segmented based on the contours drawn by the manual labels. The cross entropy is introduced as the loss function to quantify the difference between the predicted label and the manual label. By adjusting the weight parameters of the neural network nodes, the predicted label output by the next round of training is closer to the manual label. In order to prevent the training from converging to the local minimum, the model learning step size can be adjusted by using methods such as cosine annealing learning rate, so that the training may jump out of the local optimal trap and thus be more likely to converge to the global minimum. At the same time, measures can be taken to avoid overfitting and improve the generalization ability of the training results.

[0143] In order to realize the automatic segmentation algorithm of the ventricular wall and endocardium based on the deep learning model, the segmentation algorithm needs to identify three areas during modeling: the ventricular area, the endocardial area, and the background area. In terms of modeling, the widely used UNet network structure is used as our segmentation algorithm framework. The UNet network framework consists of an encoder and a decoder, and there are short connections between the two parts layer by layer. This allows UNet to learn rich ultrasound image features and accurately identify different areas such as the ventricles and endocardium, and to fully combine deep semantic features and shallow spatial features to achieve accurate positioning and segmentation of edge areas. Existing research experience has shown that the accurate positioning of the edges between different areas of echocardiograms is of great significance for the subsequent estimation of cardiac parameters based on automatic segmentation results. The basic operator of the network is composed of 3x3conv-BN-ReLU:

[0144]

[0145] ReLUout(x)=max(0,x)

[0146] In actual training, the area of ​​the endocardium is relatively small compared to the other two areas, which can easily lead to insufficient learning. To address this problem, we use the following solution: before training, we count the areas of the three areas in the training set and use the inverse ratio of the area as the weight of the cross entropy loss function to balance the learning of the three different areas. The cross entropy loss function used in this study is defined as follows:

[0147]

[0148] Due to the large number of parameters in deep learning neural network models, it is very easy to fall into local minima during the training and optimization process. In order to improve the quality of model optimization and make the final training results enter a minimum area with sufficient generalization ability, we use the cosine annealing learning rate adjustment method. This method realizes the periodic increase and decrease of the learning rate. When the learning rate increases, the optimization algorithm searches for the minimum point in a large range with a larger step size; when the learning rate decreases, the optimization algorithm tends to achieve local exploration in the neighborhood of the minimum point to achieve better performance. The positioning of the cosine annealing method used in this study is:

[0149]

[0150] The validation set is input to verify the effectiveness of the convolutional neural network model.

[0151] Verification of segmentation accuracy: To verify the accuracy of ventricular septum segmentation, the manually annotated segmented area was compared with the automatic segmentation results, and the Fβ, Dice, Hausdorff distance (HD [mm]), and mean absolute distance (MAD [mm]) indicators were used to evaluate the segmentation performance of the model.

[0152] Calculation of ventricular septum thickness: Obtain the coordinates of the outer arc of the predicted ventricular wall contour, and use the least squares method to fit the outer arc of the ventricular septum. Calculate the pixel distance between the inner and outer arcs based on the center of the arc. According to the physical distance represented by each pixel defined by the embedded TAG in the DICOM file, the wall thickness of the predicted ventricular septum contour can be calculated. In the validation set, the quantitative results based on automatic segmentation were compared with the results measured by senior physicians using a paired t-test to evaluate the consistency of the maximum thickness, minimum thickness, average thickness and ventricular septum length of each area measured by the two. P < 0.05 was considered statistically significant.

[0153] Obtain the ultrasound heart rate of the test object Figure 2 dimensional dynamic image and perform preprocessing.

[0154] For example, the fuzzy enhancement image processing method can be used to improve the shortcomings of unclear and insufficient contrast of echocardiographic images and improve the image quality of the data set.

[0155] The subjects were examined by ultrasound cardiology Figure 2 dimensional dynamic image input convolutional neural network model.

[0156] The convolutional neural network model automatically recognizes and outputs the end-diastolic image frames of each standard section of the heart of the detection object and the image data of each characteristic anatomical structure.

[0157] The convolutional neural network model compares the automatic measurement data with the manual measurement data to quantitatively evaluate the measurement accuracy of the system.

[0158] like Figure 6 As shown, in this embodiment, the convolutional neural network is used to process the echocardiogram image, which can quickly, accurately and objectively locate the structures such as the atria and ventricles, and provide quantitative indicators for each section of the heart. This not only facilitates the rapid and accurate identification of each standard section and structure of the heart by assisting doctors during ultrasound examinations, but also avoids the technical problem of large scoring deviations due to excessive subjective factors.

[0159] Figure 8 The present invention is based on a block diagram of a device of an intelligent recognition system for cardiac ultrasound examination. The device can be provided as a server. It includes a processing component, which further includes one or more processors; it also includes a memory resource represented by a memory, which is used to store instructions that can be executed by the processing component, such as an application program, and the application program stored in the memory may include one or more modules each corresponding to a set of instructions; the device may also include a power supply component to perform power management of the device; the device also has a wired or wireless input interface connected to an ultrasound diagnostic instrument for image input; the device further has an output interface for outputting image processing results. The device can operate based on an operating system stored in the memory.

[0160] For a better understanding of the present disclosure, Figure 7 As shown, the ultrasonic diagnostic instrument obtains dynamic ultrasound cardiogram images and stores the obtained ultrasound cardiogram images in a database. After the stored ultrasound cardiogram images are preprocessed, training sample images are obtained. After the training sample images are input into the convolutional neural network for training, a trained convolutional neural network model, i.e., a recognition model, is obtained.

[0161] When performing echocardiographic analysis and evaluation, the ultrasonic diagnostic instrument is also used to obtain dynamic echocardiographic images, and the trained recognition model is used for image processing. After the phase of each frame of the echocardiographic image to be tested is identified, the relevant standard section image frames are extracted, and the structure of each atrioventricular structure and endocardium of the heart is identified and located. By locating the endocardium of the left and right ventricles, the relevant data of the ventricular wall thickness and the internal diameter of each atrioventricular chamber are measured, and finally quantitative indicators are obtained.

[0162] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. The patent protection scope of the present invention shall be based on the claims. All equivalent structural changes made using the contents of the description and drawings of the present invention should also be included in the protection scope of the present invention.

Claims

1. A cardiac ultrasound intelligent recognition system for myocardial ablation, characterized by: It includes a cardiac ultrasound cardiogram acquisition module, which is used to acquire a two-dimensional dynamic image of the ultrasound cardiogram of the object to be detected; Neural network building module, used to build convolutional neural network models; Neural network training module, which is used to train convolutional neural network models; A neural network recognition module uses a trained convolutional neural network model to process and recognize the two-dimensional dynamic image of the echocardiogram, and automatically selects the end-diastolic frame image of each standard section; The image processing module processes each frame of the image, automatically identifies, locates, and marks the mitral valve, tricuspid valve, aortic valve, and left and right atria and chambers on the image, and measures the inner diameters of the left and right atria and chambers and the thickness of each chamber wall on a standard section; The image quality evaluation module evaluates and analyzes the input two-dimensional dynamic images of echocardiography, and gives quantitative scores based on whether the end-diastolic frame images of the standard sections required for cardiac ultrasound examination are included, the clarity of the images, and the fit of the measurement data; The cardiac ultrasound cardiogram acquisition module comprises an ultrasound diagnostic device and an ablation device, wherein the lower end of the ultrasound diagnostic device is fixedly connected to a plurality of vertical poles, the upper ends of the vertical poles are fixedly connected to a circular track, and the ablation device is arranged on a mounting mechanism; The mounting mechanism comprises a placement plate for mounting the ablation device, a first driving assembly for driving the placement plate to rotate, and a second driving assembly for driving the first driving assembly to slide on a circular track.

2. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 1, characterized in that: The neural network training module specifically includes the following units: A training sample acquisition unit, used for acquiring training samples; A neural network training unit, used for inputting the training set in the training sample into the initial convolutional neural network to obtain an initial convolutional neural network model; An output result acquisition unit, used for inputting the verification set in the training sample into the initial convolutional neural network model for processing to obtain an output result; A result comparison unit, used to compare the output result with the verification set image label to obtain a comparison result; A parameter optimization unit is used to optimize the parameters of the initial convolutional neural network model using the obtained comparison results to obtain a trained convolutional neural network model.

3. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 1, characterized in that: The neural network building module specifically includes the following units: The convolution layer construction unit applies a preset convolution kernel according to the received echocardiographic data, performs convolution calculation according to the convolution step size, and obtains a convolution result; The pooling layer construction unit will perform aggregate statistical processing based on the received convolution results to obtain the pooling results; The fully connected layer construction unit combines the pooling results of the received local information to obtain the global information.

4. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 1, characterized in that: The image processing module specifically includes the following units: An image preprocessing unit, used for preprocessing the ultrasound cardiogram obtained from the object to be detected to obtain a preprocessed image; An image recognition unit, used for recognizing and extracting corresponding cardiac standard section frames from the preprocessed image according to the image features of each cardiac standard section; An image structure positioning unit, used to determine the positions of the mitral valve, tricuspid valve, aorta, left and right atrium and ventricle cavities of the heart structures in the preprocessed image according to the image features of each standard section of the heart, and to calibrate the corresponding structures; The image structure measurement unit is used to measure the inner diameters of the left and right atria and ventricles and the thickness of the ventricular septum of each structure of the heart in the preprocessed image according to the image features of each standard section of the heart.

5. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 1, characterized in that: The second driving component includes an arc-shaped slider, a mounting plate, an insert plate plugged into the side of the arc-shaped slider, and a second driving module for driving the arc-shaped slider to slide. The lower end of the arc-shaped slider is provided with a slot that can be plugged into the annular track, the side of the annular track is provided with a slide groove for the insert plate to be plugged into, the upper half of the arc-shaped slider is provided with a through hole, the mounting plate is slidably arranged in the through hole along the up and down directions, and the driving module is arranged at the lower end of the mounting plate located in the annular track.

6. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 5, characterized in that: A fixing rod is fixedly connected to the lower end of the mounting plate outside the annular track, a through groove for inserting the fixing rod is provided at one end of the plug plate away from the slide groove, the lower end of the fixing rod is an inverted right-angled trapezoid, a tension spring is fixedly connected to the upper end of the mounting plate, and the upper end of the tension spring is fixedly connected to the upper end of the arc-shaped slider.

7. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 6, characterized in that: The second driving module comprises a first motor fixedly connected to the lower end of the mounting plate, a driving gear fixedly connected to the output shaft of the first motor, and an inner gear ring fixedly connected to the inner side of the annular track, wherein the inner gear ring is meshingly connected to the driving gear; When the fixing rod is temporarily fixed to the mounting plate, the inner gear ring is meshed and connected with the driving gear; when the fixing rod is not temporarily fixed, the inner gear ring is not meshed with the driving gear.

8. The cardiac ultrasound intelligent recognition system for myocardial ablation according to claim 7, characterized in that: The first driving assembly includes a circular plate, a rectangular groove provided on the circular plate, a telescopic rod fixed in the rectangular groove, a second motor fixed in the arc-shaped slider, an inner polygonal sleeve rotatably connected to the middle of the circular plate, and a sector gear fixedly connected to the outer side of the inner polygonal sleeve; The placement plate is provided with a circular hole for matching the circular plate installation, and the placement plate at the upper end of the circular hole is provided with an annular tooth groove for meshing with the sector gear; The upper end of the mounting plate is fixedly connected with an insertion rod which can be inserted into the rectangular groove, and the insertion rod is provided with a limiting groove for the telescopic rod to be inserted; The output shaft of the second motor is fixedly connected with a polygonal shaft for fitting with the inner polygonal sleeve; The upper end of the arc-shaped sliding block is provided with a mounting groove for mounting the second motor; A lower ring plate is fixedly connected to the placement plate at the lower end of the circular hole, and an upper ring plate is fixedly connected to the upper end of the outer surface of the circular plate, and the upper ring plate is used to limit the lower ring plate on the arc-shaped slider; A bandage for installing the ablation device is fixed on the placement board.

9. A cardiac ultrasound intelligent recognition method for myocardial ablation, comprising the cardiac ultrasound intelligent recognition system according to any one of claims 1 to 8, characterized in that: The following steps are involved: Step S1, moving the ultrasonic diagnostic device to the treatment position, using the ultrasonic diagnostic device to determine the position of the lesion, and then controlling the first motor and the second motor to adjust the position of the ablation device, and planning the path of the ablation needle or catheter; Step S2, disinfecting and locally anesthetizing the puncture site; Step S3: Under ultrasound guidance, the ablation needle or catheter is accurately inserted into the lesion area, the ablation instrument is started, the ablation parameters are set according to the treatment plan, and the ablation process begins. During the entire ablation process, the ultrasound diagnostic device is continuously used for monitoring to ensure that the ablation needle or catheter is positioned correctly, and the change of tissue temperature is monitored at the same time; Step S4, obtaining a two-dimensional dynamic image of an echocardiogram of the test object; Step S5, inputting the two-dimensional dynamic image of the ultrasonic cardiogram of the detection object into the convolutional neural network model, automatically identifying and locating the features of each segment of the ultrasonic image, capturing the end-diastolic image frame of each standard section, and marking the mitral valve, tricuspid valve, aortic valve, left and right atrioventricular cavities, measuring the thickness of each atrioventricular cavity and ventricular wall and automatically marking them.

Images