An artificial intelligence-based image-guided system for cardiovascular surgery

Abstract

The present invention relates to the field of medical artificial intelligence imaging technology, and specifically to an artificial intelligence cardiovascular surgery image guidance system, comprising the following modules: a video acquisition module for collecting real-time video in cardiovascular surgery; a processing module for preprocessing videos to generate cardiovascular structure images of different sizes and shapes; a deep learning model for identifying and marking abnormal areas in cardiovascular structure images; a fusion module mainly for fusing multimodal image data to generate cardiovascular abnormal area analysis reports; a simulated surgery module mainly for generating three-dimensional cardiovascular surgery simulation scenes and path planning; an evaluation module mainly for evaluating the simulated surgery process and feeding back the results to the deep learning model; and a guidance module mainly for providing real-time guidance during surgery to assist doctors in performing surgery. The present invention can generate three-dimensional cardiovascular surgery simulation scenes and surgical path planning, and can perform sufficient simulation training and evaluation before surgery to improve the success rate of surgery.

Description

Technical Field

[0001] The present invention relates to the field of medical artificial intelligence imaging technology, and in particular to an artificial intelligence cardiovascular surgery image guidance system. Background Art

[0002] Cardiovascular surgery image guidance systems play a vital role in the medical field. By providing real-time surgical images and data analysis, they help doctors more accurately diagnose the condition, make surgical plans and perform operations. With the rapid development of artificial intelligence technology, these systems have made significant progress and improved the accuracy and safety of surgery in many aspects.

[0003] Although 3D visualization technology is relatively mature, many systems do not develop detailed 3D simulation surgical plans based on the patient's specific conditions before actual surgery, which may cause doctors to be unable to accurately judge the location of the lesion and the surgical path during surgery, thereby increasing surgical risks and prolonging surgery time.

[0004] In summary, how to solve the problem that before the actual operation, many systems do not formulate detailed three-dimensional simulated surgical plans according to the patient's specific conditions, which may cause doctors to be unable to accurately judge the lesion location and surgical path during the operation, thereby increasing the surgical risk and prolonging the operation time has become a difficult problem that needs to be urgently solved in this field. Therefore, it is necessary to propose an artificial intelligence cardiovascular surgery image guidance system. Summary of the invention

[0005] To solve the above problems, the present invention provides an artificial intelligence cardiovascular surgery image guidance system. The system uses three-dimensional cardiovascular surgery simulation scenes and three-dimensional surgical path planning to simulate surgical scenes for training and evaluation in advance, and finally assists doctors in providing surgical path planning and key step prompts in actual surgery based on the three-dimensional surgical path planning and evaluation results.

[0006] In order to achieve the above-mentioned purpose, the technical solution of the present invention is as follows: an artificial intelligence cardiovascular surgery image guidance system, comprising:

[0007] The video image acquisition module is used to acquire real-time video of the vascular structure, heart valves and coronary arteries during cardiovascular surgery through a camera.

[0008] The processing module is used to pre-process and segment the real-time video of the vascular structure, heart valve and coronary artery in cardiovascular surgery using a multi-scale image segmentation algorithm to generate cardiovascular structure images of different sizes and shapes.

[0009] The deep learning model is used to utilize the YOLOv10 image learning model, introduce a pruning algorithm, and adopt an incremental learning mechanism to input cardiovascular structure images of different sizes and shapes into the YOLOv10 image learning model for learning and training to detect abnormal areas in cardiovascular structure images of different sizes and shapes, and generate corresponding abnormal area data information of cardiovascular structure images of different sizes and shapes; the abnormal area data information includes stenosis areas, calcification areas, and aneurysm areas.

[0010] The fusion module is used to fuse and analyze the abnormal area data information in the CT, MRI and ultrasound image data and the abnormal area data information of cardiovascular structure images of different sizes and shapes generated by the deep learning model to obtain a multimodal cardiovascular abnormal area analysis report.

[0011] The simulation surgery module is used to utilize cardiovascular structure images of different sizes and shapes combined with multi-modal cardiovascular abnormality area analysis reports, and use OpenGL 3D rendering technology to generate 3D cardiovascular surgery simulation scenes and 3D surgical path planning.

[0012] The evaluation module is used to evaluate the simulated surgical process in the simulated surgical module, generate evaluation results, and feed the evaluation results back to the deep learning model for auxiliary learning.

[0013] The guidance module is used to provide real-time guidance during surgery using three-dimensional surgical path planning.

[0014] The technical principles of the above scheme are as follows:

[0015] Through the use of several cameras deployed in the cardiovascular surgery operating room, the real-time video of the vascular structure, heart valve and coronary artery during the operation is captured by an image segmentation algorithm to remove useless images, thereby obtaining cardiovascular structure images of different sizes and shapes; the cardiovascular structure images of different sizes and shapes are then denoised and enhanced, and the enhanced cardiovascular structure images of different sizes and shapes are then learned and trained through the YOLOv10 image learning model, so that the YOLOv10 image learning model can learn, identify and annotate the stenosis areas, calcification areas and aneurysm areas in the cardiovascular structure images of different sizes and shapes, and then generate a three-dimensional cardiovascular surgery simulation scene and a three-dimensional surgical path planning by combining CT, MRI and ultrasound image data using OpenGL three-dimensional rendering technology; doctors use the three-dimensional cardiovascular surgery simulation scene and three-dimensional surgical path planning to simulate the surgical scene for training and evaluation in advance, and finally, according to the three-dimensional surgical path planning and evaluation results, assist doctors in providing surgical path planning and key step prompts in actual surgery.

[0016] Further, in the processing module, the multi-scale image segmentation algorithm includes the following steps:

[0017] Step 1: segment the real-time video of the vascular structure, heart valve and coronary artery in cardiovascular surgery into single-frame images, perform noise suppression and contrast enhancement on the single-frame images, and obtain enhanced single-frame images.

[0018] The contrast enhancement is calculated by the following formula:

[0019] Enhanced(x,y)=α(I(x,y)-Imin) / (Imax-Imin)+β (1).

[0020] Among them, I(x,y) is the pixel value of the original image at the coordinate (x,y), Imin and Imax are the minimum and maximum pixel values ​​in the image, α and β are the contrast enhancement coefficient and brightness adjustment coefficient, and Ienhanced(x,y) is the pixel value of the enhanced image at the coordinate (x,y).

[0021] Step 2: Based on the grayscale distribution of the image, an appropriate initial threshold T0 is selected for the enhanced single-frame image, and the image is divided into two parts, the cardiovascular structure and the background, based on the threshold T0.

[0022] Step 3: The threshold T0 is automatically calculated based on the grayscale histogram of the image through the adaptive Otsu's algorithm.

[0023] T0=argtmax[ω0(t)×σ02(t)+ω1(t)×σ12(t)] (2).

[0024] Among them, ω0(t) and ω1(t) are the pixel ratios of foreground and background after segmentation with threshold t, and σ02(t) and σ12(t) are the variances of foreground and background, respectively.

[0025] Furthermore, in the deep learning model, the pruning algorithm includes the following steps:

[0026] Step 1: Use the Hessian matrix to determine the weights in the YOLOv10 image learning model.

[0027] Step 2: Set the pruning threshold based on the weight and remove the weights below the pruning threshold.

[0028] Step 3: According to the pruning threshold, remove all weights below the threshold in the YOLOv10 image learning model, and reconnect the remaining weights to update the YOLOv10 image learning model.

[0029] Step 4: Use cardiovascular structure images of different sizes and shapes to input into the YOLOv10 image learning model for retraining, adjust the learning rate, and output pruned cardiovascular structure images of different sizes and shapes.

[0030] Furthermore, in the deep learning model, the incremental learning mechanism is to set the learning parameters of the YOLOv10 image learning model to learn only the newly added and changed pruned cardiovascular structure images of different sizes and shapes.

[0031] Furthermore, in the deep learning model, cardiovascular structure images of different sizes and shapes are input into the YOLOv10 image learning model for learning and training, including the following steps:

[0032] Step 1: Data preprocessing: normalizing and resizing the pruned cardiovascular structure images of different sizes and shapes to obtain preprocessed cardiovascular structure images.

[0033] Step 2: Data annotation: Annotate the preprocessed cardiovascular structure images, mark the location and category of the abnormal areas, and generate a labeled data set for training the YOLOv10 model; the abnormal areas include stenosis areas, calcification areas, and aneurysm areas.

[0034] Step 3: Model initialization: Initialize the pruned and reconnected YOLOv10 image learning model, and set the model’s learning rate, batch size, and number of iterations.

[0035] Step 4: Model training: Input the labeled dataset into the YOLOv10 image learning model, calculate the loss through forward propagation, and update the model weights through backpropagation for training.

[0036] Step 5: Model evaluation: After the training is completed, the YOLOv10 image learning model is evaluated using other pruned cardiovascular structure image sets of different sizes and shapes to generate evaluation results. Based on the evaluation results, the model parameters are fine-tuned to obtain a trained and evaluated YOLOv10 image learning model.

[0037] Step 6: Model deployment: Deploy the trained and evaluated YOLOv10 image learning model into the cardiovascular surgery image guidance system for real-time detection and analysis of abnormal areas in cardiovascular structural images.

[0038] Furthermore, in the evaluation module, the evaluation results include the error rate of simulated surgery, the rationality of surgical path planning, the accuracy of estimated surgical time, and the accuracy of abnormal area detection by the YOLOv10 image learning model.

[0039] Furthermore, in the simulated surgery module, OpenGL 3D rendering technology combines cardiovascular structure images of different sizes and shapes with multimodal cardiovascular abnormality area analysis reports through texture mapping, lighting simulation and shadow processing steps to generate 3D models of cardiovascular structures, abnormal areas and surgical instruments in the 3D cardiovascular surgery simulation scene.

[0040] Furthermore, the evaluation module includes a user feedback unit, which is used to collect doctors' feedback on the simulated surgical process, surgical path planning, surgical time estimation, and abnormal area detection accuracy, and send the feedback to the deep learning model.

[0041] Furthermore, the evaluation module also includes a postoperative evaluation unit, which is used to use the video image acquisition module to collect postoperative surgical area images, use the YOLOv10 image learning model to learn and analyze the postoperative surgical area images, and generate a visualization report; the visualization report includes the surgical effect and subsequent treatment plan.

[0042] Furthermore, in the guidance module, real-time guidance includes surgical path planning and key step prompts.

[0043] The above scheme has the following beneficial effects:

[0044] 1. This invention integrates multi-scale image segmentation algorithm and advanced YOLOv10 image learning model, combined with pruning algorithm and incremental learning mechanism, to achieve efficient preprocessing and accurate abnormal area detection of real-time video images in cardiovascular surgery. This not only improves the accuracy and safety of surgery, but also significantly reduces surgical risks and operational complexity.

[0045] 2. The present invention generates detailed 3D cardiovascular surgery simulation scenes and surgical path planning by fusing CT, MRI and ultrasound image data and combining OpenGL 3D rendering technology. This enables doctors to conduct sufficient simulation training and evaluation before surgery, optimize surgical plans, improve surgical success rates, and reduce unnecessary surgical time and risks.

[0046] 3. The evaluation module of the present invention can not only evaluate the simulated surgical process and generate evaluation results including error rate, rationality of surgical path planning, accuracy of surgical time estimation, etc., but also collect actual usage feedback from doctors through the user feedback unit, which can be used to further optimize the deep learning model and surgical guidance system.

[0047] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1Schematic diagram of the structure of an artificial intelligence cardiovascular surgery image guidance system in an embodiment of the present invention.

[0049] Figure 2 Schematic diagram of the steps of the method for learning and training the YOLOv10 image learning model in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following is further described in detail through specific implementation methods:

[0051] As attached Figure 1 Shown: An artificial intelligence cardiovascular surgery image guidance system mainly consists of a video image acquisition module, a processing module, a deep learning model, a fusion module, a simulation surgery module, an evaluation module and a guidance module.

[0052] Among them, the video acquisition module is mainly used to collect real-time videos during cardiovascular surgery; the processing module is mainly used to preprocess videos and generate cardiovascular structure images of different sizes and shapes; the deep learning model is mainly used to identify and mark abnormal areas in cardiovascular structure images; the fusion module is mainly used to fuse multimodal image data and generate cardiovascular abnormal area analysis reports; the simulation surgery module is mainly used to generate three-dimensional cardiovascular surgery simulation scenes and path planning; the evaluation module is mainly used to evaluate the simulated surgery process and feedback the results to the deep learning model; the guidance module is mainly used to provide real-time guidance during surgery to assist doctors in performing surgery.

[0053] The following is a detailed explanation of the functions of each module:

[0054] The video image acquisition module includes several cameras arranged in the cardiovascular surgery operating room, and the cameras are used to acquire real-time video of the vascular structure, heart valve and coronary artery during cardiovascular surgery.

[0055] The processing module is used to pre-process and segment the real-time video of the vascular structure, heart valve and coronary artery in cardiovascular surgery using a multi-scale image segmentation algorithm to generate cardiovascular structure images of different sizes and shapes.

[0056] The multi-scale image segmentation algorithm includes the following steps:

[0057] Step 1: segment the real-time video of the vascular structure, heart valve and coronary artery in cardiovascular surgery into single-frame images, perform noise suppression and contrast enhancement on the single-frame images, and obtain enhanced single-frame images;

[0058] The contrast enhancement is calculated by the following formula:

[0059] Enhanced(x,y)=α(I(x,y)-Imin) / (Imax-Imin)+β (1).

[0060] Among them, I(x,y) is the pixel value of the original image at the coordinate (x,y), Imin and Imax are the minimum and maximum pixel values ​​in the image, α and β are the contrast enhancement coefficient and brightness adjustment coefficient, and Ienhanced(x,y) is the pixel value of the enhanced image at the coordinate (x,y).

[0061] Step 2: Based on the grayscale distribution of the image, an appropriate initial threshold T0 is selected for the enhanced single-frame image, and the image is divided into two parts, the cardiovascular structure and the background, based on the threshold T0.

[0062] Step 3: The threshold T0 is automatically calculated based on the grayscale histogram of the image through the adaptive Otsu's algorithm.

[0063] T0=argtmax[ω0(t)×σ02(t)+ω1(t)×σ12(t)](2).

[0064] Among them, ω0(t) and ω1(t) are the pixel ratios of foreground and background after segmentation with threshold t, and σ02(t) and σ12(t) are the variances of foreground and background, respectively.

[0065] Specifically, first, high-definition cameras are placed in the cardiovascular surgery operating room, such as above the operating table, next to the surgical instruments, and in the main viewing angle of the doctor's operation, to record the cardiovascular surgery process and capture real-time video of the vascular structure, heart valves, and coronary arteries during cardiovascular surgery.

[0066] The processing module is responsible for receiving the real-time video transmitted by the video image acquisition module, and performing preprocessing and segmentation extraction on it; specifically, first, the real-time video is segmented into single-frame images; then, the single-frame images are subjected to noise suppression and contrast enhancement; noise suppression helps to remove useless information in the single-frame images and improve the quality of the single-frame images; contrast enhancement makes the cardiovascular structure in the single-frame images clearer and more discernible by adjusting the brightness and contrast of the single-frame images.

[0067] Based on the grayscale distribution of a single-frame image, an initial threshold T0 is manually pre-set; the initial threshold T0 is used to divide the single-frame image into two parts: cardiovascular structure and background.

[0068] Then, the threshold T0 is automatically calculated based on the grayscale histogram of the single-frame image through the adaptive Otsu's algorithm; Otsu's algorithm automatically determines the optimal threshold T0 based on the grayscale histogram of the image to maximize the inter-class variance between the foreground and background.

[0069] The specific formula for automatically determining the optimal threshold T0 is: T0 = argtmax[ω0(t)×σ02(t)+ω1(t)×σ12(t)], where ω0(t) and ω1(t) are the pixel ratios of the foreground and background after segmentation with threshold t, and σ02(t) is the variance of the foreground and background.

[0070] Subsequently, the calculated threshold T0 is used to segment the single frame image and separate the cardiovascular structure from the background.

[0071] The processing module then performs morphological operations on the segmented single-frame images to optimize the shape and edges of cardiovascular structures.

[0072] Finally, cardiovascular structure images of different sizes and shapes are generated for use by subsequent modules.

[0073] The deep learning model is used to utilize the YOLOv10 image learning model, introduce a pruning algorithm, and adopt an incremental learning mechanism to input cardiovascular structure images of different sizes and shapes into the YOLOv10 image learning model for learning and training to detect abnormal areas in cardiovascular structure images of different sizes and shapes, and generate corresponding abnormal area data information of cardiovascular structure images of different sizes and shapes; the abnormal area data information includes stenosis areas, calcification areas, and aneurysm areas.

[0074] The pruning algorithm includes the following steps:

[0075] Step 1: Use the Hessian matrix to determine the weights in the YOLOv10 image learning model.

[0076] Step 2: Set the pruning threshold based on the weight and remove the weights below the pruning threshold.

[0077] Step 3: According to the pruning threshold, remove all weights below the threshold in the YOLOv10 image learning model, and reconnect the remaining weights to update the YOLOv10 image learning model.

[0078] Step 4: Use cardiovascular structure images of different sizes and shapes to input into the YOLOv10 image learning model for retraining, adjust the learning rate, and output pruned cardiovascular structure images of different sizes and shapes. The pruning algorithm can significantly reduce the parameters of the YOLOv10 image learning model, reduce computing resource consumption, and maintain high recognition accuracy.

[0079] The incremental learning mechanism sets the learning parameters of the YOLOv10 image learning model to learn only the newly added and changed pruned cardiovascular structure images of different sizes and shapes. The incremental learning strategy can further reduce the overall computational complexity of the YOLOv10 image learning model.

[0080] In the deep learning model, cardiovascular structure images of different sizes and shapes are input into the YOLOv10 image learning model for learning and training, including the following steps:

[0081] Step 1: Data preprocessing: normalizing and resizing the pruned cardiovascular structure images of different sizes and shapes to obtain preprocessed cardiovascular structure images.

[0082] Step 2: Data annotation: Annotate the preprocessed cardiovascular structure images, mark the location and category of the abnormal areas, and generate a labeled data set for training the YOLOv10 model; the abnormal areas include stenosis areas, calcification areas, and aneurysm areas.

[0083] Step 3: Model initialization: Initialize the pruned and reconnected YOLOv10 image learning model, and set the model’s learning rate, batch size, and number of iterations.

[0084] Step 4: Model training: Input the labeled dataset into the YOLOv10 image learning model, calculate the loss through forward propagation, and update the model weights through backpropagation for training.

[0085] Step 5: Model evaluation: After the training is completed, the YOLOv10 image learning model is evaluated using other pruned cardiovascular structure image sets of different sizes and shapes to generate evaluation results. Based on the evaluation results, the model parameters are fine-tuned to obtain a trained and evaluated YOLOv10 image learning model.

[0086] Step 6: Model deployment: Deploy the trained and evaluated YOLOv10 image learning model into the cardiovascular surgery image guidance system for real-time detection and analysis of abnormal areas in cardiovascular structural images.

[0087] The fusion module is used to fuse and analyze the abnormal area data information in the CT, MRI and ultrasound image data and the abnormal area data information of cardiovascular structure images of different sizes and shapes generated by the deep learning model to obtain a multimodal cardiovascular abnormal area analysis report.

[0088] Specifically, first, the YOLOv10 image learning model uses cardiovascular structure images of different sizes and shapes for learning and training, which are pre-processed and segmented from real-time videos by the processing module; for example, in a cardiovascular surgery video, the processing module successfully extracts detailed images of heart valves, coronary arteries and surrounding vascular structures from the video frames. These images show the complex structure of cardiovascular organs at different angles and under different lighting conditions, providing rich training materials for the YOLOv10 model.

[0089] After the YOLOv10 model is trained, it can accurately identify and mark abnormal areas in cardiovascular structural images, including vascular stenosis, calcification, and aneurysm areas. The data information of these abnormal areas will be recorded and fused with image data from other imaging technologies.

[0090] At the same time, the cardiovascular surgery image guidance system also receives CT, MRI and ultrasound image data from patients before or during surgery; these image data each provide information on different aspects of cardiovascular structure. For example, CT images can clearly show the calcification of the blood vessel wall, MRI images can show the functional state of the heart and the perfusion of the myocardium, and ultrasound images can show the movement of the heart and blood flow velocity in real time.

[0091] The fusion module preprocesses and aligns the CT, MRI and ultrasound image data to ensure their consistency in space and time; then, the fusion module uses the image fusion algorithm to accurately match and superimpose the abnormal area data information generated by the deep learning model with the corresponding information in the CT, MRI and ultrasound images.

[0092] During the fusion process, the cardiovascular surgery image guidance system applies image enhancement and visualization technology to improve the visibility and identification of abnormal areas. For example, by adjusting the brightness, contrast and color mapping of the image, stenosis, calcification and aneurysm areas can be made more prominent and easier for doctors to identify.

[0093] Finally, the fusion module generates a cardiovascular abnormal area analysis report. The analysis report contains image data of all modalities and fused multimodal images. The report also lists all detected abnormal areas and provides their precise location, size, shape and possible pathological diagnosis. In addition, the report may also include suggestions for surgical options to help doctors develop more accurate and effective treatment plans.

[0094] For example, in an actual analysis report, the system may detect an obvious stenosis area in the patient's coronary artery; by fusing CT, MRI and ultrasound image data, the doctor can clearly see the position and shape of the stenotic area in three-dimensional space, as well as its relationship with surrounding blood vessels and tissues; based on this report, the doctor can develop a more precise surgical plan, such as selecting an appropriate surgical route, determining the size and position of the implanted stent, etc., thereby improving the success rate of the operation and the patient's recovery effect.

[0095] The simulated surgery module is used to utilize cardiovascular structure images of different sizes and shapes, abnormal area data information of cardiovascular structure images of different sizes and shapes, and multimodal cardiovascular abnormal area analysis reports, and uses OpenGL three-dimensional rendering technology to generate three-dimensional cardiovascular surgery simulation scenes and three-dimensional surgical path planning.

[0096] In the simulated surgery module, OpenGL 3D rendering technology combines cardiovascular structure images of different sizes and shapes with multimodal cardiovascular abnormality area analysis reports through texture mapping, lighting simulation and shadow processing steps to generate 3D models of cardiovascular structures, abnormal areas and surgical instruments in the 3D cardiovascular surgery simulation scene.

[0097] The evaluation module is used to evaluate the simulated surgical process in the simulated surgical module, generate evaluation results, and feed the evaluation results back to the deep learning model for auxiliary learning.

[0098] In the evaluation module, the evaluation results include the error rate of simulated surgery, the rationality of surgical path planning, the accuracy of surgical time estimation, and the accuracy of abnormal area detection by the YOLOv10 image learning model.

[0099] The evaluation module includes a user feedback unit, which is used to collect doctors' feedback on the simulated surgical process, surgical path planning, surgical time estimation, and abnormal area detection accuracy, and send the feedback to the deep learning model.

[0100] The evaluation module also includes a postoperative evaluation unit, which is used to use the video image acquisition module to collect postoperative surgical area images, use the YOLOv10 image learning model to learn and analyze the postoperative surgical area images, and generate a visualization report; the visualization report includes the surgical effect and subsequent treatment plan.

[0101] The guidance module is used to provide real-time guidance during surgery by using three-dimensional surgical path planning and evaluation results; real-time guidance includes surgical path planning and key step prompts.

[0102] Specifically, first, the simulated surgery module uses OpenGL three-dimensional rendering technology to combine cardiovascular structure images of different sizes and shapes, abnormal area data information of cardiovascular structure images of different sizes and shapes, and multi-modal cardiovascular abnormal area analysis reports to generate a highly realistic three-dimensional cardiovascular surgery simulation scene; in this scene, doctors can intuitively see the three-dimensional structure of the patient's cardiovascular system, including blood vessels, heart valves and key parts of coronary arteries, as well as abnormal areas in these parts, such as stenosis areas, calcification areas or aneurysm areas.

[0103] For example, when simulating a surgery for coronary artery stenosis, the simulation surgery module will generate a three-dimensional model containing the stenotic coronary artery based on the cardiovascular structure images extracted by the processing module and the abnormal area data annotated by the deep learning model. This three-dimensional model containing the stenotic coronary artery not only shows the specific location and morphology of the stenotic site, but also simulates the structure of the surrounding blood vessels and tissues, allowing doctors to plan and rehearse the surgical path in a virtual environment.

[0104] At the same time, the simulated surgery module will also integrate the image data of different imaging technologies such as CT, MRI and ultrasound into the three-dimensional model based on the multi-modal analysis report of cardiovascular abnormality areas, providing more comprehensive and accurate information support. Doctors can observe and analyze cardiovascular structures from multiple angles and levels during the simulated surgery to better formulate surgical plans.

[0105] When generating 3D surgical path planning, the surgery simulation module considers multiple factors, such as the accessibility of surgical instruments, the safety of the surgical path, and the degree of damage to surrounding tissues. By simulating the effects of different surgical paths, the module can help doctors choose the best surgical plan, reduce surgical risks, and improve the success rate of surgery.

[0106] In addition, the simulated surgery module can also simulate various situations during surgery, such as ruptured blood vessels, bleeding and other emergencies, to help doctors train and improve their emergency response capabilities. In this way, when encountering similar situations in actual surgery, doctors can respond more calmly and ensure the safety and smooth progress of the operation.

[0107] After the simulated surgery is completed, the evaluation module will evaluate the entire simulated surgery process and generate detailed evaluation results. These results include the error rate of the simulated surgery, the rationality of the surgical path planning, the accuracy of the estimated surgery time, and the accuracy of the abnormal area detection of the YOLOv10 image learning model. Through the evaluation results, doctors can understand their own surgical skills and knowledge level, identify existing problems and deficiencies, and make targeted improvements and enhancements.

[0108] At the same time, the user feedback unit in the evaluation module will also collect doctors' feedback on the simulated surgical process, surgical path planning, surgical time estimation, and abnormal area detection accuracy. These feedbacks will be sent to the deep learning model for further optimization and training of the model. Through continuous learning and improvement, the recognition accuracy and generalization ability of the deep learning model will be improved, providing doctors with more accurate and reliable image guidance.

[0109] Finally, in terms of real-time guidance during surgery, the guidance module will provide doctors with real-time surgical guidance based on the three-dimensional surgical path planning and evaluation results. These guidance include prompts for the surgical path, reminders of key steps, and suggestions for the use of surgical instruments. Through real-time surgical guidance, doctors can perform surgical operations more accurately, reduce surgical risks, and improve surgical results. At the same time, the guidance module can also dynamically adjust and optimize the surgical path according to the actual situation during the operation to ensure the smooth progress of the operation.

[0110] The present invention generates detailed 3D cardiovascular surgery simulation scenes and surgical path planning by fusing CT, MRI and ultrasound image data and combining OpenGL 3D rendering technology. This enables doctors to conduct sufficient simulation training and evaluation before surgery, optimize surgical plans, improve surgical success rates, and reduce unnecessary surgical time and risks.

[0111] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based cardiovascular surgery image guidance system, characterized in that: include: A video image acquisition module, including a plurality of cameras arranged in a cardiovascular surgery operating room, the cameras are used to acquire real-time video of the vascular structure, heart valves and coronary artery parts during cardiovascular surgery; A processing module, for preprocessing and segmenting the real-time video of the vascular structure, heart valve and coronary artery in cardiovascular surgery using a multi-scale image segmentation algorithm, and generating cardiovascular structure images of different sizes and shapes; A deep learning model is used to use the YOLOv10 image learning model, introduce a pruning algorithm, and adopt an incremental learning mechanism to input cardiovascular structure images of different sizes and shapes into the YOLOv10 image learning model for learning and training to detect abnormal areas in cardiovascular structure images of different sizes and shapes, and generate corresponding abnormal area data information of cardiovascular structure images of different sizes and shapes; the abnormal area data information includes stenosis areas, calcification areas, and aneurysm areas; A fusion module is used to fuse and analyze the abnormal area data information in the CT, MRI and ultrasound image data and the abnormal area data information of the cardiovascular structure images of different sizes and shapes generated by the deep learning model to obtain a multi-modal cardiovascular abnormal area analysis report; The simulation surgery module is used to generate a three-dimensional cardiovascular surgery simulation scene and three-dimensional surgery path planning by using cardiovascular structure images of different sizes and shapes, abnormal area data information of cardiovascular structure images of different sizes and shapes, and multi-modal cardiovascular abnormal area analysis reports, and using OpenGL three-dimensional rendering technology; An evaluation module is used to evaluate the simulated surgical process in the simulated surgical module, generate evaluation results, and feed the evaluation results back to the deep learning model for auxiliary learning; The guidance module is used to provide real-time guidance during surgery using three-dimensional surgical path planning and evaluation results.

2. The artificial intelligence cardiovascular surgery image guidance system according to claim 1, characterized in that: In the processing module, the multi-scale image segmentation algorithm includes the following steps: Step 1: segment the real-time video of the vascular structure, heart valve and coronary artery in cardiovascular surgery into single-frame images, perform noise suppression and contrast enhancement on the single-frame images, and obtain enhanced single-frame images; The contrast enhancement is calculated by the following formula: Enhanced(x,y)=α(I(x,y)-Imin) / (Imax-Imin)+β (1); Where I(x,y) is the pixel value of the original image at the coordinate (x,y), Imin and Imax are the minimum and maximum pixel values ​​in the image, α and β are the contrast enhancement coefficient and brightness adjustment coefficient, and Ienhanced(x,y) is the pixel value of the enhanced image at the coordinate (x,y); Step 2: Based on the grayscale distribution of the image, an appropriate initial threshold T0 is selected for the enhanced single-frame image, and the image is divided into two parts, the cardiovascular structure and the background, based on the threshold T0; Step 3, using the adaptive Otsu's algorithm, the threshold T0 is automatically calculated according to the grayscale histogram of the image; T0=argtmax[ω0(t)×σ02(t)+ω1(t)×σ12(t)] (2); Among them, ω0(t) and ω1(t) are the pixel ratios of foreground and background after segmentation with threshold t, and σ02(t) and σ12(t) are the variances of foreground and background, respectively.

3. The artificial intelligence cardiovascular surgery image guidance system according to claim 2, characterized in that: In the deep learning model, the pruning algorithm includes the following steps: Step 1: Use the Hessian matrix to determine the weights in the YOLOv10 image learning model; Step 2: Set the pruning threshold according to the weight and remove the weight below the pruning threshold; Step 3: According to the pruning threshold, remove all weights below the threshold in the YOLOv10 image learning model, and reconnect the remaining weights to update the YOLOv10 image learning model; Step 4: Use cardiovascular structure images of different sizes and shapes to input into the YOLOv10 image learning model for retraining, adjust the learning rate, and output pruned cardiovascular structure images of different sizes and shapes.

4. The artificial intelligence cardiovascular surgery image guidance system according to claim 3, characterized in that: In the deep learning model, the incremental learning mechanism is to set the learning parameters of the YOLOv10 image learning model to learn only the newly added and changed pruned cardiovascular structure images of different sizes and shapes.

5. The artificial intelligence cardiovascular surgery image guidance system according to claim 4, characterized in that: In the deep learning model, cardiovascular structure images of different sizes and shapes are input into the YOLOv10 image learning model for learning and training, including the following steps: Step 1, data preprocessing: normalizing and resizing the pruned cardiovascular structure images of different sizes and shapes to obtain preprocessed cardiovascular structure images; Step 2: Data annotation: annotate the preprocessed cardiovascular structure images, mark the location and category of abnormal areas, and generate annotated data sets for training the YOLOv10 model; abnormal areas include stenosis areas, calcification areas, and aneurysm areas; Step 3: Model initialization: Initialize the pruned and reconnected YOLOv10 image learning model, and set the learning rate, batch size, and number of iterations of the model; Step 4: Model training: Input the labeled dataset into the YOLOv10 image learning model, calculate the loss through forward propagation, and update the model weights through back propagation for training; Step 5: Model evaluation: After the training is completed, the YOLOv10 image learning model is evaluated using other pruned cardiovascular structure image sets of different sizes and shapes to generate evaluation results. Based on the evaluation results, the model parameters are fine-tuned to obtain a trained and evaluated YOLOv10 image learning model. Step 6: Model deployment: Deploy the trained and evaluated YOLOv10 image learning model into the cardiovascular surgery image guidance system for real-time detection and analysis of abnormal areas in cardiovascular structural images.

6. The artificial intelligence cardiovascular surgery image guidance system according to claim 5, characterized in that: In the evaluation module, the evaluation results include the error rate of simulated surgery, the rationality of surgical path planning, the accuracy of surgical time estimation, and the accuracy of abnormal area detection by the YOLOv10 image learning model.

7. The artificial intelligence-based cardiovascular surgery image guidance system according to claim 6, characterized in that: In the simulated surgery module, OpenGL 3D rendering technology combines cardiovascular structure images of different sizes and shapes with multimodal cardiovascular abnormality area analysis reports through texture mapping, lighting simulation and shadow processing steps to generate 3D models of cardiovascular structures, abnormal areas and surgical instruments in the 3D cardiovascular surgery simulation scene.

8. The artificial intelligence cardiovascular surgery image guidance system according to claim 7, characterized in that: The evaluation module includes a user feedback unit; The user feedback unit is used to collect doctors' feedback on the simulated surgical process, surgical path planning, surgical time estimation, and abnormal area detection accuracy, and send the feedback to the deep learning model.

9. The artificial intelligence cardiovascular surgery image guidance system according to claim 8, characterized in that: The assessment module also includes a postoperative assessment unit; The postoperative evaluation unit is used to collect postoperative surgical area images using the video image acquisition module, and use the YOLOv10 image learning model to learn and analyze the postoperative surgical area images to generate a visual report; Visual reports include surgical results and follow-up treatment options.

10. The artificial intelligence cardiovascular surgery image guidance system according to claim 9, characterized in that: In the guidance module, real-time guidance includes surgical path planning and key step prompts.

Images