Intubation fluorescent image feature learning system for hepatic artery chemoembolization

Abstract

The invention relates to the technical field of medical image processing and interventional therapy assistance, and discloses an intubation fluorescent image feature learning system for hepatic artery chemoembolization. Comprising an image acquisition module which is configured to acquire a distortionless two-dimensional fluorescent blood vessel image in a hepatic artery chemoembolization super-selective intubation process; the control module is used for extracting surface geometric features of the blood vessel from the two-dimensional fluorescent blood vessel image; inputting the surface geometric features to a preset spatial reasoning model, and outputting a spatial distribution probability matrix of each blood vessel part; capturing a difference signal between the spatial distribution probability matrix and angiography feedback data in an actual intubation process in real time, and forming an iterative training data set to optimize the spatial reasoning model; and identifying a potential complex blood vessel region according to the spatial distribution probability matrix, generating an intubation path early warning signal, and generating guide parameters of X-ray release opportunity and dose based on the early warning signal so as to reduce radiation exposure.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing and interventional therapy auxiliary technology, and in particular to a catheterization fluoroscopic image feature learning system for transcatheter arterial chemoembolization. BACKGROUND

[0002] Transcatheter arterial chemoembolization (TACE) is the core technology of liver cancer interventional therapy, and its key is to accurately insert a microcatheter into the tumor feeding artery branch through superselective catheterization to achieve targeted drug delivery and reduce damage to normal liver tissue. In clinical practice, two-dimensional digital subtraction angiography (DSA) fluoroscopic images are the main guiding means for the catheterization process due to their strong real-time performance and convenient operation, but they have inherent defects: due to the two-dimensional projection characteristics, complex and tortuous hepatic artery vessels are prone to image overlap, making it difficult for physicians to accurately determine the three-dimensional spatial relationship of the vessels (such as the front and back positions of the branches, the depth difference of the vessel shape, etc.), which affects the accuracy of the catheterization path planning. To solve the problem of image overlap, traditional solutions rely on cone beam CT (CBCT) or micro-ultrasound and other auxiliary technologies for three-dimensional vessel reconstruction, but such methods have significant limitations: limited application scenarios: the liver is adjacent to the stomach, intestines and other active organs, and their free movement will cause distortion of the CBCT reconstruction image, and the radioactivity of some TACE drug delivery and the microbiological safety requirements of interventional surgery further limit the use of auxiliary equipment; high cost and equipment threshold: micro-ultrasound and other technologies require additional hardware support, increasing the cost and complexity of the operation, making it difficult to popularize in primary hospitals; insufficient real-time performance: the image acquisition and reconstruction of auxiliary equipment need to interrupt the surgical procedure, which cannot meet the real-time guidance requirements of superselective catheterization.

[0003] In the prior art, the information mining of two-dimensional fluoroscopic images is limited to single vessel segmentation or morphological analysis, and there is a lack of effective reasoning methods for the spatial relationship of overlapping vessels.

[0004] Therefore, there is an urgent need to design a technical solution to solve at least one of the above technical problems. SUMMARY

[0005] The present application provides a catheterization fluoroscopic image feature learning system for transcatheter arterial chemoembolization, which aims to solve the problem that in the prior art, the information mining of two-dimensional fluoroscopic images is limited to single vessel segmentation or morphological analysis, and there is a lack of effective reasoning methods for the spatial relationship of overlapping vessels.

[0006] In a first aspect, the present application provides a catheterization fluoroscopic image feature learning system for transcatheter arterial chemoembolization, comprising:

[0007] An image acquisition module configured to acquire undistorted two-dimensional fluorescent blood vessel images containing complex blood vessel structures of a patient in a superselective catheterization process of transcatheter arterial chemoembolization (TACE);

[0008] A control module connected to the image acquisition module, configured to extract surface geometric features of blood vessels from the two-dimensional fluorescent blood vessel images, the surface geometric features including blood vessel branch morphology, tube diameter variation, and edge profile; input the surface geometric features into a preset spatial reasoning model to combine preset blood vessel anatomical prior knowledge, distinguish pixel differences of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of blood vessel branches, and output a spatial distribution probability matrix of each blood vessel part; capture difference signals between the spatial distribution probability matrix and angiography feedback data in an actual catheterization process in real time, form an iterative training data set to optimize the spatial reasoning model; identify potential complex blood vessel regions according to the spatial distribution probability matrix, generate a catheterization path early warning signal, and generate X-ray release timing and dose guidance parameters based on the early warning signal to reduce radiation exposure;

[0009] In some embodiments, the surface geometric features and spectral features are jointly analyzed to infer the spatial distribution relationship of complex hepatic artery blood vessels and to guide the catheterization process in real time without relying on cone beam CT or microscopic ultrasound assistance.

[0010] In some embodiments, the two-dimensional fluorescent blood vessel images are acquired in the superselective catheterization process of TACE by performing perspective projection calibration through a multi-angle calibration plate preset in a DSA device, combining an image distortion correction algorithm based on Zhang Zhengyou's calibration method to compensate for geometric distortion of original fluorescent images, and suppressing abdominal organ tissue noise through a blood vessel enhancement filtering algorithm to retain high-frequency detail features of blood vessel edges to highlight complex blood vessel structures to generate the two-dimensional fluorescent blood vessel images.

[0011] In some embodiments, the surface geometric features of blood vessels are extracted from the two-dimensional fluorescent blood vessel images by using a blood vessel segmentation model based on a U-Net convolutional neural network to perform pixel-level semantic segmentation on preprocessed images, identifying blood vessel branch nodes through a graph theory algorithm after extracting blood vessel skeletons, calculating branch angles, tube diameter variation rates, and Fourier descriptors of blood vessel edges, and forming a feature vector group containing blood vessel morphological parameters as the surface geometric features.

[0012] In some embodiments, the real-time capturing of the difference signal of the spatial distribution probability matrix and the angiography feedback data in the actual intubation process comprises: mapping the position sensor coordinates of the microcatheter tip to the fluoroscopic image coordinate system, tracking the catheter motion trajectory based on the optical flow method, comparing the actual blood vessel development gray value of the region passed by the catheter motion trajectory with the spectral feature probability distribution in the prediction matrix, and calculating the pixel-level cosine similarity deviation of the overlapping region as the difference signal.

[0013] In some embodiments, the forming of the iterative training data set to optimize the spatial reasoning model comprises: constructing a dynamic data cache queue, spatiotemporally aligning the image slices corresponding to the real-time captured difference signal, the surface geometric feature vector and the corrected spatial distribution label, using a transfer learning strategy to fine-tune the initially trained spatial reasoning model online, and balancing the training weights of new and old data through an adaptive learning rate adjustment mechanism.

[0014] In some embodiments, the identifying of the potential complex blood vessel region according to the spatial distribution probability matrix and the generation of the intubation path early warning signal comprise: setting a blood vessel spatial distribution probability entropy value threshold, performing region growing segmentation on the probability matrix, marking a region with a curvature greater than a preset threshold and a branch density more than 2 times the normal liver blood vessel anatomy model as a complex blood vessel region, superimposing a pseudo-color translucent marker on the two-dimensional image through a three-dimensional reconstruction algorithm, and generating voice warning information containing blood vessel tortuosity and branch risk level.

[0015] In some embodiments, the generating of the X-ray release timing and dose guidance parameters based on the early warning signal comprises: establishing a radiation dose-image signal-to-noise ratio optimization model, switching the X-ray exposure mode to pulse fluoroscopy for the complex blood vessel region, dynamically adjusting the pulse frequency according to the uncertainty of the blood vessel spatial distribution probability matrix, and generating personalized exposure time window suggestions in combination with a historical operation habit database.

[0016] In a second aspect, the present application provides an intubation fluoroscopic image feature learning method for transcatheter arterial chemoembolization, characterized in that it is applied to the control module of the intubation fluoroscopic image feature learning system for transcatheter arterial chemoembolization provided in any of the embodiments of the present application, and the method comprises:

[0017] The image acquisition module acquires undistorted two-dimensional fluoroscopic blood vessel images during the superselective intubation process of transcatheter arterial chemoembolization; the two-dimensional fluoroscopic blood vessel images contain complex blood vessel structures of patients with isomerism and flexion;

[0018] extracting surface geometric features of the blood vessels from the two-dimensional fluorescent vascular images, the surface geometric features including vascular branch morphology, tube diameter variation and edge profile; inputting the surface geometric features to a preset spatial reasoning model to combine preset vascular anatomical prior knowledge, distinguishing pixel differences of overlapping blood vessels through spectral feature analysis, constructing a three-dimensional spatial relationship prediction model of the blood vessel branches, and outputting a spatial distribution probability matrix of each blood vessel part;

[0019] real-time capturing of difference signals between the spatial distribution probability matrix and angiography feedback data in an actual intubation process, forming an iterative training data set to optimize the spatial reasoning model; identifying potential complex blood vessel regions according to the spatial distribution probability matrix, generating an intubation path warning signal, and generating guidance parameters of X-ray release timing and dosage based on the warning signal to reduce radiation exposure; wherein, through joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of the complex blood vessels of the hepatic artery is inferred and the real-time guidance of the intubation process is realized without relying on cone beam CT or microscopic ultrasound assistance.

[0020] In a third aspect, the present application provides a catheterization fluorescent image feature learning device for transcatheter arterial chemoembolization, which is applied to the control module of the catheterization fluorescent image feature learning system for transcatheter arterial chemoembolization provided in any of the embodiments of the present application, and the device comprises:

[0021] an image acquisition unit configured to acquire undistorted two-dimensional fluorescent vascular images obtained by the image acquisition module during the superselective intubation process of the transcatheter arterial chemoembolization; the two-dimensional fluorescent vascular images contain complex blood vessel structures of the patient isomers and flexures;

[0022] a matrix output unit configured to extract surface geometric features of the blood vessels from the two-dimensional fluorescent vascular images, the surface geometric features including vascular branch morphology, tube diameter variation and edge profile; inputting the surface geometric features to a preset spatial reasoning model to combine preset vascular anatomical prior knowledge, distinguishing pixel differences of overlapping blood vessels through spectral feature analysis, constructing a three-dimensional spatial relationship prediction model of the blood vessel branches, and outputting a spatial distribution probability matrix of each blood vessel part;

[0023] a guidance implementation unit configured to real-time capture difference signals between the spatial distribution probability matrix and angiography feedback data in an actual intubation process, form an iterative training data set to optimize the spatial reasoning model; identify potential complex blood vessel regions according to the spatial distribution probability matrix, generate an intubation path warning signal, and generate guidance parameters of X-ray release timing and dosage based on the warning signal to reduce radiation exposure; wherein, through joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of the complex blood vessels of the hepatic artery is inferred and the real-time guidance of the intubation process is realized without relying on cone beam CT or microscopic ultrasound assistance.

[0024] In a fourth aspect, the present application provides a control module, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and realize the method provided by any embodiment of the present application when executing the computer program.

[0025] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer readable instruction is executed by the processor to make one or more processors execute the method provided by any embodiment of the present application.

[0026] The present application belongs to the field of medical image processing and interventional therapy auxiliary technology, and particularly relates to a two-dimensional fluoroscopic image-based superselective intubation auxiliary system for transcatheter arterial chemoembolization (TACE), which solves the problems of complex blood vessel spatial distribution estimation and precise intubation guidance through image feature learning technology. The system comprises: an image acquisition module: obtaining two-dimensional fluoroscopic blood vessel images without distortion, retaining the details of complex blood vessel structure of isomerism and flexion; a control module: integrating surface geometric feature extraction (branch morphology, tube diameter change, edge contour, etc.), spatial reasoning model based on deep learning (combining anatomical prior knowledge and spectral feature analysis, constructing a three-dimensional spatial relationship prediction model, and outputting a blood vessel distribution probability matrix), differential signal capture (optimizing the model by comparing the prediction and actual contrast data), and early warning guidance (identifying complex blood vessel regions and generating X-ray release parameters), etc. Through the joint analysis of surface geometric features and spectral features, the system realizes blood vessel spatial distribution estimation and real-time intubation guidance based on two-dimensional fluoroscopic images without the aid of cone beam CT or microscopic ultrasound.

[0027] The system overcomes the problems of abdominal cavity organ interference, radioactivity safety, high cost, etc., and can complete complex blood vessel spatial relationship analysis without additional precise contrast equipment; through feature learning dynamic optimization of blood vessel distribution prediction model, the system can early warn potential complex areas and assist physicians in planning superselective intubation path; based on the prediction result, the system intelligently adjusts the X-ray release strategy, reduces the radiation exposure of medical staff and patients, and balances safety and effectiveness.

[0028] In summary, in the prior art, the information mining of two-dimensional fluorescence images is limited to single blood vessel segmentation or morphological analysis, and there is a lack of effective reasoning method for the spatial relationship of overlapping blood vessels. The present application creatively proposes a solution without relying on external auxiliary equipment: by jointly learning the surface geometric features (branch morphology, diameter change, etc.) and spectral features (pixel difference analysis) of two-dimensional fluorescence images, a blood vessel three-dimensional spatial distribution prediction model is constructed by combining a deep learning model, and is dynamically optimized through real-time contrast feedback data, which first realizes the intelligent guidance of complex blood vessel spatial relationship prediction and intubation path based on two-dimensional fluorescence images. This scheme breaks through the dependence on auxiliary equipment in traditional technology, solves the contradiction between abdominal cavity intervention, cost control and real-time performance, provides a new method of high efficiency, safety and universality for TACE superselective intubation, and fills the gap in the field of "two-dimensional image three-dimensional space reasoning and precise guidance without additional equipment" in the prior art. Through image feature learning technology, the spatial information value of two-dimensional fluorescence images is reconstructed, and the breakthrough of the blood vessel overlapping problem which is difficult to solve by traditional technology is realized without introducing complex auxiliary equipment, which significantly improves the safety and effectiveness of TACE surgery.

[0029] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0031] Figure 1 is a structural schematic diagram of an intubation fluorescence image feature learning system for hepatic arterial chemoembolization provided by an embodiment of the present application;

[0032] Figure 2 is a step schematic flow chart of an intubation fluorescence image feature learning method for hepatic arterial chemoembolization provided by an embodiment of the present application;

[0033] Figure 3 is a structural schematic block diagram of a control module provided by an embodiment of the present application.

[0034] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. DETAILED DESCRIPTION

[0035] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the present application.

[0036] The flowcharts shown in the drawings are only illustrative, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be further decomposed, combined or partially merged, so that the actual execution order can be changed according to actual conditions.

[0037] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second", etc. are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. also do not necessarily mean different.

[0038] It should be understood that the terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0039] It should also be understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0040] Some embodiments of the present application will be described in detail below with reference to the drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0041] Transcatheter arterial chemoembolization (TACE) is the core technology of interventional treatment for liver cancer, and its key is to accurately insert the microcatheter into the tumor feeding artery branch through superselective intubation, so as to achieve targeted drug delivery and reduce damage to normal liver tissue. In clinical practice, two-dimensional digital subtraction angiography (DSA) fluorescence images have become the main guiding means for intubation process due to their strong real-time performance and convenient operation, but they have inherent defects: due to the limitation of two-dimensional projection characteristics, complex and tortuous hepatic artery vessels are prone to image overlap, making it difficult for physicians to accurately determine the three-dimensional spatial relationship of the vessels (such as the front and back positions of the branches, the depth difference of the vessel shape, etc.), which affects the accuracy of intubation path planning. To solve the problem of image overlap, traditional solutions rely on cone beam CT (CBCT) or micro-ultrasound and other auxiliary technologies for three-dimensional vessel reconstruction, but such methods have significant limitations: limited application scenarios: the liver is adjacent to the stomach, intestines and other active organs, and their free movement will cause distortion of the CBCT reconstruction image, and the radioactivity of some TACE drug delivery and the microbiological safety requirements of interventional surgery further limit the use of auxiliary equipment; high cost and equipment threshold: micro-ultrasound and other technologies require additional hardware support, increasing the cost and complexity of the operation, making it difficult to popularize in primary hospitals; lack of real-time performance: the image acquisition and reconstruction of auxiliary equipment need to interrupt the surgical procedure, which cannot meet the real-time guidance requirements of superselective intubation.

[0042] In the prior art, information mining of two-dimensional fluorescence images is limited to single vessel segmentation or morphological analysis, and there is a lack of effective reasoning methods for the spatial relationship of overlapping vessels.

[0043] Therefore, there is an urgent need to design a technical solution to solve at least one of the above technical problems.

[0044] It should be noted that any information involved in the present application is obtained with the permission of the relevant users and in accordance with relevant policies. It will not infringe on the personal information of the users.

[0045] To solve the above problems, please refer to Figure 1The application provides a catheterization fluorescence image feature learning system for transcatheter arterial chemoembolization, comprising: an image acquisition module configured to acquire undistorted two-dimensional fluorescence blood vessel images during the superselective catheterization process of transcatheter arterial chemoembolization, wherein the two-dimensional fluorescence blood vessel images contain complex blood vessel structures of patient isomorphism and inflection; a control module connected to the image acquisition module, used to extract the surface geometric features of the blood vessels from the two-dimensional fluorescence blood vessel images, wherein the surface geometric features include blood vessel branch morphology, tube diameter change and edge contour; input the surface geometric features into a preset spatial reasoning model to combine the preset blood vessel dissection priori knowledge, distinguish the pixel difference of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of blood vessel branches, and output the spatial distribution probability matrix of each blood vessel part; capture the difference signal of the spatial distribution probability matrix and the angiography feedback data in the actual catheterization process in real time, form an iterative training data set to optimize the spatial reasoning model; identify the potential complex blood vessel area according to the spatial distribution probability matrix, generate a catheterization path early warning signal, and generate X-ray release timing and dose guidance parameters based on the early warning signal to reduce radiation exposure; wherein through the joint analysis of the surface geometric features and the spectral features, the speculation of the spatial distribution relationship of the complex hepatic artery and the real-time guidance of the catheterization process are realized without relying on cone beam CT or microscopic ultrasound assistance.

[0046] Specifically, the system proposes a real-time guidance scheme based on joint analysis of fluorescence image features to solve the three-dimensional spatial relationship judgment problem caused by two-dimensional digital subtraction angiography (DSA) image overlap in transcatheter arterial chemoembolization (TACE).

[0047] Traditional two-dimensional DSA images cannot accurately reflect the three-dimensional spatial relationship of complex hepatic arteries (such as branch position before and after, shape depth difference) due to projection overlap, while auxiliary technologies such as cone beam CT (CBCT) and microscopic ultrasound are limited by organ motion artifacts, high cost, and insufficient real-time performance, making it difficult to meet the precise guidance needs of superselective catheterization. Existing technologies only stop at single blood vessel segmentation or morphological analysis, lacking reasoning ability for spatial relationship of overlapping blood vessels.

[0048] The system realizes three-dimensional spatial relationship speculation without additional hardware dependence through a closed loop of "image acquisition-feature extraction-spatial reasoning-real-time optimization-guidance output", and the specific modules include: an image acquisition module that acquires undistorted two-dimensional fluorescence blood vessel images, which completely retains complex blood vessel structures such as bifurcation angle and inflection path.

[0049] The control module extracts geometric features, quantifies blood vessel branch morphology (bifurcation angle, branch number), tube diameter change (diameter gradient), edge profile (curvature, smoothness), etc. Spatial reasoning model: integrate blood vessel anatomical prior knowledge (such as standard anatomical atlas of hepatic artery, common variation type) and spectral feature analysis (pixel gray level / spectral difference in overlapping area), construct a three-dimensional spatial relationship prediction model, output the spatial distribution probability matrix of blood vessel branches (such as the probability of a branch being located in the "front" or "rear" of another branch). Real-time iterative optimization: capture the difference between the prediction matrix and the actual data through catheter actual operation feedback (such as catheter resistance, contrast agent diffusion path), dynamically update the training data set, and continuously optimize the model reasoning accuracy. Guiding parameter generation: based on the spatial distribution probability matrix, identify high overlap risk areas (such as complex bifurcation, deep migration curve segment), generate catheterization path warning signals (such as suggesting adjusting the catheter angle), and calculate the X-ray release time and dose (such as reducing unnecessary exposure) combined with radiation safety requirements.

[0050] Image acquisition and preprocessing includes: distortion-free image acquisition: preoperatively calibrate the DSA device through the calibration plate, correct the X-ray source angle and detector position deviation, and ensure that the two-dimensional image has no perspective distortion; intraoperatively use respiratory gating technology or motion compensation algorithm to reduce the interference of gastrointestinal peristalsis on blood vessel morphology, and obtain stable and clear fluorescent sequence images. ROI demarcation: automatically identify the hepatic artery region (based on liver contour positioning and blood vessel enhancement algorithm), and focus on the blood vessel segments of interest (such as hepatic proper artery, tumor blood supply branch).

[0051] Vessel segmentation: using an improved U-Net network or a dynamic contour model based on threshold (Snake model), combined with the blood vessel enhancement characteristics of DSA images, accurately segmenting the main stem and branches of blood vessels, and outputting binary blood vessel masks. Geometric parameter calculation: branch morphology: based on graph theory, build a blood vessel tree structure, calculate bifurcation point coordinates, branch angle (such as acute / obtuse bifurcation), and branch order (distinguish between primary and secondary branches). Tube diameter change: along the blood vessel centerline, sample, fit the blood vessel cross-sectional diameter through Gaussian, and generate a tube diameter change curve (identify stenosis or expansion segment). Edge profile: extract the Fourier descriptor or curvature histogram of the blood vessel edge, and quantify the edge smoothness and tortuosity.

[0052] The spatial reasoning model construction includes: anatomical prior knowledge embedding: construct a hepatic artery anatomical atlas knowledge base, including the typical spatial distribution probability of each branch (such as the right hepatic artery usually located in front of the right branch of the portal vein), and the variation type (such as the shape rule of the replacement hepatic artery), as the constraint condition of model reasoning. Spectral feature analysis: in the overlapping area, based on the dynamic change of pixel gray scale of multiple image (such as the difference of contrast agent filling time sequence), extract the spectral feature vector (such as time-density curve), and distinguish the pixel signal difference of different depth blood vessels through transfer learning (assuming that the deep blood vessels show lower gray value due to tissue attenuation). Model architecture: adopt graph neural network (GNN) or 3D convolutional neural network (3D-CNN), input geometric feature matrix and spectral feature vector, and output the spatial position probability distribution of each blood vessel segment (such as the probability density function of X / Y / Z axis in three-dimensional coordinate system).

[0053] Difference signal capture: during the catheter insertion, the actual position of the catheter is obtained through the pressure sensor or electromagnetic positioning device, and the spatial distribution of the model prediction is registered to calculate the position deviation and the abnormal value of the path resistance, which is used as the model prediction error signal. Online learning strategy: using small batch stochastic gradient descent (SGD) algorithm, the real-time error signal is back propagated to the spatial reasoning model, and the network weight is dynamically updated to adapt to the vascular anatomical variation of individual patients (such as abnormal branch shape). Complex region identification: set the spatial distribution probability threshold (such as when the probability difference of branch before and after is greater than 0.7, it is marked as high overlap risk area), and superimpose visual warning on DSA image (such as red highlight display of potential path blind area). Radiation optimization algorithm: based on the spatial distribution probability matrix of blood vessels, the minimum exposure field and time required for catheterization operation are predicted, and X-ray machine control instructions (such as pulse video frequency adjustment, exposure area dynamic scaling) are generated, reducing the radiation exposure of doctors and patients.

[0054] No CBCT / ultrasound hardware: only relying on conventional DSA equipment, avoiding the distortion problem caused by gastrointestinal movement, at the same time meeting the requirements of microbiological safety and radiation protection of interventional surgery (without introducing additional equipment to increase the risk of infection). Basic hospital applicability: reduce hardware cost and operation complexity, so that superselective catheterization technology can be popularized in medical institutions with limited equipment configuration.

[0055] Combined feature analysis advantage: through the cross verification of surface geometric features (reflecting blood vessel morphology) and spectral features (reflecting pixel signal difference), combined with anatomical prior knowledge constraint, the three-dimensional structure ambiguity caused by two-dimensional image overlap is effectively solved, and the reasoning accuracy is significantly higher than that of single morphological analysis method. Real-time closed-loop optimization: real-time training model using operation feedback data during operation, dynamically adapting to individual vascular variation, avoiding the delay problem of traditional offline reconstruction technology, meeting the demand of superselective catheterization for "real-time decision" (such as immediate guidance of millimeter level position adjustment of catheter head).

[0056] Precise path planning: By clarifying the hierarchical relationship of blood vessel branches through spatial distribution probability matrix, it helps doctors to predict the direction of catheter advancement (such as avoiding misentry into deep overlapping branches), reduces the number of trial insertion, and reduces the risk of vascular endothelial injury. Radiation safety optimization: Based on real-time inference results, intelligent control of X-ray exposure is expected to reduce 30%-50% of invalid radiation exposure, in line with the principle of "radiation dose optimization" (ALARA) of interventional surgery. Enhanced treatment effect: Targeted delivery of drugs to tumor feeding arteries reduces normal liver tissue drug exposure, increases local drug concentration while reducing systemic side effects, and potentially improves TACE efficacy and patient prognosis. For the first time, the "geometric-spectral feature joint inference" framework based on two-dimensional fluorescence images is proposed, providing a new idea for image guidance of vascular interventional surgery, which can be extended to interventional treatment of other complex blood vessel systems such as renal artery and cerebral blood vessels, with cross-department application potential.

[0057] The system deeply mines the multi-dimensional features of two-dimensional fluorescence images, fuses anatomical priori and real-time feedback, and realizes the inference of three-dimensional spatial relationship of complex hepatic artery and real-time guidance of intubation without additional hardware assistance, effectively solving the application bottleneck of traditional technology, with technical innovation, clinical practicability and cost effectiveness, providing key support for the precision development of TACE surgery.

[0058] In some embodiments, during the superselective intubation process of transcatheter arterial chemoembolization, the undistorted two-dimensional fluorescence blood vessel image is obtained, including: through perspective projection calibration of a multi-angle calibration plate preset in a DSA device, combining an image distortion correction algorithm based on Zhang Zhengyou's calibration method, compensating for the geometric distortion of the original fluorescence image, and suppressing the noise of abdominal organ tissue through a blood vessel enhancement filtering algorithm, the high-frequency detail features of the blood vessel edge are retained to highlight the complex blood vessel structure, to generate the two-dimensional fluorescence blood vessel image.

[0059] In the X-ray projection path of the DSA device, a multi-angle calibration plate containing a concentric circle array or a grid pattern is placed at a fixed position of the patient examination bed. The calibration plate is made of high-density metal (such as titanium alloy) and low-density plastic arranged alternately, forming clear perspective contrast marks.

[0060] The calibration plate images under different X-ray incident angles (such as 0°, 30°, 60°, -30°, etc.) are collected before surgery, and the feature point coordinates (such as circle centers, grid intersection points) on the calibration plate are extracted through image processing to establish the geometric projection model of the device and calculate the calibration parameters of the X-ray source position and the detector plane parameters (such as offset, rotation angle).

[0061] Based on Zhang Zhengyou's calibration algorithm, a perspective imaging model of the DSA device is established to map the pixel coordinates of the original fluorescence image to the world coordinate system.

[0062] For radial distortion (barrel / pincushion distortion) and tangential distortion caused by X-ray perspective, distortion coefficients (k1, k2, p1, p2) are solved by least square method, and geometric correction is performed on each pixel:

[0063]

[0064] where (x raw , y raw ) is the distorted pixel coordinate, . A multi-scale difference of Gaussian filter (DOG) is used to process the corrected image. By difference operation of Gaussian kernel with different standard deviations (σ1=1.0, σ2=2.0), the high-frequency details of the blood vessel edge are enhanced: IDOG=Gσ1*I-Gσ2*I;

[0065] where Gσ is the Gaussian kernel, and I is the input image. Combined with the adaptive threshold segmentation algorithm (Niblack method), the noise of abdominal organ tissue is suppressed. The threshold is dynamically set according to the local gray mean and variance, and the high-contrast signal of the blood vessel region is preserved.

[0066] By multi-angle calibration and Zhang Zhengyou algorithm, the geometric distortion error of DSA image is controlled within 0.5 pixels, ensuring the measurement accuracy of blood vessel branch angle and length, and avoiding spatial relationship misjudgment caused by distortion. Multi-scale filtering enhances the continuity of the blood vessel edge, suppresses the motion artifacts and soft tissue noise caused by gastrointestinal peristalsis, and makes the fine branches of tortuous blood vessels (such as tumor feeding arteries with a diameter of <1 mm) clear and identifiable, providing a high-quality data source for subsequent feature extraction. The calibration process does not require patient cooperation and can be integrated into the self-checking process of the DSA device, without increasing the additional operation time, and is suitable for emergency or routine TACE surgery.

[0067] By multi-angle calibration and Zhang Zhengyou algorithm, the geometric distortion error of DSA image is controlled within 0.5 pixels, ensuring the measurement accuracy of blood vessel branch angle and length, and avoiding spatial relationship misjudgment caused by distortion. Multi-scale filtering enhances the continuity of the blood vessel edge, suppresses the motion artifacts and soft tissue noise caused by gastrointestinal peristalsis, and makes the fine branches of tortuous blood vessels (such as tumor feeding arteries with a diameter of <1 mm) clear and identifiable, providing a high-quality data source for subsequent feature extraction. The calibration process does not require patient cooperation and can be integrated into the self-checking process of the DSA device, without increasing the additional operation time, and is suitable for emergency or routine TACE surgery.

[0068] In some embodiments, the surface geometric features of the blood vessels are extracted from the two-dimensional fluorescent vascular image, including: adopting a blood vessel segmentation model based on a U-Net convolutional neural network to perform pixel-level semantic segmentation on the preprocessed image, identifying blood vessel branch nodes by a graph theory algorithm after extracting a blood vessel skeleton, calculating a branch angle, a pipe diameter change rate, and a Fourier descriptor of a blood vessel edge, and forming a feature vector group containing blood vessel morphological parameters as the surface geometric features.

[0069] The U-Net convolutional neural network blood vessel segmentation is performed by constructing a U-Net model with a depth of 5 layers, the input layer is a single-channel fluorescent image (size 256*256), the encoding path adopts 3*3 convolution + ReLU + 2*2 maximum pooling, the decoding path adopts deconvolution + jump connection, and the output layer generates a blood vessel probability map (pixel value 0-1) through a Sigmoid activation function. The model is trained using a Dice loss function and an Adam optimizer, and the data set contains 1000 DSA images (annotated blood vessel mask) of TACE surgery, and the enhancement strategy includes rotation, flipping, and brightness adjustment to ensure the generalization ability of the model to individual blood vessel differences. The preprocessed image is segmented, the probability map is binarized by the Otsu threshold method, and a binary blood vessel mask M(x, y) (blood vessel region is 1 and background is 0) is generated.

[0070] The blood vessel skeleton extraction and branch node identification are performed by using a Hessian matrix blood vessel skeleton thinning algorithm to iteratively erode the binary mask, retain a single-pixel width blood vessel centerline, and generate a skeleton image S(x, y).

[0071] The blood vessel tree structure is constructed based on graph theory, the pixel points on the skeleton are defined as nodes of the graph, and adjacent pixels are edges. The branch nodes are identified by calculating the degree (number of connected edges) of the nodes: nodes with a degree of ≥3 are bifurcation points, and nodes with a degree of =1 are end points.

[0072] Morphological parameter calculation: branch angle: for each bifurcation point, the vector angle of the two branch centerlines is calculated, and the vector dot product formula is used , the result is mapped to 0-180°. The tube diameter change rate: along the centerline of the blood vessel, every 0.5mm is sampled, the blood vessel diameter is measured by local threshold method, the ratio of the diameter difference of adjacent sampling points to the upstream diameter (such as Δd / dupstream) is calculated, which reflects the degree of gradual change of the tube diameter. The Fourier descriptor converts the blood vessel edge contour coordinates into frequency domain coefficients by discrete Fourier transform, retains the first 20 low frequency coefficients (removes high frequency noise), reconstructs the edge shape feature vector, and quantifies the blood vessel tortuosity and smoothness. The Dice coefficient of the U-Net model is greater than or equal to 0.92, which can accurately segment the edge of the overlapping blood vessels, especially for the low-contrast micro-branches (such as tumor neovascularization), which has robustness and avoids the missed detection problem of traditional threshold segmentation. Through graph theory and Fourier analysis, the complex blood vessel morphology is converted into calculable numerical features (such as branch angle, tube diameter change rate), which provides structured input for spatial reasoning model and solves the subjectivity problem of traditional qualitative analysis. The processing time of a single frame of image is less than 50ms, which meets the feature updating requirement of high frequency (15 frames per second) in real-time intubation process, and avoids the influence of calculation delay on real-time guidance.

[0073] In some embodiments, the real-time capture of the difference signal between the spatial distribution probability matrix and the angiography feedback data in the actual intubation process includes: mapping the position sensor coordinates of the microcatheter tip to the fluoroscopic image coordinate system, tracking the catheter motion trajectory based on the optical flow method, comparing the actual blood vessel development gray value of the region passed by the catheter motion trajectory with the spectral feature probability distribution in the prediction matrix, and calculating the pixel-level cosine similarity deviation of the overlapping region as the difference signal.

[0074] The microcatheter position sensor coordinate mapping is achieved by integrating a miniature electromagnetic positioning sensor (such as Polhemus Fastrak) at the tip of the microcatheter, with a sensor accuracy of less than or equal to 0.5mm, and by establishing a conversion matrix T between the sensor coordinate system and the DSA image coordinate system through preoperative calibration: (ximage, yimage) = T*(xsensor, ysensor, zsensor); wherein (x sensor, y sensor, z sensor) are three-dimensional coordinates of the sensor, which are mapped to two-dimensional image plane to obtain the catheter tip position (ximage, yimage).

[0075] The optical flow method tracks the catheter motion trajectory by using Lucas-Kanade optical flow algorithm to process the continuous frame of fluoroscopic images, marking feature points (such as metal markers) at the tip of the catheter, tracking their displacement vectors (dx, dy) in adjacent frames, generating the catheter motion trajectory P(t) = [(x1, y1), (x2, y2), …, (xn, yn)], with a time resolution of 30ms.

[0076] The pixel-level cosine similarity deviation calculation includes extracting the actual image gray value vector Vreal = [v1, v2, …, vm] (the local region pixel gray scale of length m) and the spectral feature probability distribution vector Vpred = [p1, p2, …, pm] (the probability is normalized to 0-1) predicted by the spatial reasoning model for the overlapping blood vessel region passed by the track.

[0077] The cosine similarity of the two is calculated The difference signal is defined as Δ = 1-cosθ, and the model update signal is triggered when Δ>0.3.

[0078] By combining the electromagnetic sensor with the optical flow method, sub-millimeter-level tracking of the catheter position is realized, solving the error problem of traditional naked eye judgment of the catheter position (especially in the overlapping blood vessel region), and providing real operation data calibration for the model. The gray scale signal difference is converted into a calculable numerical index by cosine similarity, which sensitively captures the prediction deviation in the overlapping region (such as the significant gray scale value difference when the deep blood vessel is misjudged as shallow), avoiding the ambiguity of traditional qualitative feedback. The calculation delay of the optical flow algorithm is <10ms, which is synchronized with the frame rate (15-30 frames / second) of the DSA device, ensuring that the difference signal capture is real-time aligned with the catheter movement, meeting the needs of rapid adjustment in super-selective intubation.

[0079] In some embodiments, the forming of the iterative training data set to optimize the spatial reasoning model includes: constructing a dynamic data cache queue, spatiotemporally aligning the image slices corresponding to the difference signals captured in real time, the surface geometric feature vectors, and the corrected spatial distribution labels, using a transfer learning strategy to fine-tune the initially trained spatial reasoning model online, and balancing the training weights of new and old data through an adaptive learning rate adjustment mechanism.

[0080] The dynamic data cache queue is constructed by establishing a first-in-first-out (FIFO) cache queue with a capacity of 500 cases, storing the training samples captured in real time, each sample including: image slices: cutting a 50x50 pixel ROI region centered on the catheter tip position; surface geometric feature vectors: including 12-dimensional parameters such as branch angle, pipe diameter change rate, and Fourier descriptor; and corrected spatial distribution labels: real spatial relationships (such as branch A being located in front of branch B) artificially labeled according to the actual contrast agent diffusion path or generated through multi-modal fusion (such as ultrasound short-time assistance). The spatiotemporal alignment and data enhancement are performed by time stamp calibration on samples at different time points to ensure the time consistency of the image slices, feature vectors, and labels; and data enhancement methods such as elastic deformation and Gaussian noise addition are used to expand the sparse real-time samples (such as when the sample amount of complex bifurcation regions is insufficient), avoiding overfitting.

[0081] Transfer learning and online fine-tuning are based on a pre-trained spatial reasoning model (trained based on 1000 historical data), the first 3 convolutional layers are frozen (to retain anatomical prior knowledge), and the last 2 fully connected layers are fine-tuned online; the loss function uses cross-entropy loss combined with KL divergence to constrain the difference between the predicted probability distribution and the true label:

[0082] L = -∑y i logp i + αD KL (q|p);

[0083] wherein yi is the true label, pi is the predicted probability, q is the prior distribution of the pre-trained model, and a is the balance coefficient. Adaptive learning rate adjustment is achieved by using the AdamW optimizer, with an initial learning rate of 1e-4. When the average difference signal of 10 consecutive batches decreases by less than 5%, the learning rate is reduced by 0.5 times to avoid excessive parameter oscillation during early training, while ensuring the model's sensitivity to new data.

[0084] By fine-tuning the model with real-time operation data, the prediction bias problem of individual patient vascular anatomy variation (such as alternative hepatic artery shape) is solved, and the model reasoning accuracy is improved from 75% in offline state to more than 88% in online state. Dynamic cache queue combined with transfer learning can update the model with only a small amount of real-time samples (about 50-100 effective samples per operation), avoiding the burden of large data storage and computation during the operation, and adapting to the tight time window of interventional surgery. Freezing the base layer parameters and introducing KL divergence constraint prevents online training from destroying the learned anatomical prior knowledge, ensuring that the model can make reasonable inferences based on prior knowledge when there is insufficient new data, and avoiding the blindness of relying entirely on real-time data.

[0085] In some embodiments, the identification of potential complex vascular regions according to the spatial distribution probability matrix and the generation of intubation path warning signals include: setting a vascular spatial distribution probability entropy threshold, region growing segmentation of the probability matrix, marking regions with curvature greater than a preset threshold and branch density exceeding 2 times the normal liver vascular anatomy model as complex vascular regions, superimposing pseudo-color translucent markers on two-dimensional images through three-dimensional reconstruction algorithms, and generating voice warning information containing vascular tortuosity and branch risk level.

[0086] Spatial distribution probability entropy value calculation is performed on the probability matrix P(x, y, z) output by the spatial reasoning model to calculate the entropy value H = -∑pi log pi. The higher the entropy value, the stronger the uncertainty of the vascular spatial relationship in that region (such as severely overlapping regions). Set the entropy threshold Hth = 1.5 (based on Shannon entropy normalization to 0-2) to screen out high-uncertainty regions as candidate complex regions.

[0087] Region growing segmentation and anatomical model matching By taking high-entropy pixels as seed points, a region growing algorithm (growth condition: adjacent pixel entropy value > 1.2 and geometric feature similarity > 0.8) is used to segment the continuous complex blood vessel region. The average curvature (through the second derivative of the skeleton curve) and branch density (branch node number per unit area) of the region are calculated, and compared with the normal anatomical model of the liver blood vessels. When the curvature is > 0.3 / mm and the branch density is > 2 times the normal threshold, it is marked as a high-risk complex region.

[0088] Visualization and voice warning generation includes pseudo-color semi-transparent marking: map the three-dimensional spatial distribution probability to the two-dimensional image, encode the depth information using the HSV color space (e.g. red for shallow layer, blue for deep layer), superimpose a 50% transparent pseudo-color mask on the complex region, and retain the original image details while highlighting the spatial hierarchy. Voice warning information: generate personalized voice prompts based on region characteristics, such as "high tortuosity, suggest reducing catheter advancement speed" "abnormal branch density, pay attention to distinguishing front and back layer blood vessels", the warning content includes risk level (I / II / III level) and operation suggestion.

[0089] Through double screening of entropy value and anatomical model, the complex region (such as three-dimensional crossing blood vessel loop) that is difficult to identify by traditional two-dimensional image is accurately located, and the missed detection rate is reduced from 40% of the traditional method to less than 15%. Pseudo-color marking combined with voice prompts provides visual and auditory feedback, reducing visual fatigue caused by long-term concentration of physicians, especially in emergency operation scenarios (such as catheter impaction), providing immediate decision support. Based on the quantitative indicators of curvature and branch density, a unified complex blood vessel evaluation standard is established to avoid differences in subjective judgments by different physicians and improve the tacit understanding of the surgical team.

[0090] In some embodiments, the generation of X-ray release timing and dose guidance parameters based on the warning signal includes: establishing a radiation dose-image signal-to-noise ratio optimization model, switching the X-ray exposure mode to pulse fluoroscopy for complex blood vessel regions, dynamically adjusting the pulse frequency according to the uncertainty of the blood vessel spatial distribution probability matrix, and generating personalized exposure time window recommendations in combination with a historical operation habit database.

[0091] By establishing a mathematical model D = f (SNR, uncertainty), where D is the radiation dose, SNR is the image signal-to-noise ratio (target value > 20 dB), and uncertainty is the uncertainty of the spatial distribution probability matrix (range 0-1 after standardization of entropy value). For low uncertainty regions (entropy value < 1.0), a low dose mode (exposure dose reduced by 30%) is used; for high uncertainty regions (entropy value > 1.5), the dose is automatically increased to 1.2 times the standard value to ensure the clarity of blood vessel visualization in complex regions.

[0092] The dynamic switching of the pulsed fluoroscopy mode includes the use of 2 frames per second of pulsed fluoroscopy in the conventional region, and the dynamic adjustment of the pulse frequency to 6 frames per second in the complex region according to the update frequency of the probability matrix (such as detecting once every 50 ms), while limiting the single exposure time < 200 ms to avoid the cumulative radiation of continuous exposure.

[0093] The exposure control instructions are sent in real time through the API interface of the DSA device to realize the linkage of the exposure mode and the risk level of the region (such as automatically switching to a high-frequency pulsed mode when entering a complex region).

[0094] The personalized exposure time window suggestion is achieved by constructing a historical operation habit database to record the average exposure time, dose preference and other parameters of different physicians in similar blood vessel structures, and generating a personalized time window suggestion for the current physician through a K-NN algorithm (such as extending the exposure time by 10% for physicians who prefer accurate operation).

[0095] The basic dose is adjusted in combination with the patient's body size parameters (BMI, liver depth) according to the formula: Dadjusted = Dbase x (1 + 0.1 x log(BMI / 22)); to ensure that the blood vessel imaging quality of obese patients is not reduced, while avoiding excessive exposure.

[0096] Through dynamic dose adjustment and pulsed fluoroscopy, the radiation dose of a single operation is expected to be reduced by 35%-45% compared with the traditional method, especially in the high-frequency exposure scene of complex blood vessel regions, effectively reducing the risk of cataract for physicians and the probability of radiation-induced cancer for patients. Based on the uncertainty of dose optimization, while ensuring clear imaging of blood vessels in complex regions, it avoids excessive exposure in conventional regions, solves the low efficiency problem of the traditional "one-size-fits-all" exposure mode. Personalized time window suggestion combined with physician habits reduces the parameter adjustment time in the operation process, so that the physician can focus more on catheter control, especially for junior physicians to provide standardized radiation safety guidance and reduce the learning curve difficulty.

[0097] Please refer to Figure 2 , Figure 2 is a schematic flowchart of a catheterization fluoroscopy image feature learning method for hepatic arterial chemoembolization provided in an embodiment of the present application. The execution device of the method is a control module of a catheterization fluoroscopy image feature learning system for hepatic arterial chemoembolization provided in any embodiment of the present application.

[0098] As shown in Figure 2 , the provided method includes steps S101 to S103. The control module can be a handheld terminal, a notebook computer, a wearable device, or a robot, etc. The steps S101 to S103 and the corresponding embodiments for implementing them are provided.

[0099] Step S101. Obtain a two-dimensional undistorted fluorescent blood vessel image during the superselective intubation process of the hepatic arterial chemoembolization; the two-dimensional undistorted fluorescent blood vessel image contains complex blood vessel structures of the patient isomers and flexures;

[0100] Step S102. Extract the surface geometric features of the blood vessels from the two-dimensional fluorescent blood vessel image, including the blood vessel branch morphology, the tube diameter change and the edge contour; input the surface geometric features into a preset spatial reasoning model to combine the preset blood vessel anatomical prior knowledge, distinguish the pixel differences of the overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of the blood vessel branches, and output a spatial distribution probability matrix of each blood vessel part;

[0101] Step S103. Real-time capture the difference signal of the spatial distribution probability matrix and the angiography feedback data in the actual intubation process, form an iterative training data set to optimize the spatial reasoning model; identify the potential complex blood vessel area according to the spatial distribution probability matrix, generate an intubation path warning signal, and generate X-ray release timing and dose guidance parameters based on the warning signal to reduce radiation exposure; wherein, through the joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of the complex blood vessels of the hepatic artery is speculated and the real-time guidance of the intubation process is realized without relying on cone beam CT or micro-ultrasound assistance.

[0102] In some embodiments, during the superselective intubation process of the hepatic arterial chemoembolization, the two-dimensional undistorted fluorescent blood vessel image is obtained, including: performing perspective projection calibration through a multi-angle calibration plate preset in the DSA device, combining an image distortion correction algorithm based on Zhang Zhengyou's calibration method to compensate for the geometric distortion of the original fluorescent image, and suppressing the abdominal organ tissue noise through a blood vessel enhancement filtering algorithm to retain the high-frequency detail features of the blood vessel edge to highlight the complex blood vessel structure to generate the two-dimensional fluorescent blood vessel image.

[0103] In some embodiments, the surface geometric features of the blood vessels are extracted from the two-dimensional fluorescent blood vessel image, including: using a blood vessel segmentation model based on a U-Net convolutional neural network to perform pixel-level semantic segmentation on the preprocessed image, identifying the blood vessel branch nodes through a graph theory algorithm after extracting the blood vessel skeleton, calculating the branch angle, the tube diameter change rate and the Fourier descriptor of the blood vessel edge, and forming a feature vector group containing the morphological parameters of the blood vessels as the surface geometric features.

[0104] In some embodiments, the real-time capturing of the difference signal of the spatial distribution probability matrix and the angiography feedback data in the actual intubation process comprises: mapping the position sensor coordinates of the microcatheter tip to the fluoroscopic image coordinate system, tracking the catheter motion trajectory based on the optical flow method, comparing the actual blood vessel development gray value of the region passed by the catheter motion trajectory with the spectral feature probability distribution in the prediction matrix, and calculating the pixel-level cosine similarity deviation of the overlapping region as the difference signal.

[0105] In some embodiments, the forming of the iterative training data set to optimize the spatial reasoning model comprises: constructing a dynamic data cache queue, spatiotemporally aligning the image slices corresponding to the real-time captured difference signal, the surface geometric feature vector, and the corrected spatial distribution label, using a transfer learning strategy to online fine-tune the initially trained spatial reasoning model, and balancing the training weights of new and old data through an adaptive learning rate adjustment mechanism.

[0106] In some embodiments, the identifying of the potential complex blood vessel region according to the spatial distribution probability matrix and the generation of the intubation path early warning signal comprise: setting a blood vessel spatial distribution probability entropy threshold, performing region growing segmentation on the probability matrix, marking a region with a curvature greater than a preset threshold and a branch density exceeding 2 times of a normal liver blood vessel anatomical model as a complex blood vessel region, superimposing a pseudo-color translucent marker on the two-dimensional image through a three-dimensional reconstruction algorithm, and generating voice warning information containing blood vessel tortuosity and branch risk level.

[0107] In some embodiments, the generating of the X-ray release timing and dose guidance parameters based on the early warning signal comprises: establishing a radiation dose-image signal-to-noise ratio optimization model, switching the X-ray exposure mode to pulse fluoroscopy for the complex blood vessel region, dynamically adjusting the pulse frequency according to the uncertainty of the blood vessel spatial distribution probability matrix, and generating personalized exposure time window suggestions in combination with a historical operation habit database.

[0108] It should be noted that, for the convenience and brevity of description, the specific working processes of the above-described intubation fluoroscopic image feature learning method for hepatic arterial chemoembolization and each step can be clearly understood by those skilled in the art, which can refer to the corresponding processes in the above-described intubation fluoroscopic image feature learning system embodiments for hepatic arterial chemoembolization, which will not be described here.

[0109] The embodiments of the present application also provide a catheterization fluoroscopy image feature learning device for hepatic arterial chemoembolization. The catheterization fluoroscopy image feature learning device for hepatic arterial chemoembolization is used to execute the steps of the catheterization fluoroscopy image feature learning method for hepatic arterial chemoembolization shown in the above embodiments. The catheterization fluoroscopy image feature learning device for hepatic arterial chemoembolization can be a single server or a server cluster, or the catheterization fluoroscopy image feature learning device for hepatic arterial chemoembolization can be a terminal, which can be a handheld terminal, a notebook computer, a wearable device, or a robot, etc.

[0110] The catheterization fluoroscopy image feature learning device for hepatic arterial chemoembolization comprises:

[0111] An image acquisition unit is configured to acquire undistorted two-dimensional fluoroscopy blood vessel images obtained by the image acquisition module during the superselective catheterization process of the hepatic arterial chemoembolization. The two-dimensional fluoroscopy blood vessel images contain complex blood vessel structures of patient isomerism and inflection.

[0112] A matrix output unit is configured to extract surface geometric features of blood vessels from the two-dimensional fluoroscopy blood vessel images, wherein the surface geometric features include blood vessel branch morphology, tube diameter change, and edge profile. The surface geometric features are input to a preset spatial reasoning model to combine preset blood vessel dissection prior knowledge, distinguish pixel differences of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of blood vessel branches, and output a spatial distribution probability matrix of each blood vessel part.

[0113] A guidance implementation unit is configured to capture difference signals between the spatial distribution probability matrix and angiography feedback data in the actual catheterization process in real time, form an iterative training data set to optimize the spatial reasoning model, identify potential complex blood vessel regions according to the spatial distribution probability matrix, generate a catheterization path early warning signal, and generate guidance parameters of X-ray release timing and dose based on the early warning signal to reduce radiation exposure. Through joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of the complex blood vessels of the hepatic artery is inferred and the real-time guidance of the catheterization process is realized without relying on cone beam CT or microscopic ultrasound assistance.

[0114] It should be noted that, for the convenience and brevity of description, the specific working processes of the catheterization fluoroscopy image feature learning device for hepatic arterial chemoembolization and each unit described above can be referred to the corresponding processes in the catheterization fluoroscopy image feature learning method for hepatic arterial chemoembolization described in the above embodiments, which will not be described here in detail.

[0115] The method for learning the fluoroscopic image features of the catheterization for hepatic arterial chemoembolization described above is implemented in the form of a computer program that can run on the device described above.

[0116] Please refer to Figure 3 , Figure 3 is a structural schematic block diagram of the control module provided in the embodiments of the present application. The control module includes a processor, a memory and a network interface connected through a device bus, wherein the memory can include a storage medium and an internal memory.

[0117] The storage medium can store an operating device and a computer program. The computer program includes program instructions that, when executed, can cause the processor to execute any one of the embodiments of the method for learning the fluoroscopic image features of the catheterization for hepatic arterial chemoembolization.

[0118] The processor is used to provide computing and control capabilities to support the operation of the entire control module.

[0119] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium, which, when executed by the processor, can cause the processor to execute any one of the embodiments of the method for learning the fluoroscopic image features of the catheterization for hepatic arterial chemoembolization.

[0120] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the terminal to which the scheme of the present application is applied. The specific control module can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] It should be understood that the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0122] In one embodiment, the processor is used to run the computer program stored in the memory to implement the following steps:

[0123] An image acquisition module acquires a two-dimensional undistorted fluorescent blood vessel image in a superselective catheterization process of a transcatheter hepatic arterial chemoembolization; the two-dimensional undistorted fluorescent blood vessel image contains complex blood vessel structures of a patient isomerism and flexion;

[0124] Surface geometric features of the blood vessels are extracted from the two-dimensional undistorted fluorescent blood vessel image, the surface geometric features including blood vessel branch morphology, tube diameter change and edge profile; the surface geometric features are input to a preset spatial reasoning model to combine preset blood vessel anatomical prior knowledge, distinguish pixel differences of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of blood vessel branches, and output a spatial distribution probability matrix of each blood vessel part;

[0125] A difference signal between the spatial distribution probability matrix and angiographic feedback data in an actual catheterization process is captured in real time to form an iterative training data set to optimize the spatial reasoning model; potential complex blood vessel regions are identified according to the spatial distribution probability matrix to generate a catheterization path early warning signal, and guidance parameters of X-ray release timing and dose are generated based on the early warning signal to reduce radiation exposure; wherein, through joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of complex blood vessels of the hepatic artery is inferred and the catheterization process is guided in real time without relying on cone beam CT or micro-ultrasound assistance.

[0126] It should be noted that, for the convenience and brevity of description, the specific working process of the processor described above can refer to the corresponding process in the method embodiments described in the above embodiments, which will not be described here.

[0127] The embodiments of the present application also provide a computer readable storage medium storing a computer program, the computer program including program instructions, and the processor executes the program instructions to implement the steps of the catheterization fluorescent image feature learning method for transcatheter hepatic arterial chemoembolization provided by the above embodiments of the present application.

[0128] The computer readable storage medium can be an internal storage unit of the control module, such as a hard disk or a memory of the control module. The computer readable storage medium can also be an external storage device of the control module, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0129] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A catheterization fluoroscopic image feature learning system for transhepatic arterial chemoembolization, characterized by, The application relates to a real-time guiding system for superselective catheterization in transcatheter arterial chemoembolization (TACE), comprising: an image acquisition module configured to acquire undistorted two-dimensional fluorescent blood vessel images during superselective catheterization in TACE, wherein the two-dimensional fluorescent blood vessel images contain complex blood vessel structures of patients with isomerism and flexion; a control module connected to the image acquisition module, configured to extract surface geometric features of blood vessels from the two-dimensional fluorescent blood vessel images, wherein the surface geometric features include blood vessel branch morphology, tube diameter change and edge contour; input the surface geometric features into a preset spatial reasoning model to combine preset blood vessel anatomical prior knowledge, distinguish pixel differences of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of blood vessel branches, and output a spatial distribution probability matrix of each blood vessel part; capture difference signals between the spatial distribution probability matrix and angiography feedback data in the actual catheterization process in real time, form an iterative training data set to optimize the spatial reasoning model; identify potential complex blood vessel regions according to the spatial distribution probability matrix, generate a catheterization path early warning signal, and generate guiding parameters of X-ray release time and dose based on the early warning signal to reduce radiation exposure; wherein the joint analysis of the surface geometric features and the spectral features realizes the prediction of the spatial distribution relationship of the hepatic artery complex blood vessels and the real-time guidance of the catheterization process without relying on cone beam CT or microscopic ultrasound assistance.

2. The system of claim 1, wherein, The application relates to a real-time guiding system for superselective catheterization in transcatheter arterial chemoembolization (TACE), comprising: perspective projection calibration is performed through a multi-angle calibration plate preset on a DSA device, original fluorescent images are geometrically distorted and compensated through an image distortion correction algorithm based on Zhang Zhengyou's calibration method, abdominal cavity organ tissue noise is inhibited through a blood vessel enhancement filtering algorithm, high-frequency detail features of blood vessel edges are reserved to highlight complex blood vessel structures, and the two-dimensional fluorescent blood vessel images are generated.

3. The system of claim 1, wherein, The application relates to a real-time guiding system for superselective catheterization in transcatheter arterial chemoembolization (TACE), comprising: a blood vessel segmentation model based on a U-Net convolutional neural network is used for pixel-level semantic segmentation of the preprocessed images, blood vessel branch nodes are identified through a graph theory algorithm after the blood vessel skeleton is extracted, branch angle, tube diameter change rate and Fourier descriptors of blood vessel edges are calculated, a feature vector group containing blood vessel morphological parameters is formed as the surface geometric features.

4. The system of claim 1, wherein, The application relates to a real-time guiding system for superselective catheterization in transcatheter arterial chemoembolization (TACE), comprising: a position sensor coordinate at the end of a microcatheter is mapped to a fluorescent image coordinate system, a catheter motion trajectory is tracked based on an optical flow method, actual blood vessel development gray values of regions passed through by the catheter motion trajectory are compared with spectral feature probability distribution in the prediction matrix, and pixel-level cosine similarity deviation of the overlapping regions is calculated as the difference signal.

5. The system of claim 1, wherein, The application relates to a real-time guiding system for superselective catheterization in transcatheter arterial chemoembolization (TACE), comprising: The dynamic data cache queue is constructed, the image slices, surface geometric feature vectors and corrected spatial distribution labels corresponding to the real-time captured difference signals are spatio-temporally aligned, a transfer learning strategy is used to online fine-tune the initially trained spatial reasoning model, and an adaptive learning rate adjustment mechanism is used to balance the training weights of new and old data.

6. The system of claim 1, wherein, The potential complex blood vessel region is identified according to the spatial distribution probability matrix, and a catheterization path early warning signal is generated, which comprises: A blood vessel spatial distribution probability entropy threshold is set, the probability matrix is regionally grown and segmented, a region with a curvature greater than a preset threshold and a branch density more than twice that of a normal liver blood vessel anatomical model is marked as a complex blood vessel region, a three-dimensional reconstruction algorithm is used to superimpose a pseudo-color semi-transparent mark on a two-dimensional image, and voice early warning information containing blood vessel tortuosity and branch risk level is generated.

7. The system of claim 1, wherein, The X-ray release timing and dose guidance parameters are generated based on the early warning signal, which comprises: A radiation dose-image signal-to-noise ratio optimization model is established, the X-ray exposure mode is switched to pulse fluoroscopy for the complex blood vessel region, the pulse frequency is dynamically adjusted according to the uncertainty of the blood vessel spatial distribution probability matrix, and a personalized exposure time window suggestion is generated in combination with a historical operation habit database.

8. A method for learning the features of catheter-directed fluorescence images used in hepatic artery chemoembolization, characterized in that, The control module is applied to the catheterization fluoroscopic image feature learning system for hepatic arterial chemoembolization in any one of claims 1-7, and the method comprises: An image acquisition module acquires undistorted two-dimensional fluoroscopic blood vessel images during the superselective catheterization process of the hepatic arterial chemoembolization; the two-dimensional fluoroscopic blood vessel images contain complex blood vessel structures of patients with isomorphism and flexion; Surface geometric features of the blood vessels are extracted from the two-dimensional fluoroscopic blood vessel images, the surface geometric features include blood vessel branch morphology, tube diameter change and edge contour; the surface geometric features are input into a preset spatial reasoning model to combine preset blood vessel anatomical prior knowledge, distinguish pixel differences of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of blood vessel branches, and output a spatial distribution probability matrix of each blood vessel part; A difference signal between the spatial distribution probability matrix and angiographic feedback data in the actual catheterization process is captured in real time to form an iterative training data set to optimize the spatial reasoning model; a potential complex blood vessel region is identified according to the spatial distribution probability matrix, a catheterization path early warning signal is generated, and X-ray release timing and dose guidance parameters are generated based on the early warning signal to reduce radiation exposure; wherein, through joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of the complex blood vessels of the hepatic artery is inferred and the catheterization process is guided in real time without relying on cone beam CT or microscopic ultrasound assistance.

9. A catheter fluorescence image feature learning device for hepatic artery chemoembolization, characterized in that, The control module is applied to the catheterization fluoroscopic image feature learning system for hepatic arterial chemoembolization in any one of claims 1-7, and the device comprises: An image acquisition unit is configured to acquire undistorted two-dimensional fluoroscopic blood vessel images during the superselective catheterization process of the hepatic arterial chemoembolization; the two-dimensional fluoroscopic blood vessel images contain complex blood vessel structures of patients with isomorphism and flexion; A matrix output unit is configured to extract surface geometric features of blood vessels from the two-dimensional fluorescent vascular image, wherein the surface geometric features include vascular branch morphology, tube diameter variation, and edge profile; input the surface geometric features into a preset spatial reasoning model, combine preset vascular anatomical prior knowledge, distinguish pixel differences of overlapping blood vessels through spectral feature analysis, construct a three-dimensional spatial relationship prediction model of vascular branches, and output a spatial distribution probability matrix of each blood vessel part; A guidance implementation unit is configured to capture a difference signal between the spatial distribution probability matrix and angiography feedback data in an actual intubation process in real time, form an iterative training data set to optimize the spatial reasoning model; identify a potential complex blood vessel region according to the spatial distribution probability matrix, generate an intubation path early warning signal, and generate guidance parameters of X-ray release timing and dose based on the early warning signal to reduce radiation exposure; wherein, through joint analysis of the surface geometric features and the spectral features, the spatial distribution relationship of complex blood vessels of the hepatic artery is inferred and the real-time guidance of the intubation process is realized without relying on cone beam CT or micro-ultrasound assistance.

10. A control module, characterized by The control module comprises a memory and a processor; The memory is configured to store a computer program; The processor is configured to execute the computer program and implement the method of claim 8 when executing the computer program.

Images