X-ray image visual prompt system for atrial septum puncture surgery

Abstract

The invention relates to the technical field of medical images, in particular to an X-ray image visual prompt system for atrial septum puncture surgery, which comprises a data acquisition and preprocessing module for acquiring preoperative X-ray video to obtain an anatomical structure segmentation mask; the data analysis and generation module is used for constructing a puncture point decision model, outputting a puncture area probability graph and generating candidate puncture points based on the puncture area probability graph; the data processing module is used for tracking the anatomical structure segmentation mask, determining a space average value, optimizing a space-time model through a trajectory-semantic decoupling mechanism, obtaining a trajectory consistency loss function and outputting a prediction result; the data prediction module is used for constructing a safety puncture area probability graph and generating a dynamic visual prompt graph; the data enhancement and decision-making module is used for superposing and displaying the dynamic visual prompt graph and the safety puncture area probability graph, and real-time feedback is carried out by a computing terminal, so that collaborative operation of people and the system is realized; and through AI auxiliary analysis and suggestion, the reliability of the puncture operation is improved.

Description

Technical Field

[0001] The present invention relates to the field of medical imaging technology, and in particular to an X-ray image visual prompting system for atrial septal puncture surgery. Background Art

[0002] Atrial septal puncture is a key step in various cardiac interventional procedures, such as transcatheter mitral valve clipping, left atrial appendage occlusion, and arrhythmia ablation. Its purpose is to safely and accurately introduce a catheter through the atrial septum into the left atrium. Although this procedure has been widely used, it has certain technical challenges. In particular, in patients with complex anatomy or congenital lesions, the operation risk is increased, which may lead to serious complications such as cardiac perforation and pericardial tamponade. As a traditional imaging method, X-ray fluoroscopy remains the most commonly used guidance method for atrial septal puncture surgery worldwide, excluding North America, due to its advantages such as instant imaging, simple and intuitive operation, and the lack of additional training for ICE catheter operation. In particular, in some centers without transesophageal ultrasound guidance, X-ray-guided atrial septal puncture remains the only option.

[0003] Traditional TSP (Transseptal Puncture) operations mainly rely on the surgeon's experience and real-time judgment of X-ray images. However, this method requires high spatial imagination, image interpretation skills, and hand-eye coordination abilities of the surgeon, has a steep learning curve, and limits the safety and efficiency of the operation. If the judgment is biased, it may lead to serious complications or even the death of the patient. In recent years, artificial intelligence technology, especially deep learning, has made significant progress in medical image recognition and decision support. Currently, some auxiliary puncture technology solutions have been proposed, but most of them use additional mechanical devices to achieve three-dimensional imaging to improve the positioning accuracy of the puncture. The cost is high and the operation requirements are difficult. It is difficult to deploy in grassroots or conventional operating room environments, which limits clinical promotion and implementation. The capabilities and demand response are insufficient; the introduced AI systems are still in their infancy. For example, the previous case CN118319486B proposed an artificial intelligence-based image guidance system for cardiovascular interventional surgery, which used deep learning methods to analyze a variety of cardiovascular interventional images, constructed a semi-automatic intelligent annotation tool and a cardiovascular interventional surgery image guidance system based on vascular structure segmentation and lesion area segmentation, and performed risk assessment on the surgery; it did not establish a spatial model across cardiac chamber structures, nor did it involve the design of exclusive algorithms such as puncture window identification and optimal puncture angle evaluation. It lacked the design of special technical modules for atrial septal puncture, and the design of precision control and error tolerance mechanism was weak. It lacked image enhancement strategies for the detailed structure of the atrial septum and the ability of dynamic adjustment during surgery. Summary of the Invention

[0004] In order to solve the technical problems of existing solutions for atrial septal puncture requiring additional auxiliary equipment, resulting in insufficient demand response, high cost, lack of dedicated technical module design for atrial septal puncture, lack of dynamic adjustment capability, and incompatibility with real-time changes in surgery, the purpose of the present invention is to provide an X-ray image visual prompt system for atrial septal puncture surgery. The technical solutions adopted are as follows:

[0005] Data acquisition and preprocessing module, used to: collect preoperative X-ray video and preprocess it to obtain anatomical structure segmentation mask;

[0006] The data analysis and generation module is used to: construct a puncture point decision model, input the anatomical structure segmentation mask into the puncture point decision model, output a puncture area probability map, and generate candidate puncture points based on the puncture area probability map;

[0007] The data processing module is used to track the anatomical structure segmentation mask, construct a trajectory set, perform trajectory encoding on the trajectory set to obtain an embedding vector set, determine the spatial average, optimize the spatiotemporal model through the trajectory-semantic decoupling mechanism, combine the spatial average and the anatomical structure segmentation mask to obtain a trajectory consistency loss function, output the prediction result based on the optimized spatiotemporal model, and use the trajectory consistency loss function to determine whether the prediction result matches the candidate puncture point;

[0008] The data prediction module is used to: construct a safe puncture area probability map based on multi-expert integration, and generate a dynamic visual prompt map based on the prediction results that match the candidate puncture points;

[0009] The data enhancement and decision-making module is used to: use augmented reality technology to overlay the dynamic visual prompt map with the safe puncture area probability map, and assist in the real-time feedback of the computing terminal to achieve collaborative operation between people and systems.

[0010] Preferably, collecting preoperative X-ray video and preprocessing it to obtain an anatomical structure segmentation mask includes:

[0011] Collect preoperative X-ray video, extract each frame image and unify the size;

[0012] Based on the recognition of anatomical structure region boundaries in each frame image, the main loss and auxiliary loss are introduced to determine the overall loss function. The basic model of image segmentation is used to fine-tune the anatomical structure region boundaries to obtain the anatomical structure segmentation mask.

[0013] Preferably, the main loss and the auxiliary loss are introduced respectively to determine the overall loss function, and the basic model of image segmentation is used to fine-tune the boundary of the anatomical structure region to obtain the anatomical structure segmentation mask, including:

[0014] The calculation formula for the main loss is:

[0015]

[0016] in, Represents the subject loss; N represents the total number of pixels in the image; i represents the i-th pixel; y i represents the true label of the i-th pixel; Indicates the mask probability output by the basic model based on image segmentation corresponding to the i-th pixel;

[0017] The calculation formula corresponding to the auxiliary loss is:

[0018]

[0019] in, represents auxiliary loss; ∈ represents a constant to avoid the denominator being zero;

[0020] Determine the overall loss function, the corresponding calculation formula is:

[0021]

[0022] in, Represents the total loss function; λ1 and λ2 represent the adjustment weights corresponding to the main loss and auxiliary loss, respectively.

[0023] Preferably, a puncture point decision model is constructed, the anatomical structure segmentation mask is input into the puncture point decision model, a puncture area probability map is output, and candidate puncture points are generated based on the puncture area probability map, including:

[0024] The puncture point decision model adopts a lightweight U-Net network;

[0025] The anatomical structure segmentation mask is used as a visual cue to generate a position cue map. The input data is determined based on each frame image combined with the position cue map. The input data is input into the puncture point decision model to output a puncture area probability map.

[0026] The target area is delineated based on the puncture area probability map, and the puncture point decision model is trained. The trained puncture point decision model is used to input the preoperative X-ray video to generate candidate puncture points and form a recommended puncture window.

[0027] Preferably, the anatomical structure segmentation mask is used as a visual cue to generate a position cue map, input data is determined based on each frame image combined with the position cue map, the input data is input into the puncture point decision model, and a puncture area probability map is output, including:

[0028] The anatomical structure segmentation mask is used as a visual cue to generate a position cue map. The corresponding calculation formula is:

[0029]

[0030] Among them, M prompt Indicates a location prompt map; The segmentation binary mask images represent the heart contour, spinal midline and coronary sinus electrode catheter anatomical structure segmentation masks respectively;

[0031] Define each frame image as I, and determine the input data in combination with the position prompt map. The corresponding calculation formula is:

[0032]

[0033] Where X represents the input data;

[0034] The input data is fed into the puncture point decision model, and the puncture area probability map is output.

[0035] Preferably, the target area is delineated based on the puncture area probability map, and the puncture point decision model is trained, including:

[0036] Based on the puncture area probability map, any real puncture point is selected as the center of the circle, and the radius is defined to delineate the candidate area circle as the target area. The corresponding logical expression is:

[0037]

[0038] Among them, (x, y) represents the center coordinates of the target area detected by the puncture point decision model; (x gt ,y gt ) represents the coordinates of the actual puncture point; r represents the radius of the target area.

[0039] Preferably, the anatomical structure segmentation mask is tracked to construct a trajectory set, the trajectory set is trajectory-encoded to obtain an embedding vector set, the spatial average is determined, the spatiotemporal model is optimized through a trajectory-semantic decoupling mechanism, the spatial average and the anatomical structure segmentation mask are combined to obtain a trajectory consistency loss function, a prediction result is output based on the optimized spatiotemporal model, and the trajectory consistency loss function is used to determine whether the prediction result matches the candidate puncture point, including:

[0040] Perform continuous frame tracking on the anatomical structure segmentation mask to determine the trajectory in any frame, obtain the trajectories in all frames to construct a trajectory set, perform trajectory encoding on the trajectory set to obtain an embedding vector set, and determine the spatial average of the trajectory of any anatomical structure in the preoperative X-ray video;

[0041] The spatiotemporal model is optimized through the trajectory-semantic decoupling mechanism, the semantic center coordinates are determined based on the anatomical structure segmentation mask, and the trajectory consistency loss function is obtained by combining the spatial average value.

[0042] According to the optimized spatiotemporal model, the prediction results are fused and output through the cross-attention mechanism;

[0043] A judgment threshold is set and the trajectory consistency loss function is used to determine whether the prediction result matches the candidate puncture point. If the trajectory consistency loss function is less than the judgment threshold, it means that the prediction result matches the trajectory of the candidate puncture point.

[0044] Preferably, to determine the trajectory within any frame, the corresponding calculation formula is:

[0045]

[0046] Among them, τ i represents the trajectory of the anatomical structure in the i-th frame; represents the coordinates of the i-th pixel corresponding to the anatomical structure in the i-th frame; T represents the total number of frames corresponding to the preoperative X-ray video.

[0047] Preferably, the spatiotemporal model is optimized through the trajectory-semantic decoupling mechanism, the semantic center coordinates are determined based on the anatomical structure segmentation mask, and the trajectory consistency loss function is obtained by combining the spatial average value, including:

[0048] Define the anatomical structure segmentation mask as Determine the semantic center coordinates, and the corresponding calculation formula is:

[0049]

[0050] in, Represents the semantic center coordinates (x, y);

[0051] Combined with the spatial average value, the trajectory consistency loss function is obtained, and the corresponding calculation formula is:

[0052]

[0053] in, represents the trajectory consistency loss function; i represents the i-th pixel; Indicates the semantic center coordinates corresponding to the i-th pixel; Represents the spatial average value corresponding to the i-th pixel.

[0054] The present invention has the following beneficial effects:

[0055] This system does not require additional surgical equipment and is adapted to the real intraoperative environment. That is, it does not rely on three-dimensional modeling equipment, intracardiac ultrasound or electromagnetic navigation systems. It directly performs reasoning and prompts based on low-dose X-ray fluoroscopy images commonly used during surgery, which is convenient for widespread clinical implementation and is particularly suitable for resource-constrained surgical environments. It identifies the boundaries of anatomical regional structures and fine-tunes the basic image segmentation model. It generates high-confidence candidate puncture points through visual guidance to form a recommended puncture window, thereby improving the interpretability and stability of the puncture point decision model positioning. By simulating the motion trajectory of the anatomical structure, the spatiotemporal model is optimized based on the attention mechanism and the motion-semantic decoupling embedding module to achieve the dynamic positioning of the atrial septum puncture point. Prediction; by learning the puncture point annotations of multiple experts, the system can predict the range of reliable puncture points instead of a single puncture point, and link the generated safe puncture area probability map with the dynamic visual prompt map for visual prompting, to achieve personalized risk prompts and operation accuracy during surgery, and enhance the reliability of the system in clinical use; through augmented reality technology, the safe puncture area probability map and the dynamic visual prompt map are superimposed and displayed in the surgeon's field of view, realizing the "what you see is what you suggest" auxiliary operation mode, lowering the learning threshold, improving the operation confidence and safety of young surgeons, enhancing reality visual prompts, optimizing human-computer collaboration efficiency, and supporting surgeons to make more accurate and safer individualized decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0057] Figure 1 A schematic diagram showing the display of anatomical structure region boundaries of an X-ray image visual prompt system for atrial septal puncture surgery provided by one embodiment of the present invention;

[0058] Figure 2 A schematic diagram showing a suggested puncture window of an X-ray image visual prompt system for atrial septal puncture surgery provided by one embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of the results output by a data prediction module of an X-ray image visual prompt system for atrial septal puncture surgery provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0060] To further illustrate the technical means and effectiveness of the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effectiveness of an X-ray imaging visual prompt system for transseptal puncture surgery proposed by the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0061] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0062] The following specifically describes a specific solution of an X-ray image visual prompt system for atrial septal puncture surgery provided by the present invention in conjunction with the accompanying drawings. The system includes:

[0063] Data acquisition and preprocessing module, used to: collect preoperative X-ray video and preprocess it to obtain anatomical structure segmentation mask;

[0064] The data analysis and generation module is used to: construct a puncture point decision model, input the anatomical structure segmentation mask into the puncture point decision model, output a puncture area probability map, and generate candidate puncture points based on the puncture area probability map;

[0065] The data processing module is used to track the anatomical structure segmentation mask, construct a trajectory set, perform trajectory encoding on the trajectory set to obtain an embedding vector set, determine the spatial average, optimize the spatiotemporal model through the trajectory-semantic decoupling mechanism, combine the spatial average and the anatomical structure segmentation mask to obtain a trajectory consistency loss function, output the prediction result based on the optimized spatiotemporal model, and use the trajectory consistency loss function to determine whether the prediction result matches the candidate puncture point;

[0066] The data prediction module is used to: construct a safe puncture area probability map based on multi-expert integration, and generate a dynamic visual prompt map based on the prediction results that match the candidate puncture points;

[0067] The data enhancement and decision-making module is used to: use augmented reality technology to overlay the dynamic visual prompt map with the safe puncture area probability map, and assist in the real-time feedback of the computing terminal to achieve collaborative operation between people and systems.

[0068] As an optional implementation, in this embodiment, the three major anatomical structures of the heart contour, spinal midline and coronary sinus electrode catheter are analyzed, and a video of the anatomical structure is obtained based on X-ray imaging technology, and an image is extracted to provide a predicted atrial septum surgical puncture area, thereby improving the safety and success rate of the operation.

[0069] See also Figure 1 , which shows a schematic diagram of the display of anatomical structure area boundaries of an X-ray image visual prompt system for atrial septal puncture surgery provided by an embodiment of the present invention, wherein the red line indicates the area boundary of the heart contour; the yellow line indicates the area boundary of the spinal midline; and the green line indicates the area boundary of the coronary sinus electrode catheter.

[0070] Furthermore, the data acquisition and preprocessing module includes:

[0071] Step S11: Collect preoperative X-ray video, extract each frame image and unify the size.

[0072] Preferably, in this embodiment, the data sample collected is a standard preoperative low-dose X-ray fluoroscopy dynamic video during surgery, which records the key moments before the surgery and shows the real-time dynamic changes of organs and tissues in the patient's body. By using low-dose X-rays, it ensures that while obtaining high-quality imaging data, the radiation exposure to patients and medical staff is minimized.

[0073] Specifically, standard preoperative low-dose X-ray fluoroscopic dynamic videos of interventional surgery were collected, and each frame of the video was extracted and uniformly cropped and normalized to 512×512.

[0074] Step S12: Identify the anatomical structure region boundary based on each frame image, introduce the main loss and auxiliary loss to determine the overall loss function, use the basic model of image segmentation to fine-tune the anatomical structure region boundary, and obtain the anatomical structure segmentation mask.

[0075] Preferably, the basic model of image segmentation includes three modules: image encoder, bounding box hint encoder and segmentation mask decoder. At the same time, in order to adapt to the grayscale distribution and weak boundary characteristics of anatomical structures in low-dose X-ray images, a channel separation attention mechanism is added to the segmentation mask decoder during fine-tuning to enhance the model's perception of areas with weak texture, such as the boundaries of the heart contour area.

[0076] Furthermore, step S12 includes:

[0077] The calculation formula for the main loss is:

[0078]

[0079] in, Represents the subject loss; N represents the total number of pixels in the image; i represents the i-th pixel; y i represents the true label of the i-th pixel; Indicates the mask probability output by the basic model based on image segmentation corresponding to the i-th pixel;

[0080] The calculation formula corresponding to the auxiliary loss is:

[0081]

[0082] in, represents auxiliary loss; ∈ represents a constant to avoid the denominator being zero;

[0083] Determine the overall loss function, the corresponding calculation formula is:

[0084]

[0085] in, Represents the total loss function; λ1 and λ2 represent the adjustment weights corresponding to the main loss and auxiliary loss, respectively.

[0086] Specifically, based on the fine-tuning of the basic model of image segmentation and combined with expert image annotation, the X-ray image visual prompt system for atrial septal puncture surgery is guided to identify the boundaries of the three major anatomical structure regions: the heart contour, the spinal midline, and the coronary sinus electrode catheter in a single frame image of the X-ray fluoroscopy video. During fine-tuning, the total loss function is used to simultaneously optimize the structural region segmentation accuracy and boundary accuracy to obtain a more accurate anatomical structure segmentation mask. Among them, the main loss adopts binary cross entropy, and the auxiliary loss is the structural consistency loss function to ensure the consistency and accuracy of the obtained region boundary with the real boundary, so as to facilitate the subsequent provision of more accurate visual prompts for atrial septal puncture surgery.

[0087] See also Figure 2 , which shows a display schematic diagram of a recommended puncture window of an X-ray image visual prompt system for atrial septal puncture surgery provided by an embodiment of the present invention; wherein the marked frame portion is the recommended puncture window.

[0088] Furthermore, the data analysis and generation module includes:

[0089] The puncture point decision model uses a lightweight U-Net network; its unique symmetrical structure enables feature information to be effectively transmitted in the network to improve segmentation accuracy. The lightweight U-Net network optimizes the traditional U-Net structure, reduces the number of parameters, and the model is simple and efficient while maintaining high segmentation accuracy. That is, fast and efficient image segmentation is achieved while ensuring accuracy.

[0090] Step S21: Using the anatomical structure segmentation mask as a visual cue, a position cue map is generated, input data is determined based on each frame image combined with the position cue map, the input data is input into the puncture point decision model, and a puncture area probability map is output.

[0091] Furthermore, step S21 includes:

[0092] Step S211: Use the anatomical structure segmentation mask as a visual cue to generate a position cue map. The corresponding calculation formula is:

[0093]

[0094] Among them, M prompt Indicates a location prompt map; The segmentation binary mask images represent the heart contour, spinal midline and coronary sinus electrode catheter anatomical structure segmentation masks respectively;

[0095] Step S212: define each frame image as I, and determine the input data in combination with the position prompt map. The corresponding calculation formula is:

[0096]

[0097] Where X represents the input data;

[0098] Step S213: input the input data into the puncture point decision model and output a puncture area probability map.

[0099] It is explained that the anatomical structure segmentation mask is used as a visual cue to generate a position cue map. The input data is determined based on each frame of the image in combination with the position cue map, and the specific position of the input data can be accurately obtained. After processing by the puncture point decision model, a puncture area probability map can be output to show the possibility of puncture in various areas of any anatomical structure. It can more intuitively see which parts have a high puncture success rate and which parts have risks or uncertainties.

[0100] Step S22: Delineate the target area based on the puncture area probability map, train the puncture point decision model, use the trained puncture point decision model to input the preoperative X-ray video to generate candidate puncture points, and form a recommended puncture window.

[0101] Preferably, the training puncture point decision model also uses binary cross entropy as the main loss function.

[0102] Furthermore, in step S22, the target area is delineated based on the puncture area probability map, and the puncture point decision model is trained, including:

[0103] Based on the puncture area probability map, any real puncture point is selected as the center of the circle, and the radius is defined to delineate the candidate area circle as the target area. The corresponding logical expression is:

[0104]

[0105] Among them, (x, y) represents the center coordinates of the target area detected by the puncture point decision model; (x gt ,y gt ) represents the coordinates of the actual puncture point; r represents the radius of the target area.

[0106] Specifically, the target area is set as the learning target of the puncture point decision model, and the model is trained in combination with the subject loss function. Then, the preoperative X-ray video is input into the trained puncture point decision model to identify the area with the highest probability in the puncture area probability map, generate a series of candidate puncture points, and define the recommended puncture window for the candidate puncture points, providing reference and guidance for the actual surgical process, effectively improving the accuracy and safety of the puncture operation, and enhancing the overall effect of the operation.

[0107] It is understandable that although low-dose X-rays have significant advantages in reducing radiation exposure for patients or surgeons, their imaging quality is affected to a certain extent, especially in anatomical structure areas such as the right outer edge of the heart, where blurred boundaries may occur; therefore, the present application combines visual trajectory modeling on each frame of the video to improve the diagnostic accuracy of X-ray fluoroscopic images based on the trajectory or running path of the anatomical structure; specifically, a detailed analysis of the heart movement and catheter path is performed, and the position changes of the catheter head end and the changes of the boundary points of the outer edge of the heart in multiple frames of the video are extracted to form a quasi-long-range trajectory to capture the movement laws of the heart and catheter in the dynamic process; the target recognition method of the dynamic video is simulated, and the quasi-long-range trajectory is input into the spatiotemporal model to model the dynamic changes of each anatomical structure and its interactive relationship in space, thereby assisting in improving the relative position stability of the spatiotemporal model in detecting the boundaries of the surrounding structures of the anatomical structure and predicting the puncture point position, that is, significantly improving the imaging quality of low-dose X-ray fluoroscopic images in key anatomical structure areas such as the heart contour, providing more reliable and accurate imaging support for clinical diagnosis and interventional surgery.

[0108] Furthermore, the data processing module includes:

[0109] Step S31: Continuously track the anatomical structure segmentation mask, determine the trajectory in any frame, obtain the trajectories in all frames to construct a trajectory set, perform trajectory encoding on the trajectory set to obtain an embedding vector set, and determine the spatial average value of the trajectory of any anatomical structure in the preoperative X-ray video.

[0110] Specifically, in this embodiment, trajectory tracking is performed on the heart contour and the coronary sinus electrode catheter, that is, the activity trajectory of the outer edge of the heart and the trajectory of the electrode catheter tip are tracked in consecutive frames, and the trajectory in any frame is determined to integrate the trajectories in all frames to construct a trajectory set.

[0111] Furthermore, in step S31, the trajectory in any frame is determined, and the corresponding calculation formula is:

[0112]

[0113] Among them, τ i represents the trajectory of the anatomical structure in the i-th frame; represents the coordinates of the i-th pixel corresponding to the anatomical structure in the i-th frame; T represents the total number of frames corresponding to the preoperative X-ray video.

[0114] Trajectory encoding is performed on the trajectory set to obtain an embedded vector set, denoted as {f1, f2}, which is used to describe the activity of the outer edge of the heart and the movement pattern of the electrode catheter. At the same time, when predicting the puncture area, the position of the anatomical structure output from the actual trajectory observation should not deviate. That is, when the trajectory of the anatomical structure is used for prediction based on the spatiotemporal model, the position of the candidate puncture point obtained by the data analysis and generation module cannot deviate to avoid errors in the subsequent puncture surgery. The spatial average value of the corresponding anatomical structure is then confirmed by the trajectory confirmed in each frame and the total number of frames.

[0115] Step S32: Optimize the spatiotemporal model through the trajectory-semantic decoupling mechanism, determine the semantic center coordinates based on the anatomical structure segmentation mask, and obtain the trajectory consistency loss function by combining the spatial average.

[0116] It is explained that the trajectory-semantic decoupling mechanism, namely the motion-semantic decoupling embedding technology, enables the spatiotemporal model to process trajectory motion features and semantic structure features separately, where the trajectory motion features are the set of embedding vectors output by the trajectory of the anatomical structure; the semantic structure features are the semantic embeddings of the spinal midline and the cardiac outer edge position generated after fine-tuning based on the image segmentation basic model, so as to achieve highly robust dynamic structure positioning.

[0117] Furthermore, step S32 includes:

[0118] Step S321: Define the anatomical structure segmentation mask as Determine the semantic center coordinates, and the corresponding calculation formula is:

[0119]

[0120] in, Represents the semantic center coordinates (x, y);

[0121] Step S322: Combine the spatial average value to obtain the trajectory consistency loss function, and the corresponding calculation formula is:

[0122]

[0123] in, represents the trajectory consistency loss function; i represents the i-th pixel; Indicates the semantic center coordinates corresponding to the i-th pixel; Represents the spatial average value corresponding to the i-th pixel.

[0124] It can be explained that the introduction of the trajectory consistency loss function can ensure that the predicted puncture point area output by the optimized spatiotemporal model and the actual outer edge of the heart or the center of gravity of the coronary sinus electrode catheter trajectory will not deviate too far between adjacent frames. That is, the trajectory consistency loss function can ensure that the spatiotemporal model maintains stability and coherence in the time dimension, avoids drastic fluctuations or unreasonable jumps in the prediction results between adjacent frames, and improves the reliability and accuracy of the spatiotemporal model prediction results.

[0125] Step S33: The prediction results are outputted through the cross-attention mechanism according to the optimized spatiotemporal model; that is, the optimized spatiotemporal model adopts a dual-branch structure to process the two modalities of trajectory motion features and semantic structure features, and then fuses the outputs of the two modalities through the cross-attention mechanism to obtain the prediction results, that is, by analyzing the correlation between trajectory motion features and semantic structure features, the contribution of each modality to the final prediction result is dynamically adjusted to ensure that the optimized spatiotemporal model can capture the most relevant feature information while ignoring irrelevant or noise data, so as to make full use of the complementarity between different modalities and improve the accuracy and reliability of the prediction results.

[0126] Step S34: setting a judgment threshold and using a trajectory consistency loss function to judge whether the prediction result matches the candidate puncture point. If the trajectory consistency loss function is less than the judgment threshold, it means that the prediction result matches the trajectory of the candidate puncture point.

[0127] It is explained that the judgment threshold is set according to the specific needs of the actual application; when the final value obtained by the trajectory consistency loss function is less than the judgment threshold, it means that the smaller the Euclidean distance between the semantic center coordinates and the spatial average value, the more reasonable the output prediction result is, indicating that the prediction result matches the true trajectory, that is, the trajectory of the candidate puncture point. It can be seen that the prediction result is relatively consistent with the actual observed trajectory in spatial position, verifying the accuracy of the prediction result; on the contrary, if the trajectory consistency loss function is greater than the judgment threshold, the Euclidean distance between the semantic center coordinates and the spatial average value is large, indicating that there is a deviation between the prediction result and the true trajectory, and the optimized spatiotemporal model needs to be adjusted to ensure that the output prediction result is reasonable.

[0128] See also Figure 3, which shows a schematic diagram of the results output by a data prediction module of an X-ray image visual prompt system for atrial septal puncture surgery provided by an embodiment of the present invention; wherein, frame 1, frame 2 and frame 3 refer to images extracted from any frame in the preoperative X-ray video, and the color areas corresponding to the heart contour, spine and coronary sinus electrode catheter are dynamic visual prompt images corresponding to the anatomical structure; the ideal puncture area is a probability map of the safe puncture area, and the high-risk puncture area is a high-risk area; different frame numbers indicate different movement changes of the anatomical structure, and at this time, the corresponding prompt area in each frame image also has corresponding changes.

[0129] Understandably, each expert has slightly different strategies for selecting puncture points during atrial septal puncture. Although these puncture points are roughly concentrated in a certain central area, the actual puncture point location may deviate from the center due to the different habits and experience of each expert. Therefore, in order to more accurately simulate the puncture decision-making process in this actual operation and provide the operator with a more reliable range of puncture point candidates, rather than just predicting a single puncture point location, the puncture point annotations of multiple experienced operators on the same pre-puncture X-ray fluoroscopy video are adopted to train and simulate the integrated model of multiple experts, and a safe puncture area probability map is constructed to reflect the most likely area for puncture point selection under the experience and habits of different experts; that is, combined with historical puncture path data, a safe puncture area probability map is constructed based on multi-expert integration to be closer to the patient's actual situation.

[0130] Specifically, a regional risk assessment mechanism is introduced for the probability map of the safe puncture area, and a heat map warning is issued for high-risk areas such as the aorta adjacent to the atrial septum, that is, the high-risk puncture area is determined, and then a dynamic visual prompt map corresponding to the anatomical structure is generated according to the prediction result confirmed in step S34, so as to help the surgeon better identify and avoid potential risk areas when performing the puncture operation, thereby significantly improving the safety and success rate of the operation.

[0131] As an optional implementation, in this embodiment, in order to achieve real-time feedback of data and spatiotemporal model prediction results, a computing terminal is configured independently of the surgical instrument. The real-time images captured by the camera are input into the computing terminal in real time via wireless transmission. The computing terminal outputs the anatomical structure of the detected continuous frames as a visual prompt, and the prediction results are uploaded to the augmented reality terminal.

[0132] Furthermore, augmented reality technology is used to overlay dynamic visual prompt images with safe puncture area probability maps, and assist in real-time feedback from the computing terminal to achieve collaborative operation between humans and the system. Through augmented reality technology, the acquired dynamic visual prompt images, safe puncture area probability maps, and high-risk puncture areas are superimposed in real time on the X-ray fluoroscopy images in the operator's field of view in the form of images, so that the operator can see various important information more intuitively. That is, augmented reality technology is used to overlay visual prompts on the real-time X-ray images observed by the operator to achieve visual puncture point markings, warning boundary prompts, and navigation arrow feedback. The prediction result output is enhanced through the computing terminal, and real-time feedback is combined with visual prompts, so that the operator can make decisions through terminal feedback, realizing a human-centered "intelligent augmented reality" system that does not rely on external three-dimensional imaging equipment or navigation platforms. Through lightweight software and visual prompt interaction, it achieves low-threshold deployment, which is convenient for general application to routine atrial septal puncture operation procedures at the grassroots and large centers.

[0133] Understandably, this system does not require additional surgical equipment and is adapted to the real intraoperative environment, that is, it does not rely on three-dimensional modeling equipment, intracardiac ultrasound or electromagnetic navigation systems, and directly performs reasoning and prompts based on low-dose X-ray fluoroscopy images commonly used during surgery, which is convenient for widespread clinical implementation and is particularly suitable for resource-constrained surgical environments; it identifies the structural boundaries of anatomical regions and fine-tunes the basic image segmentation model, generates high-confidence candidate puncture points through visual guidance, forms a recommended puncture window, and improves the interpretability and stability of the puncture point decision model positioning; by simulating the motion trajectory of the anatomical structure, it optimizes the spatiotemporal model based on the attention mechanism and the motion-semantic decoupling embedding module to achieve atrial septal puncture points. Dynamic prediction; by learning the puncture point annotations of multiple experts, the system can predict the range of reliable puncture points instead of a single puncture point, and link the generated safe puncture area probability map with the dynamic visual prompt map for visual prompting, to achieve personalized risk prompts and operation accuracy during surgery, and enhance the reliability of the system in clinical use; through augmented reality technology, the safe puncture area probability map and the dynamic visual prompt map are superimposed and displayed in the operator's field of view, realizing the "what you see is what you suggest" auxiliary operation mode, lowering the learning threshold, improving the operation confidence and safety of young operators, enhancing reality visual prompts, optimizing human-computer collaboration efficiency, and supporting operators to make more accurate and safer individualized decisions.

[0134] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0135] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. An X-ray image visual prompt system for atrial septal puncture surgery, characterized in that: The system comprises: Data acquisition and preprocessing module, used to: collect preoperative X-ray video and preprocess it to obtain anatomical structure segmentation mask; The data analysis and generation module is used to: construct a puncture point decision model, input the anatomical structure segmentation mask into the puncture point decision model, output a puncture area probability map, and generate candidate puncture points based on the puncture area probability map; The data processing module is used to track the anatomical structure segmentation mask, construct a trajectory set, perform trajectory encoding on the trajectory set to obtain an embedding vector set, determine the spatial average, optimize the spatiotemporal model through the trajectory-semantic decoupling mechanism, combine the spatial average and the anatomical structure segmentation mask to obtain a trajectory consistency loss function, output the prediction result based on the optimized spatiotemporal model, and use the trajectory consistency loss function to determine whether the prediction result matches the candidate puncture point; The data prediction module is used to: construct a safe puncture area probability map based on multi-expert integration, and generate a dynamic visual prompt map based on the prediction results that match the candidate puncture points; The data enhancement and decision-making module is used to: use augmented reality technology to overlay the dynamic visual prompt map with the safe puncture area probability map, and assist in the real-time feedback of the computing terminal to achieve collaborative operation between people and systems.

2. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 1, characterized in that: Collect preoperative X-ray video and preprocess it to obtain anatomical structure segmentation masks, including: Collect preoperative X-ray video, extract each frame image and unify the size; Based on the recognition of anatomical structure region boundaries in each frame image, the main loss and auxiliary loss are introduced to determine the overall loss function. The basic model of image segmentation is used to fine-tune the anatomical structure region boundaries to obtain the anatomical structure segmentation mask.

3. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 1, characterized in that: The main loss and auxiliary loss are introduced to determine the overall loss function. The basic model of image segmentation is used to fine-tune the boundaries of the anatomical structure region to obtain the anatomical structure segmentation mask, including: The calculation formula for the main loss is: in, Represents the subject loss; N represents the total number of pixels in the image; i represents the i-th pixel; y i represents the true label of the i-th pixel; Indicates the mask probability output by the basic model based on image segmentation corresponding to the i-th pixel; The calculation formula corresponding to the auxiliary loss is: in, represents auxiliary loss; ∈ represents a constant to avoid the denominator being zero; Determine the overall loss function, the corresponding calculation formula is: in, Represents the total loss function; λ1 and λ2 represent the adjustment weights corresponding to the main loss and auxiliary loss, respectively.

4. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 1, characterized in that: Construct a puncture point decision model, input the anatomical structure segmentation mask into the puncture point decision model, output the puncture area probability map, and generate candidate puncture points based on the puncture area probability map, including: The puncture point decision model adopts a lightweight U-Net network; The anatomical structure segmentation mask is used as a visual cue to generate a position cue map. The input data is determined based on each frame image combined with the position cue map. The input data is input into the puncture point decision model to output a puncture area probability map. The target area is delineated based on the puncture area probability map, and the puncture point decision model is trained. The trained puncture point decision model is used to input the preoperative X-ray video to generate candidate puncture points and form a recommended puncture window.

5. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 4, characterized in that: The anatomical structure segmentation mask is used as a visual cue to generate a position cue map. The input data is determined based on each frame image combined with the position cue map. The input data is input into the puncture point decision model, and the puncture area probability map is output, including: The anatomical structure segmentation mask is used as a visual cue to generate a position cue map. The corresponding calculation formula is: Among them, M prompt Indicates a location prompt map; The segmentation binary mask images represent the heart contour, spinal midline and coronary sinus electrode catheter anatomical structure segmentation masks respectively; Define each frame image as I, and determine the input data in combination with the position prompt map. The corresponding calculation formula is: Where X represents the input data; The input data is fed into the puncture point decision model, and the puncture area probability map is output.

6. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 4, characterized in that: Delineate the target area based on the puncture area probability map and train the puncture point decision model, including: Based on the puncture area probability map, any real puncture point is selected as the center of the circle, and the radius is defined to delineate the candidate area circle as the target area. The corresponding logical expression is: Among them, (x, y) represents the center coordinates of the target area detected by the puncture point decision model; (x gt ,y gt ) represents the coordinates of the actual puncture point; r represents the radius of the target area.

7. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 1, characterized in that: The anatomical structure segmentation mask is tracked to construct a trajectory set. The trajectory set is trajectory-encoded to obtain an embedding vector set, and the spatial average is determined. The spatiotemporal model is optimized through the trajectory-semantic decoupling mechanism. The spatial average and the anatomical structure segmentation mask are combined to obtain a trajectory consistency loss function. The prediction result is output based on the optimized spatiotemporal model. The trajectory consistency loss function is used to determine whether the prediction result matches the candidate puncture point, including: Perform continuous frame tracking on the anatomical structure segmentation mask to determine the trajectory in any frame, obtain the trajectories in all frames to construct a trajectory set, perform trajectory encoding on the trajectory set to obtain an embedding vector set, and determine the spatial average of the trajectory of any anatomical structure in the preoperative X-ray video; The spatiotemporal model is optimized through the trajectory-semantic decoupling mechanism, the semantic center coordinates are determined based on the anatomical structure segmentation mask, and the trajectory consistency loss function is obtained by combining the spatial average value. According to the optimized spatiotemporal model, the prediction results are fused and output through the cross-attention mechanism; A judgment threshold is set and the trajectory consistency loss function is used to determine whether the prediction result matches the candidate puncture point. If the trajectory consistency loss function is less than the judgment threshold, it means that the prediction result matches the trajectory of the candidate puncture point.

8. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 7, characterized in that: To determine the trajectory within any frame, the corresponding calculation formula is: Among them, τ i represents the trajectory of the anatomical structure in the i-th frame; represents the coordinates of the i-th pixel corresponding to the anatomical structure in the i-th frame; T represents the total number of frames corresponding to the preoperative X-ray video.

9. The X-ray image visual prompt system for atrial septal puncture surgery according to claim 7, characterized in that: The spatiotemporal model is optimized through the trajectory-semantic decoupling mechanism. The semantic center coordinates are determined based on the anatomical structure segmentation mask. The trajectory consistency loss function is obtained by combining the spatial average, including: Define the anatomical structure segmentation mask as Determine the semantic center coordinates, and the corresponding calculation formula is: in, Represents the semantic center coordinates (x, y); Combined with the spatial average value, the trajectory consistency loss function is obtained, and the corresponding calculation formula is: in, represents the trajectory consistency loss function; i represents the i-th pixel; Indicates the semantic center coordinates corresponding to the i-th pixel; Represents the spatial average value corresponding to the i-th pixel.

Images