Image display method and device, equipment and storage medium

Abstract

The invention provides an image display method and device, equipment and a storage medium to achieve accurate positioning of a guide wire position and reduce the path deviation risk, and the method comprises the steps that preoperative three-dimensional image data of a focus area of a patient is obtained, three-dimensional reconstruction processing is conducted on the three-dimensional image data, and a focus area three-dimensional model is generated; in the guide wire puncture operation process, a multi-axial image of a puncture part of a patient is collected, and the multi-axial image comprises a plurality of axial two-dimensional images; determining a projection transformation relationship between the plurality of axial two-dimensional images and a three-dimensional space; identifying and extracting guide wire features in the two-dimensional image, and determining two-dimensional feature information of the guide wire; according to the projection transformation relation, converting multiple pieces of axial guide wire two-dimensional feature information into three-dimensional space coordinates, and generating guide wire three-dimensional data representing the spatial position of the guide wire; and according to the three-dimensional data of the guide wire, generating a three-dimensional model of the guide wire, and displaying the three-dimensional model of the guide wire on the three-dimensional model of the focus area in an overlapping manner.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image display method, apparatus, device and storage medium. Background Technology

[0002] In medical procedures such as cardiac surgery that require guidewire insertion, precise guidewire positioning is crucial for ensuring surgical safety and effectiveness. Surgeons need to monitor the guidewire's exact location within the patient's body in real time and use this information to plan the subsequent guidewire movement path, gradually guiding the guidewire to the target lesion. Currently, the commonly used guidewire positioning technology in the medical industry involves periodically acquiring two-dimensional images of the patient's body using digital subtraction angiography (DSA) during the insertion process. Surgeons observe these images to determine the approximate location of the guidewire within the patient's body, and then make decisions and execute subsequent guidewire insertion procedures.

[0003] However, existing technologies cannot meet the high requirements for accuracy and safety in clinical surgery. For example, the limitations of two-dimensional image information make positioning unintuitive, requiring doctors to rely on their own clinical experience and professional knowledge for indirect inference. This reliance on doctors' experience increases the risk of judgment errors, as different doctors have varying abilities to interpret two-dimensional images and infer spatial information, which can easily lead to deviations in guidewire position judgment. Summary of the Invention

[0004] In view of this, the present disclosure provides an image display method, apparatus, device and storage medium to achieve accurate positioning of the guide wire and reduce the risk of path deviation.

[0005] In a first aspect, an image display method is provided, comprising: acquiring preoperative three-dimensional image data of a patient's lesion area; performing three-dimensional reconstruction processing on the three-dimensional image data to generate a three-dimensional model of the lesion area; during guidewire puncture, acquiring multi-axial images of the patient's puncture site, the multi-axial images including multiple axial two-dimensional images; determining the projection transformation relationship between the multiple axial two-dimensional images and three-dimensional space; identifying and extracting guidewire features in the two-dimensional images to determine guidewire two-dimensional feature information; converting the multiple axial guidewire two-dimensional feature information into three-dimensional spatial coordinates according to the projection transformation relationship to generate guidewire three-dimensional data characterizing the guidewire's spatial position; generating a guidewire three-dimensional model based on the guidewire three-dimensional data; and superimposing the guidewire three-dimensional model on the lesion area three-dimensional model.

[0006] In one embodiment, the 3D model of the lesion region includes a 3D model of the lesion and surrounding vascular structures; the step of performing 3D reconstruction processing on the 3D image data to generate the 3D model of the lesion region includes: performing voxelization processing on the acquired preoperative 3D image data of the patient's lesion region to generate a 3D voxel dataset; applying a surface extraction algorithm to the 3D voxel dataset to extract the vascular surface and construct a 3D mesh model of the blood vessels; and performing topology optimization and texture mapping processing on the 3D mesh model to obtain a 3D model of the lesion region including the lesion and surrounding vascular structures.

[0007] In one embodiment, before acquiring multiaxial images of the patient's puncture site, the method further includes: extracting vascular orientation features and lesion location information based on the three-dimensional model of the lesion area; determining configuration parameters for multiaxial image acquisition based on the vascular orientation features and lesion location information, and determining position configuration information of the image acquisition device based on the configuration parameters; and driving multiple detectors of the image acquisition device to adjust to the positions indicated by the position configuration information.

[0008] In one embodiment, the configuration parameters for multi-axial image acquisition are determined based on the vascular orientation features and lesion location information, including: determining vascular imaging clarity evaluation information at different acquisition angles based on the vascular orientation features and lesion location information, and determining the configuration parameters for multi-axial image acquisition based on the vascular imaging clarity evaluation information; the configuration parameters are used to indicate the target acquisition angles corresponding to multiple axial images.

[0009] In one embodiment, the multiaxial imaging includes biaxial imaging, which includes an anteroposterior two-dimensional image and a lateral two-dimensional image; the acquisition of multiaxial images of the patient's puncture site is performed using a dual-plate digital subtraction angiography (DSA) device, which includes a radiation generator and two detectors, the two detectors being arranged in an anteroposterior and lateral configuration; or, the acquisition of multiaxial images of the patient's puncture site is performed collaboratively using a main DSA device and an auxiliary C-arm device.

[0010] In one embodiment, when acquiring multiaxial images of the patient's puncture site is performed collaboratively by a main DSA device and an auxiliary C-arm device, before acquiring the multiaxial images of the patient's puncture site, the method further includes: acquiring calibration images of the patient's puncture site through the main DSA device and the auxiliary C-arm device, extracting feature markers from the calibration images to obtain image feature point data; and calculating the spatial coordinate transformation matrix between the main DSA device and the auxiliary C-arm device based on the image feature point data.

[0011] In one embodiment, the method further includes: when the auxiliary C-arm device is a mobile C-arm device, extracting feature marker points from the two-dimensional image acquired by the mobile C-arm device during each image acquisition, and recalculating the real-time spatial coordinate transformation matrix between the mobile C-arm device and the main DSA device.

[0012] In one embodiment, the method further includes: generating a guidewire travel guide trajectory based on the vascular structure information in the three-dimensional model of the lesion region; the step of overlaying the guidewire three-dimensional model on the three-dimensional model of the lesion region includes: overlaying the guidewire three-dimensional model and the guidewire travel guide trajectory on the three-dimensional model of the lesion region.

[0013] In one embodiment, the method further includes: receiving an interactive operation command from a user in a display window, the interactive operation command including a rotation command, a zoom command, or a point selection command; adjusting the display view of the three-dimensional model or locating the target position according to the interactive operation command; when a point selection command is detected to select a specific point on the guide wire three-dimensional model, extracting the three-dimensional coordinates of the specific point, marking the corresponding projection position in a two-dimensional image auxiliary window, thereby realizing the linkage display of the three-dimensional model and the two-dimensional image.

[0014] In one embodiment, the method further includes: recording the three-dimensional position coordinate data of the guidewire during the guidewire puncture operation in a time sequence to generate a guidewire motion trajectory time-series database; recording early warning events occurring during the guidewire puncture operation, extracting the early warning time, early warning type, and deviation value, and generating an early warning event log; after the guidewire puncture operation is completed, calculating the average deviation value, maximum deviation value, and operation duration of the guidewire based on the guidewire motion trajectory time-series database and the early warning event log, and generating an operation review report; and / or, reconstructing the three-dimensional motion process of the guidewire in chronological order according to the guidewire motion trajectory time-series database.

[0015] Secondly, an image display device is provided, comprising: The acquisition module is used to acquire preoperative three-dimensional image data of the patient's lesion area, perform three-dimensional reconstruction processing on the three-dimensional image data, and generate a three-dimensional model of the lesion area. The image acquisition and processing module is used to acquire multi-axial images of the patient's puncture site during the guidewire puncture procedure. The multi-axial images include two-dimensional images along multiple axes. The module also determines the projection transformation relationship between the two-dimensional images along multiple axes and three-dimensional space. The feature processing module is used to identify and extract guidewire features in the two-dimensional image, determine guidewire two-dimensional feature information, and convert guidewire two-dimensional feature information of multiple axes into three-dimensional spatial coordinates according to the projection transformation relationship, thereby generating guidewire three-dimensional data characterizing the spatial position of the guidewire. The three-dimensional model display module is used to generate a three-dimensional model of the guidewire based on the guidewire three-dimensional data, and to overlay the guidewire three-dimensional model on the three-dimensional model of the lesion area.

[0016] Thirdly, an electronic device is provided, including a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is configured to execute the machine-readable instructions stored in the memory, wherein when the machine-readable instructions are executed by the processor, the processor performs the steps of the image display method described in any of the above embodiments.

[0017] Fourthly, a computer-readable storage medium is provided, including instructions stored thereon, wherein when the instructions are executed by a processor, the image display method described in any of the above embodiments is executed.

[0018] In the image display method provided in this embodiment, a three-dimensional model is reconstructed based on three-dimensional image data of the lesion area before surgery. During surgery, multi-axial two-dimensional images are acquired, and combined with projection transformation relationships, the two-dimensional features of the guidewire are converted into three-dimensional spatial coordinates to generate three-dimensional guidewire data. Ultimately, the precise quantification of the three-dimensional position of the guidewire in the patient's body is achieved, eliminating reliance on the doctor's experience and avoiding positioning deviations caused by missing depth information. By overlaying the three-dimensional model of the guidewire with the three-dimensional model of the lesion area, doctors do not need to switch between multiple two-dimensional images for inference. They can directly and intuitively observe the spatial morphology and travel position of the guidewire through the fused images, clearly grasp the relative relationship between the guidewire and the blood vessel wall, lesion, and surrounding tissues, and thus quickly determine whether the guidewire has deviated from the target blood vessel or is close to a dangerous area, significantly reducing the complexity of intraoperative observation and decision-making and improving operational efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an image display method; Figure 2 This is a schematic diagram of the equipment deployment for acquiring dual-axis images; Figure 3 This is a schematic diagram illustrating the synchronized display of real-time 2D images and processed 3D models; Figure 4 This is a schematic diagram of an image display device illustrated in an exemplary embodiment of the present disclosure; Figure 5 This is a schematic diagram of a computer device illustrated in an exemplary embodiment of this disclosure. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0021] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0023] Furthermore, the symbol “ / ” in this disclosure indicates that there is an “or” relationship between the related objects before and after the symbol, or that the related objects have an exemplary relationship that can coexist.

[0024] like Figure 1 The diagram shown is a flowchart of an image display method provided in an embodiment of this disclosure, including: S101: Obtain preoperative three-dimensional image data of the patient's lesion area, perform three-dimensional reconstruction processing on the three-dimensional image data, and generate a three-dimensional model of the lesion area.

[0025] Here, the preoperative three-dimensional imaging data refers to the image dataset containing spatial information of the lesion and surrounding tissues, obtained by using specialized medical imaging acquisition equipment (such as a dual-panel DSA device, or a combination device of DSA combined with a floor-standing C-arm or a mobile C-arm) to perform a comprehensive scan of the area where the patient's lesion is located before the formal start of the guidewire-guided interventional surgery.

[0026] In practice, specialized medical imaging equipment (such as dual-panel DSA equipment) can be used to acquire omnidirectional images of the patient's lesion area (such as target areas like cardiac vessels and peripheral vessels). During acquisition, image information of the lesion and surrounding tissues (such as blood vessels and adjacent organs) is captured from different angles, ensuring complete anatomical coverage of the lesion, and the acquisition accuracy must meet clinical requirements (e.g., 0.05mm accuracy) to provide high-quality raw data for 3D reconstruction. Then, the image information from different angles is integrated, and the spatial three-dimensional structure of the lesion and surrounding tissues is reconstructed through calculation, transforming the scattered image data into a 3D digital model. The final generated 3D model of the lesion area clearly presents the location and shape of the lesion, as well as anatomical details such as the direction and diameter of surrounding blood vessels, providing an intuitive and accurate spatial reference for intraoperative system operations such as automatically analyzing blood vessel direction, calculating the optimal acquisition angle, and planning the optimal guidewire path.

[0027] In practice, after acquiring the aforementioned three-dimensional image data, the three-dimensional image data is processed for three-dimensional reconstruction to construct a preoperative three-dimensional model that can clearly present the anatomical details of the lesion and surrounding blood vessels, providing a reliable spatial basis for intraoperative guidewire path planning and equipment angle optimization.

[0028] In one embodiment, the 3D model of the lesion region may include a 3D model of the lesion and surrounding vascular structures; the step of performing 3D reconstruction processing on the 3D image data to generate a 3D model of the lesion region may include: performing voxelization processing on the acquired preoperative 3D image data of the patient's lesion region to generate a 3D voxel dataset; applying a surface extraction algorithm to the 3D voxel dataset to extract the vascular surface and construct a 3D mesh model of the blood vessels; and performing topology optimization and texture mapping processing on the 3D mesh model to obtain a 3D model of the lesion region including the lesion and surrounding vascular structures.

[0029] Here, the preoperative image data of the lesion area (including information on the lesion, blood vessels, and adjacent tissues) is first converted into a computer-recognizable three-dimensional voxel dataset. A voxel can be understood as a pixel in three-dimensional space, with each voxel corresponding to a specific anatomical location and grayscale value (reflecting differences in tissue density). Through voxelization, the originally scattered two-dimensional images are integrated into a continuous three-dimensional digital space, completely preserving the spatial relationship between the lesion and blood vessels, laying the data foundation for subsequent extraction of vascular structures. This step also ensures that subsequent processing maintains the high accuracy of the original images, consistent with the accuracy requirements of preoperative three-dimensional reconstruction (e.g., 0.05 mm). Then, surface extraction algorithms (such as the marching cubes algorithm) are used to selectively separate the surface contours of vascular tissue from the three-dimensional voxel dataset. This algorithm, based on the differences in voxel grayscale values ​​(blood vessels differ from bones and soft tissues in density, resulting in clear distinctions in grayscale values), eliminates interference from non-vascular tissues and accurately extracts the surface information of the inner and outer walls of blood vessels. Subsequently, the extracted vascular surface information can be converted into a 3D mesh model (a three-dimensional structure composed of numerous polygonal patches), clearly presenting the direction, branches, diameter, and attachment location of lesions on the blood vessels (such as stenosis and plaque locations). Next, the 3D reconstruction model is optimized and its visualization enhanced, involving two core operations: first, topology optimization, which uses algorithms to repair redundant patches and topological defects in the 3D mesh model (such as mesh breaks at vascular branches), and simplifies unnecessary details (such as minor protrusions that do not affect the overall structure), improving computer processing efficiency while ensuring model accuracy and avoiding stuttering during real-time intraoperative display; second, texture mapping, which maps tissue features from preoperative images (such as density differences in the lesion area and the texture of the blood vessel wall) onto the surface of the 3D mesh model, and uses specific visual rules to annotate key information (such as red highlighting of the lesion area and numerical annotation of the blood vessel diameter), making the model more consistent with the clinical observation habits of doctors. The final generated 3D model of the lesion area can accurately restore the anatomical structure and, through visualization enhancement, assist doctors in quickly identifying key information, providing intuitive spatial basis for intraoperative angle calculation and path planning.

[0030] S102: During the guidewire puncture procedure, multi-axial images of the patient's puncture site are acquired, including two-dimensional images of multiple axes; and the projection transformation relationship between the two-dimensional images of multiple axes and three-dimensional space is determined.

[0031] This step, through multi-angle image acquisition and spatial mapping, lays the foundation for subsequently deducing the three-dimensional position of the guidewire from two-dimensional images. In practice, specialized imaging equipment (such as dual-plate DSA, DSA + floor-standing C-arm, or DSA + mobile C-arm) can be used to acquire multiple axial two-dimensional images (usually anteroposterior and lateral views) of the puncture site in real time during guidewire insertion. During acquisition, it is necessary to ensure that the angles between the images in each axis are reasonable (greater than 30°, with spatial information being more accurate when closer to 80°-90°), and the acquisition frequency must meet the requirements of real-time surgery (e.g., 30 frames / second). Simultaneously, preprocessing such as noise reduction and contrast enhancement will be performed on the images to ensure clarity.

[0032] Next, algorithms such as Scale-Invariant Feature Transform (SIFT) feature point extraction + Random Sample Consensus (RANSAC) matching can be used to establish mapping rules (i.e., projection transformation relationships) between pixels on two-dimensional images and the spatial coordinates of the three-dimensional model of the lesion region (generated through preoperative reconstruction). Using this relationship, the projection information of the guidewire in multiple axial two-dimensional images can be jointly and inversely converted into the actual position information of the guidewire in three-dimensional space, providing a spatial mapping basis for the generation of the guidewire's three-dimensional model.

[0033] In practice, to ensure that subsequent multiaxial images accurately match the anatomical structure of the lesion area and effectively support the three-dimensional positioning of the guidewire, the appropriate parameters for image acquisition and the device location can be determined based on the three-dimensional model of the lesion area generated preoperatively. In one implementation, before acquiring multiaxial images of the puncture site, the precise configuration of the image acquisition device is completed through the following process: Based on the three-dimensional model of the lesion area, the vascular orientation features and lesion location information are extracted; according to the vascular orientation features and lesion location information, the configuration parameters for multi-axial image acquisition are determined, and the position configuration information of the image acquisition device is determined according to the configuration parameters; the multiple detectors of the image acquisition device are driven to adjust to the positions indicated by the position configuration information.

[0034] Here, key anatomical information in the model is automatically parsed, including extracting vascular features (such as the curvature of the vessel, branching direction, and main trunk extension path) and locating the specific position of the lesion (such as the attachment point of the lesion on the vessel and its relative distance to surrounding vessels). Based on the extracted anatomical information, the core configuration parameters for multi-axial image acquisition can be further calculated. Key parameters include the acquisition angle, the acquisition field of view (which must completely cover the lesion and guidewire puncture path), and the detector resolution adaptation value. Simultaneously, these configuration parameters can be converted into positional configuration information for the image acquisition equipment. For example, if a dual-plate DSA device is used, the specific placement angle of the two detectors (anteroposterior or lateral view, or a position conforming to the optimal angle) and their distance from the patient's puncture site will be determined. If a DSA + floor-standing C-arm / mobile C-arm combination is used, the spatial coordinate matching relationship between the devices can be calculated in advance using the SIFT + RANSAC algorithm to ensure that the positions of different devices can be coordinated to adapt to the anatomical features. Based on the calculated positional configuration information, multiple detectors of the image acquisition device (such as the two detectors of a dual-plate DSA, or the detectors of a floor-mounted C-arm / mobile C-arm) can be automatically driven to move to the designated position. The entire adjustment process avoids physical interference between the device and the patient in real time (the included angle can be adaptively optimized within the range of 30-90°). Finally, the position of the detector will accurately match the blood vessel course and lesion location, ensuring that the subsequent multi-axial two-dimensional images can capture guidewire projection information from the optimal angle, avoiding image information loss or redundancy due to detector position deviation.

[0035] To ensure that multi-axial images clearly present vascular structures and lesion details, and to provide a high-quality foundation for subsequent extraction of guidewire projection information and establishment of two-dimensional and three-dimensional spatial mapping relationships from the images, the acquisition angle can be optimized by combining the vascular orientation features and lesion location information in the three-dimensional model of the lesion area. In one implementation, the specific logic for determining the multi-axial image acquisition configuration parameters may include: Based on the vascular orientation characteristics and the lesion location information, vascular imaging clarity evaluation information is determined at different acquisition angles, and the configuration parameters for the multi-axial image acquisition are determined according to the vascular imaging clarity evaluation information; the configuration parameters are used to indicate the target acquisition angles corresponding to multiple axial images.

[0036] Here, different acquisition angles can be simulated (covering an adjustable range of the device, such as 30-90 degrees). Combining the extracted vascular orientation and lesion location information, the clarity of vascular imaging at each simulated angle is quantitatively evaluated. Evaluation dimensions mainly include: whether the blood vessel is obscured by bone / soft tissue images (e.g., avoiding overlap between the blood vessel and rib images due to the acquisition angle), whether the lesion area is fully presented within the imaging field of view (e.g., ensuring no truncation at the lesion edge), whether the blood vessel diameter and stenosis segment can be clearly distinguished (avoiding vascular projection distortion caused by the angle), and whether the area covered by the guidewire puncture path is fully visualized. For example, when the blood vessel is C-shaped, the evaluation considers whether there is projection overlap in the curved segment at different angles, selecting angles that allow the entire curved segment to be clearly visible. This process essentially uses anatomical information to predict the imaging effect and exclude angles that would lead to information loss or interference. Then, from the evaluation results of all simulated angles, the angle with the clearest vascular imaging, the most complete lesion presentation, and no key area obstruction is selected as the target acquisition angle. These angle values ​​constitute the configuration parameters for multi-axial image acquisition. The finalized configuration parameters (target acquisition angle) are directly used to guide the adjustment of the detector position of the image acquisition equipment, ensuring that high-quality images can be accurately captured during actual acquisition.

[0037] In one embodiment, the multiaxial imaging includes biaxial imaging, which comprises an anteroposterior two-dimensional image and a lateral two-dimensional image. Multiaxial images of the patient's puncture site are acquired using a dual-plate digital subtraction angiography (DSA) device, which includes a radiation generator and two detectors arranged in an anteroposterior and lateral configuration. Alternatively, multiaxial images of the patient's puncture site can be acquired collaboratively using a main DSA device and an auxiliary C-arm device; for example, the main DSA device acquires anteroposterior images, and the auxiliary C-arm device acquires lateral images.

[0038] like Figure 2The diagram shows a schematic of the equipment deployment for acquiring biaxial images. By using anteroposterior (frontal) and lateral (side) two-dimensional images that reflect different dimensions of the puncture site, the spatial information gaps in a single two-dimensional image can be filled, laying the foundation for establishing a two-dimensional-to-three-dimensional mapping relationship. Specifically, there are several hardware deployment methods. One method uses a dedicated dual-panel DSA device, which includes a radiation generator and two detectors, pre-configured for anteroposterior and lateral views (one anteroposterior and one lateral). With this configuration, the device can simultaneously acquire anteroposterior and lateral two-dimensional images of the puncture site in a single exposure, eliminating the need for multiple adjustments to the device position. Furthermore, the detector angle can be dynamically optimized using a preoperative three-dimensional model, further improving image quality. Another method involves the main DSA device working in conjunction with an auxiliary arm device (a floor-standing C-arm or a mobile C-arm). The main DSA and the auxiliary C-arm are responsible for acquiring anteroposterior and lateral images, respectively. Before acquisition, the spatial coordinates of the two need to be calibrated by an algorithm (such as SIFT+RANSAC algorithm) to ensure that the images acquired by the two devices can accurately correspond to the same puncture area. During acquisition, the two are exposed synchronously, and the preprocessing process is the same as that of dual-plate DSA, and finally, the anteroposterior and lateral two-dimensional images that meet the requirements are obtained.

[0039] In one embodiment, when acquiring multiaxial images of the patient's puncture site is performed collaboratively by a main DSA device and an auxiliary C-arm device, before acquiring the multiaxial images of the patient's puncture site, the method further includes: acquiring calibration images of the patient's puncture site through the main DSA device and the auxiliary C-arm device, extracting feature markers from the calibration images to obtain image feature point data; and calculating the spatial coordinate transformation matrix between the main DSA device and the auxiliary C-arm device based on the image feature point data.

[0040] In specific implementation, when calculating the spatial coordinate transformation matrix between the main DSA device and the auxiliary C-arm device based on the image feature point data, the SIFT algorithm can be used to extract feature point descriptors and the RANSAC algorithm can be used to match feature points to calculate the spatial coordinate transformation matrix between the main DSA device and the auxiliary C-arm device.

[0041] Here, since the main DSA device and the auxiliary C-arm device (floor-mounted C-arm or mobile C-arm) are two independent devices, their spatial coordinate systems are initially inconsistent. That is, the same anatomical point (such as a segment of blood vessel or the edge of a lesion) will correspond to different pixel coordinates in the images of the two devices. Directly acquiring images will result in inaccurate correlation between the anteroposterior and lateral images. Therefore, it is necessary to first acquire calibration images of the patient's puncture site simultaneously using both devices (consistent with the patient's position in subsequent surgical acquisition), and then extract feature markers from the calibration images (anatomical markers of the human body can be selected, such as spinous processes of the spine, rib endpoints, or positioning stickers pasted on the body surface). These feature markers must meet the condition that they can be clearly identified in the calibration images of both devices, thereby forming image feature point data that can be used for matching.

[0042] The SIFT algorithm described above can process feature markers in calibration images from two devices, extracting descriptors for each feature point (containing unique information such as the feature point's shape and grayscale distribution). Even if the size and angle of the feature point differ slightly between the two device images (due to different shooting angles), the descriptors remain consistent, thus achieving preliminary identification of the same anatomical marker in the images from both devices. Due to factors such as image noise and soft tissue interference, the SIFT algorithm may produce mismatches (i.e., classifying different anatomical points in the two device images as the same feature point). The RANSAC algorithm can use random sampling verification to filter out valid matching points that conform to spatial geometric laws, eliminating mismatches and ensuring the accuracy of feature point matching. Based on the valid matching point pairs optimized by RANSAC (i.e., the coordinates of the feature point in the main DSA image and the coordinates of the corresponding anatomical point in the auxiliary C-arm image), a spatial coordinate transformation matrix is ​​generated through spatial geometric operations. This matrix can accurately convert the image pixel coordinates acquired by the auxiliary C-arm device into the coordinate system of the main DSA device, thereby achieving the unification of coordinates between the two devices. When acquiring subsequent anteroposterior and lateral images, the images from the auxiliary C-arm will automatically align with the main DSA images through this matrix, ensuring that both reflect the same anatomical location at the same puncture site for the patient.

[0043] In one embodiment, when the auxiliary C-arm device is a mobile C-arm device, during each image acquisition, feature marker points in the two-dimensional image acquired by the mobile C-arm device are extracted, and the real-time spatial coordinate transformation matrix between the mobile C-arm device and the main DSA device is recalculated.

[0044] Here, because the mobile C-arm device can be flexibly adjusted in position, it may slightly shift during the procedure due to operational needs, causing a change in its relative spatial position to the main DSA device. If the previously calculated fixed coordinate transformation matrix is ​​used, image coordinate misalignment will occur, failing to accurately correspond to the same puncture site. Therefore, each time a two-dimensional image is acquired using the mobile C-arm, feature markers (such as spinous processes of the spine, surface positioning stickers, etc.) are extracted from the current image simultaneously. Based on these real-time feature points, the spatial coordinate transformation matrix between the two (i.e., the real-time matrix) is recalculated to ensure that each acquired mobile C-arm image can be aligned with the coordinates of the main DSA device through this real-time matrix. This ensures that the biaxial images accurately reflect the same anatomical location, providing accurate data support for the three-dimensional inference of the guidewire position.

[0045] S103: Identify and extract guidewire features from the two-dimensional image to determine guidewire two-dimensional feature information; according to the projection transformation relationship, convert guidewire two-dimensional feature information of multiple axes into three-dimensional spatial coordinates to generate guidewire three-dimensional data characterizing the spatial position of the guidewire.

[0046] In practice, planar feature information such as the boundary and centerline of the guidewire can be accurately extracted from multi-axial two-dimensional images. Based on the projection relationship between the image and three-dimensional space, these two-dimensional features are converted into three-dimensional coordinates to generate three-dimensional data representing the spatial position of the guidewire. Before feature extraction, the acquired two-dimensional images can be preprocessed, such as using Gaussian filtering for noise reduction to eliminate the interference of image noise on guidewire identification; or contrast enhancement processing to strengthen the grayscale difference between the guidewire and the background, making the guidewire outline clearer.

[0047] When identifying and extracting guide wire features from the two-dimensional image and determining the two-dimensional feature information of the guide wire, the guide wire region can be accurately segmented first, and then a pixel-level guide wire segmentation mask can be generated (i.e., accurately marking whether each pixel in the image belongs to the guide wire region). Based on the guide wire mask obtained from the segmentation, the guide wire boundary coordinates (the set of pixel coordinates of the contour boundary of the guide wire in the two-dimensional image) and the guide wire centerline coordinates (the set of pixel coordinates of the central axis of the guide wire (which is the core basis for subsequent three-dimensional reconstruction)) are extracted through morphological processing and contour extraction algorithms.

[0048] Based on the device calibration parameters (such as detector position and acquisition angle), the projection transformation relationship between each two-dimensional image plane and three-dimensional space can be established (i.e., the mapping rules between two-dimensional pixel coordinates and three-dimensional spatial coordinates are clearly defined), and the projection transformation relationships of multi-axis images are interrelated (e.g., the angle between the projection planes of frontal and lateral images is 80-90 degrees to ensure spatial reconstruction accuracy). In this way, based on this projection transformation relationship, the two-dimensional feature information of the guidewire in the extracted images can be converted into three-dimensional spatial coordinates to obtain the three-dimensional data of the guidewire.

[0049] To accurately separate guide wires from background tissue in 2D images and extract effective features, efficient processing can be achieved using a trained U-Net++ semantic segmentation model: First, the 2D image is input into the encoder network of the semantic segmentation model to extract multi-scale feature maps. Then, the decoder network of the semantic segmentation model upsamples and fuses the multi-scale feature maps to generate a pixel-level guide wire segmentation mask. Finally, based on the guide wire segmentation mask, combined with morphological processing and contour extraction algorithms, the boundary coordinates and centerline coordinates of the guide wire are determined as 2D feature information.

[0050] In practice, clinical surgical images of various types of blood vessels, guidewire models, and lesions can be collected as raw training data. The raw training data is then labeled with pixel-level masks of the guidewire region in each image to generate a labeled training dataset containing the target number of clinical surgical images. A U-Net++ network architecture is then constructed, and supervised learning is performed using the labeled training dataset to obtain a trained U-Net++ semantic segmentation model.

[0051] Here, we collect clinical surgical images covering different vascular scenarios, guidewire types, and lesion conditions as raw data. We annotate the guidewire region with pixel-level masks in each image (clearly indicating the pixel location of the guidewire) to form a target number of annotated training datasets. Then, we build a U-Net++ network architecture and use the annotated dataset for supervised learning training to finally obtain a model that can accurately identify guidewires.

[0052] To achieve accurate reconstruction of multi-axial two-dimensional guidewire features into three-dimensional spatial positions, the conversion can be accomplished in the following way: for guidewire feature points in two-dimensional images of each axis, feature point matching is performed with the help of epipolar constraints to determine the corresponding feature point pairs. Then, combined with the preset projection transformation relationship, the three-dimensional spatial coordinates of each pair of feature points are calculated using triangulation methods, and finally, three-dimensional point set data representing the spatial position of the guidewire are generated.

[0053] Here, during feature point matching (epidural constraint), the epipolar constraint rules in multi-view geometry are used to limit the projection range of the same guidewire feature point in different axial images, avoiding erroneous matching across images and ensuring that the selected feature point pairs correspond to the same physical point on the guidewire (such as the guidewire tip or inflection point), providing a reliable foundation for 3D coordinate calculation. During 3D coordinate calculation (triangulation), combined with the established projection transformation relationship (clarifying the mapping rules between 2D images and 3D space), a triangulation algorithm (similar to the depth perception principle of human binocular vision) is used to reverse-engineer the unique coordinates of the same physical point in 3D space based on the 2D coordinates of the same physical point from different viewpoints. Finally, the results are output: the 3D coordinate set of all feature point pairs forms the guidewire 3D point set data, fully representing the spatial morphology and position of the guidewire, providing core data support for subsequent guidewire 3D model optimization and overlay display.

[0054] S104: Based on the guidewire three-dimensional data, generate a guidewire three-dimensional model, and overlay the guidewire three-dimensional model onto the lesion area three-dimensional model.

[0055] In practice, a smooth three-dimensional geometric model of the guidewire can be generated using the guidewire's three-dimensional coordinate data. This model is then overlaid on the preoperatively reconstructed three-dimensional model, which includes the lesion and surrounding blood vessels, and presented synchronously in a visualization window, allowing doctors to intuitively grasp the real-time spatial position of the guidewire in the lesion area.

[0056] In one implementation, before generating a three-dimensional model of the guidewire based on the guidewire three-dimensional data, the guidewire three-dimensional data can be optimized to generate optimized guidewire three-dimensional data. Specifically, the optimization process for the guidewire three-dimensional data may include: initializing the guidewire three-dimensional data as source data, constructing a guidewire geometric constraint model, and establishing initial correspondences; calculating the distance from each point in the source data to the nearest point in the geometric constraint model, solving for the optimal rigid body transformation matrix using the least squares method, and updating the source data; determining the error between the updated source data and the geometric constraint model; if the error is greater than a convergence threshold, repeating the above transformation steps; if the error is less than the convergence threshold, terminating the iteration, and generating optimized guidewire three-dimensional data.

[0057] Here, through iterative fitting and error convergence, noise and jitter in the guidewire 3D data are eliminated, making the data more closely match the actual physical shape of the guidewire and laying the foundation for generating a smooth and accurate guidewire 3D model. First, the previously generated guidewire 3D point set (the original 3D data representing the guidewire's spatial position) is used as the source data. Simultaneously, a guidewire geometric constraint model is constructed, and an initial correspondence between the two is established. The guidewire geometric constraint model can be constructed based on the guidewire's physical characteristics (slender, continuous, smooth), typically a straight line or smooth curve model, used to limit the reasonable morphological range of the guidewire data. The 3D points in the source data can be initially matched with the constraint model to determine the reference position of each point on the model, providing a benchmark for subsequent adjustments. Next, data adjustment and matrix solving are performed: First, the distance from each 3D point in the source data to the nearest point in the guidewire geometric constraint model is calculated (i.e., the degree of deviation of each point from the model). Then, the optimal rigid body transformation matrix is ​​solved using the least squares method, and the source data is updated based on this matrix. The calculation of the nearest point distance here quantifies the deviation of each original data point, clarifying the direction and magnitude of adjustment. The least squares method finds the transformation rule that minimizes the total deviation of all points from the constraint model through mathematical fitting (rigid body transformation ensures that the overall shape of the guidewire remains unchanged, only correcting local deviations). The source data update is performed by adjusting the coordinates of each 3D point according to the optimal rigid body transformation matrix, making it closer to the constraint model and reducing discrete deviations caused by noise. Next, iterative convergence judgment is performed. The overall error between the updated source data and the guidewire geometric constraint model is calculated and compared with the preset convergence threshold (i.e., the maximum allowable deviation). If the error > the convergence threshold, it means that the data still has large jitter, and the steps of calculating the nearest point distance → solving the optimal transformation matrix → updating the source data are repeated. If the error < the convergence threshold, it means that the data has matched the actual shape of the guidewire, the iteration is terminated, and the optimized 3D data of the guidewire is output.

[0058] To achieve an intuitive presentation of the guidewire's spatial morphology and its positional association with the lesion area, the following method can be used to construct and overlay a three-dimensional model of the guidewire: read the coordinate information of the guidewire's three-dimensional data, use spline interpolation algorithm to generate a smooth guidewire center curve to construct a three-dimensional geometric model, perform segmented color rendering of the front navigation segment (red) and the main body segment (blue) to obtain a color-coded model, and then overlay this model with the three-dimensional model of the lesion area to output a fused image.

[0059] To allow doctors to intuitively grasp the blood flow status of the blood vessels at the puncture site and to assist the guidewire in accurately adapting to the vascular environment, a hemodynamic information fusion process can be added: Before the operation, blood flow image data of the puncture site is collected by ultrasound equipment, blood flow velocity and blood flow direction information are extracted to generate hemodynamic parameters, and then these parameters are mapped to the corresponding blood vessel positions in the three-dimensional model of the lesion area, and the blood flow velocity distribution is superimposed in the display window with a gradient color (such as from blue to red).

[0060] Here, preoperative ultrasound equipment is used to selectively acquire blood flow images at the puncture site. Key information is extracted from these images: blood flow velocity (such as the velocity in different areas of the blood vessel) and blood flow direction (such as flowing along the vessel or reflux). This information is quantified into standardized hemodynamic parameters, providing a data foundation for subsequent visualization. The generated hemodynamic parameters are mapped one-to-one to the corresponding vascular structures in the preoperatively reconstructed 3D model of the lesion area according to the spatial location of the blood vessel. A gradient color coding rule, such as blue-green-red (blue represents slow blood flow velocity, and red represents fast blood flow velocity), is used to overlay these parameters in the display window, allowing doctors to easily distinguish the differences in blood flow velocity in different areas of the blood vessel. This implementation method helps doctors assess vascular patency preoperatively, avoid abnormal blood flow (such as excessively fast flow velocity or reflux) or stenotic areas during the procedure, further improving the accuracy of guidewire puncture. It is particularly suitable for complex vascular scenarios such as vascular stenosis and slow blood flow, reducing the risk of vascular injury.

[0061] In one embodiment, the method further includes: generating a guidewire travel guide trajectory based on the vascular structure information in the three-dimensional model of the lesion region; the step of overlaying the guidewire three-dimensional model on the three-dimensional model of the lesion region includes: overlaying the guidewire three-dimensional model and the guidewire travel guide trajectory on the three-dimensional model of the lesion region.

[0062] Here, in order to provide clear guidance for guidewire puncture and reduce the difficulty of operation for doctors, a guidewire trajectory can be generated based on the vascular structure information in the 3D model of the lesion area. When overlaid, the 3D model of the guidewire and the guide trajectory are presented together on the 3D model of the lesion area.

[0063] When generating the guidewire trajectory, the preoperatively reconstructed 3D model of the lesion area can be used as a basis to extract key information about the vascular structure (such as the vessel centerline, diameter, curvature, and lesion location), and a path planning algorithm (such as...) can be employed. The algorithm comprehensively considers the curvature of the blood vessel, the adaptability of the diameter, and the need to avoid surrounding dangerous tissues (nerves, vital organs), automatically extracts the center path of the target blood vessel, generates the optimal guiding trajectory of the guidewire, and clarifies the ideal direction and path range of the guidewire.

[0064] Specifically, the topological structure of the blood vessel centerline is extracted from the 3D model of the lesion area, and the blood vessel centerline is discretized into a set of nodes to construct a blood vessel path map; the starting position node of the guidewire (the current location of the blood vessel) and the target position node (the lesion or target blood vessel area that the guidewire needs to reach) are set, and a path cost function that comprehensively considers the path length, blood vessel curvature, blood vessel diameter and distance from dangerous tissue is defined, and the open list and closed list are initialized; the node with the smallest cost function value in the open list is selected as the current node, the adjacent nodes of the current node are expanded and the cost value is calculated, the open list and closed list are updated, and the expansion steps are repeated until the target node is selected; the path from the starting node to the target node is traced back to generate the guidewire travel guidance trajectory.

[0065] Here, the topological structure of the blood vessel centerline (i.e., the core axis of the blood vessel, reflecting the direction and branching relationship of the blood vessel) is first extracted from the 3D model of the lesion area. Then, this continuous centerline is discretized into a set of nodes, and finally a blood vessel path graph is constructed, that is, a visualized network structure with nodes as the core and the connectivity between nodes as the edges. The path cost function comprehensively considers four key factors: path length (the shorter the better), blood vessel curvature (the smoother the better, reducing guidewire bending damage), blood vessel diameter (adapting to guidewire size and avoiding narrow areas), and distance from dangerous tissues (the farther the safer, avoiding nerves and vital organs). Each factor is assigned a corresponding weight, and finally, the total cost of each potential path is calculated through the function. The open list is used to store nodes to be evaluated, i.e., unexplored potential path nodes, and the closed list is used to store nodes that have been evaluated, i.e., nodes whose optimal paths have been determined, avoiding repeated calculations or path loops.

[0066] Next, iteratively search for the optimal node: select the node with the smallest cost function value from the open list as the current node (i.e., the node most likely to lead to the optimal path); expand all adjacent nodes of the current node (i.e., the next node directly connected to the current node), and calculate the total cost value of each adjacent node using the cost function; update the open and closed lists: add adjacent nodes not in either list to the open list, and update the cost value of an adjacent node already in the open list (preserving the smaller cost), and move the processed current node into the closed list; repeat the above steps, continuously expanding the nodes until the target node is selected (i.e., the target node enters the closed list). Finally, backtrack to generate the guiding trajectory: backtrack from the target node, sequentially associating its previous optimal parent node, grandparent node, ... until the starting node, forming a complete start-target node chain.

[0067] In one optional implementation, the operation can be monitored in real time during surgery, and alarm prompts can be issued based on operational errors. For example, the front-end coordinates and front-end direction vector of the guidewire 3D model are obtained, the shortest path point is searched on the optimal travel trajectory, and the Euclidean distance is calculated as the distance deviation value and the angle between the direction vectors is calculated as the angle deviation value to obtain deviation parameters; the distance deviation value is compared with a preset distance deviation threshold, and the angle deviation value is compared with a preset angle deviation threshold. If the deviation value exceeds the corresponding threshold, a path deviation warning message is generated, the deviation area is highlighted in the display window, and a buzzer prompt is triggered.

[0068] For example, the spatial distance between the tip of the guidewire and the pre-marked location of dangerous tissue is detected, and the spatial distance is compared with a preset safe distance threshold. When the spatial distance is less than the safe distance threshold, the dangerous area is highlighted in red in the display window and a high-intensity continuous buzzer alarm is triggered.

[0069] In one embodiment of this disclosure, two-dimensional images in frontal and lateral views can be displayed synchronously with a processed three-dimensional image, such as... Figure 3 As shown.

[0070] In one implementation, an interactive response function of the interface can be provided. Specifically, the interface receives interactive operation commands from the user in the display window, including rotation commands, zoom commands, or point selection commands. Based on the interactive operation commands, the display view of the 3D model is adjusted or the target position is located. When a point selection command is detected to select a specific point on the guide wire 3D model, the 3D coordinates of the specific point are extracted, and the corresponding projection position is marked in the 2D image auxiliary window, thereby realizing the linkage display of the 3D model and the 2D image.

[0071] In this implementation, the system receives user commands to rotate, zoom, and select points, and adjusts the display perspective of the 3D model of the lesion area and guidewire accordingly (e.g., rotating to view different orientations, zooming to focus on details), or quickly locating the target position through point selection, thereby meeting the diverse observation needs of doctors. When a user selects a specific point on the 3D model of the guidewire, the system automatically extracts the 3D coordinates of that point, and then, through projection transformation rules, marks the corresponding projection position of that point in a 2D image auxiliary window. This allows doctors to simultaneously grasp the 3D spatial information and the original 2D image reference of the same location, avoiding the hassle of switching between windows for inference.

[0072] In practical implementation, the guidewire puncture procedure can be traced, and postoperative effects can be evaluated and optimized. Specifically, the three-dimensional position coordinates of the guidewire during the puncture procedure can be recorded and stored in a time sequence to generate a time-series database of the guidewire movement trajectory; warning events occurring during the guidewire puncture procedure can be recorded, and the warning time, warning type, and deviation value can be extracted to generate a warning event log; after the procedure is completed, based on the time-series database of the guidewire movement trajectory and the warning event log, the average deviation value, maximum deviation value, and procedure duration of the guidewire can be calculated to generate a procedure review report. After generating the procedure review report, the time-series database of the guidewire movement trajectory can be loaded to reconstruct the three-dimensional movement process of the guidewire in chronological order, realizing a three-dimensional animation playback of the procedure; it can also receive user-defined playback speed parameters, adjust the time scaling ratio of the animation playback, and output an adjustable-speed playback video of the procedure.

[0073] In practice, the following can be extracted from the guidewire motion trajectory time series database: the guidewire's three-dimensional position coordinates sorted by timestamp, the surgery start timestamp (first data time), and the surgery end timestamp (last data time); and the following can be extracted from the warning event log: the warning occurrence time, the warning type (deviation warning / danger warning), the corresponding deviation value (distance deviation / angle deviation), and the danger zone association information. Based on the timestamp, the two types of data are associated to ensure that the guidewire position at each time node corresponds to the warning event.

[0074] Next, the actual 3D coordinates of the guidewire at each time point are extracted; the optimal guidewire trajectory data generated preoperatively is used to obtain the ideal path 3D coordinates at the same time point; the distance deviation between the actual and ideal coordinates at each time point is calculated using the Euclidean distance formula; the arithmetic mean of the distance deviations at all time points is calculated to obtain the average deviation value. The distance deviation values ​​at all the above time points are sorted, and the maximum value is selected as the maximum deviation value; at the same time, the warning event log is linked to mark whether the maximum deviation triggers a warning, the corresponding warning time and scenario (such as at the bend of the blood vessel, near the dangerous tissue area). In addition, the total operation time can be determined based on the duration between the operation end time stamp and the operation start time stamp; if it is necessary to further subdivide the duration of each step (such as guidewire advancement, adjustment, pause, etc.), it can be further subdivided in conjunction with the doctor's operation timeline (built-in data in the database). The final post-operative review report can include: basic information (surgery name, patient information, equipment model); core indicators (average deviation value, maximum deviation value, total operation time); guidewire trajectory diagram (connecting the actual coordinates of the guidewire in time sequence, overlaying the ideal path for comparison); details of warning events (arranged in chronological order by warning type, occurrence time, deviation value, and handling result); and analysis of key nodes (such as the surgical scenario when the maximum deviation occurred, and the operational adjustments after the warning was triggered). Through the review report, doctors can be provided with objective data to evaluate surgical outcomes, helping to optimize operational procedures, reduce subsequent surgical risks, and it can also serve as case study material for medical teaching.

[0075] It can load a time-series database of guidewire motion trajectories, arranging all guidewire 3D coordinate data in ascending order of timestamps to ensure the continuity of the time series. For the guidewire 3D coordinate point set at each timestamp, a spline interpolation algorithm can be used for curve fitting to generate a smooth guidewire center curve for that moment. Then, following the previous 3D model construction rules, the curve can be segmented and colored (red for the front navigation segment and blue for the main body segment) to construct the guidewire 3D geometric model at that time point, ensuring that the morphology is consistent with the real-time intraoperative display. The guidewire 3D models at each moment are continuously concatenated to form a frame sequence (10 frames per second, matching the data acquisition frequency). It receives the playback speed parameters set by the user (e.g., 0.5x-2x speed), and outputs a smooth 3D animation playback by adjusting the playback time interval of the frame sequence (e.g., shortening the frame interval at 2x speed and lengthening the frame interval at 0.5x speed). During playback, it supports switching perspectives, rotating and scaling the model, and simultaneously associating with a 2D image auxiliary window to achieve linked display of the 3D motion process and the original 2D image, facilitating detailed observation. Thus, through three-dimensional motion reconstruction, the entire process of guidewire movement can be intuitively restored, making it easier to trace the cause of deviations or provide early warnings, and providing a visual basis for the iteration of surgical techniques.

[0076] like Figure 4 As shown, this disclosure provides an image display device 400, including: The three-dimensional reconstruction module 41 is used to acquire preoperative three-dimensional image data of the patient's lesion area, perform three-dimensional reconstruction processing on the three-dimensional image data, and generate a three-dimensional model of the lesion area. The image space projection module 42 is used to acquire multi-axial images of the patient's puncture site during the guidewire puncture procedure, the multi-axial images including two-dimensional images of multiple axes; and to determine the projection transformation relationship between the two-dimensional images of multiple axes and three-dimensional space. The guidewire 3D data generation module 43 is used to identify and extract guidewire features in the 2D image, determine guidewire 2D feature information, and convert guidewire 2D feature information of multiple axes into 3D spatial coordinates according to the projection transformation relationship to generate guidewire 3D data characterizing the spatial position of the guidewire. The three-dimensional model construction module 44 is used to generate a three-dimensional model of the guidewire based on the three-dimensional data of the guidewire, and to overlay the three-dimensional model of the guidewire onto the three-dimensional model of the lesion area.

[0077] The specific implementation process of the functions and roles of each unit in the above-mentioned device can be found in the implementation process of the corresponding steps in the above-mentioned image display method, and will not be repeated here.

[0078] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0079] Based on the same technical concept, this disclosure also provides an electronic device 500, referring to... Figure 5 The diagram shown is a schematic representation of the structure of an electronic device according to an exemplary embodiment of this disclosure, comprising: The processor 510, memory 520, and bus 530 are included. The memory 520 is used to store execution instructions and includes main memory 521 and external memory 522. The main memory 521, also known as internal memory, is used to temporarily store the operation data in the processor 510 and the data exchanged with external memory 522 such as hard disk. The processor 510 exchanges data with external memory 522 through main memory 521.

[0080] In this embodiment, the memory 520 is specifically used to store application code that executes the scheme of this disclosure, and its execution is controlled by the processor 510. That is, when the electronic device 500 is running, the processor 510 communicates with the memory 520 through the bus 530, or the processor 510 communicates with the memory 520 through other means, so that the processor 510 executes the application code stored in the memory 520, and then executes the steps of the image display method described in any of the foregoing embodiments.

[0081] The memory 520 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0082] Processor 510 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0083] It is understood that the structures illustrated in the embodiments of this disclosure do not constitute a specific limitation on the electronic device 500. In other embodiments of this disclosure, the electronic device 500 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0084] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image display method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0085] This disclosure also provides a computer program product, which stores a computer program. When the computer program is run by a processor, it executes the steps of the image display method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0086] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0087] Furthermore, embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0088] The processing and logic flows described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flows can also be executed by dedicated logic circuits—such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), and the device can also be implemented as dedicated logic circuits.

[0089] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device such as a Universal Serial Bus (USB) flash drive, to name a few.

[0090] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0091] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0092] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0093] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0094] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image display method, characterized in that, include: Preoperative three-dimensional image data of the lesion area of ​​the patient is obtained, and the three-dimensional image data is processed for three-dimensional reconstruction to generate a three-dimensional model of the lesion area. During the guidewire puncture procedure, multi-axial images of the patient's puncture site are acquired, including two-dimensional images along multiple axes; and the projection transformation relationship between the two-dimensional images along multiple axes and three-dimensional space is determined. Identify and extract guidewire features from the two-dimensional image to determine guidewire two-dimensional feature information; based on the projection transformation relationship, convert guidewire two-dimensional feature information of multiple axes into three-dimensional spatial coordinates to generate guidewire three-dimensional data characterizing the spatial position of the guidewire. Based on the guidewire 3D data, a guidewire 3D model is generated, and the guidewire 3D model is overlaid on the lesion area 3D model for display.

2. The method according to claim 1, characterized in that, The 3D model of the lesion region includes a 3D model of the lesion and surrounding vascular structures; the 3D reconstruction processing of the 3D image data to generate the 3D model of the lesion region includes: The preoperative three-dimensional image data of the patient's lesion area were voxelized to generate a three-dimensional voxel dataset. A surface extraction algorithm is applied to the three-dimensional voxel dataset to extract the surface of blood vessels, and a three-dimensional mesh model of the blood vessels is constructed. The three-dimensional mesh model is subjected to topology optimization and texture mapping to obtain a three-dimensional model of the lesion region containing the lesion and surrounding vascular structures.

3. The method according to claim 1, characterized in that, Before acquiring multiaxial images of the patient's puncture site, the following steps are also included: Based on the three-dimensional model of the lesion area, the vascular orientation features and lesion location information are extracted; Based on the blood vessel orientation characteristics and lesion location information, the configuration parameters for multi-axial image acquisition are determined, and the location configuration information of the image acquisition device is determined based on the configuration parameters. The multiple detectors of the driving image acquisition device are respectively adjusted to the positions indicated by the position configuration information.

4. The method according to claim 3, characterized in that, Based on the described vascular orientation characteristics and lesion location information, the configuration parameters for multi-axial image acquisition are determined, including: Based on the vascular orientation characteristics and the lesion location information, vascular imaging clarity evaluation information is determined at different acquisition angles, and the configuration parameters for the multi-axial image acquisition are determined according to the vascular imaging clarity evaluation information; the configuration parameters are used to indicate the target acquisition angles corresponding to multiple axial images.

5. The method according to claim 1, characterized in that, The multi-axial image includes a dual-axial image, which includes a frontal two-dimensional image and a lateral two-dimensional image. Multiaxial images of the patient's puncture site are acquired using a dual-plate digital subtraction angiography (DSA) device, which includes a radiation generator and two detectors arranged in a frontal and lateral configuration; alternatively, multiaxial images of the patient's puncture site are acquired using a main DSA device and an auxiliary C-arm device in tandem.

6. The method according to claim 5, characterized in that, When acquiring multiaxial images of the patient's puncture site using a combination of the main DSA device and an auxiliary C-arm device, the following steps are included before acquiring multiaxial images of the patient's puncture site: The main DSA device and the auxiliary C-arm device are used to acquire calibration images of the patient's puncture site, and feature markers in the calibration images are extracted to obtain image feature point data. Based on the image feature point data, calculate the spatial coordinate transformation matrix between the main DSA device and the auxiliary C-arm device.

7. The method according to claim 6, characterized in that, Also includes: When the auxiliary C-arm device is a mobile C-arm device, during each image acquisition, feature marker points in the two-dimensional image acquired by the mobile C-arm device are extracted, and the real-time spatial coordinate transformation matrix between the mobile C-arm device and the main DSA device is recalculated.

8. The method according to claim 2, characterized in that, The method further includes: Based on the vascular structure information in the three-dimensional model of the lesion area, a guide wire travel trajectory is generated. The step of overlaying the guidewire 3D model onto the lesion area 3D model includes: The guidewire 3D model and the guidewire travel trajectory are superimposed and displayed on the lesion area 3D model.

9. The method according to claim 1, characterized in that, The method further includes: Receive user interaction commands in the display window, including rotation commands, zoom commands, or point-and-click commands, and adjust the display view of the 3D model or locate the target position according to the interaction commands. When a point selection command is detected to select a specific point on the guidewire 3D model, the 3D coordinates of that specific point are extracted, and the corresponding projection position is marked in the 2D image auxiliary window, so as to realize the linkage display of the 3D model and the 2D image.

10. The method according to claim 1, characterized in that, The method further includes: Record the three-dimensional position coordinate data of the guidewire during the guidewire puncture operation in a time series to generate a time series database of guidewire movement trajectory; and record the early warning events that occur during the guidewire puncture operation, extract the early warning time, early warning type and deviation value, and generate an early warning event log; After the guidewire puncture operation is completed, the average deviation value, maximum deviation value and operation time of the guidewire are calculated based on the guidewire motion trajectory time series database and the early warning event log, and an operation review report is generated; and / or, the three-dimensional motion process of the guidewire is reconstructed in chronological order according to the guidewire motion trajectory time series database.

11. An image display device, characterized in that, include: The acquisition module is used to acquire preoperative three-dimensional image data of the patient's lesion area, perform three-dimensional reconstruction processing on the three-dimensional image data, and generate a three-dimensional model of the lesion area. The image acquisition and processing module is used to acquire multi-axial images of the patient's puncture site during the guidewire puncture procedure. The multi-axial images include two-dimensional images along multiple axes. The module also determines the projection transformation relationship between the two-dimensional images along multiple axes and three-dimensional space. The feature processing module is used to identify and extract guidewire features in the two-dimensional image, determine guidewire two-dimensional feature information, and convert guidewire two-dimensional feature information of multiple axes into three-dimensional spatial coordinates according to the projection transformation relationship, thereby generating guidewire three-dimensional data characterizing the spatial position of the guidewire. The three-dimensional model display module is used to generate a three-dimensional model of the guidewire based on the guidewire three-dimensional data, and to overlay the guidewire three-dimensional model on the three-dimensional model of the lesion area.

12. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-readable instructions executable by the processor, the processor executing the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the processor performs the steps of the image display method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, Includes instructions stored thereon, wherein, when the instructions are executed by a processor, the image display method as described in any one of claims 1-10 is performed.

Images