A surgical navigation method, system, medium and electronic device based on DSA image

Abstract

The present invention provides a surgical navigation method, system, medium and electronic device based on DSA images, and the method includes the following steps: Step 1, obtaining the patient's CT angiography data, extracting the vascular structure therefrom, and constructing a 3D vascular model. The present invention significantly reduces the situation of interactive overlap of vascular positions by constructing the preoperative CT angiography data into a 3D vascular model, and aligning it with the DSA image in real time during the operation, thereby avoiding the need for multiple projections, thereby reducing the patient's irradiation dosage, examination time, contrast agent dosage and economic burden, and also reducing equipment loss; by processing the DSA image and predicting the camera parameters through the CNN neural network model, the accuracy and effectiveness problems existing in the registration of two-dimensional images and three-dimensional images are effectively overcome. By simulating the generation and training of DSA images, the CNN model can accurately predict the camera parameters corresponding to the intraoperative DSA image, thereby achieving high-precision image registration and improving the safety and success rate of the operation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a surgical navigation method, system, medium and electronic equipment based on DSA images. Background Art

[0002] After the vascular interventional surgery robot is put into use, the company will build a data storage platform to record patient disease data, imaging data, surgical planning, real-time navigation, surgical operations and other information. This information can be used for learning by the vascular interventional surgery robot (deep learning technology). When the amount of data is large enough, the vascular interventional surgery robot will learn tens of thousands or hundreds of thousands of surgeries to achieve more efficient and accurate assessment of surgical risks, formulate safer surgical plans, and implement more accurate real-time navigation. The vascular interventional surgery robot will develop towards full automation. With the development of automation technology, vascular interventional surgery robots will gradually become fully intelligent, and combined with imaging equipment, image post-processing technology, surgical real-time navigation technology, and automation technology, the full automation of vascular interventional surgery robots will be gradually realized. However, there are still certain problems:

[0003] First, in conventional DSA angiography images, the positions of blood vessels often overlap and require multiple projections to determine, which forces patients to increase irradiation dosage, examination time, contrast agent dosage, economic burden, and equipment loss, and cannot provide navigation guidance for interventional robots;

[0004] Second, the current navigation technology based on 3D images is mainly achieved by image registration between preoperative 3D images and intraoperative 2D images. This technology has two main problems:

[0005] The first is that there may be differences between preoperative three-dimensional images and the actual morphology during surgery;

[0006] The second is that the registration of two-dimensional and three-dimensional images has problems with accuracy and effectiveness;

[0007] Since the navigation technology based on two-dimensional images lacks one dimension of spatial information, the lack of information causes incomplete functions. Therefore, the current navigation technology based on two-dimensional images or three-dimensional images has problems such as poor accuracy and poor effectiveness.

[0008] 3. In existing vascular interventional surgeries, there is no real-time vascular registration method for DSA and preoperative CTA. Doctors need to rely on experience to determine the real-time position of the guidewire in the human body. Inexperienced doctors find it difficult to perform vascular interventional surgeries independently.

[0009] To this end, a surgical navigation method, system, medium and electronic equipment based on DSA images are proposed. Summary of the invention

[0010] In view of this, the embodiments of the present invention hope to provide a surgical navigation method, system, medium and electronic device based on DSA images to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial choice.

[0011] In order to solve the above technical problems, a technical solution adopted in this application is: a surgical navigation method based on DSA images, comprising the following steps:

[0012] Step 1: Obtain the patient's CT angiography data, extract the vascular structure from it, and construct a 3D vascular model;

[0013] Step 2: Adjust the camera parameters, project the 3D blood vessel model to generate multiple simulated DSA images, and record the camera parameter set corresponding to each simulated DSA image;

[0014] Step 3, obtaining the patient's intraoperative DSA image, and preprocessing the simulated DSA image and the intraoperative DSA image;

[0015] Step 4: Build a CNN neural network model and train the CNN neural network model using simulated DSA images and their corresponding camera parameter sets;

[0016] Step 5: Input the patient's intraoperative DSA image into the trained CNN neural network model to predict the corresponding camera parameter set;

[0017] Step 6: According to the predicted camera parameter set, the 3D vascular model constructed before the operation is adjusted to be registered with the intraoperative DSA image, and the registered 3D vascular model is displayed in real time in the surgical navigation system;

[0018] Step 7: Evaluate the registration results and optimize the CNN neural network model, training data, and preprocessing methods based on the evaluation results;

[0019] Step 8: Build a user interaction interface, through which the doctor can view and adjust the position, posture and scaling of the 3D vascular model in real time during the operation.

[0020] As a further preferred embodiment of the present technical solution, in step 1, before extracting the vascular structure, the CT angiography data is subjected to denoising and contrast adjustment processing.

[0021] As a further preferred embodiment of the present technical solution, in step 2, the camera parameters include position parameters, posture parameters, focal length, field of view angle and image resolution, and the camera parameter set includes camera parameters for generating a DSA image.

[0022] As a further preferred embodiment of the present technical solution, in step three, the preprocessing of the simulated DSA image and the intraoperative DSA image includes denoising, contrast adjustment, normalization and size adjustment.

[0023] As a further preferred embodiment of the present technical solution, in step four, the CNN neural network model utilizes a deep learning architecture, including a convolutional layer, a pooling layer, an activation function layer and a fully connected layer, and the CNN neural network model is optimized using a cross entropy loss function during training.

[0024] As a further preferred embodiment of the present technical solution, in step five, the process of predicting the camera parameter set also includes a confidence evaluation of the prediction result, and when the prediction confidence is lower than a preset threshold, a recalibration or manual intervention mechanism is automatically triggered.

[0025] As a further preferred embodiment of the present technical solution, in step six, the registration of the 3D vascular model with the intraoperative DSA image utilizes an iterative closest point algorithm or a feature-based registration method.

[0026] To solve the above technical problems, another technical solution adopted in this application is: a surgical navigation system based on DSA images, comprising: a data acquisition and processing module, a simulated DSA generation module, an image preprocessing module, a CNN neural network training module, a camera parameter prediction module, an image registration and display module, a user interaction module and an optimization and feedback module;

[0027] The data acquisition and processing module is used to acquire the patient's CT angiography data and perform denoising and contrast enhancement processing;

[0028] The simulated DSA generation module is used to generate multiple simulated DSA images by projecting the 3D blood vessel model by adjusting camera parameters, and record the camera parameter set corresponding to each image;

[0029] The image preprocessing module is used to preprocess the simulated DSA image and the intraoperative DSA image acquired in real time during the operation;

[0030] The CNN neural network training module is used to build and train a CNN neural network model based on deep learning;

[0031] The camera parameter prediction module is used to input the intraoperative DSA image into the trained CNN neural network model, predict the corresponding camera parameter set, and perform confidence assessment on the prediction result;

[0032] The image registration and display module is used to register the 3D vascular model constructed before surgery with the intraoperative DSA image according to the predicted camera parameter set;

[0033] The user interaction module is used to construct a user interaction interface, through which the doctor can view and adjust the position, posture and scaling of the 3D blood vessel model in real time during the operation;

[0034] The optimization and feedback module is used to evaluate the registration results and optimize the CNN neural network model, training data, and preprocessing methods based on the evaluation results.

[0035] To solve the above-mentioned technical problems, another technical solution adopted in the present application is: an electronic device, comprising a processor and a memory coupled to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the steps of a surgical navigation method based on DSA images as described above.

[0036] In order to solve the above technical problems, another technical solution adopted in the present application is: a storage medium storing program instructions capable of implementing a surgical navigation method based on DSA images as described above.

[0037] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution:

[0038] 1. The present invention constructs a 3D vascular model from preoperative CT angiography data and registers it with the DSA image in real time during the operation, which significantly reduces the overlap of vascular positions and avoids the need for multiple projections, thereby reducing the patient's radiation dosage, examination time, contrast agent usage and economic burden, and also reduces equipment loss;

[0039] 2. The present invention processes DSA images and predicts camera parameters through a CNN neural network model, effectively overcoming the accuracy and effectiveness problems existing in the registration of two-dimensional and three-dimensional images. By simulating the generation and training of DSA images, the CNN model can accurately predict the camera parameters corresponding to the intraoperative DSA images, thereby achieving high-precision image registration;

[0040] 3. The present invention solves the problem of lack of real-time registration means in existing vascular interventional surgery through the real-time vascular registration method of intraoperative DSA and preoperative CTA. Doctors do not need to rely on experience to judge the position of the guidewire, which improves the safety and success rate of the operation.

[0041] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic diagram of a surgical navigation method based on DSA images according to the present invention;

[0044] Figure 2 A schematic diagram of functional modules of a surgical navigation system based on DSA imaging according to the present invention;

[0045] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0047] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0048] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0049] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0050] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0051] Figure 1 FIG. 1 is a flowchart of a surgical navigation method based on DSA images according to an embodiment of the present invention. It should be noted that if there are substantially the same results, the method of the present application is not limited to the method of FIG. Figure 1 The process sequence shown is limited. Figure 1 As shown: A surgical navigation method based on DSA images, comprising the following steps:

[0052] Step 1: Obtain the patient's CT angiography data, extract the vascular structure from it, and construct a 3D vascular model. Specifically, first, use a high-end CT scanner to perform angiography scans on the patient to obtain high-quality CT angiography data, then use methods such as Gaussian filtering or median filtering to remove noise in the CT image, and use histogram equalization or adaptive contrast enhancement algorithm to improve the contrast between blood vessels and surrounding tissues, and then use image segmentation technology (such as threshold segmentation, region growing, level set method, etc.) to extract the vascular structure from the preprocessed CT image. Finally, use three-dimensional reconstruction software or algorithms (such as Marching Cubes, Surface Nets, etc.) to convert the extracted vascular structure into a 3D vascular model;

[0053] Step 2: Adjust the camera parameters, project the 3D vascular model to generate multiple simulated DSA images, and record the camera parameter set corresponding to each simulated DSA image. Specifically, first, preset a series of parameter combinations of camera position, posture, focal length, field of view and image resolution. Then, use the projection transformation matrix in computer graphics to project the 3D vascular model onto a two-dimensional plane according to each set of camera parameters to generate multiple simulated DSA images. Finally, record the corresponding camera parameter set for each simulated DSA image, including position parameters, posture parameters, focal length, field of view and image resolution.

[0054] Step 3: Obtain the patient's intraoperative DSA image, and pre-process the simulated DSA image and the intraoperative DSA image. Specifically, during the operation, use the DSA device to obtain the patient's intraoperative DSA image in real time, remove the noise in the intraoperative DSA image, and enhance the image contrast to make the vascular structure clearer. Then, normalize the image pixel values ​​to the same range to eliminate the brightness difference between different images, and adjust the image size to match the CNN neural network model input requirements.

[0055] Step 4: Construct a CNN neural network model and use the simulated DSA images and their corresponding camera parameter sets to train the CNN neural network model. Specifically, first, design a CNN neural network model based on deep learning, including multiple convolutional layers, pooling layers, activation function layers (such as ReLU) and fully connected layers. Then, use the simulated DSA images and their corresponding camera parameter sets as training data. Finally, use the cross entropy loss function as the optimization target and adjust the network weights through the back propagation algorithm so that the model can accurately predict the camera parameter set.

[0056] Step 5: Input the patient's intraoperative DSA image into the trained CNN neural network model to predict the corresponding camera parameter set. Specifically, first, input the intraoperative DSA image into the trained CNN neural network model, and the model outputs the predicted camera parameter set. Then, the prediction result is evaluated for confidence, and the probability distribution of the prediction result is output using a softmax layer or other mechanism. When the prediction confidence is lower than a preset threshold, a recalibration mechanism (such as reacquiring the intraoperative DSA image and re-predicting) or a manual intervention mechanism (such as notifying the doctor to make manual adjustments) is automatically triggered.

[0057] Step 6: According to the predicted camera parameter set, the 3D vascular model constructed before the operation is adjusted to be registered with the intraoperative DSA image, and the registered 3D vascular model is displayed in real time in the surgical navigation system. Specifically, first, according to the predicted camera parameter set, the position, posture and other parameters of the 3D vascular model constructed before the operation are adjusted, and the 3D vascular model is registered with the intraoperative DSA image using an iterative closest point algorithm (I CP) or a feature-based registration method (such as SI FT, SURF feature matching), and then the registered 3D vascular model is displayed in real time in the surgical navigation system to provide doctors with intuitive surgical navigation information;

[0058] Step 7: Evaluate the registration results. According to the evaluation results, optimize the CNN neural network model, training data, and preprocessing methods. Specifically, first, use quantitative indicators (such as registration error, similarity measurement, etc.) to evaluate the registration results. Then, according to the evaluation results, optimize the CNN neural network model, training data, preprocessing methods, etc., which may include adjusting the network structure, increasing the amount of training data, improving the preprocessing algorithm, etc.

[0059] Step 8. Build a user interaction interface. The doctor can view and adjust the position, posture and scaling of the 3D vascular model in real time during the operation through the user interaction interface. Specifically, the user interaction interface is designed by using a graphical user interface (GUI) library (such as Qt, WPF, etc.) to realize the functions of real-time display of the 3D vascular model, adjustment of the model position, posture and scaling, and provide intuitive operation controls (such as sliders, buttons, mouse dragging, etc.) for doctors to use.

[0060] In one embodiment, specifically, in step 1, before extracting the vascular structure, the CT angiography data is subjected to denoising and contrast adjustment processing, wherein the denoising processing uses a Gaussian filter to smooth the image, and the Gaussian filter reduces noise by calculating the weighted average of each pixel in the image and its neighboring pixels. This method can effectively remove random noise while retaining the image edge; the contrast enhancement processing method is specifically as follows: first, a global or local histogram analysis is performed on the CT image to understand the contrast distribution of the image, and then, a suitable contrast enhancement method is selected according to the analysis results. If the overall contrast of the image is low, global histogram equalization can be used; if the contrast of different regions in the image is greatly different, an adaptive contrast enhancement method is used, and finally, a contrast enhancement algorithm is applied to process the CT image to make the vascular structure clearer.

[0061] In one embodiment, specifically, in step 2, the camera parameters include position parameters, posture parameters, focal length, field of view angle, and image resolution, and the camera parameter set includes camera parameters for generating a DSA image, wherein:

[0062] Position parameters: describe the position of the camera in three-dimensional space, usually expressed by three coordinate values: X, Y, and Z. These parameters determine the spatial position of the camera relative to the 3D blood vessel model;

[0063] Posture parameters: describe the orientation and tilt of the camera, usually expressed by rotation matrix, Euler angle or quaternion. These parameters determine the angle and direction of the camera observing the 3D blood vessel model;

[0064] Focal length: The focal length of a camera determines the magnification and field of view of the image. The longer the focal length, the larger the objects in the image appear, but the smaller the field of view. The shorter the focal length, the smaller the objects in the image appear, but the larger the field of view.

[0065] Field of view: related to focal length, it is the maximum angle range that the camera can capture. The size of the field of view directly affects the visibility and proportional relationship of objects in the image;

[0066] Image resolution: refers to the number of pixels in an image, usually expressed as width and height (such as 1920x1080). Image resolution determines the clarity and detail of the image.

[0067] In one embodiment, specifically, in step three, the preprocessing of simulated DSA images and intraoperative DSA images includes denoising, contrast adjustment, normalization and resizing, by removing noise in the image to reduce the impact of noise on image feature extraction and model training; by improving the contrast between vascular structures and other tissues in the image, the blood vessels are made more clearly visible; by normalizing the pixel values ​​of the image to the same range (such as 0-1 or -1 to 1) to eliminate the differences between different images caused by factors such as brightness and contrast; by adjusting the image size to match the input requirements of the CNN neural network model to ensure that the model can correctly process the input image.

[0068] In one embodiment, specifically, in step 4, the CNN neural network model uses a deep learning architecture, including a convolution layer, a pooling layer, an activation function layer and a fully connected layer. The CNN neural network model is optimized using a cross-entropy loss function during the training process. The training process of the CNN neural network model is specifically as follows: the pre-processed simulated DSA image and its corresponding camera parameter set are used as training data. The data usually needs to be divided into a training set, a validation set and a test set so as to evaluate and select the model during the training process; the initial parameters of the CNN neural network model are set, including the weights and biases of the convolution kernel, the weights and biases of the fully connected layer, etc. These parameters are usually initialized randomly or pre-trained; the training data is input into the CNN model, and the prediction results (i.e., the predicted camera parameters) are obtained through the calculation of the convolution layer, the pooling layer, the activation function layer and the fully connected layer; the difference between the prediction result and the true value is calculated using the cross-entropy loss function. The cross-entropy loss function is a common method for measuring the difference between the predicted probability distribution and the true probability distribution in the classification problem; according to the gradient information of the loss function, the parameters of the CNN model are updated by the back propagation algorithm. The backpropagation algorithm uses the chain rule to calculate the gradient of the loss function with respect to the model parameters, and updates the parameter values ​​through optimization algorithms such as gradient descent; the process of forward propagation, loss calculation, and backpropagation is repeated until the preset training rounds (Epochs) are reached or other stopping conditions are met (such as the validation set loss no longer decreases).

[0069] In one embodiment, specifically, in step five, the process of predicting the camera parameter set also includes a confidence assessment of the prediction results. When the prediction confidence is lower than a preset threshold, a recalibration or manual intervention mechanism is automatically triggered. The confidence assessment can be implemented based on a variety of methods, such as internal uncertainty estimation of the model (such as Bayesian neural network), statistical characteristics of the prediction results (such as the distribution range and standard deviation of the predicted values), or comparison with other known information (such as the range of camera parameters planned before surgery); recalibration includes adjusting the physical position and posture of the camera or re-capturing images to obtain more accurate intraoperative DSA images and more reliable prediction results; when automatic recalibration is not possible or the recalibration effect is not good, a manual intervention mechanism can be triggered, and an experienced doctor or technician can analyze and judge based on the intraoperative DSA images and prediction results, and manually adjust the camera parameters or take other remedial measures.

[0070] In one embodiment, specifically, in step six, the registration of the 3D vascular model and the intraoperative DSA image utilizes an iterative closest point algorithm or a feature-based registration method, wherein the iterative closest point algorithm is a three-dimensional object alignment algorithm based on a pure geometric model, which is mainly used to solve the registration problem between three-dimensional point sets, that is, given two three-dimensional data point sets from different coordinate systems, the two point sets can be matched in space by iteratively calculating the optimal rotation and translation transformation; the feature-based registration method mainly utilizes significant features (such as corners, edges, textures, etc.) in the image or three-dimensional model for registration. These features have high distinguishability and stability in the image or model, and therefore can be used as the basis for registration.

[0071] In summary, the DSA image-based surgical navigation method provided by the present invention not only relies on advanced image processing technology and deep learning algorithms, but also focuses on the design and implementation of the user interaction interface. It can provide real-time and accurate vascular structure navigation during surgery, assisting doctors in performing more precise and safe surgical operations.

[0072] Figure 2 is a functional module diagram of a surgical navigation system based on DSA images according to an embodiment of the present application, such as Figure 2 As shown, a surgical navigation system based on DSA images includes: a data acquisition and processing module, a simulated DSA generation module, an image preprocessing module, a CNN neural network training module, a camera parameter prediction module, an image registration and display module, a user interaction module and an optimization and feedback module;

[0073] The data acquisition and processing module is used to obtain the patient's CT angiography data and perform denoising and contrast enhancement processing to improve the accuracy and efficiency of subsequent processing;

[0074] The simulated DSA generation module is used to generate multiple simulated DSA images by projecting the 3D vascular model by adjusting the camera parameters, and record the camera parameter set corresponding to each image for subsequent training and verification;

[0075] The image preprocessing module is used to preprocess the simulated DSA images and the intraoperative DSA images acquired in real time during the operation to improve the image quality and the accuracy of subsequent processing;

[0076] The CNN neural network training module is used to build and train a CNN neural network model based on deep learning. The CNN neural network model is used to predict the camera parameter set corresponding to the intraoperative DSA image.

[0077] The camera parameter prediction module is used to input the intraoperative DSA image into the trained CNN neural network model, predict the corresponding camera parameter set, and conduct confidence assessment on the prediction results. If the confidence is lower than the preset threshold, a recalibration or manual intervention mechanism is triggered;

[0078] The image registration and display module is used to register the 3D vascular model constructed before surgery with the intraoperative DSA image according to the predicted camera parameter set, and display the registration results in real time on the display device for the doctor to view;

[0079] The user interaction module is used to construct a user interaction interface, through which doctors can view and adjust the position, posture and scaling of the 3D vascular model in real time during surgery. The interaction between doctors and the system is realized through programming, allowing doctors to adjust the status of the 3D vascular model in real time during surgery and view the registration results;

[0080] The optimization and feedback module is used to evaluate the registration results and optimize the CNN neural network model, training data, and preprocessing methods based on the evaluation results to improve the overall performance and accuracy of the system.

[0081] In summary, the surgical navigation system based on DSA images provided by the present invention can effectively realize the surgical navigation function based on DSA images, provide doctors with real-time and accurate surgical guidance information, thereby improving the success rate and safety of the operation.

[0082] For other details about the technical solutions for implementing each module in the system of the above embodiment, please refer to the description of a surgical navigation method based on DSA images in the above embodiment, which will not be repeated here.

[0083] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0084] like Figure 3 The present invention provides a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which is suitable for implementing the electronic device in the embodiment of the present invention. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0085] like Figure 3As shown, the electronic device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and data required for the operation of the electronic device are also stored. The processor, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0086] Typically, the following devices can be connected to the I / O interface: input devices such as sensors or visual information acquisition devices; output devices such as display screens; storage devices such as magnetic tapes and hard disks; and communication devices. The communication device can allow the electronic device to communicate with other devices (such as edge computing devices) wirelessly or by wire to exchange data. Figure 3 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0087] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processor, all or part of the steps of a surgical navigation method based on DSA images of an embodiment of the present disclosure are executed.

[0088] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0089] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of a surgical navigation method based on DSA images in the above-mentioned embodiments of the present disclosure are executed.

[0090] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0091] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0092] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0093] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0094] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0095] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0096] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0097] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0098] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A surgical navigation method based on DSA images, characterized in that: The following steps are involved: Step 1: Obtain the patient's CT angiography data, extract the vascular structure from it, and construct a 3D vascular model; Step 2: Adjust the camera parameters, project the 3D blood vessel model to generate multiple simulated DSA images, and record the camera parameter set corresponding to each simulated DSA image; Step 3, obtaining the patient's intraoperative DSA image, and preprocessing the simulated DSA image and the intraoperative DSA image; Step 4: Build a CNN neural network model and train the CNN neural network model using simulated DSA images and their corresponding camera parameter sets; Step 5: Input the patient's intraoperative DSA image into the trained CNN neural network model to predict the corresponding camera parameter set; Step 6: According to the predicted camera parameter set, the 3D vascular model constructed before the operation is adjusted to be registered with the intraoperative DSA image, and the registered 3D vascular model is displayed in real time in the surgical navigation system; Step 7: Evaluate the registration results and optimize the CNN neural network model, training data, and preprocessing methods based on the evaluation results; Step 8: Build a user interaction interface, through which the doctor can view and adjust the position, posture and scaling of the 3D vascular model in real time during the operation.

2. The surgical navigation method based on DSA images according to claim 1, characterized in that: In step 1, before extracting the vascular structure, the CT angiography data is subjected to denoising and contrast adjustment processing.

3. The surgical navigation method based on DSA images according to claim 1, characterized in that: In step 2, the camera parameters include position parameters, posture parameters, focal length, field of view angle and image resolution, and the camera parameter set includes camera parameters for generating DSA images.

4. The surgical navigation method based on DSA images according to claim 1, characterized in that: In step three, the simulated DSA image and intraoperative DSA image preprocessing includes denoising, contrast adjustment, normalization and size adjustment.

5. The surgical navigation method based on DSA images according to claim 1, characterized in that: In step 4, the CNN neural network model uses a deep learning architecture, including a convolutional layer, a pooling layer, an activation function layer and a fully connected layer. The CNN neural network model is optimized using a cross entropy loss function during training.

6. The surgical navigation method based on DSA images according to claim 1, characterized in that: In step five, the process of predicting the camera parameter set also includes a confidence assessment of the prediction result. When the prediction confidence is lower than a preset threshold, a recalibration or manual intervention mechanism is automatically triggered.

7. The surgical navigation method based on DSA images according to claim 1, characterized in that: In step six, the 3D vascular model is registered with the intraoperative DSA image using an iterative closest point algorithm or a feature-based registration method.

8. A surgical navigation system based on DSA images, characterized in that: include: Data acquisition and processing module, simulated DSA generation module, image preprocessing module, CNN neural network training module, camera parameter prediction module, image registration and display module, user interaction module and optimization and feedback module; The data acquisition and processing module is used to acquire the patient's CT angiography data and perform denoising and contrast enhancement processing; The simulated DSA generation module is used to generate multiple simulated DSA images by projecting the 3D blood vessel model by adjusting camera parameters, and record the camera parameter set corresponding to each image; The image preprocessing module is used to preprocess the simulated DSA image and the intraoperative DSA image acquired in real time during the operation; The CNN neural network training module is used to build and train a CNN neural network model based on deep learning; The camera parameter prediction module is used to input the intraoperative DSA image into the trained CNN neural network model, predict the corresponding camera parameter set, and perform confidence assessment on the prediction result; The image registration and display module is used to register the 3D vascular model constructed before surgery with the intraoperative DSA image according to the predicted camera parameter set; The user interaction module is used to construct a user interaction interface, through which the doctor can view and adjust the position, posture and scaling of the 3D blood vessel model in real time during the operation; The optimization and feedback module is used to evaluate the registration results and optimize the CNN neural network model, training data, and preprocessing methods based on the evaluation results.

9. An electronic device, characterized in that: The electronic device comprises: a processor, and a memory connected to the processor in communication; wherein, The memory stores instructions executable by the processor, and the instructions are executed by the processor so that the processor can execute a DSA image-based surgical navigation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute a DSA image-based surgical navigation method as described in any one of claims 1-7.

Images