Intelligent neural navigation system for neurosurgery department and use method of intelligent neural navigation system

Abstract

The invention relates to an intelligent neural navigation system for neurosurgery and a use method thereof.The intelligent neural navigation system comprises an image collecting and processing unit, an intelligent planning unit, a positioning and tracking unit, a visual interaction unit and a data management and updating unit. The optimal operation scheme is selected through operation simulation operation, so that a doctor is helped to master the condition of a patient more clearly, personalized adjustment can be carried out in real time, the operation scheme better fits the actual condition of the patient and the operation habit of the doctor, and the success rate and the quality of an operation are improved. Meanwhile, an optical positioning sensor and an electromagnetic positioning sensor are integrated, accurate position information of the surgical instrument and the area, needing to be treated, of the patient is obtained in real time, operation of the surgical instrument is accurately guided, then it is ensured that the surgical instrument is accurately matched with the preoperative planned path, and mistaken damage to surrounding normal tissue in the surgical process is reduced; and the accuracy and the safety of the operation are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to an intelligent neuronavigation system for neurosurgery and a method of using the system. Background Art

[0002] In the field of neurosurgery, the complexity and high risk of surgery require doctors to accurately locate the lesion, plan the best surgical path, and monitor the progress of the surgery in real time during the operation to ensure the safety and effectiveness of the operation. With the rapid development of medical imaging technology, computer technology and artificial intelligence, intelligent neuronavigation systems have gradually become an indispensable auxiliary tool in neurosurgery. However, the existing neuronavigation systems often have relatively single functions, which may only focus on one aspect of the function such as image acquisition or surgical positioning, and lack comprehensive support for the entire surgical process, resulting in doctors needing to use multiple different devices and software during the operation, which is cumbersome and inefficient, affecting the efficiency and accuracy of treatment; in addition, the accuracy of image processing, segmentation and fusion of the existing system needs to be improved. The segmentation algorithm cannot accurately distinguish between neural tissue, lesion tissue and other important anatomical structures, which is easy to cause errors in subsequent analysis and affect the subsequent operation and treatment; during the operation, it is also impossible to locate and navigate the surgical instruments in real time, resulting in inaccurate relative positioning of the surgical instruments and the area to be treated; the tracking algorithm may not be able to calculate the position changes of the surgical instruments and the patient's head in a timely and accurate manner, affecting the accuracy of the operation. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] In view of the deficiencies in the prior art, the present invention aims to provide an intelligent neuronavigation system for neurosurgery and a method of using the same. After comprehensively collecting multiple modality images (MRI, CT, PET, etc.), a three-dimensional model is established, and multiple feasible surgical path plans are automatically generated. Through surgical simulation operations, the best surgical plan is selected to treat the patient, helping doctors to more clearly understand the location and morphology of the lesion and its relationship with the surrounding tissues. At the same time, personalized adjustments can be made based on the doctor's experience and preferences, so that the surgical plan is more in line with the patient's actual situation and the doctor's operating habits, thereby improving the success rate and quality of the surgery. At the same time, the optical positioning sensor and the electromagnetic positioning sensor are integrated to obtain the accurate position information of the surgical instrument and the area to be treated by the patient in real time. According to the positioning data and the preoperative planning path, the operation of the surgical instrument can be accurately guided, and the error can be corrected in real time, further improving the positioning and tracking accuracy, ensuring that the surgical instrument is accurately matched with the preoperative planning path, reducing accidental damage to the surrounding normal tissues during the operation, and improving the accuracy and safety of the operation.

[0005] (II) Technical solution

[0006] To achieve the above object, the present invention provides the following technical solution: an intelligent neuronavigation system for neurosurgery, comprising:

[0007] Image acquisition and processing unit: used to collect the patient's neurological imaging data of different modalities, and perform preprocessing, segmentation, registration and other operations on the image data to obtain clear and accurate neurological imaging information that can be used for subsequent analysis;

[0008] Intelligent planning unit: fuses the image information provided by the image acquisition and processing unit, establishes a three-dimensional model, and performs surgical path planning, risk assessment, and surgical plan formulation to provide the best treatment plan;

[0009] Positioning and tracking unit: Tracks the position of surgical instruments and the position changes of the patient's treatment area in real time, and positions the surgical instruments to accurately match the path and position planned before surgery;

[0010] Visualization interaction unit: visualizes the image information of the image acquisition and processing unit, the planning results of the intelligent planning unit, and the real-time data of the positioning and tracking unit, and presents them to the surgeon in an intuitive manner, while providing a human-computer interaction interface to facilitate the surgeon's operation;

[0011] Data management and update unit: manage, store and back up all types of data throughout the entire surgical process. At the same time, update the system's algorithms and models based on new clinical cases and research results, set up data encryption, and only enable access and operation permissions to authorized users.

[0012] Preferably, the image acquisition and processing unit includes:

[0013] Image acquisition interface: It has acquisition interfaces for a variety of image acquisition devices, including magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET) and other imaging devices of different modalities, to achieve comprehensive acquisition of patients' neurological images; having acquisition interfaces for imaging devices of different modalities means that the advantages of various devices can be fully utilized to obtain patients' neurological images, so as to fully understand the patient's neurological condition from multiple angles through these imaging devices of different modalities.

[0014] Preprocessing module: Perform denoising, enhancement, grayscale correction and other preprocessing operations on the collected raw image data; the raw image data may contain interference caused by factors such as equipment and environment, which may cause certain errors in the image. Direct use will affect the accuracy of treatment. Therefore, preprocessing operations are performed on the collected raw image data to improve the image quality and availability, making diagnosis and treatment more reliable and effective.

[0015] Segmentation module: Use image segmentation algorithms to segment nerve tissue, diseased tissue and other important anatomical structures from images;

[0016] Registration module: registers images collected at different times, in different modalities or at different viewing angles;

[0017] Data transmission interface: The data under different modes are transmitted to the intelligent planning unit through their corresponding transmission interfaces for subsequent processing. The data of different modes are transmitted through the corresponding transmission interfaces, and the images of different modes are separated to a certain extent, which can avoid the confusion of data from different modes and improve the stability and reliability of the system.

[0018] Furthermore, the segmentation module includes:

[0019] Tissue segmentation: distinguishing different tissue types in medical images;

[0020] Lesion detection and segmentation: Detect and segment lesion areas in medical images;

[0021] Structural annotation: Labeling and annotating important anatomical structures in medical images.

[0022] Preferably, the intelligent planning unit includes:

[0023] Data fusion module: Comprehensively process the registered image data of different modalities, and the fusion is performed at the pixel level, feature level or decision level;

[0024] 3D reconstruction module: Uses the fused image data to construct a 3D model of the patient's neural structure through a variety of 3D reconstruction algorithms;

[0025] Disease analysis module: conducts comprehensive analysis of the patient's imaging information and clinical medical records to determine the key information of the lesion;

[0026] Surgical Pathway Planning Module: Automatically generates multiple feasible surgical path plans, and performs evaluation and analysis;

[0027] Risk assessment module: predicts and assesses the risks that may be encountered during surgery and provides suggestions for countermeasures;

[0028] Surgical plan formulation module: Comprehensively consider the condition analysis, surgical path planning and risk assessment results to formulate a detailed surgical plan;

[0029] Surgical simulation operation module: Based on the generated surgical path and surgical plan, simulated drills are performed in a virtual environment.

[0030] Furthermore, the data fusion module includes:

[0031] Pixel-level fusion: Wavelet transform is used to directly combine the pixel values ​​of different modal images to generate a fused image. For medical images of different modalities, wavelet transform is first performed on them separately. Then, their sub-bands are combined according to certain rules in the wavelet domain.

[0032] Feature-level fusion: Extract features from images of different modalities and combine and select them. The extracted feature vectors need to be connected in series to form a new comprehensive feature vector. Then, principal component analysis (PCA) is used to select the most representative features and remove noise and redundant information.

[0033] Decision-level fusion: Fusion of decision results from different modalities of imaging, including judgments on the presence of lesions, the type of lesions, and the grading of lesions.

[0034] Furthermore, the 3D reconstruction module integrates multiple 3D reconstruction algorithms, including:

[0035] Voxel-based reconstruction algorithm: Dual Contouring algorithm is used to generate high-quality three-dimensional surfaces by interpolating the boundaries of voxels.

[0036] Surface-based reconstruction algorithm: The Poisson reconstruction algorithm is used, based on the indicator function and the Poisson equation. The gradient field of the indicator function is constrained and the Poisson equation is used to solve the function representation of the reconstructed surface.

[0037] Reconstruction algorithm based on deep learning: A reconstruction algorithm based on a convolutional neural network is used to reconstruct by learning the relationship between a large number of two-dimensional image slices and the corresponding target three-dimensional models. It contains multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer is used to extract features from the image, the pooling layer is used to reduce the dimension of the data, and the fully connected layer is used to integrate the extracted features.

[0038] Preferably, the positioning tracking unit includes:

[0039] Positioning sensor: including optical positioning sensor and electromagnetic positioning sensor, which can obtain the position information of surgical instruments and patient's head in real time;

[0040] Optical camera: equipped with multiple cameras, focusing the image of the surgical area onto the optical positioning sensor;

[0041] Electromagnetic generator: generates an electromagnetic field of a specific frequency. The electromagnetic positioning sensor determines the position of the surgical instrument and the patient's treatment area by sensing the electromagnetic field.

[0042] Automatic navigation module: Automatically navigate surgical instruments to perform surgery based on the data collected by the positioning sensor and the path planned before surgery;

[0043] Error compensation module: Use error compensation algorithm to compensate for errors that may occur during positioning and tracking in real time.

[0044] Preferably, the visual interaction unit includes:

[0045] Visualization module: Use virtual reality (VR) technology to present relevant information in three-dimensional form;

[0046] Interactive interface module: uses a human-computer interactive interface to adjust imaging and surgical planning parameters and control surgical instruments;

[0047] Information display module: used to display important surgery-related information in real time on a visual interface.

[0048] Preferably, the data management and updating unit includes:

[0049] Data management module: classifies, stores and manages various types of data, establishes data index and query mechanisms, and supports data backup and recovery functions;

[0050] Data update module: regularly connect to the medical database or research institution's server to obtain the latest information and update and optimize the system;

[0051] User authority management module: Different levels of user authority are set, and only authorized users are given the authority to access and operate corresponding data and functions.

[0052] A method for using an intelligent neuronavigation system for neurosurgery, comprising the following steps:

[0053] S1. The intelligent planning unit collects the patient's neurological imaging data of different modalities through different acquisition interfaces. The collected neurological imaging data is processed preliminarily by the preprocessing module, the separation module and the registration module in sequence. The image data after registration is transmitted to the intelligent planning unit through the corresponding transmission interfaces.

[0054] S2. The fusion module in the intelligent planning unit performs comprehensive fusion processing on the registered image data of different modalities, and the 3D reconstruction module uses a variety of 3D reconstruction algorithms to construct a 3D model of the patient's neural structure using the fused data;

[0055] S3. The condition analysis module determines the patient's condition based on the image data, 3D model and the patient's own condition. The surgical path module generates multiple feasible surgical paths based on the condition analysis results and surgical goals and principles.

[0056] S4. The risk assessment module predicts and assesses possible risks during surgery, issues early warnings, and provides countermeasures. The surgical plan formulation module formulates a detailed surgical plan based on the condition analysis, surgical path, and risk assessment results.

[0057] S5. The surgical simulation module performs simulation exercises on the model based on the generated surgical path and surgical plan. The doctor selects and verifies the best surgical plan based on the simulation results and his own needs;

[0058] S6. After the best surgical plan is formulated, the medical staff performs surgery on the patient according to the best surgical plan, and the positioning tracking unit tracks the position changes of the surgical instrument and the patient's treatment area in real time, and transmits the data to the visualization interaction unit and the intelligent planning unit;

[0059] S7. The visual interaction unit integrates the real-time data and preoperative planning information in real time, and provides feedback on the surgical process and surgical data for medical staff to understand the progress of the surgery and provide guidance. The intelligent planning unit adjusts and optimizes the surgical plan in real time based on the real-time position and the progress of the surgery, and feeds back the results to the visual interaction unit so that medical staff can grasp and respond in real time;

[0060] S8. After the operation is completed, the image acquisition and processing unit collects the patient's neurological imaging data again and compares and analyzes it with the neurological imaging data collected before the operation to evaluate the surgical effect and the patient's recovery. These imaging data and surgical process data are integrated and stored in the data management module.

[0061] By collecting neurological imaging data of different modalities (such as MRI, CT, PET, etc.) and fusing them, the advantages of various images can be combined to present the patient's neural structure and lesion information more comprehensively and accurately, providing rich data support for subsequent disease analysis and surgical planning. Using a variety of 3D reconstruction algorithms to build a 3D model of the patient's neural structure, doctors can observe the spatial relationship between the lesion and the surrounding tissue from a three-dimensional perspective, which helps doctors to more accurately grasp the complex anatomical structure of the surgical site. The disease analysis module combines imaging data, 3D models and the patient's own conditions to determine the condition. The surgical path module generates a variety of feasible surgical paths. The surgical plan formulation module further considers the risk assessment results to formulate a detailed surgical plan and conducts simulation exercises, providing doctors with multiple options, allowing them to choose the most suitable surgical plan based on the patient's specific conditions and their own experience. At the same time, they are familiar with the surgical process, observe possible problems, and improve the accuracy and flexibility of surgical planning.

[0062] During the operation, the positioning tracking unit tracks the position changes of surgical instruments and the parts of the patient to be treated in real time, and transmits the data to other units. This ensures that medical staff can understand the position of surgical instruments in real time to avoid the instruments deviating from the planned path and damaging important tissues. The visual interaction unit integrates real-time data and preoperative planning information for display, so that medical staff can intuitively see the progress of the operation and adjust the operation in time according to the feedback information. At the same time, the intelligent planning unit adjusts and optimizes the surgical plan in real time according to the real-time position and the progress of the operation. This dynamic adjustment mechanism can better cope with various changes that occur during the operation, such as patient position movement, differences between the lesion tissue and the preoperative image, etc., further ensuring the smooth progress of the operation.

[0063] After the operation, the image acquisition and processing unit collects the patient's neurological imaging data again and compares and analyzes it with the preoperative data, which can objectively evaluate the surgical effect and the patient's recovery. All imaging data and surgical process data are integrated and stored in the data management module, which can be used for subsequent clinical research, medical quality assessment and technical improvement. By analyzing a large amount of case data, we can summarize experience and lessons, optimize surgical procedures and technologies, and promote the continuous improvement of neurosurgery medical level.

[0064] (III) Beneficial effects

[0065] (1) Through the acquisition interfaces of various imaging devices, comprehensive acquisition of images of different modalities is achieved, providing doctors with multi-angle disease information. The images are then fully processed and analyzed, allowing doctors to more accurately judge the severity of the disease, the stage of development, and the relationship between the lesion and the surrounding tissue, thereby improving the accuracy of early diagnosis and buying time for timely treatment. At the same time, the data fusion module of the intelligent planning unit combines the advantages of different images, the three-dimensional reconstruction module intuitively displays the relationship between the neuroanatomical structure and the spatial position of the lesion, and the disease analysis module integrates the image information and clinical medical records to determine the key disease information, so that doctors can have a more comprehensive and in-depth understanding of the patient's condition.

[0066] (2) The surgical path planning module automatically generates multiple feasible surgical path plans and evaluates and analyzes them. The risk assessment module predicts surgical risks and provides countermeasures. At the same time, the surgical simulation operation module conducts simulation exercises in a virtual environment to provide doctors with decision-making references and improve the safety, effectiveness and feasibility of the surgery. In addition, the positioning tracking unit tracks the position changes of surgical instruments and the patient's head in real time to ensure that the surgical instruments are accurately matched with the preoperative planned path and position. The automatic navigation module accurately navigates the surgical instruments in real time and accurately adjusts the errors to reduce damage to surrounding normal tissues and improve surgical accuracy.

[0067] (3) The visualization module of the visualization interaction unit uses virtual reality technology to present neurological imaging information, surgical planning results and real-time tracking data in three-dimensional form, so that doctors can more intuitively understand the anatomical structure of the surgical site and the progress of the operation. The interactive interface module realizes image operation and surgical planning parameter adjustment, surgical instrument control, and the information display module displays important information related to the operation in real time, which makes it convenient for doctors to grasp the operation situation at any time and improve the safety and reliability of the operation.

[0068] (4) After the operation, the imaging data is collected again and compared with the preoperative data to evaluate the surgical effect and the patient's recovery. The data management and update unit integrates and stores the postoperative imaging data and surgical process data for analysis and summary. The system is updated and improved based on the postoperative data analysis results and new clinical research results to improve system performance and accuracy and provide better support for subsequent treatment. (IV) Description of the drawings

[0069] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0070] Figure 1 This is a schematic diagram of the equipment and module composition of an intelligent neuronavigation system for neurosurgery and a method of using the same in an embodiment of the present application;

[0071] Figure 2 This is a schematic diagram of the composition of a segmentation module in an intelligent neuronavigation system for neurosurgery and a method for using the same in an embodiment of the present application;

[0072] Figure 3 This is a schematic diagram of the composition of a data fusion module in an intelligent neuronavigation system for neurosurgery and a method for using the system according to an embodiment of the present application;

[0073] Figure 4 This is a schematic diagram of the composition of a three-dimensional reconstruction module in an intelligent neuronavigation system for neurosurgery and a method of using the system according to an embodiment of the present application. (V) Specific implementation methods

[0074] The following will be combined with the attached examples of the present invention Figure 1 -Attached Figure 4 A clear and complete description is given to the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0075] Embodiment 1: An intelligent neuronavigation system for neurosurgery, comprising:

[0076] Image acquisition and processing unit: used to collect neurological imaging data of different modalities of patients, and perform preprocessing, segmentation, registration and other operations on the image data to obtain clear, accurate neurological imaging information that can be used for subsequent analysis; the effects of different modalities collected initially have errors and other unclear situations, so they need to be preprocessed, including but not limited to image denoising, contrast enhancement and image distortion improvement. Denoising can remove noise interference caused by equipment or environmental factors and improve image clarity; contrast enhancement helps to highlight the differences between neural tissue and surrounding structures, making it easier for doctors to observe; correcting image distortion ensures the accuracy of the image and avoids misleading due to equipment errors. Performing image segmentation at the same time not only helps doctors understand the location and range of lesions more clearly, but also provides accurate basic data for subsequent three-dimensional model establishment and surgical planning. At the same time, the registration operation also provides a basis for multimodal image fusion, so that the advantages of different images can be fully utilized.

[0077] Intelligent planning unit: fuses the image information provided by the image acquisition and processing unit, establishes a three-dimensional model, and performs surgical path planning, risk assessment, and surgical plan formulation to provide the best treatment plan; image fusion processing is to fuse image information from different modalities, and comprehensively utilize the advantages of structural images (such as CT, MRI) and functional images (such as PET) to provide more comprehensive information for surgical path planning. The establishment of a three-dimensional model is to generate a three-dimensional model of the patient's neural tissue by three-dimensionally reconstructing the processed image data. Doctors can observe the relationship between the lesion and the surrounding important structures from different angles, and better understand the complexity and risks of the operation. The three-dimensional model can also be rotated, scaled, and other operations to facilitate preoperative discussions and teaching by doctors.

[0078] Positioning and tracking unit: real-time tracking of the position of surgical instruments and the position changes of the patient's treatment area, positioning surgical instruments so that they accurately match the path and position planned before surgery; the positioning and tracking unit accurately matches the real-time acquired position information with the path and position planned before surgery, which is particularly important during surgery, providing doctors with real-time navigation information. If the surgical instrument deviates from the planned path, the system will promptly issue an alarm to remind the doctor to make adjustments. During surgery, the patient's position may change, or the surgical instrument may be disturbed by external forces. The error correction function of the positioning and tracking unit ensures that the surgical instrument is always consistent with the preoperative plan through continuous monitoring and adjustment.

[0079] Visualization interaction unit: The image information of the image acquisition and processing unit, the planning results of the intelligent planning unit, and the real-time data of the positioning and tracking unit are visualized and presented to the surgeon in an intuitive manner, and a human-computer interaction interface is provided to facilitate the doctor's operation; visualization processing means visualizing the image information of the image acquisition and processing unit, the planning results of the intelligent planning unit, and the real-time data of the positioning and tracking unit, and using advanced graphics rendering technology to generate realistic three-dimensional images. Doctors can clearly see the structure of the patient's nerve tissue, the location of the lesion, the location of the surgical instrument and other information through a high-resolution display screen, thereby improving the accuracy and safety of the operation. At the same time, an intuitive and convenient human-computer interaction interface is provided, and doctors can operate through devices such as a touch screen, a mouse, and a keyboard. Interface functions include but are not limited to image zooming, rotation, switching between different perspectives, marking important structures, etc. At the same time, it can display information such as the progress of the operation and risk warnings, which is convenient for doctors to monitor and make decisions in real time.

[0080] Data management and update unit: manages, stores and backs up all kinds of data during the entire surgical process. At the same time, the system's algorithms and models are updated according to new clinical cases and research results, and data encryption is set to open access and operation permissions only to authorized users. The data management and update unit uses an advanced data management system to ensure the security, integrity and traceability of the data. Doctors can query and call historical data at any time to provide reference for subsequent treatment and research.

[0081] It should be noted that due to the privacy of medical and patient treatment information, an encryption algorithm needs to be set up so that only authorized users can review and retrieve the data. In other words, access and operation permissions are only open to authorized users to prevent unauthorized personnel from accessing and tampering with the data. At the same time, user operations are recorded and audited for traceability and management.

[0082] The image acquisition and processing unit includes:

[0083] Image acquisition interface: It has acquisition interfaces for a variety of image acquisition devices, including magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET) and other imaging devices of different modalities, to achieve comprehensive acquisition of patients' neurological images; MRI can provide high-resolution soft tissue images and clearly display the structure of neural tissue; CT has advantages in displaying bone structure; PET can reflect the metabolic activity of tissues, which is very important for the activity assessment of lesions such as tumors. Through these imaging devices of different modalities, the patient's neurological condition can be fully understood from multiple angles. Different imaging modalities can reveal different pathological characteristics and physiological functions. Specifically, for brain lesions, MRI can show the location and morphology of the lesions, CT can help determine whether there is bleeding or calcification, and PET can evaluate the metabolic activity of the lesions. Comprehensive acquisition of these imaging data helps to diagnose and plan surgery more accurately.

[0084] Preprocessing module: Perform preprocessing operations such as denoising, enhancement, and grayscale correction on the collected raw image data; denoising is due to the presence of noise in the raw image data due to factors such as equipment and environment, and noise will affect the doctor's observation and judgment of the lesion. Denoising can improve the readability of the image. Enhancement processing is to improve the contrast and brightness of the image through enhancement operations, making the nerve tissue, diseased tissue, etc. more prominent. The enhanced image makes it easier for doctors to distinguish different structures and helps to find tiny lesions. Grayscale correction processing is to ensure that the grayscale value of the image is within a reasonable range to avoid affecting the doctor's judgment due to uneven grayscale. The grayscale of the corrected image is more uniform, which helps to improve the accuracy of segmentation and registration.

[0085] Segmentation module: Use image segmentation algorithms to segment nerve tissue, diseased tissue, and other important anatomical structures from images. In neurosurgery, it is often necessary to avoid damaging important nerve structures. Accurately segmenting nerve tissue from images can help doctors understand the distribution and morphology of nerves. Accurate segmentation of diseased tissues, such as tumors and hemorrhages, can determine the scope and location of the lesions, which helps doctors develop surgical plans and determine the scope of resection. In addition to nerve tissue and diseased tissue, other important anatomical structures, such as blood vessels and skulls, can also be segmented to help with diagnosis and treatment.

[0086] Registration module: Register images collected at different times, in different modalities or from different perspectives; for patients who need to track changes in their condition, images collected at different times can be registered. This allows the development of lesions to be compared and the treatment effect to be evaluated. By registering images of different modalities, the advantages of various images can be comprehensively utilized. In other words, by registering MRI and PET images, the structure of nerve tissue and the metabolic activity of lesions can be seen at the same time, providing more comprehensive information for diagnosis and treatment. By collecting images from different perspectives, such as multi-angle CT scans, these images from different perspectives can be integrated through the registration module to generate a more complete three-dimensional model.

[0087] Data transmission interface: Data in different modes are transmitted to the intelligent planning unit through their corresponding transmission interfaces for subsequent processing. Data in different modes are transmitted through their corresponding transmission interfaces, which helps to ensure the accuracy and integrity of the data.

[0088] The segmentation module includes:

[0089] Tissue segmentation: threshold segmentation, region growing method and edge detection method are used in sequence to distinguish different tissue types in medical images; in medical images, different tissue types have different characteristics, including but not limited to density, texture, signal strength, etc. This is of great significance for doctors to accurately understand the patient's body structure, assess the condition and formulate treatment plans. The specific tissue segmentation method is to first use threshold segmentation to preliminarily distinguish the skull and brain tissue, and then use the region growing method to further subdivide different neural tissue areas within the brain tissue, and combine the edge detection method to accurately locate the boundary between the lesion and normal tissue, thereby providing the most accurate and effective tissue segmentation results for neurosurgery.

[0090] The threshold segmentation method determines a grayscale threshold that can better distinguish the skull and brain tissue through statistical analysis of a large amount of clinical imaging data. For areas above the threshold, they are judged as skulls; for areas below the threshold, they are preliminarily identified as brain tissue. In practical applications, it is necessary to perform morphological operations (erosion, expansion, etc.) on the areas that have been preliminarily distinguished to optimize the distinction results. That is, when there are some isolated high grayscale value points (probably caused by noise) in the area preliminarily determined to be brain tissue, these points can be removed through corrosion operations to make the division of brain tissue areas more accurate.

[0091] The region growing method starts with one or more seed points in the image, and gradually merges adjacent pixels or voxels into the seed region according to specific similarity criteria until a certain stopping condition is met. For different neural tissue regions within the brain tissue, such as gray matter, white matter, cerebrospinal fluid, etc., representative seed points can be selected. Specifically, a seed point is selected in the white matter region, and then the surrounding pixels similar to it are gradually merged into the white matter region based on the gray value, texture and other characteristics of the white matter. Similarity criteria include but are not limited to the degree of proximity of gray values, similarity of texture, etc. Through continuous growth, different neural tissue regions can eventually be subdivided.

[0092] At the boundary between lesions and normal tissues, there are usually obvious changes in grayscale or signal intensity. The edge detection method is to determine the contour of the tissue by detecting the grayscale change, gradient change and other features of the tissue boundary in the image. Specifically, the edge images obtained by the Sobel operator and the Canny operator are weighted fused, that is, the gradient amplitude or other edge features of the edge images obtained by the Sobel operator and the Canny operator are calculated respectively. Then they are added according to a certain weight to obtain the final fused edge image.

[0093] Specifically, first calculate the gradient magnitude or other edge features of the edge image:

[0094] For each pixel in the image, the Sobel operator performs a convolution operation with the image through two convolution kernels (one for calculating the horizontal gradient and the other for calculating the vertical gradient). The horizontal gradient convolution kernel is usually:

[0095]

[0096] The vertical gradient convolution kernel is:

[0097]

[0098] Calculate the horizontal gradient G x (x, y) and the vertical gradient G y (x, y), the gradient amplitude G of the pixel point (x, y) Sobel The formula for (x, y) is:

[0099]

[0100] Next is the edge feature extraction of the Canny operator. First, a two-dimensional Gaussian function is used to smooth the image and remove noise, and then the gradient magnitude and direction are calculated. Similar to the Sobel operator, it also obtains the gradient magnitude G of each pixel. canny (x, y), and at the same time, the gradient direction will be obtained for subsequent non-maximum suppression.

[0101] The Gaussian function has the form:

[0102]

[0103] Where σ is the standard deviation of the Gaussian function, which determines the degree of filtering.

[0104] In the non-maximum suppression process, only the pixel points whose gradient amplitude is the local maximum in its gradient direction will be retained as candidate edge points, and the gradient amplitudes of other points are set to 0. Finally, the final edge image is obtained through double threshold detection and edge connection, and the gradient amplitude of its edge points also represents the edge strength information.

[0105] After the gradient amplitude or other edge features are calculated, the weights must be determined and added to obtain a fused edge image. The weights are determined through empirical and experimental methods. When the weight combination that makes the evaluation index optimal is found, this weight can be determined for subsequent edge detection, that is, addition fusion.

[0106] Specifically, assume that the weight of the Sobel operator has been determined to be w Sobel , the weight of the Canny operator is w Canny , and w Sobel +w Canny =1. For each pixel (x, y) in the image, the calculation formula for the fused edge strength is: G fusion (x,y)=w Sobel ×G Sobel (x,y)+w Canny ×G Canny (x,y)

[0107] According to the strength of the fused edge, a threshold is determined through experiments or experience. When the strength is greater than the threshold, the pixel is considered to belong to the fused edge image.

[0108] Lesion detection and segmentation: Detect and segment the lesion area in medical images; Lesion detection and segmentation can accurately detect and segment the lesion area, provide key information such as the location, size, shape, etc. of the lesion, and provide assistance for early diagnosis of the disease, condition assessment and formulation of treatment plans. The specific segmentation method based on region growing can provide doctors with accurate lesion detection and segmentation results, helping them to better plan surgical plans and perform surgical operations.

[0109] Structural annotation: Mark and annotate important anatomical structures in medical images. The annotation content includes but is not limited to organ structure annotation, blood vessel annotation and nerve annotation. Specifically, it is necessary to annotate major organs such as the liver, gallbladder, pancreas, spleen, and kidney. The annotation of organs should include the boundary, morphology and location information of the organs. For each organ, its internal substructures can be further annotated, such as the liver lobe and liver segment of the liver, the renal pelvis and renal calyx of the kidney, etc. In terms of blood vessel annotation, including the annotation of arteries and veins, it is necessary to clarify the starting point, running path, branching, and diameter of the blood vessels. In nerve annotation, for peripheral nerves, their running routes and their relationship with surrounding tissues are annotated; specifically, in cerebral angiography images, the branches and distribution of blood vessels such as the middle cerebral artery and the anterior cerebral artery are accurately annotated. In addition to organs, blood vessels and nerves, there are some special anatomical structures that need to be annotated. That is to say, in skeletal system images, structures such as articular surfaces and bone marrow cavities are annotated; in eye ultrasound images, structures such as lenses and retina are annotated. These structures are also of great significance for the diagnosis and treatment of related diseases. Structural annotation is a key step in surgical planning. Based on the annotated anatomical structure information, the system accurately annotates important anatomical structures, plans the location of surgical incisions and surgical paths, avoids damage to important blood vessels, nerves and other structures, and helps doctors detect lesions more quickly and accurately. In addition, structurally annotated medical images provide a wealth of teaching resources for medical education. That is, by observing the annotated images, we can better understand the morphology, location and performance of human anatomical structures in images.

[0110] The intelligent planning unit includes:

[0111] Data fusion module: The image data of different modalities after registration are processed comprehensively, and the fusion is performed at the pixel level, feature level and decision level respectively; medical images of different modalities contain unique detailed information about the human neural structure. Pixel-level fusion directly integrates these details together, so there is no need to frequently switch and compare between images of different modalities. The comprehensive situation of neural structure and lesions can be directly observed, which helps to improve diagnostic efficiency and make diagnostic decisions quickly even in emergency situations. At the same time, after pixel-level fusion, the lesion characteristics under different modalities can be comprehensively presented, making the boundary, internal structure and other information of the lesion clearer, thereby improving the accuracy of lesion detection and reducing the possibility of missed diagnosis and misdiagnosis.

[0112] Feature-level fusion is to integrate the different features of different modal images and give full play to their complementary role. For complex neural structures and lesions, feature-level fusion can construct more representative feature vectors to describe the direction, integrity and relationship of nerve fiber bundles and surrounding lesions, effectively avoiding damage to nerve fiber bundles. In addition, through feature-level fusion, the mutual constraints and complementary relationships between multiple modal features can be used to reduce the impact of individual differences and imaging equipment differences to a certain extent, making the features more stable and reliable.

[0113] Decision-level fusion integrates independent decisions based on different modalities, giving full play to the advantages of each modality. By combining the decisions of multiple modalities, the error of single-modality image decisions can be reduced, the reliability and accuracy of decisions can be improved, and the accuracy of judgment of the nature of lesions can be improved. At the same time, the importance of each modality in the diagnosis of neurosurgical diseases (such as brain tumors, vascular lesions, trauma, etc.) can be adjusted and fused in real time to better adapt to various complex and changing diagnostic needs.

[0114] 3D reconstruction module: The fused image data is used to construct a 3D model of the patient's neural structure through a variety of 3D reconstruction algorithms; the 3D model of the patient's neural structure generated by the 3D reconstruction module can intuitively display the morphology, position and spatial relationship between complex neural structures such as the brain and spinal cord and the lesions. Doctors can clearly see the specific location of the lesion through the 3D model, and during the operation, the 3D model in the intelligent neuronavigation system can be combined with the position information of the surgical instrument to achieve real-time navigation. During the operation, the doctor can observe the position of the surgical instrument in the 3D model at any time to ensure that the surgical instrument approaches the lesion along the predetermined safe path to avoid accidental damage to other key structures, greatly improving the safety and accuracy of the operation.

[0115] In addition, the 3D model can present the condition in a more intuitive and understandable way. Doctors can use the 3D model to explain to patients the location, size and shape of the lesion, as well as the general process and risks of the operation. At the same time, the 3D reconstruction model provides a unified visualization platform to facilitate the sharing of the patient's neural structure and lesion information among team members. Different members can better understand their responsibilities in the operation based on the 3D model, thereby improving the collaboration efficiency of the entire surgical team.

[0116] Condition analysis module: Comprehensively analyze the patient's imaging information and clinical medical records to determine the key information of the lesion; the diagnosis of neurosurgical diseases cannot rely solely on imaging information, but also needs to be combined with clinical medical records. Therefore, by integrating the patient's imaging characteristics (such as the location, size, morphology, density / signal intensity of the lesion in brain CT or MRI, etc.) and clinical symptoms (such as headache, limb weakness, sensory impairment, etc.), medical history (such as previous history of brain trauma, tumor history, etc.) and other examination results (such as blood tests, cerebrospinal fluid examination, etc.), a comprehensive assessment of the patient's condition is conducted.

[0117] The key details in determining the lesion are the determination of the nature of the lesion, the location and range of the lesion, and the analysis of the impact of the lesion on the surrounding tissues: the determination of the nature of the lesion is to determine whether the lesion is tumorous, vascular, inflammatory or traumatic through comprehensive analysis. That is, for a space-occupying lesion, the clarity of the lesion boundary in the image, the presence or absence of calcification, the edema of the surrounding tissues, and the duration of the patient's symptoms are combined to determine whether the lesion is a benign tumor or a malignant tumor.

[0118] The lesion location and scope are clearly defined to accurately locate the lesion in the neural structure and determine the extent of the lesion. This is crucial for subsequent surgical planning, because the choice of surgical scope and path depends largely on the location and size of the lesion. Specifically, for small lesions deep in the brain, surgical path planning needs to be more sophisticated to ensure that the lesion can be accurately reached and damage to surrounding normal tissues can be minimized.

[0119] Analysis of the impact of lesions on surrounding tissues is to assess whether the lesions compress or invade surrounding nerves, blood vessels, tissues, etc., and the possible functional impairment caused by such impact. Specifically, it is to analyze whether the tumor compresses the surrounding nerve fiber bundles, causing limb movement or sensory dysfunction; or to determine whether vascular lesions affect blood supply, thereby causing ischemic symptoms.

[0120] Surgical Path Planning Module: Automatically generates multiple feasible surgical path plans, and conducts evaluation and analysis; that is, based on the three-dimensional reconstructed neural structure model, combined with the lesion location, range and surrounding tissue conditions determined by the disease analysis module, it generates paths to the lesion site from different angles and directions, and uses computer algorithms to automatically generate multiple possible surgical paths. These path plans take into account avoiding important nerve and vascular structures, while minimizing the surgical distance to reduce surgical trauma and time.

[0121] Risk assessment module: predicts and assesses the risks that may be encountered during surgery and provides suggestions for countermeasures;

[0122] Predict various operational risks that may occur during surgery. For example, predict the risk of bleeding during surgery based on the relationship between the lesion and the blood vessel; estimate the risk of nerve damage based on the proximity of the lesion to the nerve; and assess the risk of infection by considering factors such as the size and location of the surgical incision and the duration of the operation. For neurosurgery, these risks may have a serious impact on the patient's prognosis, and early prediction helps to prepare for them.

[0123] At the same time, a safety assessment is conducted on each generated surgical path, mainly considering the proximity of the path to important blood vessels and nerves. By calculating parameters such as the distance and angle between the path and key structures, the risk of damaging these structures during surgery is assessed. And the feasibility of the surgical path in actual operation is assessed, including whether the surgical instruments can smoothly reach the lesion site and whether the surgical field of view is good. In addition, possible complications after surgery are predicted, including but not limited to cerebral edema, cerebrospinal fluid leakage, epileptic seizures, etc. This requires comprehensive consideration of factors such as the nature of the lesion (such as the malignancy of the tumor), the surgical method, and the patient's physical condition (such as age, underlying diseases, etc.). Subsequently, specific countermeasures are provided for the various predicted risks. These countermeasures can effectively reduce surgical risks, ensure the safety of patients, and help medical staff respond to emergencies and take timely measures.

[0124] Surgical plan formulation module: Comprehensively consider the results of condition analysis, surgical path planning and risk assessment to formulate a detailed surgical plan; that is, comprehensively consider the results of condition analysis, surgical path planning and risk assessment, weigh various factors, and formulate a detailed and comprehensive surgical plan. Specifically, it is necessary to balance the relationship between the thoroughness of lesion resection and surgical risks, and make the best choice based on the patient's specific situation. The formulation of the surgical plan includes selecting the appropriate surgical method according to the patient's condition and physical condition, such as craniotomy, minimally invasive surgery, endoscopic surgery, etc., as well as detailed planning of each step of the operation, including the location and size of the surgical incision, surgical approach, order of lesion resection, hemostasis and suturing, and other specific methods of operation.

[0125] Surgical simulation operation module: Based on the generated surgical path and surgical plan, simulated exercises are performed in a virtual environment. By simulating each surgical path and plan, the feasibility and safety of each surgical plan can be verified, so as to select the best surgical plan and surgical path, and verify whether there are any problems with the surgical plan and surgical path. If problems are found in the surgical path, the surgical plan and path are adjusted and optimized in time to improve the success rate of the operation. In addition, it can also provide a surgical operation training platform for neurosurgeons. Doctors can repeatedly practice surgical operation steps in a simulated environment, familiarize themselves with the surgical process and the use of instruments, and improve their surgical skills.

[0126] The data fusion module includes:

[0127] Pixel-level fusion: Wavelet transform is used to directly combine the pixel values ​​of different modal images to generate a fused image;

[0128] Wavelet transform is a time-frequency analysis method that can decompose image signals into subbands of different frequencies. In medical image processing, wavelet transform can be used to convert the pixel information of an image from the spatial domain to the frequency domain. Specifically, after wavelet transform, an image will be decomposed into low-frequency subbands (approximate components) and high-frequency subbands (detail components). The low-frequency subbands mainly contain information such as the contour and basic shape of the image, while the high-frequency subbands contain detailed information such as the edge and texture of the image. For medical images of different modalities, they are first subjected to wavelet transforms. Then, their subbands are combined according to certain rules in the wavelet domain. That is, the information in the low-frequency subbands of the CT image and the information in the low-frequency subbands of the MRI image are weighted averaged to synthesize the contour information of the two. For the high-frequency subbands, the method of selecting coefficients with larger absolute values ​​can be adopted, that is, the coefficients that can better reflect the details are selected from the high-frequency subbands of the CT and MRI images to construct the fused high-frequency subbands. Finally, the combined subbands are converted back to the spatial domain through inverse wavelet transform to obtain the fused image.

[0129] The formula for the weighted average portfolio is:

[0130] F L (i,j)=w1C L (i,j)+w2M L (i,j)

[0131] Among them C L (i, j) is the low-frequency subband coefficient of the CT image after wavelet transformation, M L (i, j) is the low-frequency subband coefficient of the MRI image after wavelet transformation, w1 and w2 are weight coefficients, and w1+w2=1.

[0132] The method for comparing absolute values ​​is:

[0133] If |F H (i,j)|>|M H (i,j)|, then F H (i,j)=C H (i,j); otherwise F H (i,j)=M H (i,j).

[0134] After obtaining the fused subband coefficients by weighted average combination of low-frequency subband coefficients and selecting appropriate high-frequency subband coefficients, these fused subband coefficients (including low-frequency subband coefficients and high-frequency subband coefficients, etc.) are used as input and converted from the frequency domain back to the spatial domain through the inverse wavelet transform algorithm. In other words, the inverse wavelet transform will reconstruct the pixel values ​​of the image based on the fused subband coefficients, so that the image can be restored in the spatial domain.

[0135] The fused image obtained by inverse wavelet transform combines the advantages of CT and MRI images. In terms of contour information, the advantages of both are combined by weighted averaging, which can clearly show the overall structure of organs and lesions; in terms of detail information, by selecting appropriate high-frequency sub-band coefficients, the fused image contains the most significant details. Such fused images provide doctors with more comprehensive and accurate information, which helps to better perform tasks such as disease diagnosis and surgical planning in fields such as neurosurgery. For example, in brain surgery planning, doctors can use fused images to more clearly see the relationship between the tumor and the surrounding brain tissue, blood vessels, and skull, so as to formulate a more accurate surgical plan.

[0136] Feature-level fusion: Extract features from different modal images and combine and select them. The extracted features include but are not limited to shape features, density features, texture features, signal strength features, etc. The extracted feature vectors need to be connected in series to form a new comprehensive feature vector. This comprehensive feature vector also contains information from different modalities for subsequent classification, recognition and other tasks. It should be noted that the combined feature vector has a high dimension and contains some redundant information, so feature selection and dimensionality reduction are required. The specific method used is to use principal component analysis (PCA) to select the most representative features and remove noise and redundant information.

[0137] Specifically, PCA first calculates the covariance matrix of the data, which reflects the correlation between the various features. Then, by solving the eigenvalues ​​and eigenvectors of the covariance matrix, the eigenvector with the larger eigenvalue is selected to form a new projection space.

[0138] Decision-level fusion: Fusion of decision results of different modal images. These decision results include judgments on whether a lesion exists, the type of lesion, and the grade of the lesion. The main method of fusion is probabilistic fusion, that is, using a probabilistic model (Bayesian classifier) ​​to make decisions on different modal images. For each modality image, there is a probability estimate of the existence of a lesion. The Bayesian formula is:

[0139]

[0140] Among them, P(D) is the probability of existence of lesions after fusion, P CT (-D) and PMRI (-D) are the probabilities of CT and MRI images judging the absence of lesions, respectively.

[0141] This fusion method takes into account the uncertainty of the decision of each modality image, obtains a comprehensive decision probability through probability calculation, and makes the final fusion decision based on this probability.

[0142] The 3D reconstruction module integrates multiple 3D reconstruction algorithms, including:

[0143] Voxel-based reconstruction algorithm: The Dual Contouring algorithm is used to generate high-quality three-dimensional surfaces by interpolating the boundaries of voxels. The main considerations are the adjacency relationship between voxels and the geometric information inside the voxels. For each voxel, the Dual Contouring algorithm determines its boundary based on the state of the surrounding voxels (that is, whether it belongs to the target object or the background). For objects with irregular internal structures, such as some complex neural tissue lesions, it can accurately reconstruct the shape of the object. And by accurately calculating the intersection points at the boundary of the voxels, the generated three-dimensional surface is relatively smooth and can well preserve the detailed features of the object, that is, it can clearly show the external contour of the tumor and the boundary with the surrounding normal brain tissue, helping doctors better understand the spatial position and morphology of the tumor.

[0144] Specifically, it looks for intersections on the boundary parts of voxels, which form the basic elements of the reconstructed surface.

[0145] Because the Dual Contourin algorithm is based on voxel information, it can make full use of the details in CT or MRI scan data. It can accurately capture the shape and position information of irregular tumor boundaries or complex vascular branching structures. The three-dimensional model generated by this algorithm can help doctors better understand the spatial relationship between diseased tissue and surrounding normal tissue, thereby providing strong support for surgical planning and diagnosis.

[0146] Surface-based reconstruction algorithm: The Poisson reconstruction algorithm is used, which is mainly based on the indicator function and the Poisson equation. The indicator function is used to represent the inner and outer parts of an object, which is 1 inside the object and 0 outside. In other words, there is an implicit function whose gradient field can be estimated from the set of object surface points obtained by the scanning device. This function is then found by solving the Poisson equation, and the zero isosurface of this function is the reconstructed object surface.

[0147] First, the algorithm starts with the input point cloud data and constructs a gradient field, that is, for each point, its normal vector is estimated. These points and their corresponding normal vectors form a vector field, which is actually an approximation of the gradient field of the reconstructed surface function. The formula for the vector field is:

[0148]

[0149] The point cloud dataset is P = {pi}, i = 1, 2, ..., n, for each point p i =(x i ,y i , z i ), whose normal vector is n i =(n i，x , n i，y , n i，z ), w i is a weight function, usually determined based on the distance between points, Represents the outer product operation.

[0150] After the gradient field is constructed, the Poisson equation is solved. The general form of the Poisson equation is:

[0151]

[0152] in is the divergence operator, and V is the gradient field constructed previously.

[0153] After solving the Poisson equation to obtain the function f, the isosurface of f is extracted as the reconstructed three-dimensional surface, that is, f=0 is used as the reconstructed surface.

[0154] Since the algorithm is based on solving the Poisson equation to generate a functional representation of the reconstructed surface, the resulting surface model has good geometric properties. It can generate a relatively closed and smooth surface. When reconstructing structures such as human organs or blood vessels, the smooth surface helps doctors observe their morphology and structure more intuitively, and is more accurate in subsequent analysis.

[0155] The Poisson reconstruction algorithm can use the point cloud data of the blood vessel surface (obtained through angiography and other methods) to generate a smooth blood vessel surface model. Doctors can use this model to evaluate the morphology, course, and relationship of the blood vessels with the surrounding nerve tissue, which is very important for avoiding vascular damage during surgical planning.

[0156] Reconstruction algorithm based on deep learning: It uses a reconstruction algorithm based on convolutional neural network (CNN). In 3D reconstruction, it reconstructs by learning the relationship between a large number of 2D image slices and the corresponding target 3D model. It contains multiple convolutional layers, pooling layers and fully connected layers. The convolutional layer is used to extract features in the image, that is, to learn the texture, shape and other characteristic patterns of different tissues in the image. The pooling layer is used to reduce the dimension of the data while retaining the main feature information. The fully connected layer is used to integrate the extracted features and finally generate a representation of the 3D model. It can perform effective 3D reconstruction for different organs, different imaging modalities (such as MRI, CT, etc.) and different types of lesions.

[0157] In the field of neurosurgery, for brain structural changes caused by some neurodegenerative diseases, the CNN model trained on a large amount of case imaging data can quickly and accurately reconstruct a three-dimensional model of the diseased brain, providing doctors with a more intuitive basis for diagnosis and treatment.

[0158] The location tracking unit includes:

[0159] Positioning sensor: including optical positioning sensor and electromagnetic positioning sensor, which can obtain the position information of surgical instruments and patient's head in real time; optical positioning is based on optical principles to determine the position of objects. Usually infrared light or visible light is used to place markers (reflective balls or light-emitting diodes) on surgical instruments and the parts of patients to be treated. These markers can reflect or emit light signals and be received by optical positioning sensors. The sensor calculates the position of surgical instruments and the parts of patients to be treated based on the principle of triangulation. The principle of triangulation is to determine the three-dimensional spatial position of the marker by using trigonometric function relationships based on the known positions of two or more sensors and the angles of the markers relative to these sensors.

[0160] Optical camera: It has multiple cameras to focus the image of the surgical area on the optical positioning sensor. The optical camera adjusts the focal length of the lens to clearly focus the image of the surgical area on the optical positioning sensor, similar to the focusing operation of a camera. By changing the distance between the lens and the image sensor, the light in the surgical area is accurately focused on the optical positioning sensor, thereby obtaining a clear image with markers. Different cameras can observe the surgical area from different angles, thus avoiding the blind spots that may exist in a single camera. This ensures that all surgical instruments in the surgical area and related markers on the patient's head can be observed.

[0161] Electromagnetic generator: generates an electromagnetic field of a specific frequency. The electromagnetic positioning sensor determines the position of the surgical instrument and the patient's treatment area by inducing the electromagnetic field; this electromagnetic field is distributed around the surgical area. The electromagnetic positioning sensor will generate an induced current or voltage change in this electromagnetic field. According to the law of electromagnetic induction, the induced electromotive force is related to factors such as the rate of change of the magnetic field and the number of turns of the sensor. There is an induction coil inside the electromagnetic positioning sensor. When the magnetic field changes, an induced electromotive force is generated at both ends of the coil. By measuring the size and direction of this electromotive force, and combining it with the intensity distribution law and frequency of the magnetic field generated by the electromagnetic generator, the position of the sensor can be calculated, so that the position of the surgical instrument and the patient's treatment area can be determined in three-dimensional space, and it has unique advantages for positioning deep tissues.

[0162] Automatic navigation module: Automatically navigate surgical instruments to perform surgery based on the data collected by the positioning sensor and the preoperatively planned path; based on the real-time position data of the surgical instruments and the patient's treatment area collected by the positioning sensor, combined with the preoperatively planned surgical path. Before the operation, the best surgical path has been determined through image data processing and surgical planning software. This path includes a series of spatial position points and directions that the surgical instruments need to pass through. The automatic navigation module will compare the real-time position data with the planned path and calculate the deviation between the current position of the surgical instrument and the target position. Then, by controlling the movement of the surgical instrument, the position and direction of the surgical instrument are adjusted so that it moves along the planned path.

[0163] Error compensation module: It uses error compensation algorithm to compensate for the errors that may occur in the positioning and tracking process in real time, and analyzes the distribution law and statistical characteristics of the errors by collecting a large amount of positioning data.

[0164] The positioning and tracking unit can accurately determine the position of surgical instruments and the patient's treatment area in real time, allowing the surgical instruments to operate along the optimal path planned before surgery. Through precise positioning and tracking, the tumor can be accurately reached to avoid damaging the surrounding important nerve tissue and blood vessels, thereby improving the success rate of the operation. In addition, the position information of the surgical instruments and the patient's head is obtained in real time, so that the doctor can understand the relative position relationship between the surgical instruments and the surrounding important structures at any time. When the surgical instrument approaches important neurovascular structures, the system can issue an alarm to remind the doctor to pay attention and reduce the risk of surgery. During the operation, the patient's head position may change due to physiological factors such as breathing, heartbeat, and muscle tension. These changes can be monitored in real time through the positioning and tracking unit, and the positioning information can be adjusted through the error compensation module to ensure that the surgical instrument can still accurately align with the target position.

[0165] The visual interaction unit includes:

[0166] Visualization module: Use virtual reality (VR) technology to present relevant information in a three-dimensional form. That is, the image information of the image acquisition and processing unit, the planning results of the intelligent planning unit, and the real-time data of the positioning and tracking unit are integrated and converted into a three-dimensional model. Rendering is performed through VR technology, allowing doctors to observe the situation of the surgical site from multiple angles and more intuitively.

[0167] Interactive interface module: A human-computer interactive interface is used to adjust the image and surgical planning parameters and control the surgical instruments; an intuitive graphical user interface (GUI) is used, combined with touch, voice, gesture and other interactive methods. At the same time, considering the particularity of the surgical environment, the design of the interactive interface must comply with the principle of aseptic operation. Doctors can adjust the display mode of the image through the interactive interface, such as changing parameters such as brightness, contrast, and color, so as to better observe the characteristics of different tissues. Images can also be scaled, rotated, and translated to view lesions and surrounding tissues from different angles. Doctors can also modify the parameters of surgical planning, such as surgical path and resection range, on the interactive interface. In other words, when it is found during the operation that the preoperatively planned path needs to be adjusted due to individual differences in the patient's anatomical structure, the doctor can modify the surgical path in real time through the interactive interface, and the system will re-navigate according to the new path.

[0168] In addition, the interactive interface can also realize the control of surgical instruments. By integrating with surgical robots or automated surgical equipment, doctors can remotely control the movement, rotation, cutting and other operations of surgical instruments on the interface and perform surgical operations more accurately.

[0169] Information display module: used to display important information related to surgery in real time on the visual interface, including but not limited to the real-time position of surgical instruments, surgical progress (completed surgical steps, remaining surgical steps, etc.), patient's physiological parameters (blood pressure, heart rate, etc.) and error information fed back by the positioning tracking unit, etc. It should be noted that all information should be updated in a timely manner to ensure that doctors can make decisions based on the latest situation, and the information should be displayed in a clear and intuitive manner to avoid information overload for doctors.

[0170] The visualization interactive unit presents information in an intuitive way, so doctors do not need to spend a lot of time interpreting complex two-dimensional image data. Through the three-dimensional presentation and information display module, doctors can quickly locate key information, so that they can perform surgical operations more quickly and improve surgical efficiency. In addition, the three-dimensional presentation allows doctors to clearly see the location of important neurovascular structures. Through the visualization interactive unit, doctors can better avoid damaging important structures such as the spinal cord and nerve roots, and reduce the occurrence of surgical complications.

[0171] The data management and update unit includes:

[0172] Data management module: classify, store and manage various types of data, establish data index and query mechanism, support data backup and recovery functions; patient imaging data (CT, MRI, PET and other modalities), surgical planning data (including surgical path, resection range and other planning information), real-time data during surgery (surgical instrument position, patient physiological parameters, etc.) and postoperative evaluation data. The data management module will classify data according to factors such as data type, source, and time. Specifically, all imaging data of the same patient are stored in a specific folder or data area and arranged in chronological order, so that doctors can check the changes in the patient's condition at any time.

[0173] Different storage structures and methods are used for different types of data. Image data is stored in the medical image file format (DICOM), which contains the image metadata (patient information, imaging device parameters, etc.) and pixel data. Surgical planning data is stored in a structured data table to record the detailed information of each surgical step. This classified storage and management can improve the organization and accessibility of data.

[0174] The data management module establishes a data index to facilitate quick search and use of data. That is, for patient imaging data, the index may contain information such as the patient's name, ID number, imaging date, imaging modality, etc. When it is necessary to find a brain MRI image of a specific patient, the system can quickly locate the corresponding data storage location through the index without traversing the entire database.

[0175] In addition, the data management module needs to have a backup function, that is, all data should be backed up regularly every week and the backup data should be stored in the cloud. When data loss occurs, the system can recover the data from the backup storage medium and restore the data to a previous point in time. At the same time, if errors occur during the data update process and some data is damaged, the recovery function can also restore the data to the state before the update to ensure data integrity and availability.

[0176] Data update module: regularly connects to the servers of medical databases or research institutions to obtain the latest information and update and optimize the system; the data update module obtains the latest information by regularly connecting to the servers of medical databases or research institutions. In other words, when new research finds that the imaging features of a certain neuropathy have new manifestations, the data update module can obtain this information from the relevant medical imaging database and integrate it into the system. For surgical planning, new algorithms may provide more accurate surgical path planning methods. The data update module will update these algorithms to the system so that the system can provide better services to doctors based on the latest knowledge and technology.

[0177] In addition to acquiring new information, the data update module is also responsible for optimizing the system, including optimizing software performance. That is, as the system usage time increases and the amount of data grows, data query and processing may slow down. The data update module can improve system performance by updating the database index structure or optimizing the data processing algorithm.

[0178] User authority management module: Different levels of user authority are set, and only authorized users are given the authority to access and operate corresponding data and functions. The user authority management module is set with different levels of user authority, and the authority levels include but are not limited to administrators, doctors, nurses, researchers and other categories. Specifically, the administrator has the highest authority and can comprehensively manage the system, including user account management, data backup and recovery settings, system updates and other operations. The doctor's authority is mainly concentrated on using the system to diagnose and operate on patients. He can access the patient's imaging data, plan operations and view real-time data during the operation. Nurses have partial authority to view the patient's basic information and operation progress in order to assist doctors in performing operations, while researchers can access clinical research-related data for data analysis and research, but may not be able to perform functions related to surgical operations.

[0179] Only authorized users can access and operate the corresponding data and functions. When a user logs in to the system, the permission management module verifies the user's identity and provides corresponding functional interfaces and data access permissions based on the user's permission level. In other words, after an ordinary nurse logs in to the system, she can only see the patient's basic information and the interface related to the surgical progress. She cannot access advanced functions such as surgical planning, nor can she modify the patient's imaging data. This strict authorization access and operation control can prevent risks such as data leakage and misoperation, and ensure the security and stable operation of the system.

[0180] A method for using an intelligent neuronavigation system for neurosurgery, comprising the following steps:

[0181] S1. The intelligent planning unit collects the patient's neurological imaging data of different modalities through different acquisition interfaces. The collected neurological imaging data is processed preliminarily by the preprocessing module, the separation module and the registration module in sequence. The image data after registration is transmitted to the intelligent planning unit through the corresponding transmission interfaces.

[0182] S2. The fusion module in the intelligent planning unit performs comprehensive fusion processing on the registered image data of different modalities, and the 3D reconstruction module uses a variety of 3D reconstruction algorithms to construct a 3D model of the patient's neural structure using the fused data;

[0183] S3. The condition analysis module determines the patient's condition based on the image data, 3D model and the patient's own condition. The surgical path module generates multiple feasible surgical paths based on the condition analysis results and surgical goals and principles.

[0184] S4. The risk assessment module predicts and assesses possible risks during surgery, issues early warnings, and provides countermeasures. The surgical plan formulation module formulates a detailed surgical plan based on the condition analysis, surgical path, and risk assessment results.

[0185] S5. The surgical simulation module performs simulation exercises on the model based on the generated surgical path and surgical plan. The doctor selects and verifies the best surgical plan based on the simulation results and his own needs;

[0186] S6. After the best surgical plan is formulated, the medical staff performs surgery on the patient according to the best surgical plan, and the positioning tracking unit tracks the position changes of the surgical instrument and the patient's treatment area in real time, and transmits the data to the visualization interaction unit and the intelligent planning unit;

[0187] S7. The visual interaction unit integrates the real-time data and preoperative planning information in real time, and provides feedback on the surgical process and surgical data for medical staff to understand the progress of the surgery and provide guidance. The intelligent planning unit adjusts and optimizes the surgical plan in real time based on the real-time position and the progress of the surgery, and feeds back the results to the visual interaction unit so that medical staff can grasp and respond in real time;

[0188] S8. After the operation is completed, the image acquisition and processing unit collects the patient's neurological imaging data again and compares and analyzes it with the neurological imaging data collected before the operation to evaluate the surgical effect and the patient's recovery. These imaging data and surgical process data are integrated and stored in the data management module.

[0189] Working principle: Before the operation, the image acquisition and processing unit comprehensively acquires the patient's neurological imaging data through various acquisition interfaces. The acquired image data enters the preprocessing module for preprocessing operations such as denoising, enhancement and grayscale correction. After the preprocessing, the image data then enters the segmentation module for segmentation and annotation. Then, the image data of different modalities are fused and analyzed in the same coordinate system through the registration module. After the registration, the data of different modalities are then transmitted to the intelligent planning unit through their respective corresponding transmission interfaces. The fusion module in the intelligent planning unit sequentially performs pixel-level fusion, feature fusion and decision-level fusion on the image data of different modalities after registration to obtain comprehensive image data.

[0190] The 3D reconstruction module uses a variety of 3D reconstruction algorithms to construct a 3D model of the patient's neural structure based on the fused data, and displays it through the visualization module; then, the condition analysis module retrieves the patient's clinical case from the data management and update unit based on the patient's imaging data for comprehensive analysis and diagnosis of the patient's condition; then, the surgical path module formulates a variety of feasible surgical paths based on the condition analysis and combined with the surgical goals and relevant surgical principles; after the surgical path is formulated, the risk assessment module predicts and estimates the risks that may arise from these surgical paths during the operation, and issues warnings and provides response measures; then, the surgical plan designation module comprehensively combines the condition analysis, surgical path and wind direction assessment to formulate a variety of feasible surgical plans; after receiving the surgical plan, the surgical operation simulation module simulates the model according to the surgical path and surgical plan, verifies and selects the best surgical plan. During this process, doctors can understand the intelligent planning process through the visual interaction unit and adjust the surgical plan that suits them as needed;

[0191] After the surgical plan is determined, the medical staff will perform surgery on the patient according to the surgical plan. During the operation, the positioning tracking unit will track the position changes of the surgical instruments and the patient's treatment area in real time, and transmit the data to the visualization interaction unit and the intelligent planning unit respectively. The visualization interaction unit will integrate and display the real-time surgical data and preoperative planning information. The doctor can observe the surgical situation and understand the real-time surgical information through the visualization interaction unit in real time; the intelligent planning unit will adjust and optimize the surgical plan in real time according to the real-time position display and the progress of the operation, and transmit the adjustment results to the visualization interaction unit for the doctor to understand and respond in time; the real-time information and data during the operation will be recorded and stored in real time by the data management and update unit;

[0192] After the operation is completed, the image acquisition and processing unit will collect the patient's neuroimaging data of different modalities again, compare and analyze it with the patient's neuroimaging data collected before the operation, and evaluate the effect of the operation and the patient's recovery. Subsequently, the data management and update unit will integrate the postoperative imaging data and surgical process data and store them in the data management module. The data update module will regularly update and improve the system according to the patient's recovery and new clinical research results.

Claims

1. An intelligent neuronavigation system for neurosurgery, characterized in that: include: Image acquisition and processing unit: used to collect the patient's neurological imaging data of different modalities, and perform preprocessing, segmentation, registration and other operations on the image data to obtain clear and accurate neurological imaging information that can be used for subsequent analysis; Intelligent planning unit: fuses the image information provided by the image acquisition and processing unit, establishes a three-dimensional model, and performs surgical path planning, risk assessment, and surgical plan formulation to provide the best treatment plan; Positioning and tracking unit: Tracks the position of surgical instruments and the position changes of the patient's treatment area in real time, and positions the surgical instruments to accurately match the path and position planned before surgery; Visualization interaction unit: visualizes the image information of the image acquisition and processing unit, the planning results of the intelligent planning unit, and the real-time data of the positioning and tracking unit, and presents them to the surgeon in an intuitive manner, while providing a human-computer interaction interface to facilitate the surgeon's operation; Data management and update unit: manage, store and back up all types of data throughout the entire surgical process. At the same time, update the system's algorithms and models based on new clinical cases and research results, set up data encryption, and only enable access and operation permissions to authorized users.

2. The intelligent neuronavigation system for neurosurgery according to claim 1, characterized in that: The image acquisition processing unit include: Image acquisition interface: It has acquisition interfaces for a variety of image acquisition devices, including magnetic resonance imaging, computer tomography, positron emission tomography and other imaging devices of different modalities, to achieve comprehensive acquisition of patient neurological images; Preprocessing module: performs preprocessing operations such as denoising, enhancement, and grayscale correction on the collected raw image data; Segmentation module: Use image segmentation algorithms to segment nerve tissue, diseased tissue and other important anatomical structures from images; Registration module: registers images collected at different times, in different modalities or at different viewing angles; Data transmission interface: The data under different modes are transmitted to the intelligent planning unit through their respective corresponding transmission interfaces for subsequent processing.

3. The intelligent neuronavigation system for neurosurgery according to claim 2, characterized in that: The segmentation module comprises: Tissue segmentation: distinguishing different tissue types in medical images; Lesion detection and segmentation: Detect and segment lesion areas in medical images; Structural annotation: Labeling and annotating important anatomical structures in medical images.

4. The intelligent neuronavigation system for neurosurgery according to claim 1, characterized in that: The intelligent planning unit comprises: Data fusion module: Comprehensively process the registered image data of different modalities, and the fusion is performed at the pixel level, feature level or decision level; 3D reconstruction module: Uses the fused image data to construct a 3D model of the patient's neural structure through a variety of 3D reconstruction algorithms; Disease analysis module: conducts comprehensive analysis of the patient's imaging information and clinical medical records to determine the key information of the lesion; Surgical Pathway Planning Module: Automatically generates multiple feasible surgical path plans, and performs evaluation and analysis; Risk assessment module: predicts and assesses the risks that may be encountered during surgery and provides suggestions for countermeasures; Surgical plan formulation module: Comprehensively consider the condition analysis, surgical path planning and risk assessment results to formulate a detailed surgical plan; Surgical simulation operation module: Based on the generated surgical path and surgical plan, simulated drills are performed in a virtual environment.

5. The intelligent neuronavigation system for neurosurgery according to claim 4, characterized in that: The data fusion module comprises: Pixel-level fusion: Wavelet transform is used to directly combine the pixel values ​​of different modal images to generate a fused image; Feature level fusion: extract features of images of different modalities and combine and select them; Decision-level fusion: Fusion of decision results from different modal images.

6. The intelligent neuronavigation system for neurosurgery according to claim 4, characterized in that: The 3D reconstruction module integrates multiple 3D reconstruction algorithms, including: Voxel-based reconstruction algorithm: using Dual Contouring algorithm; Surface-based reconstruction algorithm: Poisson reconstruction algorithm is used; Reconstruction algorithm based on deep learning: A reconstruction algorithm based on convolutional neural network (CNN) is used.

7. The intelligent neuronavigation system for neurosurgery according to claim 1, characterized in that: The positioning tracking unit comprises: Positioning sensor: including optical positioning sensor and electromagnetic positioning sensor, which can obtain the position information of surgical instruments and patient's head in real time; Optical camera: equipped with multiple cameras, focusing the image of the surgical area onto the optical positioning sensor; Electromagnetic generator: generates an electromagnetic field of a specific frequency. The electromagnetic positioning sensor determines the position of the surgical instrument and the patient's treatment area by sensing the electromagnetic field. Automatic navigation module: Automatically navigate surgical instruments to perform surgery based on the data collected by the positioning sensor and the path planned before surgery; Error compensation module: Use error compensation algorithm to compensate for errors that may occur during positioning and tracking in real time.

8. The intelligent neuronavigation system for neurosurgery according to claim 1, characterized in that: The visual interaction unit comprises: Visualization module: Use virtual reality VR technology to present relevant information in a three-dimensional form; Interactive interface module: uses a human-computer interactive interface to adjust imaging and surgical planning parameters and control surgical instruments; Information display module: used to display important surgery-related information in real time on a visual interface.

9. The intelligent neuronavigation system for neurosurgery according to claim 1, characterized in that: The data management and updating unit comprises: Data management module: classifies, stores and manages various types of data, establishes data index and query mechanisms, and supports data backup and recovery functions; Data update module: regularly connect to the medical database or research institution's server to obtain the latest information and update and optimize the system; User authority management module: Different levels of user authority are set, and only authorized users are given the authority to access and operate corresponding data and functions.

10. A method for using the intelligent neuronavigation system for neurosurgery according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1. The intelligent planning unit collects the patient's neurological imaging data of different modalities through different acquisition interfaces. The collected neurological imaging data is processed preliminarily by the preprocessing module, the separation module and the registration module in sequence. The image data after registration is transmitted to the intelligent planning unit through the corresponding transmission interfaces. S2. The fusion module in the intelligent planning unit performs comprehensive fusion processing on the registered image data of different modalities, and the 3D reconstruction module uses a variety of 3D reconstruction algorithms to construct a 3D model of the patient's neural structure using the fused data; S3. The condition analysis module determines the patient's condition based on the image data, 3D model and the patient's own condition. The surgical path module generates multiple feasible surgical paths based on the condition analysis results and surgical goals and principles. S4. The risk assessment module predicts and assesses possible risks during surgery, issues early warnings, and provides countermeasures. The surgical plan formulation module formulates a detailed surgical plan based on the condition analysis, surgical path, and risk assessment results. S5. The surgical simulation module performs simulation exercises on the model based on the generated surgical path and surgical plan. The doctor selects and verifies the best surgical plan based on the simulation results and his own needs; S6. After the best surgical plan is formulated, the medical staff performs surgery on the patient according to the best surgical plan, and the positioning tracking unit tracks the position changes of the surgical instrument and the patient's treatment area in real time, and transmits the data to the visualization interaction unit and the intelligent planning unit; S7. The visual interaction unit integrates the real-time data and preoperative planning information in real time, and provides feedback on the surgical process and surgical data for medical staff to understand the progress of the surgery and provide guidance. The intelligent planning unit adjusts and optimizes the surgical plan in real time based on the real-time position and the progress of the surgery, and feeds back the results to the visual interaction unit so that medical staff can grasp and respond in real time; S8. After the operation is completed, the image acquisition and processing unit collects the patient's neurological imaging data again and compares and analyzes it with the neurological imaging data collected before the operation to evaluate the surgical effect and the patient's recovery. These imaging data and surgical process data are integrated and stored in the data management module.

Images