Surgical path planning method and device based on multimodal image registration

Abstract

The present disclosure provides a surgical path planning method and device based on multi-modal image registration, the surgical path planning method comprising: acquiring multi-modal image data of a target cranial structure, wherein the multi-modal image data comprises a plurality of images captured for the target cranial structure, and the plurality of images are captured in different ways; performing image registration on the plurality of images to obtain registered multi-modal image data; receiving a target position input by a user for the registered multi-modal image data; and planning a path from outside the target cranial structure to the target position based on the registered multi-modal image data to obtain a planned surgical path. The present disclosure can solve the problems of time-consuming and error-prone preoperative path planning and low safety, can more accurately plan the surgical path, improve the accuracy and safety of path planning, and provide help for the preoperative preparation of the operator.

Description

Technical Field

[0001] This disclosure relates to the fields of medical image processing and computer-aided surgical planning, and particularly to a surgical path planning method and apparatus based on multimodal image registration. Background Technology

[0002] With the development of computer technology, the analysis and processing of medical images by computers can simulate and plan surgical paths before surgery. Automated path planning can provide auxiliary reference information for surgery and reduce the amount of calculation work for surgical personnel in the preoperative preparation stage.

[0003] Specifically, in related technologies, surgical path planning generally relies on manual planning by personnel before surgery. However, this method struggles to fully consider complex anatomical structures and functional area distributions, making the planning process time-consuming and prone to errors. Furthermore, some automated path planning schemes can only utilize single-modality imaging data, failing to provide comprehensive anatomical and functional information, thus affecting the accuracy and safety of path planning. Summary of the Invention

[0004] This disclosure provides a surgical path planning method and apparatus based on multimodal image registration, to at least solve the problems of time-consuming and error-prone preoperative manual path planning, and low accuracy and safety of automated path planning in related technologies. The technical solution of this disclosure is as follows:

[0005] According to a first aspect of this disclosure, a surgical path planning method based on multimodal image registration is provided. The surgical path planning method includes: acquiring multimodal image data of a target cranial structure, wherein the multimodal image data includes multiple images captured for the target cranial structure, the multiple images being captured in different ways; performing image registration on the multiple images to obtain registered multimodal image data; receiving a target location input by a user for the registered multimodal image data; and planning a path from the outside of the target cranial structure to the target location based on the registered multimodal image data to obtain a planned surgical path.

[0006] Optionally, the planned surgical path is obtained by: under preset optimization conditions, determining the entry point on the outer surface of the target cranial structure based on the registered multimodal image data; using the straight-line path from the entry point to the target location as the planned surgical path, wherein the optimization conditions include at least one of the following: the path between the entry point and the target location is the shortest; the direction of the path between the entry point and the target location is perpendicular to the skull surface of the target cranial structure; the skull thickness at the entry point is the smallest; the distance of the path between the entry point and the target location from a preset avoidance area is the largest; the distance of the path between the entry point and the target location from a preset pathway area is the smallest; and the depth of the path between the entry point and the target location through the sulcus region is the smallest.

[0007] Optionally, there are multiple optimization conditions. The step of determining the entry point on the outer surface of the target cranial structure based on the registered multimodal image data under preset optimization conditions includes: constructing an objective function based on each optimization condition and a preset weight corresponding to each optimization condition; the objective function represents the summation of scores for the entry point under each optimization condition using the preset weights; solving for the optimal solution of the objective function within the preset entry point region on the outer surface of the target cranial structure; and determining the position corresponding to the optimal solution as the entry point.

[0008] Optionally, the distance between the path from the cranial entry point to the target location and the preset avoidance area is determined by: determining multiple intermediate points on the straight path from the cranial entry point to the target location; and taking the minimum distance from each intermediate point to the avoidance area as the distance between the path and the avoidance area, wherein the distance between the path from the cranial entry point to the target location and the preset pathway area is determined by: determining multiple intermediate points on the straight path from the cranial entry point to the target location; and taking the minimum distance from each intermediate point to the pathway area as the distance between the path and the pathway area.

[0009] Optionally, the plurality of images includes CT images, MRI images, and vascular images, wherein the surgical path planning method further includes: determining a vascular surface grid of the target cranial structure based on the vascular images, and determining the avoidance area based on the vascular surface grid; and / or determining a skull surface grid of the target cranial structure based on the CT images, and determining the skull surface and skull thickness based on the skull surface grid; and / or determining a gray and white matter surface grid of the target cranial structure based on the MRI images, and determining the depth of the sulcus region based on the gray and white matter surface grid.

[0010] Optionally, the plurality of images includes CT images, MRI images, and vascular images, wherein the registered multimodal image data is obtained by: performing a first image registration on the CT images and the MRI images to obtain a first registration relationship; performing the first image registration on the vascular images and the MRI images to obtain a second registration relationship; performing a second image registration on the MRI images and a reference brain atlas template to obtain a third registration relationship; and obtaining the registered multimodal image data based on the first registration relationship, the second registration relationship, and the third registration relationship, wherein the first image registration is performed by: determining the rotational variation between the images to be registered. The rotation and translation transformations are adjusted to maximize the similarity between the images to be registered, thereby obtaining the registration relationship between the images to be registered. The second image registration is performed as follows: a first transformation relationship is determined to maximize the similarity between the MRI image and the reference brain atlas template; a second transformation relationship is determined to align each voxel between the MRI image and the reference brain atlas template; and a third registration relationship is determined based on the first and second transformation relationships, wherein the third registration relationship represents the correspondence between each brain region of the MRI image and the reference brain atlas template.

[0011] According to a second aspect of this disclosure, a surgical path planning device based on multimodal image registration is provided. The surgical path planning device includes: an acquisition unit configured to acquire multimodal image data of a target cranial structure, wherein the multimodal image data includes multiple images captured for the target cranial structure, the multiple images being captured in different ways; a registration unit configured to perform image registration on the multiple images to obtain registered multimodal image data; a receiving unit configured to receive a target location input by a user for the registered multimodal image data; and a planning unit configured to plan a path from the outside of the target cranial structure to the target location based on the registered multimodal image data to obtain a planned surgical path.

[0012] According to a third aspect of this disclosure, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to perform the surgical path planning method based on multimodal image registration according to this disclosure.

[0013] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the surgical path planning method based on multimodal image registration according to this disclosure.

[0014] According to a fifth aspect of this disclosure, a computer program product is provided, including computer-executable instructions that, when executed by at least one processor, implement the surgical path planning method based on multimodal image registration according to this disclosure.

[0015] The technical solution provided in this disclosure brings at least the following beneficial effects:

[0016] According to the surgical path planning scheme based on multimodal image registration disclosed herein, by registering multimodal image data, registered multimodal image data can be obtained. Based on the registered multimodal image data, a path from the outside of the target cranial structure to the target location can be planned according to the target location input by the user. In this way, automated surgical path planning can be realized. Furthermore, since the multimodal image data can contain more information about the target cranial structure, the surgical path can be planned more accurately, improving the accuracy and safety of path planning and providing assistance to the operator in preoperative preparation.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0019] Figure 1 This is a flowchart illustrating a surgical path planning method based on multimodal image registration according to an exemplary embodiment of the present disclosure.

[0020] Figure 2 This is a flowchart illustrating the steps of obtaining registered multimodal image data in a surgical path planning method based on multimodal image registration according to an exemplary embodiment of the present disclosure.

[0021] Figure 3 This is a flowchart illustrating the step of performing second image registration in a surgical path planning method based on multimodal image registration according to an exemplary embodiment of the present disclosure.

[0022] Figure 4This is a flowchart illustrating the step of determining the intracranial entry location in a surgical path planning method based on multimodal image registration according to an exemplary embodiment of the present disclosure.

[0023] Figure 5 This is a schematic diagram of an example of a pre-defined intracranial entry location region in a surgical path planning method based on multimodal image registration according to an exemplary embodiment of the present disclosure.

[0024] Figure 6 This is a flowchart illustrating an application example of image registration in a surgical path planning method based on multimodal image registration according to exemplary embodiments of the present disclosure.

[0025] Figure 7 This is a flowchart illustrating an application example of path planning in a surgical path planning method based on multimodal image registration according to an exemplary embodiment of the present disclosure.

[0026] Figure 8 This is a schematic block diagram of a surgical path planning device based on multimodal image registration according to exemplary embodiments of the present disclosure.

[0027] Figure 9 This is a schematic block diagram of an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0028] In order to enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0030] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0031] As mentioned earlier, among the relevant technologies, there are problems such as the time-consuming and error-prone nature of preoperative manual path planning, and the low accuracy and safety of automated path planning.

[0032] Generally speaking, existing image processing software may not be able to directly integrate multimodal image data such as CT, MRI, and TOF-MRA, as the operation is complex and prone to errors.

[0033] Furthermore, manual path planning is not only time-consuming and labor-intensive, but also fails to fully and comprehensively consider safety factors such as blood vessel avoidance, increasing surgical risks.

[0034] Furthermore, some existing path planning algorithms have high computational complexity, making it difficult to achieve real-time interaction, which affects clinical and research applications.

[0035] In view of the above problems, exemplary embodiments of this disclosure provide a surgical path planning method and apparatus, electronic device, computer-readable storage medium and computer program product based on multimodal image registration, which can solve or at least alleviate the above problems.

[0036] In a first aspect of an exemplary embodiment of this disclosure, a surgical path planning method based on multimodal image registration is provided.

[0037] The surgical path planning method according to the exemplary embodiments of the present disclosure can be applied to scenarios where a user interacts with surgical path planning software. For example, a surgical path planning software can be loaded on a user terminal, and the user can input surgical path planning instructions on the user terminal. The user terminal can obtain the final surgical path by executing the surgical path planning method according to the exemplary embodiments of the present disclosure.

[0038] The aforementioned user terminal can be such as a tablet computer, laptop computer, digital assistant, wearable device, etc. However, the implementation scenario of the above method is only an example scenario. The surgical path planning method according to the exemplary embodiments of this disclosure can also be applied to other application scenarios. For example, the user can request surgical path planning from the server, and the server can execute the method. In addition, the server can be a standalone server, a server cluster, a cloud computing platform, or a virtualization center.

[0039] Furthermore, the surgical pathway planning method according to the exemplary embodiments of this disclosure can be applied to neuroscience theoretical research in large animals such as dogs, pigs, and monkeys, for example, mapping brain atlases and studying the electrophysiological mechanisms of neural circuits; it can also be applied to preoperative pathway planning in human neurosurgery, such as intracranial surgery, providing auxiliary reference information for clinical decision-making; and it can also be used as a medical education and training tool to help doctors and researchers learn and simulate intracranial surgical pathway planning. In addition, the application scenarios of this surgical pathway planning method are flexible and can be applied according to actual needs. For example, it can also be applied to or included in other planning methods, such as the planning of drug delivery systems and electrode implantation. Furthermore, the image registration method in this method can also be applied to image analysis and processing fields requiring precise localization, such as brain tumor surgery and deep brain stimulation.

[0040] The following will refer to Figures 1 to 7 Examples of surgical path planning methods based on multimodal image registration according to embodiments of the present disclosure are described. For example... Figure 1 As shown, the surgical path planning method may include the following steps:

[0041] In step S110, multimodal image data of the target cranial structure can be acquired.

[0042] Here, multimodal imaging data can include multiple images captured targeting a specific brain structure, with each image captured in a different manner.

[0043] As an example, multiple images may include CT (Computed Tomography) images, MRI (Magnetic Resonance Imaging) images, and vascular images.

[0044] Here, CT imaging uses X-rays to obtain images of the internal structure of tissues or structures such as the skull and soft tissues; MRI imaging uses the magnetic resonance phenomenon to obtain detailed images of the internal structure of tissues or structures, such as structural MRI (sMRI) images, which can provide high-resolution information on brain tissue structure; vascular imaging can be structural images of blood vessels, such as, but not limited to, TOF-MRA (Time-of-Flight Magnetic Resonance Angiography) images, which can obtain three-dimensional structures of blood vessels such as those in the brain.

[0045] The aforementioned image data can be obtained by capturing images of the head of the target object. Here, the target object can be, for example, a human or animal body. CT images can cover the face and top of the head, providing structural information about the skull and soft tissues; MRI images can be T1-weighted or T2-weighted, providing structural information about brain tissue; and vascular images such as TOF-MRA can provide three-dimensional imaging of vascular structures.

[0046] As an example, the multimodal image data can be preprocessed before image registration. Preprocessing may include image orientation rotation, image cropping, etc.

[0047] For example, the method may further include: receiving a rotation operation from a user on any image in the multimodal image data; and rotating the image according to the rotation center and rotation angle specified by the rotation operation.

[0048] In embodiments of this disclosure, users can manually rotate CT images, MRI images, and / or vascular images to ensure that the images remain consistent in anatomical directions (e.g., head-to-tail, top-bottom, and left-to-right). This rotation can be achieved by manually adjusting the images or by inputting a rotation center, rotation axis, and / or rotation angle.

[0049] For example, suppose the rotation center received from the user input is c = [c x ,c y ,c z ] T The axis of rotation is φ, and the rotation angle is θ, where [c x ,c y ,c z Let ] denote the three-dimensional coordinates of the center of rotation, and T denote the transpose. Then the rotation matrix can be represented as R. φ(θ), the specific form of the rotation matrix can be obtained, for example, through the Rogers formula. In this case, let the original direction matrix of the rotated image be R, and the origin vector be o, then the direction matrix R′ of the image after rotation can be expressed by the following equation (1):

[0050] R′=R φ (θ)R (1)

[0051] The origin o′ of the rotated image can be represented by the following equation (2):

[0052] o′=R φ (θ)(oc)+c (2)

[0053] Here, o′ can represent the origin vector after rotation. Similarly, in this paper, the bold symbols can represent vectors.

[0054] The image rotation process described above allows the processed images to meet the standards of neuroscience research and ensures the accuracy of subsequent image registration. Furthermore, this image rotation method ensures that no interpolation errors are introduced during the rotation process, thus maintaining the accuracy of the original image.

[0055] Furthermore, during the aforementioned process, image rotation can be achieved by adjusting the direction matrix and origin. This allows for fast and accurate rotation without resampling the image, maintaining the high precision of the original image and thus improving the accuracy of subsequent path planning.

[0056] For example, the method may also include: receiving a cropping operation from a user for any image in the multimodal image data; and cropping the image according to the cropping method specified by the cropping operation. The cropping method may include, for example, cropping size, cropping shape, etc.

[0057] Specifically, users can manually crop CT images, MRI images, and / or vascular images as needed. The purpose of cropping can be, for example, to preserve the skull and brain tissue, thereby reducing the computational burden of subsequent registration and allowing focus on key brain regions.

[0058] In step S120, image registration can be performed on multiple images to obtain registered multimodal image data.

[0059] In this step, the acquired original image or the image after preprocessing such as rotation and cropping can be registered to obtain registered multimodal image data. As an example, registration can refer to finding the mapping relationship between images so that every point on one image has a unique corresponding point on another image, and these two points should correspond to the same anatomical location.

[0060] As an example, in an example involving multiple images including CT images, MRI images, and vascular images, such as... Figure 2 As shown, registered multimodal image data can be obtained in the following ways:

[0061] In step S210, a first image registration can be performed on the CT image and the MRI image to obtain a first registration relationship.

[0062] In step S220, a first image registration can be performed on the vascular image and the MRI image to obtain a second registration relationship.

[0063] In the above steps, the first image registration can be performed, for example, by determining the rotation and translation transformation amounts between the images to be registered, adjusting the rotation and translation transformation amounts to maximize the similarity between the images to be registered, and obtaining the registration relationship between the images to be registered.

[0064] Specifically, to ensure spatial alignment of images of different modalities, CT images can be registered with MRI images (e.g., sMRI images), and vascular images can be registered with MRI images (e.g., sMRI images). Here, the optimal transformation can be found by optimizing the similarity metric S to achieve the best spatial consistency between two images A and B. The rotation transformation amount can be represented by a rotation matrix, and the translation transformation amount can be represented by a translation vector. For example, the first image registration can be expressed as the following equation (3):

[0065]

[0066] Among them, R A→B Let t represent the rotation matrix from image A to B. A→B Let R represent the translation vector from image A to B, and max represent the result of adjusting the rotation matrix R. A→B Translation vector t A→B Find the maximum value of the similarity measure S. Here, the similarity measure S can be, for example, but not limited to, mutual information.

[0067] The above registration process can be implemented using various rigid body registration algorithms. Specifically, for equation (3) above, the transformation that maximizes the similarity metric can be found by continuously adjusting the rotation matrix R and the translation vector t. In this way, the optimal rotation matrix R and translation vector t can be directly applied to the orientation matrix and origin coordinates of the image, avoiding resampling of the image data and effectively preventing the loss of details. To accelerate the registration process, image pyramid technology can be used to gradually optimize the registration results from low resolution to high resolution. Each level contains different downsampling factors and smoothing parameters to balance speed and accuracy.

[0068] In step S230, a second image registration can be performed between the MRI image and the reference brain atlas template to obtain a third registration relationship.

[0069] Here, a brain atlas can be, for example, a standard model of brain anatomy and functional partitioning, which can be mapped onto individual images to locate and label brain regions. The reference brain atlas template can be a pre-defined standard brain atlas that characterizes the standard brain atlas of the group to which the target subject belongs. By registering the individual MRI images of the target subject's cranial structure with this reference brain atlas target, it can be ensured that the anatomical labels of the brain atlas are accurately mapped to the individual space, thereby accurately delineating the various brain regions in the MRI images.

[0070] As an example, such as Figure 3 As shown, the second image registration can be performed in the following manner:

[0071] In step S310, a first transformation relationship that maximizes the similarity between the MRI image and the reference brain atlas template can be determined.

[0072] Here, the first transformation relation can be, for example, an affine transformation matrix. Specifically, the optimal affine transformation matrix T can be solved using affine transformations. affjne This maximizes the similarity measure between stationary images (e.g., sMRI images of the target object) and moving images (e.g., reference brain atlas templates). The affine transformation matrix T affine For example, the form can be expressed by the following formula (4):

[0073]

[0074] Where R1 can represent an n×n (e.g., 3×3) rotation / shearing / scaling matrix, t1=[t x ,t y ,t z ] T It can represent a translation vector, where [t] x ,t y ,t z ] represents the coordinates of the translation vector on the three coordinate axes, and 0 represents [0,0,0].

[0075] However, the transformation methods that maximize the similarity between MRI images and reference brain atlas templates are not limited to the affine transformation methods described above. Correspondingly, the first transformation relationship is not limited to the affine transformation matrix described above, and other transformation methods and corresponding transformation relationships can also be selected.

[0076] In step S320, a second transformation relationship can be determined to align the voxels between the MRI image and the reference brain atlas template.

[0077] Here, the first transformation relationship can be a global registration between the MRI image and the reference brain atlas template (e.g., image-level registration), while the second transformation relationship can be a detailed registration between the MRI image and the reference brain atlas template (e.g., voxel-level registration). Specifically, after obtaining the first transformation relationship, the details of the image can be further aligned.

[0078] As an example, the second transformation relation can be a nonlinear deformation field. Specifically, after completing the affine transformation, the details between the MRI image and the reference brain atlas template can be aligned by estimating the nonlinear deformation field. As shown in Equation (5), the deformation field can be described by the displacement vector field D(x) to represent the nonlinear transformation of each voxel:

[0079] x ′ =x + D(x) (5)

[0080] Where x represents the coordinates of any voxel on the moving image, x ′ Let x represent the coordinates of any voxel in the registered moving image, and let D represent the coordinate offset of x in the anatomical direction, which can be determined by an optimization algorithm such as the Demons algorithm. Here, the deformation field can be progressively adjusted by an optimization algorithm such as the Demons algorithm to align the fixed image (e.g., the sMRI image of the target object) and the moving image (e.g., a reference brain atlas template) in detail. Although the process of registering an MRI image as a fixed image and a reference brain atlas template as a moving image is described herein, the embodiments of this disclosure are not limited thereto. The reference brain atlas template can also be used as a fixed image and the MRI image as a moving image for registration, as long as the transformation relationship between the two can be determined.

[0081] In step S330, the third registration relationship can be determined based on the first transformation relationship and the second transformation relationship.

[0082] Here, the third registration relationship can represent the correspondence between the MRI images and the brain regions of the reference brain atlas template. Specifically, through the above-mentioned double registration process, the transformation between the template (or group) and the individual space can be obtained from the first and second transformation relationships between the template (or group) and the individual space.

[0083] In step S330, based on the first and second transformation relationships described above, the brain atlas and gray / white matter surface mesh of the group space (e.g., a reference brain atlas template) can be mapped to the individual space (e.g., MRI images of the target cranial structure).

[0084] Specifically, the MRI image of the target cranial structure can first be transformed based on a first transformation relationship to obtain a first registered MRI image. Then, based on a second transformation relationship, the first registered MRI image can be transformed to obtain a second registered MRI image. Brain atlas labels can be added to the second registered MRI image, allowing the determination of the brain atlas and / or gray / white matter surface mesh of the target cranial structure. In this paper, the surface mesh can refer to a surface formed by vertices and edges, representing the morphology of the corresponding structure (e.g., skull, blood vessels, gray matter, white matter, skin, etc.).

[0085] For example, the template and the deformation field of the individual space can be applied to the first brain map and the first gray / white matter surface mesh of the group space to generate the second brain map and the second gray / white matter surface mesh of the individual space (i.e., the brain map and gray / white matter surface mesh of the target cranial structure). For example, the third registration relation can be represented by the following equation (6):

[0086]

[0087] Among them, L template T represents the brain map label in the template space. affine Denotes the first transformation relation, such as the affine transformation matrix, D nonlinear This represents a second transformation relation, such as for nonlinear deformation fields, where ° indicates a transformation of the matrix using this relation. Ultimately, L... individual This represents the second brain map mapped to the individual space and the second gray / white matter surface grid.

[0088] In addition, optionally, after obtaining the brain map of the target cranial structure, the brain map can be fine-tuned using an individual brain region segmentation algorithm to obtain more accurate functional area localization.

[0089] Preferably, the individual brain region segmentation algorithm can be, for example, a multimodal connectivity-based individual parcellation (MCIP) algorithm. Specifically, the individual brain region segmentation algorithm includes the following steps: acquiring functional magnetic resonance imaging (fMRI) and diffusion magnetic resonance imaging (dMRI) data of the target cranial structure; preprocessing the fMRI data to extract functional time series, wherein the preprocessing of the fMRI data may include, but is not limited to, artifact removal, slice time correction, head motion correction, sMRI-fMRI registration, standardization, and spatial filtering; preprocessing the dMRI data to extract anatomical connectivity profiles, wherein the preprocessing of the dMRI data may include, but is not limited to, motion correction, eddy current distortion correction, sMRI-dMRI registration, fiber orientation estimation, and probabilistic fiber tracing; and iteratively updating the brain region segmentation of the target cranial structure based on functional time series, anatomical connectivity profiles, spatial domain information, and brain atlases of the target cranial structure, thereby obtaining more accurate brain region segmentation results.

[0090] Specifically, the MCIP algorithm comprehensively considers the anatomical and functional connectivity information of group-level atlases and the individuals to be partitioned. Its goal is to maximize the consistency of connectivity within brain regions across modalities, while ensuring the spatial continuity of brain regions and the similarity between individual and group-level atlases. Specifically, this algorithm integrates functional time series data, anatomical connectivity spectra, spatial connectivity characteristics of vertices, and second brain atlas information, iteratively solving an energy function based on a graph cuts algorithm to achieve individual brain region partitioning.

[0091] As an example, the brain region division of a target cranial structure can be iteratively updated as follows: The brain regions of the target cranial structure can be initialized as group-level atlases; based on the current brain region division of the target cranial structure, the average functional time series and average anatomical connectivity spectrum of each brain region are calculated, and the brain region division of the target cranial structure is updated by minimizing the energy function using a graph cutting algorithm. The updated brain region division is then used as the brain region division for the target cranial structure in the next iteration. This process can be repeated until the brain region division results converge. The MCIP algorithm, by fusing multimodal data and group-level atlas information, ensures that the division results reflect both anatomical and functional connectivity consistency and capture individual-specific brain region features.

[0092] Furthermore, as an example, before performing the second image registration described above, non-uniform field correction can be performed on the MRI images (e.g., pre-processed MRI images). By performing non-uniform field correction, the signal intensity inconsistency caused by magnetic field inhomogeneity during MRI can be eliminated, thereby improving registration accuracy and ensuring more precise spatial alignment between the individual's MRI images and the standard brain atlas template.

[0093] In step S240, registered multimodal image data can be obtained based on the first registration relationship, the second registration relationship, and the third registration relationship.

[0094] For example, multimodal image data can be registered based on multiple images, including CT images, MRI images, and vascular images, as well as first, second, and third registration relationships. Through such registration, the correspondence between multimodal images can be established, thereby enabling the fusion of information from each multimodal image for comprehensive path planning.

[0095] As an example, a registered CT image can be determined based on a first registration relationship and the original CT image. For example, the original CT image can be transformed using the first registration relationship to obtain the registered CT image. Similarly, a registered MRI image can be determined based on a second registration relationship and the original MRI image. Thirdly, a brain atlas and / or gray-white matter surface grid corresponding to the original MRI image can be determined based on a third registration relationship and the original MRI image; that is, a brain atlas and / or gray-white matter surface grid of the target cranial structure. For example, the location of each brain region on the original MRI image can be determined based on the third registration relationship to construct a brain atlas and / or gray-white matter surface grid.

[0096] Here, the registered CT images, registered MRI images, brain atlases of the target cranial structure, and / or gray and white matter surface meshes can provide information on different aspects of the target cranial structure, thereby enabling the integration of this information for subsequent path planning.

[0097] Furthermore, although an example process for registration between images has been described above, the embodiments of this disclosure are not limited thereto. Registration between images can also be achieved in other ways, such as by using deep learning-based registration algorithms to improve registration speed and automation.

[0098] Furthermore, although the registration process has been described above in the order from step S210 to step S240, this disclosure does not restrict the execution order of the steps. For example, the execution order of steps S210, S220 and S230 can be arbitrary, or they can be executed in parallel.

[0099] In step S130, the target location can be received from the user for the registration of multimodal image data.

[0100] As an example, a user interface can be provided for users to input target locations, such as any intracranial location determined according to actual needs. According to embodiments of this disclosure, MRI images from multimodal imaging data can be displayed, and the user can receive the target location selected on the image. For example, the user can manually select a target location, such as the location for an injection / implantation, on an MRI image or individual brain atlas through a visual interface.

[0101] Furthermore, in the embodiments of this disclosure, target locations can also be automatically recommended. For example, based on big data and artificial intelligence technologies, optimal target points (and the preset cranial entry location range described below) can be automatically recommended to reduce manual operation by the user.

[0102] Return to reference Figure 1 In step S140, the path from the outside of the target cranial structure to the target location can be planned based on the registered multimodal image data to obtain the planned surgical path.

[0103] After obtaining the registered multimodal image data and the target location, the path planning can be accurately performed because these images can provide comprehensive information from multiple aspects.

[0104] As an example, when planning a route, certain preset planning criteria or optimization conditions can be followed to automatically optimize to the optimal path. Registered multimodal imagery data can be used to determine these planning criteria or optimization conditions.

[0105] Here, planning criteria may include, for example, the following: the shortest distance principle, which minimizes surgical trauma; the principle of perpendicularity to the skull, which ensures stability during the drilling process (e.g., based on the skull surface grid described below); the principle of traversing the thinnest part of the skull, which reduces damage to the skull (based on the skull surface grid); the principle of avoiding blood vessels, which minimizes the risk of intraoperative bleeding (e.g., based on the vascular surface grid described below); the principle of avoiding important brain regions, which reduces the risk of postoperative neurological impairment (e.g., based on the brain atlas described below); the principle of defining the route area, which can meet specific experimental needs, such as the need to traverse a pre-defined intracranial region (e.g., based on the brain atlas described below); and the principle of avoiding sulci, which reduces the risk of damage to microvessels (e.g., based on the second gray / white matter surface grid described below).

[0106] In this step, the planned surgical path can be obtained in the following way: under preset optimization conditions, based on the registered multimodal image data, the entry point on the outer surface of the target cranial structure is determined; the straight line path from the entry point to the target location is used as the planned surgical path.

[0107] Here, the optimization conditions may include at least one of the following: the path between the entry point and the target point is the shortest; the direction of the path between the entry point and the target point is perpendicular to the skull surface of the target cranial structure; the skull thickness at the entry point is the smallest; the path between the entry point and the target point is the largest distance from the preset avoidance area; the path between the entry point and the target point is the smallest distance from the preset pathway area; the path between the entry point and the target point passes through the sulcus region to the smallest depth.

[0108] Specifically, multiple optional optimization conditions can be displayed in the user interface. In response to the user selecting at least one optimization condition, the path from the outside of the target cranial structure to the target location is planned based on the user's selected optimization condition, and the planned surgical path is obtained.

[0109] The optimization conditions can be used as evaluation criteria to find the optimal intracranial access location. Here, the shortest path between the intracranial access location and the target location can be called the shortest path criterion (L1), and its optimization objective is to minimize the distance between the intracranial access location and the target location. Here, the distance between the intracranial access location and the target location can be expressed, for example, by the following equation (7):

[0110] L1(p)=||p-p0||2 (7)

[0111] Where p represents the entry point, p0 represents the target location, ||·||2 represents the 2-norm (i.e., Euclidean distance), and L1(p) represents the distance between the entry point p and the target location. The goal of optimization is to minimize the value of L1(p).

[0112] The direction of the path between the entry point and the target point, perpendicular to the skull surface of the target cranial structure, can be referred to as the perpendicularity criterion (L2). Its optimization objective is to ensure the stability of the actuating instrument, such as a surgical instrument, when entering the skull. Here, the distance between the entry point and the target point can be expressed, for example, by the following equation (8):

[0113]

[0114] Where, n pL2(p) represents the normal vector at the entry point p on the skull surface, and L2(p) represents the angle between the direction of the path between the entry point and the target position and the direction perpendicular to the skull surface of the target cranial structure. The optimization objective is to minimize the value of L2(p).

[0115] The minimum skull thickness at the entry point can be referred to as the thinnest skull criterion (L3). Its optimization goal is to select the thinnest skull section for entry to reduce surgical difficulty. Here, the skull thickness at the entry point can be expressed, for example, by the following equation (9):

[0116] L3(p)=||pS(p,p0)||2(9)

[0117] Where S(p, p0) represents the intersection of path (p, p0) and the inner surface of the skull, and L3(p) represents the skull thickness at the entry point. The optimization objective is to minimize the value of L3(p).

[0118] The maximum distance between the path from the entry point to the target point and the preset avoidance area can be called the avoidance area criterion (L4). Its optimization goal is to avoid blood vessels and important brain regions, thereby reducing intraoperative risks.

[0119] As an example, the distance between the path from the entry point to the target location and the preset avoidance area can be determined as follows: identify multiple intermediate points on the straight path from the entry point to the target location; and take the minimum distance from each intermediate point to the avoidance area as the distance between the path and the avoidance area.

[0120] Here, the thickness of the skull at the entry point can be expressed, for example, by the following formula (10):

[0121]

[0122] Among them, SDF A (·) represents the signed distance function of the target avoidance zone, A represents the target avoidance zone, and SDF A A positive t indicates that the input point is outside region A, while a negative t indicates that the input point is inside region A. The absolute value of t represents the nearest distance between the input point and the boundary of region A. A The value can be taken in the range [0, 1] according to actual needs, in order to find the SDF (Sign Distance Function) on the path between the entry position p and the target position p0. A(·) The point with the smallest value. L4(p) represents the inverse of the distance from the path between the entry point and the target point to the preset avoidance area. The optimization objective is to minimize the value of L4(p). Here, the inverse of the distance from the path between the entry point and the target point to the preset avoidance area is taken to unify the optimization objectives in each optimization condition (e.g., the optimization objectives are all to minimize or maximize the value), simplifying the optimization calculation.

[0123] As an example, the minimum distance from each intermediate point to the avoidance area can be determined iteratively by performing the following operations: determining multiple current sampling points on the current candidate path; determining the current reference sampling point among the multiple current sampling points that has the smallest distance to the avoidance area; determining a first path segment on the current candidate path that includes the current reference sampling point (e.g., centered on the reference sampling point); determining a second path segment based on the distances from the two endpoints of the first path segment to the avoidance area and the distance from the current reference sampling point to the avoidance area, wherein the second path segment is the path between one of the two endpoints of the first path segment and the current reference sampling point; using the second path segment as the candidate path for the next iteration until the number of iterations reaches a preset number, and using the distance from the reference sampling point to the avoidance area in the last iteration as the aforementioned minimum distance. Here, in the first iteration, the candidate path is the path between the entry point p and the target position p0.

[0124] As a preferred embodiment of this example, for a specific intracranial location p, a fast algorithm combining discrete sampling and binary iteration is used to solve L4(p). This algorithm has better accuracy than discrete sampling and can also overcome the shortcomings of simple binary iteration, which gets stuck in local minima and cannot reach the global minimum. The implementation process of this algorithm is as follows: First, on the line segment (p, p0) (i.e., t... A Uniform sampling is performed on the interval [0, 1]. Specifically, the interval [0, 1] is uniformly divided into N (e.g., N = 10) parts to obtain N+1 sampling points. The L4 value at each sampling point is calculated, and then the t value that minimizes L4 is found. A , denoted as t mid Then with t mid Centered on the data, we take a smaller sampling interval to the left and right as the initial upper and lower limits for the binary search iteration, for example, t. low =t mid -1 / 2N, t high =t mid +1 / 2N, then calculate and compare t low t mid and t high L4. If t low If L4 is smaller, it means the minimum value is biased towards [t]. low , t mid], then let the new t high =t mid If t high If L4 is smaller, it means the minimum value is biased towards [t]. mid , t high ], then let the new t low =t mid Otherwise, it means t mid L4 is already better than t low and t high If the minimum value is in the middle, then let the new t be smaller. low =(t low +t mid ) / 2, new = (t) mid +t high () / 2. By repeating this process multiple times within the aforementioned smaller interval, the minimum value of L4 can be obtained precisely.

[0125] The minimum distance from the path between the entry point and the target point to the preset pathway area can be called the pathway area criterion (L5). Its optimization goal is to have the path pass through a specific area to meet specific experimental requirements or to pass through a preset safety area.

[0126] As an example, the distance between the path from the entry point to the target location and the preset pathway area can be determined by: identifying multiple intermediate points on the straight path from the entry point to the target location; and using the minimum distance from each intermediate point to the pathway area as the distance between the path and the pathway area.

[0127] Here, the distance from the path between the entry point and the target point to the preset pathway region can be represented, for example, by the following formula (11):

[0128]

[0129] Among them, SDF B (.) represents the signed distance function of the area the target passes through, B represents the area the target passes through, and t B The value can be taken in the range [0, 1] according to actual needs, in order to find the SDF (Sign Distance Function) on the path between the entry position p and the target position p0. B (·) The point with the smallest value. L5(p) represents the distance from the path between the entry point and the target point to the preset pathway region. The optimization goal is to minimize the value of L5(p).

[0130] As an example, a fast algorithm combining discrete sampling and binary iteration can also be used to solve L5(p). Specifically, the minimum distance from each intermediate point to the pathway region can be determined by iteratively performing the following operations: determining multiple current sampling points on the current candidate path; determining the current reference sampling point with the smallest distance from the pathway region among the multiple current sampling points; determining a first path segment on the current candidate path that includes the current reference sampling point (e.g., centered on the reference sampling point); determining a second path segment based on the distances from the two endpoints of the first path segment to the pathway region and the distance from the current reference sampling point to the pathway region, wherein the second path segment is the path between one of the two endpoints of the first path segment and the current reference sampling point; using the second path segment as the candidate path for the next iteration until the number of iterations reaches a preset number, and using the distance from the reference sampling point to the pathway region in the last iteration as the aforementioned minimum distance. Here, in the first iteration, the candidate path is the path between the entry point p and the target position p0.

[0131] The minimum depth through which the path between the entry point and the target location traverses the sulcus region can be termed the sulcus avoidance criterion (L6). Its optimization objective is to minimize the risk of microvascular damage in the sulcus region. Here, the depth through which the path between the entry point and the target location traverses the sulcus region can be represented, for example, by the following equation (12):

[0132] L6(p)=SulcDepth(p,p0) (12)

[0133] Here, SulcDepth(p,p0) represents the depth of the path (p,p0) through the sulcus. L6(p) represents the depth of the path between the entry point and the target point through the sulcus region. The optimization objective is to minimize the value of L6(p).

[0134] The above methods can provide a variety of optimization conditions or evaluation criteria, thereby enabling the setting of multiple safety constraints (such as skull thickness, blood vessel distance, and brain sulcus depth) during the path planning process, making the final planned path safer, more accurate, and more valuable for reference.

[0135] Furthermore, although several optimization conditions have been described above by way of example, the embodiments of this disclosure are not limited thereto, and other optimization conditions may be added, such as avoiding functional nuclei, taking into account the direction of white matter fiber tracts (for example, dMRI / fMRI data can be acquired, the fMRI / dMRI data can be registered with sMRI data, the location of functional nuclei can be determined based on the registered fMRI data, the direction of white matter fiber tracts can be determined based on the registered dMRI data, and optimization conditions related to the location of functional nuclei or the direction of white matter fiber tracts can be determined), to meet a wider range of application needs.

[0136] As an example, the optimization condition can be one, in which case the optimal solution under the optimization condition can be determined as the final cranial entry location in order to plan the path from the cranial entry location to the target location.

[0137] As another example, there can be multiple optimization conditions. In an example with multiple optimization conditions, such as... Figure 4 As shown, the entry point on the outer surface of the target cranial structure can be determined in the following ways:

[0138] In step S410, an objective function can be constructed based on each optimization condition and the preset weight corresponding to each optimization condition.

[0139] Here, the objective function can be represented by summing the scores of the intracranial entry position under each optimization condition using preset weights.

[0140] Specifically, a preset weight can be set for each of the multiple optimization conditions. The weight of each optimization condition can be set according to actual needs or experience, for example, it can be input by the user.

[0141] As an example, the objective function can be expressed by the following equation (13):

[0142]

[0143] in, Let L(p) represent the normalized value of the i-th optimization condition, m represent the total number of optimization conditions, and L(p) represent the objective function. The overall optimization objective is to minimize the value of L(p).

[0144] In step S420, the optimal solution of the objective function can be solved for the position within the preset entry point region on the outer surface of the target cranial structure.

[0145] Here, the preset entry point region can be a local area on the outer surface of the cranial structure, such as a user-inputted or predefined area. By setting this region, the range for finding the optimal solution can be narrowed, reducing the amount of computation.

[0146] As an example, the preset entry point region can be determined in the following way: based on the registered multimodal image data, the outer surface of the target cranial structure (e.g., the outer surface of the skull) is determined; based on the outer surface of the target cranial structure, the target axis and its value range, the preset entry point region is determined, wherein the preset entry point region is the area of ​​the outer surface of the target cranial structure within the value range of the target axis.

[0147] Here, the centroid c of the vertices on the skull surface can be calculated, and for each vertex p among multiple vertices on the skull surface, its normal vector n can be calculated. pGiven the displacement vector pc pointing towards the centroid, calculate the product (pc) of the normal vector and the transpose of the displacement vector. T n p If the product is greater than 0, the vertex p can be identified as an inner surface; if the product is less than or equal to 0, the vertex p can be identified as an outer surface. By sequentially determining whether the surface on which the above vertices are located is an inner or outer surface, the outer surface of the target cranial structure (e.g., the outer surface of the skull) can be determined based on the set of vertices located on the outer surface.

[0148] As an example, the target axis and its value range can be user-inputted or pre-defined. For example, such as Figure 5 As shown, the preset entry point region can be determined on the outer surface of the target cranial structure based on the target axis and its value range.

[0149] Specifically, users can select the target axis plane (X / Y / Z) and its value range (e.g., 100 to 200) in the user interaction interface. In the method of the embodiments of this disclosure, in response to receiving the target axis and its value range input by the user, the set of vertices Ω of the outer surface of the skull that meet the conditions can be screened on the outer surface of the target cranial structure as the candidate range of the entry position, i.e. the above-mentioned preset entry position area.

[0150] In this example, both the target location and the preset entry point region can be manually input by the user. For example, the user can determine the target location and define a region of interest on the skull surface as the entry point range (i.e., the preset entry point region) through the user interface for the registered multimodal image data. The example of determining the preset entry point region described above is now returned to step S420, where the location that yields the optimal solution to the objective function can be found within the preset entry point region.

[0151] As an example, multiple optimization algorithms (or solvers) can be preset, and users can choose the appropriate algorithm according to their needs. In response to receiving the user's selected optimization algorithm, the system uses that algorithm to find the optimal solution to the objective function.

[0152] Here, the preset optimization algorithm may include at least one of the following: traversal search, Levenberg-Marquardt algorithm (LM algorithm), and simulated annealing algorithm. In addition, the preset optimization algorithm may also include global optimization algorithms such as particle swarm optimization (PSO) and genetic algorithm (GA) to further improve the efficiency and accuracy of the solution.

[0153] Specifically, in the traversal search method, the set of candidate entry points Ω can be exhaustively enumerated, the objective function value of each entry point can be calculated, and several optimal solutions can be selected.

[0154] The Learning Algorithm (LM) is an optimization algorithm for solving nonlinear problems. It combines the advantages of gradient descent and the Gauss-Newton method, achieving a good balance between global search and local convergence. In the initial iterations, the LM algorithm resembles gradient descent, effectively avoiding getting trapped in local optima. However, as it approaches the solution, it switches to the Gauss-Newton method, thereby accelerating convergence.

[0155] Specifically, multiple initial entry points can be selected from the vertex set Ω of the preset entry point region, and the entry points can be iteratively updated to gradually approach the optimal solution. The iterative process can be represented by the following equation (14):

[0156]

[0157] Where p represents the cranial entry position of the current step, and J is the Jacobian matrix of the objective function L(p). λ is the gradient of the objective function, and λ is an adjustment coefficient used to balance gradient descent and the Gauss-Newton method, which can be given in advance. Based on the above equation (14), the iteration is performed. If the maximum number of iterations is reached or the gradient is less than the preset threshold, the iteration can be terminated and the optimal solution obtained in the current iteration can be output.

[0158] Here, the Levenberg-Marquardt algorithm is an optimization algorithm for nonlinear least squares problems. By using this algorithm, the advantages of gradient descent and Gauss-Newton method can be combined to find the optimal solution more quickly and accurately.

[0159] Simulated annealing is a stochastic search algorithm for global optimization, particularly suitable for solving multimodal and non-convex optimization problems. By introducing randomness, simulated annealing allows for suboptimal solutions during the optimization process, thus avoiding getting trapped in local optima and ultimately approximating the global optimum.

[0160] Specifically, multiple initial entry points p0 can be selected by uniformly sampling from the vertex set Ω of the preset entry point region, where p0∈Ω. For each initial entry point p0, the entry point p and temperature T can be iteratively updated.

[0161] For example, updates can be performed as follows: A new solution p′ is randomly generated in the current neighborhood of the intracranial entry position p, and the energy difference ΔL = L(p′) - L(p) between the current intracranial entry position p and the new intracranial entry position p′ is calculated; if ΔL < 0, the new solution is accepted; otherwise, it can be updated with a preset probability (e.g., ...). Among them, T i The current temperature is used to accept new solutions. Accepting new solutions can mean replacing the current entry point with a new one. Additionally, the temperature can be updated; for example, the updated temperature could be T.i+1 =αT i , where α is the preset update coefficient.

[0162] Based on the above process, the iteration can be performed. If the maximum number of iterations is reached or the temperature is below the threshold, the iteration can be terminated. Otherwise, the above update process can be repeated until the iteration is terminated and the optimal solution is output.

[0163] Here, since the simulated annealing algorithm is a stochastic optimization algorithm that simulates the physical annealing process, it is suitable for solving the global optimum, and is particularly suitable for solving the optimum of the objective function here.

[0164] In step S430, the position corresponding to the optimal solution can be determined as the cranial entry position.

[0165] In this step, the position corresponding to the above optimal solution can be used as the cranial entry point.

[0166] By using the above methods, the optimal intracranial entry point that meets the set optimization conditions can be obtained, thus providing a more reasonable and accurate surgical path.

[0167] Furthermore, although the above describes determining the intracranial entry location under preset optimization conditions, the embodiments of this disclosure are not limited thereto. Users can also manually select the intracranial entry location, thereby planning the path from the intracranial entry location to the target location based on the registered multimodal image data, and obtaining the planned surgical path.

[0168] Once the intracranial incision location is determined, a straight path from the incision location to the target location can be used as the final planned surgical path. As an example, the resulting surgical path can be exported in a format readable by the surgical navigation system to support intraoperative reference positioning and guidance.

[0169] The above describes an example process for surgical path planning according to embodiments of the present disclosure. Below, an example of determining the reference position in each optimization condition will be described in detail.

[0170] Specifically, as mentioned above, the multiple images can include CT images, MRI images, and vascular images. On the one hand, multimodal image data registration can be performed to establish the correspondence between different images. On the other hand, the surgical path planning method can also include: surface mesh reconstruction based on multimodal image data to obtain the vascular surface mesh, skull surface mesh, and / or gray and white matter surface mesh of the target cranial structure. These meshes can be used to determine reference locations in optimization conditions, such as avoidance areas, skull surfaces, skull thickness, and sulci regions.

[0171] For example, the surgical path planning method may also include: determining the vascular surface grid of the target cranial structure based on vascular imaging; and determining the avoidance area based on the vascular surface grid.

[0172] Specifically, based on raw or preprocessed vascular images, vascular segmentation can be performed to extract the three-dimensional structure of intracranial vessels within the target brain structure and determine the surface mesh of the vessels. Furthermore, based on the three-dimensional structure of the intracranial vessels, vascular labels can be generated, which can be used to mark the location of vessels within the target brain structure. Existing vascular segmentation methods can be used for segmentation. The embodiments of this disclosure do not impose particular limitations on the segmentation method; for example, image segmentation algorithms such as U-Net can be used to improve the accuracy of vascular segmentation.

[0173] By generating a vascular surface mesh, the location information of intracranial blood vessels in the target brain structure can be obtained, providing a precise anatomical reference for path planning. This allows for the avoidance of critical blood vessels during path planning, reducing the risk of intraoperative bleeding. For example, based on the vascular surface mesh, the avoidance area in the optimization conditions can be determined, such as the target avoidance area A in the above equation (10), to avoid important blood vessels.

[0174] Alternatively or additionally, the surgical path planning method may also include: determining a skull surface grid of the target cranial structure based on CT images; and determining the skull surface and skull thickness based on the skull surface grid.

[0175] Here, since CT images can contain skin surface information and bone information, the skin surface mesh and / or skull surface mesh of the target cranial structure can be reconstructed based on CT images.

[0176] Here, the location and thickness of the skull can be determined by segmenting the CT images to obtain a skull surface mesh. The skull surface mesh can provide a precise anatomical reference for path planning. For example, based on the skull surface mesh, the skull surface and thickness in the optimization conditions can be determined, such as the skull thickness in Equation (9) above, and the skull surface used to determine the entry point. In addition, the skin surface mesh can be used for navigation in subsequent surgeries to achieve image-physical space alignment.

[0177] In the above process, existing bone segmentation methods can be used for bone segmentation. The embodiments of this disclosure do not impose any particular restrictions on the segmentation method of CT images. For example, existing image segmentation algorithms can be used to perform bone segmentation on CT images to obtain a skull surface mesh.

[0178] Alternatively or additionally, the surgical path planning method may also include: determining a gray-white matter surface grid of the target cranial structure based on MRI images; and determining the depth of the sulcus region based on the gray-white matter surface grid.

[0179] Here, software such as FreeSurfer can be used to extract the gray and white matter surface mesh of the target cranial structure. The gray and white matter surface mesh can be used to calculate the depth of the sulci. For example, the depth of the sulci region in the above equation (12) can be determined.

[0180] In addition, brain atlases of target cranial structures can be determined based on MRI images. For example, they can be obtained by image registration of MRI images with reference brain atlas templates, or by individual brain region segmentation algorithms based on fMRI / dMRI.

[0181] In the manner described above, the surgical path planning method of the present disclosure can integrate multimodal images such as sMRI structural images, CT skull data, vascular imaging (e.g., TOF-MRA) data, and individualized brain atlases, and define the multimodal input data required for path planning (e.g., sMRI, skull surface mesh, vascular surface mesh, etc.), thereby enabling a comprehensive and complete path planning scheme.

[0182] The surgical path planning method according to the embodiments of this disclosure can combine multimodal image data and automatically plan one or more candidate paths based on user-determined optimization conditions and the importance weight of each optimization condition. These paths can be provided to users in real time through a three-dimensional visualization user interface, and users can further correct and optimize the paths based on their professional experience.

[0183] Specifically, during the execution of the surgical path planning method according to the embodiments of this disclosure, the planned surgical path can be displayed in real time, allowing the user to observe the entry point, target location, and path in three-dimensional space. In response to the user's editing operations on the displayed surgical path, the surgical path can be adjusted and optimized. Thus, while achieving automated path planning, it also supports the intervention of expert experience to improve the reliability and accuracy of the final generated path.

[0184] The following will refer to... Figure 6 and Figure 7 This document describes application examples of the image registration process and the path planning process in a surgical path planning method based on multimodal image registration according to exemplary embodiments of the present disclosure.

[0185] like Figure 6As shown, in step S601, multimodal image data, such as a first CT image, a first sMRI image, and a first TOF-MRA image, can be read. In step S602, the user can manually rotate the first CT, first sMRI, and first TOF-MRA images to align them to the correct anatomical orientation. In step S603, the user can manually crop the first CT, first sMRI, and first TOF-MRA images to obtain cropped second CT, second sMRI, and second TOF-MRA images. Here, steps S602 and S603 are optional; rotation and cropping can be omitted.

[0186] In step S604, cross-modal image registration can be performed for the current individual. For example, image registration from the second CT image to the second sMRI image or from the second TOF-MRA image to the second sMRI image can be performed to obtain the corresponding image transformation relationship.

[0187] In step S605, the skull surface mesh can be reconstructed based on the second CT image, and optionally, the skin surface mesh can also be reconstructed.

[0188] In step S606, blood vessels can be segmented based on the second TOF-MRA image to obtain blood vessel labels; in step S607, blood vessel surface mesh can be reconstructed based on the blood vessel labels.

[0189] Optionally, in step S608, the second sMRI image may be subjected to non-uniform field correction.

[0190] In step S609, the second sMRI image can be image registered with the reference brain atlas template, thereby obtaining the second brain atlas and the second gray / white matter surface grid located in the individual space (corresponding to the second sMRI image) based on the first brain atlas located in the group space (corresponding to the reference brain atlas template) and the first gray / white matter surface grid.

[0191] In step S610, the target location (or target point) manually input by the user can be received based on the second brain atlas and the second sMRI image. In step S611, the region of interest on the skull surface manually defined by the user can also be received as the range for automatic planning of the entry point.

[0192] In step S612, one or more candidate paths can be automatically planned, for example, to determine the entry point into the skull, based on the skull surface mesh, the blood vessel surface mesh, and the second gray / white matter surface mesh, using evaluation criteria (i.e., the optimization conditions mentioned above) manually determined by the user and the importance weight of each evaluation criterion.

[0193] In step S613, the user's selection and / or fine-tuning of candidate paths can be received, and the final planned surgical path can be saved.

[0194] like Figure 7 As shown, in step S701, an individual brain atlas (e.g., the second brain atlas) and an sMRI image (e.g., the second sMRI image) can be determined through the above image registration; in step S702, the target location selected by the user for the individual brain atlas and the sMRI image can be received; and in step S703, the evaluation criteria (i.e., the above optimization conditions) and their weights set by the user can be received.

[0195] Here, evaluation criteria may include, for example, the shortest path criterion, the perpendicularity to the skull criterion, the thinnest passage through the skull criterion, the avoidance of blood vessels and / or specific regions criterion, the passage through specific regions criterion, and the avoidance of cerebral sulci criterion, etc., with reference to the above. Figure 6 The results of image registration in the image can be determined based on the skull surface grid, the criteria of perpendicular to the skull and the thinnest part of the skull, the criteria of avoiding blood vessels and / or specific regions can be determined based on the blood vessel surface grid and individual brain atlas, the criteria of passing through specific regions can be determined based on individual brain atlas, and the criteria of avoiding sulci can be determined based on the gray / white matter surface grid.

[0196] Based on the established evaluation criteria and their weights, the skull outer surface identified in step S704, and the user-selected entry point range received in step S705, the objective function is optimized using the solver selected by the user in step S706 to obtain multiple candidate paths. In step S707, the user-selected path and the user's adjustments and optimizations to the path can be received to obtain the final planned path.

[0197] The surgical path planning method according to the embodiments of this disclosure can utilize various medical imaging data such as structural magnetic resonance imaging (sMRI), CT, TOF-MRA to provide comprehensive anatomical and functional information. For example, it can obtain image data such as skull surface mesh (generated from CT images) and vascular surface mesh (generated from TOF-MRA or other vascular imaging technologies) to achieve the fusion of multimodal image data and support high-precision path planning.

[0198] Furthermore, the surgical path planning method according to embodiments of this disclosure can comprehensively consider multiple optimization conditions such as shortest distance, perpendicularity to the skull, thinnest skull, avoidance of blood vessels and important brain regions, passage through specific areas, and avoidance of cerebral sulci, so that the final path helps to improve the safety, success rate, and efficiency of the surgery. In addition, this method also allows users to adjust the weights of each criterion according to their needs, achieving personalized path planning.

[0199] Furthermore, the surgical path planning method according to the embodiments of this disclosure can employ efficient optimization methods such as the Levenberg-Marquardt algorithm and simulated annealing algorithm to achieve fast and accurate path planning, generate the optimal path, and meet the needs of real-time interaction.

[0200] Furthermore, the surgical path planning method according to the embodiments of this disclosure can also provide a user-friendly and intuitive three-dimensional visualization interface, support manual selection of the target location and the range of the entry point, and support real-time preview and adjustment of the planning results, thereby improving user experience and control.

[0201] Furthermore, the surgical path planning method according to the embodiments of this disclosure achieves high-precision image registration during multimodal image registration by directly adjusting the orientation matrix and origin of the image, preserving the detailed information of the original image, eliminating the need for resampling, ensuring high-precision image alignment, and avoiding the problem of accuracy loss.

[0202] Furthermore, the surgical path planning method according to the embodiments of this disclosure can realize an end-to-end workflow. Specifically, it can form a complete automated process from reading, processing, and registering multimodal images to path planning, reducing manual intervention and operational complexity, and improving work efficiency.

[0203] The surgical pathway planning method according to embodiments of this disclosure relates to the fields of neuroscience, medical image processing, and computer-aided surgical planning, particularly to preoperative pathway planning methods for intracranial drug delivery and implantation surgeries, such as those performed in large animal experiments. This method integrates multimodal brain imaging (e.g., CT, MRI, TOF-MRA, etc.) to achieve high-precision brain structure registration and personalized surgical pathway planning. By comprehensively considering multiple evaluation criteria, it automatically generates the optimal surgical pathway, improving the accuracy and safety of the surgery.

[0204] According to a second aspect of the embodiments of this disclosure, a surgical path planning device based on multimodal image registration is provided, such as... Figure 8 As shown, the surgical path planning device may include an acquisition unit, a registration unit, a receiving unit, and a planning unit.

[0205] The acquisition unit 810 is configured to acquire multimodal image data of the target cranial structure, wherein the multimodal image data includes multiple images captured of the target cranial structure, and the multiple images are captured in different ways.

[0206] The registration unit 820 is configured to perform image registration on multiple images to obtain registered multimodal image data.

[0207] The receiving unit 830 is configured to receive the target location input by the user for the registered multimodal image data.

[0208] The planning unit 840 is configured to plan the path from the outside of the target cranial structure to the target location based on the registered multimodal image data, thereby obtaining the planned surgical path.

[0209] As an example, the planning unit 840 is also configured to obtain the planned surgical path by: determining the entry point on the outer surface of the target cranial structure based on registered multimodal image data under preset optimization conditions; using the straight-line path from the entry point to the target location as the planned surgical path, wherein the optimization conditions include at least one of the following: the path between the entry point and the target location is the shortest; the direction of the path between the entry point and the target location is perpendicular to the skull surface of the target cranial structure; the skull thickness at the entry point is the smallest; the path between the entry point and the target location is the largest distance from a preset avoidance area; the path between the entry point and the target location is the smallest distance from a preset pathway area; and the path between the entry point and the target location penetrates the sulcus region to the smallest depth.

[0210] As an example, there are multiple optimization conditions. The planning unit 840 is also configured to: construct an objective function based on each optimization condition and the preset weights corresponding to each optimization condition, wherein the objective function represents the summation of the scores of the intracranial entry position under each optimization condition using the preset weights; solve for the optimal solution of the objective function for the position within the preset intracranial entry position region on the outer surface of the target cranial structure; and determine the position corresponding to the optimal solution as the intracranial entry position.

[0211] As an example, the planning unit 840 is also configured to determine the distance between the path from the cranial incision location to the target location and a preset avoidance area by: determining multiple intermediate points on the straight path from the cranial incision location to the target location; and taking the minimum distance from each intermediate point to the avoidance area as the distance between the path and the avoidance area, wherein the distance between the path from the cranial incision location to the target location and a preset pathway area is determined by: determining multiple intermediate points on the straight path from the cranial incision location to the target location; and taking the minimum distance from each intermediate point to the pathway area as the distance between the path and the pathway area.

[0212] As an example, multiple images include CT images, MRI images, and vascular images. The surgical path planning device further includes a surface mesh determination unit, which is configured to: determine the vascular surface mesh of the target cranial structure based on the vascular images, and determine the avoidance area based on the vascular surface mesh; and / or, determine the skull surface mesh of the target cranial structure based on the CT images, and determine the skull surface and skull thickness based on the skull surface mesh; and / or, determine the gray and white matter surface mesh of the target cranial structure based on the MRI images, and determine the depth of the sulcus region based on the gray and white matter surface mesh.

[0213] As an example, multiple images include CT images, MRI images, and vascular images. The registration unit 820 is further configured to obtain the registered multimodal image data by: performing a first image registration on the CT images and MRI images to obtain a first registration relationship; performing a first image registration on the vascular images and MRI images to obtain a second registration relationship; performing a second image registration on the MRI images and a reference brain atlas template to obtain a third registration relationship; and obtaining the registered multimodal image data based on the first, second, and third registration relationships, wherein the first image registration is performed by: determining the images to be registered... The rotation and translation transformations between images are adjusted to maximize the similarity between the images to be registered, thus obtaining the registration relationship between the images to be registered. The second image registration is performed as follows: a first transformation relationship is determined to maximize the similarity between the MRI image and the reference brain atlas template; a second transformation relationship is determined to align each voxel between the MRI image and the reference brain atlas template; based on the first and second transformation relationships, a third registration relationship is determined, where the third registration relationship represents the correspondence between each brain region of the MRI image and the reference brain atlas template.

[0214] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method. The training system can execute the corresponding steps in the method according to the method embodiments of the first and second aspects above, which will not be elaborated here.

[0215] Figure 9 This is a block diagram of an electronic device according to exemplary embodiments of the present disclosure. Figure 9 As shown, the electronic device 10 includes a processor 101 and a memory 102 for storing processor-executable instructions. Here, when executed by the processor, the processor executes a training method for an image segmentation model or an image segmentation method as described in the exemplary embodiments above.

[0216] As an example, electronic device 10 is not necessarily a single device, but can be a collection of any means or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 10 can also be part of an integrated control system or system manager, or can be configured to interconnect with a server locally or remotely (e.g., via wireless transmission).

[0217] In electronic device 10, processor 101 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 101 may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0218] The processor 101 can execute instructions or code stored in the memory 102, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transmission protocol.

[0219] The memory 102 may be integrated with the processor 101, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 102 may include a separate device, such as an external disk drive, a storage array, or other storage device that can be used by any database system. The memory 102 and the processor 101 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 101 to read files stored in the memory 102.

[0220] In addition, the electronic device 10 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 10 may be interconnected via a bus and / or network.

[0221] In an exemplary embodiment, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the training method or image segmentation method of the image segmentation model as described in the exemplary embodiment above. The computer-readable storage medium may be, for example, a memory including instructions. Optionally, the computer-readable storage medium may be: a read-only memory (ROM), a random access memory (RAM), a random access programmable read-only memory (PROM), an electrically erasable programmable read-only memory (EEPROM), a dynamic random access memory (DRAM), a static random access memory (SRAM), flash memory, non-volatile memory, a CD-ROM, a CD-R, a CD+R, a CD-RW, a CD+RW, a DVD-ROM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, a DVD-RAM, a BD-ROM, a BD-R, or a BD-R... LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0222] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, the computer program product including computer-executable instructions, which, when executed by at least one processor, implement a training method or an image segmentation method for an image segmentation model according to exemplary embodiments of the present disclosure.

[0223] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0224] Furthermore, it should be noted that although several examples of each step have been described above with reference to the specific accompanying drawings, it should be understood that the embodiments of this disclosure are not limited to the combinations given in the examples. The steps appearing in different drawings can be combined, and the execution order of each step can be changed, which will not be exhaustive here.

[0225] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A surgical path planning method based on multi-modal image registration, characterized in that, The surgical path planning method comprises: acquiring multi-modal image data of a target cranial structure, wherein the multi-modal image data comprises a plurality of images captured in different ways for the target cranial structure; performing image registration on the plurality of images to obtain registered multi-modal image data; receiving a target position input by a user for the registered multi-modal image data; planning a path from outside the target cranial structure to the target position based on the registered multi-modal image data to obtain a planned surgical path, wherein the planned surgical path is obtained by: determining an intracranial position on the outer surface of the target cranial structure based on the registered multi-modal image data under preset optimization conditions; regarding a straight line path from the intracranial position to the target position as the planned surgical path, wherein the optimization conditions comprise at least one of: the path between the intracranial position and the target position is the shortest; the direction of the path between the intracranial position and the target position is perpendicular to the skull surface of the target cranial structure; the skull thickness at the intracranial position is the smallest; the distance of the path between the intracranial position and the target position to a preset avoidance region is the largest; the distance of the path between the intracranial position and the target position to a preset passage region is the smallest; the depth of the path between the intracranial position and the target position passing through a sulcus region is the smallest, wherein the optimization conditions are multiple, and wherein the determination of the intracranial position on the outer surface of the target cranial structure based on the registered multi-modal image data under the preset optimization conditions comprises: constructing an objective function based on each optimization condition and a preset weight corresponding to each optimization condition, wherein the objective function represents the sum of the scores of the intracranial position under each optimization condition weighted by the preset weight; solving the optimal solution of the objective function for positions in a preset intracranial position region of the outer surface of the target cranial structure; determining the position corresponding to the optimal solution as the intracranial position.

2. The surgical path planning method of claim 1, wherein, The distance of the path between the intracranial position and the target position to a preset avoidance region is determined by: determining a plurality of intermediate points on the straight line path from the intracranial position to the target position; regarding the minimum distance among the distances from each intermediate point to the avoidance region as the distance between the path and the avoidance region, wherein the distance of the path between the intracranial position and the target position to a preset passage region is determined by: determining a plurality of intermediate points on the straight line path from the intracranial position to the target position; regarding the minimum distance among the distances from each intermediate point to the passage region as the distance between the path and the passage region.

3. The surgical path planning method of claim 1, wherein, The plurality of images comprises CT images, MRI images and blood vessel images, and the surgical path planning method further comprises: determining a blood vessel surface mesh of the target cranial structure based on the blood vessel images, determining the avoidance region based on the blood vessel surface mesh; and / or, determine a skull surface mesh of the target cranial structure based on the CT image, determine the skull surface and the skull thickness based on the skull surface mesh; and / or, determine a gray-white matter surface mesh of the target cranial structure based on the MRI image, determine the depth of the sulcus region based on the gray-white matter surface mesh.

4. The surgical path planning method of claim 1, wherein, the plurality of images include a CT image, an MRI image, and a blood vessel image, wherein the registered multi-modal image data is obtained by: performing first image registration on the CT image and the MRI image to obtain a first registration relationship; performing the first image registration on the blood vessel image and the MRI image to obtain a second registration relationship; performing second image registration on the MRI image and a reference brain atlas template to obtain a third registration relationship; obtaining the registered multi-modal image data based on the first registration relationship, the second registration relationship, and the third registration relationship, wherein the first image registration is performed by: determining a rotation transformation amount and a translation transformation amount between images to be registered, adjusting the rotation transformation amount and the translation transformation amount to maximize the similarity between the images to be registered, and obtaining a registration relationship between the images to be registered; wherein the second image registration is performed by: determining a first transformation relationship that maximizes the similarity between the MRI image and the reference brain atlas template; determining a second transformation relationship that aligns each voxel between the MRI image and the reference brain atlas template, determining the third registration relationship based on the first transformation relationship and the second transformation relationship, wherein the third registration relationship represents the correspondence relationship between each brain region of the MRI image and the reference brain atlas template.

5. A surgical path planning device based on multi-modal image registration, characterized by, The surgical path planning device includes: an acquisition unit configured to acquire multi-modal image data of a target cranial structure, wherein the multi-modal image data includes a plurality of images captured for the target cranial structure, and the plurality of images are captured in different ways; a registration unit configured to perform image registration on the plurality of images to obtain registered multi-modal image data; a receiving unit configured to receive a target position input by a user for the registered multi-modal image data; a planning unit configured to plan a path from the outside of the target cranial structure to the target position based on the registered multi-modal image data to obtain a planned surgical path, wherein the planning unit is configured to obtain the planned surgical path by: determining an intracranial position on the outer surface of the target cranial structure based on the registered multi-modal image data under a preset optimization condition; and taking a straight line path from the intracranial position to the target position as the planned surgical path. The optimization condition comprises at least one of the following: a path between the intracranial position and the target position is shortest; a direction of the path between the intracranial position and the target position is perpendicular to a skull surface of the target cranial brain structure; a skull thickness at the intracranial position is smallest; a distance of the path between the intracranial position and the target position to a preset avoidance region is largest; a distance of the path between the intracranial position and the target position to a preset passage region is smallest; a depth of the path between the intracranial position and the target position passing through a sulcus region is smallest, The optimization condition is multiple, and the planning unit is further configured to: construct a target function based on each optimization condition and a preset weight corresponding to each optimization condition, wherein the target function represents a sum of scores of the intracranial position under each optimization condition weighted by the preset weight; solve an optimal solution of the target function for positions in a preset intracranial position region of an outer surface of the target cranial brain structure; and determine a position corresponding to the optimal solution as the intracranial position.

6. An electronic device, comprising: The electronic device comprises: a processor; a memory for storing processor-executable instructions, wherein the processor-executable instructions, when executed by the processor, cause the processor to perform the surgical path planning method based on multi-modal image registration according to any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the surgical path planning method based on multi-modal image registration according to any one of claims 1 to 4.

8. A computer program product comprising computer executable instructions, characterised in that, The computer-executable instructions, when executed by at least one processor, implement the surgical path planning method based on multi-modal image registration according to any one of claims 1 to 4.

Images