Intraoperative electrode positioning method
By generating an intraoperative reference brain surface during neurosurgery and performing global constraint optimization and confidence-weighted fusion, the problem of inaccurate intraoperative positioning of electrode contacts was solved, and a robust transformation from intraoperative two-dimensional observation to preoperative medical imaging coordinate system was achieved, improving the reliability and verifiability of positioning.
Patent Information
- Application Number
- CN202611592960.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-10-15
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies cannot reliably and accurately position intraoperative electrode contacts to the preoperative medical imaging coordinate system during neurosurgery. This results in problems such as inaccurate two-dimensional coordinates, lack of pre-placement reference benchmarks, error accumulation, and unstable positioning.
By generating an intraoperative reference brain surface before electrode placement, establishing spatial constraint relationships using multiple sets of intraoperative surface observation data, performing global constraint optimization and confidence-weighted fusion, and combining imaging rays and array topological constraints, the three-dimensional positioning and transformation of electrode contacts are achieved.
In the absence of stable external tracking coordinates, the reliability and verifiability of electrode contact positioning are improved, error propagation is reduced, and the complete transformation from intraoperative two-dimensional observation to preoperative medical imaging coordinate system is ensured.
Smart Images

Figure CN123058835A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of medical image processing and computer vision, and in particular to a method for intraoperative electrode positioning. Background Technology
[0002] In neurosurgical procedures and brain-computer interface (BCI) implantation, it is crucial to accurately locate the electrode contacts placed on the patient's brain surface within the preoperative medical imaging coordinate system. This allows for the establishment of a spatial correspondence between electrophysiological signals and brain anatomical structures. Currently, acquiring intraoperative brain surface data typically involves using equipment such as stereo vision, structured light, or depth cameras to capture images containing both appearance and geometric information. Image recognition technology is then used to extract the two-dimensional coordinates of the electrode contacts from the intraoperative images. Simultaneously, medical image registration technology can utilize anatomical landmarks, surface point clouds, or vascular information to register the intraoperatively acquired surface data onto a three-dimensional model created from preoperative medical images.
[0003] However, existing solutions have the following technical problems: First, they rely solely on single or multiple intraoperative photographs for 2D image recognition and point-by-point extension, outputting only the 2D position in the photographic coordinate system. They fail to acquire brain surface depth or 3D geometric information, nor do they calculate the 3D spatial relationship between the imaging device and the brain surface. This results in different 2D coordinates for the same electrode at different shooting distances and viewing angles, making it impossible to determine the electrode's 3D position relative to the actual brain surface or directly convert it to the preoperative medical imaging coordinate system. Second, existing solutions typically process craniotomy images containing the electrodes directly after placement, without acquiring and fusing brain surface data before placement to establish an independent intraoperative reference brain surface. Since changes in the brain surface occur after electrode placement due to electrode pads, wire obstruction, and cerebrospinal fluid loss, the lack of a pre-placement reference makes it impossible to determine changes in the post-placement observation relative to the original intraoperative state. Third, for intraoperative observations acquired in multiple discrete sets, if registration is carried out by accumulating images one by one in the order of shooting, errors in intermediate steps will propagate backward, which can easily lead to cumulative drift, double-layer brain surface or texture ghosting. Moreover, when the reflection of metal contacts, transparent electrode sheets or brain sulci cause depth loss or non-zero error depth, there is a lack of multi-constraint recovery methods that combine imaging rays, deformed brain surfaces and array topology, making it difficult to guarantee the reliability of three-dimensional positioning of contacts.
[0004] Therefore, there is an urgent need for an intraoperative electrode positioning method that can robustly establish an intraoperative reference brain surface and precisely position the electrode contacts to the preoperative medical imaging coordinate system in the absence of stable external tracking coordinates. Summary of the Invention
[0005] Therefore, it is necessary to provide an intraoperative electrode positioning method to address the problem that existing technologies cannot reliably establish a reference brain surface during surgery and transform the three-dimensional coordinates of electrode contacts to the preoperative medical imaging coordinate system, so as to achieve a complete spatial transformation from intraoperative two-dimensional observation, the current brain surface, the reference brain surface before placement, to the preoperative medical imaging coordinate system.
[0006] This invention provides an intraoperative electrode positioning method, comprising: Based on the calibration parameters of the acquired imaging device, the spatial constraint relationship between multiple sets of intraoperative surface observation data before electrode placement is determined, and an intraoperative reference brain surface is generated through global constraint optimization and confidence weighted fusion; wherein, each set of intraoperative surface observation data includes appearance observation data and three-dimensional geometric observation data; The intraoperative reference brain surface is registered with the acquired preoperative medical image model to obtain the image spatial mapping relationship; Multiple sets of intraoperative surface observation data after electrode placement are repositioned relative to the intraoperative reference brain surface to determine the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface, and the state of the electrode array. The spatial relationship of the imaging device, the brain surface deformation, and the state of the electrode array are jointly estimated to obtain the candidate three-dimensional position of the electrode contact. When the depth data of the target contact area does not meet the depth reliability condition, an imaging ray is generated based on the two-dimensional observation of the target contact and the calibration parameters. The three-dimensional position of the target contact is determined by combining the spatial relationship between the imaging ray and the current brain surface or electrode array support surface, array topology constraints, and thickness or gap constraints. Based on the deformation correspondence between the current brain surface and the intraoperative reference brain surface, the three-dimensional position of each electrode contact is back-mapped to the intraoperative reference brain surface, and converted to the preoperative medical image coordinate system based on the image space mapping relationship, outputting the three-dimensional coordinates of each electrode contact and its corresponding evidence source, confidence level or processing status.
[0007] In one embodiment, the generation of an intraoperative reference brain surface through global constraint optimization and confidence-weighted fusion includes: Each set of intraoperative surface observation data before electrode placement, or the local surface formed by the observation data, is used as an observation node. Spatial constraint edges are established between observation nodes with reliable overlap to form an observation constraint map. The spatial constraint edges include constraint edges between observation nodes with adjacent shooting order, and constraint edges between observation nodes with non-adjacent shooting order but observing the same brain region. Based on the observation constraint diagram, the spatial relationship between each observation node and the unified reference coordinate system is globally optimized. The multiple sets of intraoperative surface observation data before electrode placement, which were globally optimized, were converted to the unified reference coordinate system, and weighted and fused based on the confidence level of each set of observation data to generate the intraoperative reference brain surface.
[0008] In one embodiment, the multiple sets of intraoperative surface observation data after electrode placement are repositioned relative to the intraoperative reference brain surface to determine the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface, and the state of the electrode array, and are jointly estimated, including: For each set of intraoperative surface observation data after electrode placement, the spatial relationship between the intraoperative reference brain surface and the intraoperative reference brain surface is calculated to determine the spatial relationship of the imaging device. Based on the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface and the state of the electrode array are determined; wherein, the state of the electrode array includes the overall pose and local morphology of the electrode array relative to the current brain surface. Based on the observation data of the electrode contacts, the brain surface deformation, the state of the electrode array, and the contact constraints between the contacts and the deformed current brain surface, joint optimization is performed to solve the three-dimensional position of each electrode contact on the current brain surface, which is then used as the candidate three-dimensional position.
[0009] In one embodiment, determining the three-dimensional position of the target contact point when the depth data of the target contact point region does not meet the depth reliability condition includes: Based on the two-dimensional center of the target contact point and the calibration parameters, an imaging ray is generated that passes through the target contact point from the imaging center. Based on the brain surface deformation, determine the intersection point or optimal proximity position of the imaging ray with the current brain surface or the electrode array support surface; Based on the local brain surface normal at the intersection or optimal proximity position, the electrode sheet thickness, and the thickness or gap constraint, the position of the intersection or optimal proximity position is corrected. Based on the array topology constraints and contact point spacing constraints, the three-dimensional position of the target contact point is jointly optimized.
[0010] In one embodiment, the step of globally optimizing the spatial relationships of each observation node to the unified reference coordinate system based on the observation constraint graph includes: The edge confidence weight of each of the spatial constraint edges is determined based on at least one of the following: matching interior point ratio, spatial coverage, and 3D registration residual. The translation and rotation residuals between the two sets of observations are normalized using an information matrix to construct a global optimization objective function; the expression of the global optimization objective function is as follows: , in, The objective function value for global pose optimization; The set of spatial constraint edges formed by pairs of observation nodes with reliable overlapping relationships in the observation constraint graph. The number of spatial constraint edges; This indicates traversing each pair of observation nodes in the set of spatial constraint edges; For the first Group and No. The edge confidence weights of spatial constraint edges between observation groups; This is an information matrix used to unify the dimensions of translation and rotation errors; The edge residual vector is calculated from the two sets of observation transformation relationships; A robust loss function; Fix a reference node to eliminate the overall degrees of freedom, and iteratively update the spatial relationship of each observation node until the convergence condition is met; The formula for calculating the edge credibility weight is as follows: , In the formula, For the proportion of interior points corresponding to appearance or depth, This refers to the spatial coverage index of the corresponding point. The three-dimensional registration residual between the two sets of observations, The scale parameters are determined based on the measured depth noise of the imaging device.
[0011] In one embodiment, the brain-face deformation is represented by a control node displacement field, the electrode array state is represented by a global affine component and a non-rigid component, and the jointly estimated energy function includes: The expression for the brain-face deformation is: , in, For the spatial points on the current brain surface after deformation coordinates For the first basis functions This represents the three-dimensional displacement vector of the corresponding control node. To control the total number of nodes; The expression for the brain-face deformation smoothing constraint is: , in, The set that controls the adjacency relationships of nodes. For the first The three-dimensional displacement vector of each control node. These are the regularization weight coefficients; The deformation expression for the state of the electrode array is: , in, Initial position of the contact Position after deformation The three-dimensional coordinates of the contact points under the initial regular shape of the electrode array. Let be the matrix representing the overall affine transformation of the electrode array relative to its initial regular shape. It is a translation vector. The total number of control points. It is a non-rigid deformation coefficient. For the first The location of each control point are basis functions; The joint estimation energy function includes: , in, The surface registration residual between the deformed current brain surface and the observation after placement; These are the weighting coefficients for the brain-surface deformation smoothing constraint; These are the weighting coefficients for the array topology spacing constraints; This diagram illustrates the connection relationships between adjacent contacts in the array. and These are the first two electrodes in the initial regular shape of the electrode array. The and the first The three-dimensional coordinates of each contact point and The first The and the first The position of each contact point after deformation; The nominal spacing between adjacent contacts is known. The weighting coefficients for the distance constraint between the contact point and the brain surface; Contact after deformation To the deformed current brain surface The shortest distance; This is the thickness or gap compensation value of the contact center relative to the brain surface.
[0012] In one embodiment, determining the three-dimensional position of the target contact point further includes: When the center pixel depth of the target touch point is missing, the effective depth within the preset annular neighborhood is read, and the median of the effective depth is used to replace the center depth. When an effective depth cannot be obtained within the preset annular neighborhood, the intersection point of the imaging ray and the deformed current brain surface mesh is calculated, and the effective intersection point closest to the imaging center along the ray direction is selected as the initial three-dimensional position; the expression for the imaging ray is: , , in, For the first Spatial ray vectors of each contact point The three-dimensional coordinates of the optical center of the imaging device. The unit direction vector is determined by the intrinsic parameters of the imaging device and the center of the two-dimensional image of the contact point. This is the distance parameter along the ray direction; The expression for the position correction is: , in, The first position obtained after position correction The three-dimensional coordinates of each electrode contact The initial three-dimensional position is obtained by intersecting the imaging ray with the deformed current brain surface mesh. This is the proportionality coefficient. For electrode sheet thickness, This is the estimated gap between the electrode pads and the current brain surface. The current normal to the brain surface at the intersection point; The energy function of the joint estimation is: , in, For joint estimation of the energy function; These are the weighting coefficients for the observation constraint terms; A set of touchpoints for direct observational evidence with depth or a two-dimensional center; For the first The observation confidence weight of each contact point; The first one to be optimized The three-dimensional coordinates of each contact point; For the first The initial three-dimensional position estimate of each contact point is obtained based on the replaced depth back projection or the position correction. Sure; The array spacing constraint weights; This is a diagram showing the connection relationships between adjacent contacts in the array. The nominal spacing between adjacent contacts is known. Weights for local curvature smoothing constraints; This is a set of local neighborhood triplet indexes consisting of three consecutive touchpoints in the same row or column, where... As the central contact point, and Neighboring points; The weighting coefficients for the distance constraint between the contact point and the brain surface; This is the current brain surface model after thickness and gap correction; for occluded touch points, their observation confidence weights are... Set to zero, and recover its three-dimensional position by the spacing between adjacent contacts and the array topology constraints.
[0013] In one embodiment, the confidence-weighted fusion includes: Calculate the continuity confidence for at least one of the following: pixel, depth point, feature correspondence, surface region, whole group of observations, or spatial constraint edge; The continuous confidence level is determined based on at least one of the following: depth validity, depth noise, multi-frame depth consistency, observation distance, observation angle, local surface orientation, overexposure, reflection, occlusion, and cross-observation registration residual. When the continuous confidence level is lower than the first threshold, the corresponding data is rejected from participating in the fusion; when the continuous confidence level is between the first threshold and the second threshold, the fusion weight of the corresponding data is reduced; when the continuous confidence level is higher than the second threshold, the data participates in the fusion with normal weight.
[0014] In one embodiment, each set of intraoperative surface observation data after electrode placement calculates its spatial relationship relative to the intraoperative reference brain surface based on the intraoperative reference brain surface, including: Based on direct evidence between the current observation data and the intraoperative reference brain surface or the natural landmarks preserved in the intraoperative reference brain surface, obtain or verify the spatial relationship of the current observation data; The direct evidence includes at least one of the following: corresponding quantity, spatial coverage, reprojection error, three-dimensional point-to-surface residual, and consistency with multipath constraints. When the direct evidence does not meet the preset conditions, the current observation data will be marked as downgraded, unreliable, relocated, or requiring manual verification. The positioning results of the observation data after the previous set of electrodes were placed are used as the initial values for calculating the spatial relationships of the current observation data.
[0015] In one embodiment, registering the intraoperative reference brain surface with the acquired preoperative medical image model includes: Based on at least one of the three-dimensional geometry, surface normal, curvature, sulcus structure, vascular landmarks, and surgical area prior of the intraoperative reference brain surface, the intraoperative reference brain surface is registered with the preoperative medical image model. The output of the three-dimensional coordinates of each electrode contact and its corresponding evidence source, confidence level, or processing status also includes: Output the uncertainty of the three-dimensional coordinates of each electrode contact; the uncertainty is based on at least two of the following: calibration error, depth noise, cross-observation registration error, reference brain surface fusion error, preoperative image registration error, brain surface deformation error, contact detection error, array size or thickness error, and occlusion contact estimation error. The sources of evidence include direct depth measurements, multi-observation joint reconstruction, or array topology estimation.
[0016] Compared with the prior art, the present invention has at least the following beneficial effects: The aforementioned intraoperative electrode localization method establishes spatial constraints between observations using multiple sets of intraoperative surface observation data before electrode placement. A global constraint optimization and confidence-weighted fusion are then used to generate an intraoperative reference brain surface. This reference brain surface serves simultaneously as a benchmark for registration with the preoperative medical imaging model and as a common benchmark for relocation of each set of observation data after electrode placement. This avoids incorporating post-placement brain surface changes caused by electrode occlusion or cerebrospinal fluid loss into the pre-placement reference model, thus maintaining the uniformity of the spatial benchmark even in the absence of stable external tracking coordinates. By relocating each set of post-placement observation data relative to the intraoperative reference brain surface and separately representing and jointly estimating the spatial relationships of the imaging device, brain surface deformation, and electrode array state, the mutual influence between camera motion, brain surface deformation, and electrode array deformation is decoupled, preventing error aliasing during the joint solution process. When the depth data of the target contact area fails to meet the depth reliability conditions due to metallic reflection, transparent substrate, or brain sulcus gaps, an imaging ray is generated based on two-dimensional observation and intersected with the deformed current brain surface or electrode array support surface. Then, position correction and joint optimization are performed by combining array topological constraints and thickness or gap constraints. Geometric rays, deformed surfaces, and array priors are used to replace the failed depth information, restoring the three-dimensional position of the contact. Finally, based on the deformation correspondence, the contact is backmapped to the intraoperative reference brain surface and transformed to the preoperative medical image coordinate system. Simultaneously, the evidence source, confidence level, or processing status is output, forming a complete conversion link from intraoperative two-dimensional observation to preoperative medical image three-dimensional coordinates. This reduces error propagation caused by accumulating errors according to the shooting sequence or recursively extrapolating point by point, improving the reliability and verifiability of electrode contact positioning in complex intraoperative environments. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0018] Figure 1This is a flowchart illustrating the intraoperative electrode positioning method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the process for generating an intraoperative reference brain surface according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the observation relocation and three-state joint estimation process after electrode placement in an embodiment of the present invention; Figure 4 This is a schematic diagram of the process for restoring the three-dimensional position of the contact point when the depth is unreliable according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the architecture of the intraoperative electrode positioning system according to an embodiment of the present invention. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.
[0021] The following is combined Figures 1-5 The intraoperative electrode positioning method of the present invention is described.
[0022] like Figure 1 As shown, in one embodiment, an intraoperative electrode positioning method is provided, the method comprising the following steps: Step S110: Based on the calibration parameters of the acquired imaging device, determine the spatial constraint relationship between multiple sets of intraoperative surface observation data before electrode placement, and generate an intraoperative reference brain surface through global constraint optimization and confidence weighted fusion; wherein, each set of intraoperative surface observation data includes appearance observation data and three-dimensional geometric observation data.
[0023] The calibration parameters of the imaging device include intrinsic parameters of the appearance camera, intrinsic parameters of the depth camera or 3D measurement device, lens distortion parameters, extrinsic parameters between the appearance device and the geometric device, depth units, depth scale, coordinate axis orientation, pixel alignment, and temporal correspondence between different data. After craniotomy and before electrode placement, the brain surface is in a relatively stable state. At this time, multiple sets of discrete intraoperative surface observation data with overlapping areas are collected from different positions and observation angles. For example, five to eight sets of discrete observation data can be collected; this number is only one implementation example, and in other embodiments, the number of sets can be increased or decreased according to the actual surgical field of view and equipment conditions. Each set of observation data simultaneously includes appearance observation data reflecting the brain surface texture, color, or brightness, as well as three-dimensional geometric observation data reflecting the spatial morphology of the brain surface. Appearance observation data can be acquired by ordinary color cameras, grayscale cameras, surgical microscopes, cross-polarization imaging devices, near-infrared reflectance imaging devices, or multispectral imaging devices; three-dimensional geometric observation data can be acquired by depth cameras, binocular or multi-view stereo vision devices, structured light 3D measurement devices, time-of-flight depth imaging devices, or laser scanning devices. Appearance and geometric information can be obtained simultaneously by the same composite device, or separately by different devices and then spatially correlated through pre-calibration. The intraoperative reference brain surface generated in this embodiment is a unified three-dimensional reference model established before electrode placement when the brain surface is relatively stable. It is neither a point cloud directly generated from a single photograph, nor a surface obtained from preoperative medical image segmentation, nor a hybrid model of data before and after electrode placement. By establishing this independent reference benchmark before placement, changes in the brain surface caused by subsequent electrode occlusion, wire interference, or cerebrospinal fluid loss can be avoided from contaminating the original reference model, thus providing a stable spatial benchmark for subsequent cross-modal registration and post-placement observation.
[0024] Step S120: Register the intraoperative reference brain surface with the acquired preoperative medical image model to obtain the image spatial mapping relationship.
[0025] Preoperative medical imaging models are three-dimensional models of the cortical pia mater or brain surface based on MRI or other medical imaging data acquired before surgery. Since the intraoperative reference brain surface contains the three-dimensional geometry of the actual exposed brain surface, while the preoperative medical imaging model contains individualized anatomical information of the patient, spatial differences exist between the two due to craniotomy, cerebrospinal fluid loss, and changes in body position. By registering the intraoperative reference brain surface with the preoperative medical imaging model, the transformation relationship from the coordinate system of the intraoperative reference brain surface to the coordinate system of the preoperative medical imaging model can be calculated, i.e., the image-space mapping relationship. This mapping relationship allows any three-dimensional position subsequently determined on the intraoperative reference brain surface to be transformed into the patient's preoperative anatomical coordinate system, thereby achieving cross-modal spatial alignment between intraoperative observation and preoperative planning.
[0026] Step S130: Relocate multiple sets of intraoperative surface observation data after electrode placement relative to the intraoperative reference brain surface to determine the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface, and the state of the electrode array. Then, jointly estimate the spatial relationship of the imaging device, the brain surface deformation, and the state of the electrode array to obtain the candidate three-dimensional position of the electrode contact.
[0027] After the doctor places the electrode array on the brain surface, the brain surface may shift or deform relative to its pre-placement state due to pressure from the electrode pads, further loss of cerebrospinal fluid, or gravity. Simultaneously, the electrode array itself may also undergo translation, rotation, or local bending. Therefore, multiple sets of intraoperative surface observation data acquired after electrode placement cannot be directly fused into the intraoperative reference brain surface generated in step S110 as a single rigid surface. Instead, they are processed as the current state relative to the intraoperative reference brain surface. For each set of post-placement observation data, the spatial relationship between it and the intraoperative reference brain surface is calculated to determine the spatial relationship of the current imaging device. Based on this, the brain surface deformation relative to the intraoperative reference brain surface, as well as the overall pose and local morphology of the electrode array relative to the current brain surface, are represented and estimated. Subsequently, using the observation data of the electrode contacts, brain surface deformation, electrode array state, and contact constraints between the contacts and the deformed brain surface, a joint estimation is performed to solve for the candidate three-dimensional positions of each electrode contact on the current brain surface. This approach decouples camera motion, brain-face deformation, and electrode array deformation before performing joint optimization. This avoids mixing post-placement brain-face changes and array deformation into the pre-placement reference model, and also prevents contradictory results from occurring when the three are calculated separately.
[0028] Step S140: When the depth data of the target contact area does not meet the depth reliability condition, an imaging ray is generated based on the two-dimensional observation and calibration parameters of the target contact. The three-dimensional position of the target contact is determined by combining the spatial relationship between the imaging ray and the current brain surface or electrode array support surface, array topology constraints, and thickness or gap constraints.
[0029] In real surgical environments, metal contacts are prone to specular reflection, transparent or semi-transparent electrode array substrates may prevent depth cameras from measuring accurately, and physical gaps often exist between the electrode pads and the brain surface or sulci. These factors can all lead to missing depth values for the center pixels of the target contact, or non-zero depth values that are actually incorrect. When the depth data of the target contact area does not meet the depth reliability conditions, the depth reading at that location is no longer directly relied upon. Instead, an imaging ray is constructed using the two-dimensional center coordinates of the target contact in the external image and the calibration parameters of the imaging device, originating from the optical center of the imaging device and passing through that two-dimensional center. Subsequently, the intersection or optimal proximity position of this imaging ray with the current brain surface mesh or the local support surface of the electrode array after deformation estimation is calculated, and the position is corrected by combining the local brain surface normal at the intersection, the thickness of the electrode array substrate, and the estimated gap between the array and the brain surface. Finally, the three-dimensional position of the target contact is jointly optimized using the known row and column relationships of the electrode array, the spacing between adjacent contacts, and other array topological constraints. In this way, even if the original depth information is lost, the three-dimensional position of the contact point can still be recovered by using geometric rays, deformed surfaces and array priors, ensuring the continuity of positioning under complex lighting and occlusion conditions.
[0030] Step S150: Based on the deformation correspondence between the current brain surface and the intraoperative reference brain surface, the three-dimensional position of each electrode contact is back-mapped to the intraoperative reference brain surface, and converted to the preoperative medical image coordinate system based on the image space mapping relationship, and the three-dimensional coordinates of each electrode contact and its corresponding evidence source, confidence level or processing status are output.
[0031] The three-dimensional positions of the electrode contacts determined in steps S130 and S140 are located in the coordinate system of the current brain surface. To correlate these positions with the patient's preoperative anatomical structure, the deformation correspondence between the current brain surface and the intraoperative reference brain surface estimated in step S130 is first used to backmap the three-dimensional positions of each contact back to the intraoperative reference brain surface coordinate system. Next, using the image space mapping relationship obtained in step S120, the backmapped contact coordinates are further transformed to the preoperative medical image coordinate system. The final output includes not only the three-dimensional coordinates of each electrode contact in the preoperative medical image coordinate system, but also the source of evidence for each contact position (e.g., whether the position is directly obtained based on depth measurement, recovered through multiple sets of observations, or estimated based on array topology), the corresponding confidence level, and the processing status (e.g., normal, downgraded, unreliable, requires relocation, or requires manual review). This output method with evidence chains and status indicators allows clinicians to intuitively assess the reliability of each contact location result, providing a verifiable data foundation for subsequent electrophysiological signal analysis and neuromodulation.
[0032] The intraoperative electrode localization method provided in this embodiment establishes a complete transformation link from intraoperative two-dimensional observation, the current brain surface, the pre-placement reference brain surface, to the preoperative medical image coordinate system by constructing an intraoperative reference brain surface before electrode placement. The processing method of independently relocating each set of observations after placement and performing three-state joint estimation reduces the risk of error propagation caused by accumulating errors in the order of shooting or recursively extrapolating point by point. The multi-constraint recovery strategy for deep failure scenarios ensures the reliability of three-dimensional contact localization even under complex conditions such as metallic reflections, transparent substrates, or sulci.
[0033] like Figure 2 As shown, in one embodiment, the specific implementation process of step S110 is described in detail. The process of generating an intraoperative reference brain surface through global constraint optimization and confidence-weighted fusion includes the following steps: Step S201: Take each set of intraoperative surface observation data before electrode placement or the local surface formed by the observation data as observation nodes, and establish spatial constraint edges between observation nodes with reliable overlap to form an observation constraint map; wherein, the spatial constraint edges include constraint edges between observation nodes with adjacent shooting order, and constraint edges between observation nodes with non-adjacent shooting order but observing the same brain region.
[0034] After acquiring multiple sets of discrete intraoperative surface observation data, the subsequent sets of observations are not sequentially registered to the previous sets of observations according to the order of shooting. If spatial relationships are accumulated one by one according to the shooting order, the registration error generated in an intermediate step will continue to be passed on to subsequent observations, which may eventually lead to double-layer structures, texture ghosting, or overall drift in the reconstructed brain surface, and the results will change significantly after changing the first set of reference observations. Therefore, in this embodiment, each set of intraoperative surface observation data itself, or a local surface model pre-reconstructed from a single set of observation data, is treated as an independent observation node. Subsequently, spatial constraint edges are established between all observation nodes with reliable overlapping areas, and these spatial constraint edges together constitute an observation constraint graph. The spatial constraint edges in this observation constraint graph include not only the connections between two observation nodes that are adjacent in shooting time or sequence, but also the connections between nodes that are not adjacent in shooting sequence but actually observed the same brain surface region, as well as the connections between nodes that can form closed loops or multiple alternative paths. For example, although the first set of observations and the fifth set of observations are far apart in terms of acquisition time, if both of their fields of view cover a specific sulcus or vascular branch, then a spatial constraint edge is also established between them. This non-sequential global constraint graph structure means that the spatial relationship between any two sets of observations no longer depends on a single recursive path. When an error exists on a constraint edge, other alternative paths and closed-loop constraints can detect and correct it, thereby suppressing the generation of cumulative drift at the data structure level.
[0035] Step S202: Based on the observation constraint diagram, perform global optimization of the spatial relationship between each observation node and the unified reference coordinate system.
[0036] After constructing the observation constraint graph, the transformation relationships from all observation nodes to the same unified reference coordinate system are jointly solved using global constraint structures such as pose graphs, factor graphs, or bundle adjustment networks. To measure the reliability of spatial constraints between different observation nodes, the edge confidence weight of each spatial constraint edge is determined based on at least one of the following: the proportion of matched inliers, spatial coverage, and 3D registration residuals. The proportion of matched inliers reflects the percentage of consistent data in appearance features or depth correspondences; the spatial coverage index assesses the breadth of corresponding points on the brain surface; if a large number of corresponding points are concentrated in a very small region or in repetitive textures, this index will decrease even if the number is large; the 3D registration residual reflects the magnitude of the geometric deviation after the alignment of two sets of observations. The formula for calculating the edge confidence weight is: , In the formula, For the first Group and No. The edge confidence weights of spatial constraint edges between observation groups; The proportion of interior points corresponding to appearance or depth; This refers to the spatial coverage area index of the corresponding point; The three-dimensional registration residual between the two sets of observations; The scale parameter is determined based on the measured depth noise of the imaging device; for example, it can be two to three times the measured depth noise. Using this formula, constraint edges with smaller registration residuals will receive higher weights, while the weights of constraint edges more affected by depth noise will be smoothly reduced by the exponential term.
[0037] During global optimization, the translation and rotation errors have different physical dimensions, and directly adding them will lead to an imbalance in the optimization direction. Therefore, an information matrix is used to normalize the translation and rotation residuals between the two sets of observations to construct a global optimization objective function. The expression for the global optimization objective function is: , in, The objective function value for global pose optimization; This is the set of spatial constraint edges formed by pairs of observation nodes with reliable overlapping relationships in the observation constraint graph. This represents the number of spatial constraint edges; This represents the pair of observation nodes that are traversed in the set of spatially constrained edges. For the first Group and No. The edge confidence weights of spatial constraint edges between observation groups; For the information matrix used to unify the dimensions of translation and rotation errors, a diagonal matrix form with the reciprocals of the translation error scale and the rotation error scale as diagonal elements can be adopted, for example. The edge residual vector is calculated from the two sets of observation transformation relationships; For robust loss functions, such as the Huber loss function, a quadratic loss is used to ensure smoothness when the residual is small, and linear growth is switched when the residual exceeds a set threshold, thereby reducing the pulling effect of a small number of mismatches or local outliers on the overall optimization result.
[0038] To eliminate the uncertainty of the global coordinate system, a reference node is fixed during the optimization process to eliminate the overall degrees of freedom. For example, the transformation matrix of the first set of observations or the highest-quality set of observations to the unified reference coordinate system is set as the identity matrix. Subsequently, the spatial relationships of each observation node are iteratively updated until the convergence condition is met. As a specific implementation method, the iterative convergence condition can be set to any of the following: the relative change of the objective function between two adjacent iterations is less than... The maximum pose update of all nodes is lower than the set accuracy threshold; or the number of iterations reaches one hundred. Through the above global optimization, the spatial relationship of each group of observations is determined by the entire constraint graph, avoiding the excessive influence of a single reference photo or a single sequential registration path on the final result.
[0039] Step S203: Calculate the continuity confidence for at least one of pixels, depth points, feature correspondences, surface regions, entire observation groups, or spatial constraint edges; the continuity confidence is determined based on at least one of depth validity, depth noise, multi-frame depth consistency, observation distance, observation angle, local surface orientation, overexposure, reflection, occlusion, and cross-observation registration residuals; when the continuity confidence is lower than a first threshold, the corresponding data is rejected from participating in fusion; when the continuity confidence is between the first threshold and a second threshold, the fusion weight of the corresponding data is reduced; when the continuity confidence is higher than the second threshold, the data participates in fusion with normal weights.
[0040] After completing global optimization, the observation data from each group needs to be fused to generate a unified intraoperative reference brain surface. Before fusion, the system does not perform simple binarization to remove anomalies such as reflections, overexposure, erroneous depths, and occlusions. Instead, it calculates continuous confidence scores for data at the pixel level, depth point level, feature correspondence level, surface region level, whole group observation level, and inter-observation constraint boundary level. The reason for using continuous confidence scores instead of simple valid or invalid binary judgments is that in the actual surgical environment, erroneous depths generated by depth cameras do not necessarily manifest as missing zero values; they often produce non-zero erroneous depths that deviate from the true surface. If only binary judgments are used, these non-zero erroneous depths will be written into the fusion model as normal data, thus contaminating the brain surface geometry. By calculating continuous confidence scores, the weights of these erroneous depths that are inconsistent with other high-confidence observations can be smoothly reduced.
[0041] The calculation of continuous confidence considers multiple factors. For example, for a 3D point in a set of observations, its depth validity is first determined. If the depth value is within a reasonable range, it is considered valid; otherwise, the confidence of that point is set to zero. Secondly, for multiple consecutive depth measurements acquired at the same location, the residual between the current depth and the average depth of the multiple frames is calculated. This residual, combined with the depth noise scale determined through camera calibration or actual acquisition, is converted into a depth reliability score. Thirdly, after transforming the point to other overlapping observations, the point-to-plane residual with the corresponding surface is calculated. This residual, combined with the multiple of adjacent measured depth noise as a scale parameter, yields a multi-observation registration consistency score. Finally, the depth reliability score and the multi-observation registration consistency score are weighted and summed to obtain the overall continuous confidence of the point. In addition to point-level confidence, the confidence of pixels or regions can be corrected based on factors such as excessive observation distance, excessively tilted observation angle, presence of local overexposure or specular reflection, and obstruction by gauze or equipment.
[0042] After obtaining the continuous confidence score, a tiered processing method is performed. As a specific implementation, when the continuous confidence score is below a first threshold (e.g., 0.2), the data is considered severely conflicting or completely unreliable and is rejected from subsequent fusion processes. When the continuous confidence score is between the first and second thresholds (e.g., 0.6), the data is retained but its weight in the fusion calculation is reduced. When the continuous confidence score is above the second threshold, it participates in fusion with the normal high-confidence observation weight. The specific values of the first and second thresholds mentioned above are only one implementation example and can be adjusted in practical applications based on the calibration results of the imaging device and the statistical distribution of intraoperative measured data. This tiered weighting or rejection method allows the system to automatically mitigate the negative effects of reflections, overexposure, incorrect depth, and occlusion during the fusion stage without needing to fully identify all anomalies beforehand.
[0043] Step S204: Convert the multiple sets of intraoperative surface observation data before electrode placement, which are globally optimized, to a unified reference coordinate system, and perform weighted fusion based on the confidence level of each set of observation data to generate an intraoperative reference brain surface.
[0044] Following the global optimization in step S202, each set of observation data obtained a precise spatial transformation relationship relative to a unified reference coordinate system. Using these transformation relationships, the 3D geometric data contained in each set of observations were aligned to the same coordinate system. Subsequently, the continuous confidence levels calculated in step S203 were used as fusion weights to perform weighted fusion of the aligned sets of data. The fusion process can employ 3D surface representation methods such as weighted truncated symbolic distance field (TSDF), surface element (Surfel), weighted point cloud fusion, or mesh fusion. For example, in weighted TSDF fusion, the symbolic distance value of each voxel is obtained by weighting the depth measurements of multiple sets of observations at that voxel according to their confidence levels; observations with higher confidence levels contribute more to the final voxel value.
[0045] The fused intraoperative reference brain surface preserves at least the brain surface geometry in a unified coordinate system and confidence information point-by-point or region-by-region. Preferably, it also preserves real brain surface texture, surface normal and curvature information, as well as natural landmarks such as small blood vessels and sulci shadows. To prevent simple averaging of multiple images from blurring small blood vessels and textures, the best original image with a good viewing angle, high clarity, and low reflectivity, along with its pixel coordinates, can be retained for each local region. Furthermore, the intraoperative reference brain surface can record which set of original observations each natural landmark originates from, the suitable viewing direction and scale for each landmark, the depth noise and fusion residual for each region, and the spatial relationship between each set of original observations and the intraoperative reference brain surface. This additional information plays a crucial role in benchmark transfer in subsequent preoperative medical image registration and observation repositioning after electrode placement.
[0046] like Figure 3 As shown, in one embodiment, the specific implementation process of step S130 is described in detail. This embodiment focuses on how the observation data after electrode placement is independently repositioned relative to the intraoperative reference brain surface, and how the spatial relationship of the imaging device, brain surface deformation, and electrode array state are decoupled and jointly estimated.
[0047] Step S301: For each set of intraoperative surface observation data after electrode placement, calculate its spatial relationship relative to the intraoperative reference brain surface based on the intraoperative reference brain surface, so as to determine the spatial relationship of the imaging device.
[0048] After the surgeon places the electrode array, the exposed brain surface may shift or deform relative to its pre-placement state due to factors such as electrode pressure, further loss of cerebrospinal fluid, and gravity. If multiple sets of intraoperative surface observation data collected after electrode placement are directly fused into the intraoperative reference brain surface generated in step S110 as a single rigid surface, the camera's motion trajectory, the actual deformation of the brain surface, and the bending deformation of the electrode pads themselves will be mixed together, disrupting the originally stable pre-placement reference model and affecting the accuracy of subsequent electrode contact coordinate calculations. Therefore, the observation data after electrode placement are not directly fused into the intraoperative reference brain surface; instead, they are treated as the current state relative to the intraoperative reference brain surface and processed independently group by group.
[0049] For each set of observation data after electrode placement, the spatial relationship of the current observation data needs to be obtained or verified based on direct evidence between the current observation data and the intraoperative reference brain surface or natural landmarks preserved in the intraoperative reference brain surface. Natural landmarks here include feature information such as small blood vessels, sulcus shadows, general tissue textures, and local geometry, which are preserved along with the intraoperative reference brain surface in step S204. The relocalization process mainly utilizes the still visible internal brain surface region around the electrode patch for matching, while electrodes, wires, and other occluded areas are given less weight or excluded from matching. Direct evidence includes at least one of the following: corresponding quantity, spatial coverage, reprojection error, 3D point-to-surface residual, and consistency with multipath constraints.
[0050] As a specific implementation method, observation data after a set of electrode placements is considered to have obtained sufficient direct evidence, requiring the following conditions to be met simultaneously: First, the number of correspondences established with the intraoperative reference brain surface or its preserved natural landmarks is not less than a preset lower limit, for example, no less than 30 valid correspondences; Second, the spatial coverage of the corresponding points in the brain surface region meets preset requirements, and is not concentrated in extremely small areas, single lines, or repetitive texture areas; Third, the median of reprojection error and 3D point-to-surface residual is lower than a preset threshold, for example, the median reprojection error does not exceed 2 pixels, and the median 3D residual does not exceed 2 millimeters. The specific thresholds must be calibrated based on measured data; Fourth, the current localization result does not have a significant conflict with closed loops or multiple alternative paths. When the direct evidence does not meet the preset conditions, the current observation data is marked as downgraded, unreliable, relocated, or requires manual review. This processing method avoids forcibly outputting erroneous localization results when evidence is insufficient. In addition, the localization result of the observation data after the previous set of electrode placements is used as the initial value for calculating the spatial relationship of the current observation data. That is, the calculation results of the previous group are only used to accelerate the convergence process of the current group. The current observation must obtain or verify its own spatial relationship through direct evidence between it and the intraoperative reference brain surface or its key landmarks, rather than unconditionally replacing the localization results of adjacent observations, thereby preventing the propagation of errors caused by recursion according to the shooting order.
[0051] Step S302: Based on the spatial relationship of the imaging device, determine the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface, as well as the state of the electrode array; wherein, the state of the electrode array includes the overall pose and local morphology of the electrode array relative to the current brain surface.
[0052] After repositioning the current observation relative to the intraoperative reference brain surface, the system represents and estimates three types of states. The first type is the spatial relationship between the current imaging device and the intraoperative reference brain surface, i.e., the spatial relationship of the imaging device determined in step S301. The second type is the overall movement and local deformation of the current brain surface relative to the intraoperative reference brain surface, i.e., brain surface deformation. The third type is the overall translation and rotation of the electrode array relative to the current brain surface, as well as the local bending, twisting, stretching, or contact states of the electrode array itself, collectively referred to as the electrode array state. By representing these three types of states separately, the boundaries of their respective degrees of freedom can be clearly defined in subsequent joint optimization, avoiding the misinterpretation of camera motion as brain surface deformation, or the misinterpretation of brain surface deformation as local bending of the electrode array.
[0053] Step S303: Based on the observation data of the electrode contacts, brain surface deformation, electrode array state, and contact constraints between the contacts and the deformed current brain surface, joint optimization is performed to solve the three-dimensional position of each electrode contact on the current brain surface, which is used as a candidate three-dimensional position.
[0054] Specifically, brain-face deformation is represented by the displacement field of the control nodes, and the state of the electrode array is represented by global affine components and non-rigid components. The expression for brain-face deformation is: , in, For the spatial points on the current brain surface after deformation coordinates For the first basis functions This represents the three-dimensional displacement vector of the corresponding control node. This represents the total number of control nodes. This expression describes the continuous change of the brain surface from before to after placement by limiting the displacement of the control nodes. To avoid discontinuous or unreasonable local deformations on the brain surface, a brain surface deformation smoothing constraint term is introduced, whose expression is: , in, The set that controls the adjacency relationships of nodes. For the first The three-dimensional displacement vector of each control node. For regularization weighting coefficients, the first term in this constraint term constrains the smoothness of brain-surface deformation, keeping the displacement differences between adjacent control nodes within a reasonable range; the second term restricts excessive overall deformation in areas where there is no observational evidence.
[0055] The deformation expression for the electrode array state is: , in, Initial position of the contact Position after deformation The three-dimensional coordinates of the contact points under the initial regular shape of the electrode array. Let be the matrix representing the overall affine transformation of the electrode array relative to its initial regular shape. It is a translation vector. The total number of control points. It is a non-rigid deformation coefficient. For the first The location of each control point The basis functions are defined in this expression, which decomposes the deformation of the electrode array into global affine components and non-rigid components. The global affine variation matrix is also defined. Translation vector This describes the translation, rotation, scaling, or shearing of the electrode array as a whole; while the non-rigid deformation parts are controlled by points. Non-rigid deformation coefficient Combining radial basis functions This describes the local bending or twisting that occurs when the electrode array is attached to the brain surface.
[0056] To couple the above-mentioned brain-face deformations with the electrode array state for solution, a joint energy estimation function is constructed, which includes: , in, The surface registration residual between the deformed current brain surface and the observation after placement; These are the weighting coefficients for the brain-surface deformation smoothing constraint; These are the weighting coefficients for the array topology spacing constraints; This diagram illustrates the connection relationships between adjacent contacts in the array. and These are the first two electrodes in the initial regular shape of the electrode array. The and the first The three-dimensional coordinates of each contact point and The first The and the first The position of each contact point after deformation; The nominal spacing between adjacent contacts is known. The weighting coefficients for the distance constraint between the contact point and the brain surface; Contact after deformation To the deformed current brain surface The shortest distance; This is the thickness or gap compensation value of the contact center relative to the brain surface.
[0057] In this joint energy estimation function, each term has a clear physical meaning. The first term... By utilizing the geometric or appearance differences between the image observed by the camera after placement and the current deformed brain surface, the brain surface deformation is driven to approximate the true state. (Second item) To ensure that facial deformation conforms to the continuity of biological tissues and prevents excessive distortion, the third step utilizes the connection relationship diagram of adjacent contacts in the array. and known nominal spacing This constraint ensures that the distance between any two adjacent contacts does not undergo abnormal stretching or compression after deformation, maintaining the physical structural integrity of the electrode array. The fourth term ensures that each contact after deformation... To the deformed current brain surface The shortest distance approaches the thickness or gap compensation value. This ensures that the contact points are positioned at a reasonable physical height above the brain surface.
[0058] By minimizing this joint estimated energy function, camera observations determine the direction and amplitude of brain-surface deformation. This deformation, in turn, constrains the surface location where the contacts should be situated, while array spacing and topological relationships jointly limit the relative positional relationships between the contacts. This coupled optimization method allows the three elements to cross-check each other, avoiding contradictory results that would occur when calculated independently. Ultimately, the three-dimensional positions of each electrode contact on the current brain surface are determined as candidate three-dimensional positions.
[0059] like Figure 4 As shown, in one embodiment, the specific implementation process of step S140 is described in detail. This embodiment focuses on how to recover the three-dimensional position of the contact point through multiple constraint methods when the depth data of the target contact point area does not meet the depth reliability condition. In actual surgical environments, metal contacts are prone to specular reflection, transparent or semi-transparent electrode array substrates may cause depth cameras to fail to measure correctly, and there are often physical gaps between the electrode sheet and the brain surface or sulci. These factors can lead to two typical depth failure scenarios: the first is depth loss, that is, the depth value of the center pixel of the target contact point is zero or invalid; the second is non-zero erroneous depth, that is, the depth value is numerically non-zero, but due to the interference of reflection or transparent materials, the value actually deviates from the actual position of the brain surface or electrode surface. To address the above situations, the process of determining the three-dimensional position of the target contact point includes the following steps: Step S401: When the center pixel depth of the target touch point is missing, read the effective depth within the preset annular neighborhood and replace the center depth with the median of the effective depth.
[0060] When a missing depth is detected in the center pixel of a target touch point, the system first attempts to perform local compensation using the surrounding spatial information. The system defines a pre-defined annular neighborhood centered on the two-dimensional image center of the target touch point and searches for valid depth values within this region. As a specific implementation, this pre-defined annular neighborhood can be set as an annular area with an inner radius of 5 pixels and an outer radius of 15 pixels, requiring at least 5 valid depth values to exist within this area. After obtaining these valid depths, the median is calculated and used as the replacement depth for the center pixel of the target touch point. The median is used instead of the mean because it has stronger resistance to interference from individual abnormal depth values that may remain within the annular region, and can more robustly reflect the true depth level around the touch point. Through this neighborhood depth replacement processing, a reasonable initial 3D position estimate can be quickly provided for touch points with partially missing depth without performing complex geometric intersection calculations.
[0061] Step S402: Based on the two-dimensional center and calibration parameters of the target contact point, generate an imaging ray passing through the target contact point from the imaging center; based on brain surface deformation, determine the intersection point or optimal proximity position of the imaging ray with the current brain surface or electrode array support surface.
[0062] When an effective depth cannot be obtained within the preset annular neighborhood, or when the depth value within the neighborhood is unreliable due to large-area reflection, the system switches to using geometric ray intersection to determine the spatial location of the contact point. First, based on the two-dimensional center of the target contact point and calibration parameters, an imaging ray is generated from the imaging center through the target contact point. The expression for this imaging ray is: , , in, For the first Spatial ray vectors of each contact point The three-dimensional coordinates of the optical center of the imaging device. The unit direction vector is determined by the intrinsic parameters of the imaging device and the center of the two-dimensional image of the contact point. The distance parameter is along the ray direction. This expression describes a three-dimensional straight line originating from the camera's optical center and passing through the two-dimensional center of the contact point on the image plane.
[0063] Subsequently, based on the brain-surface deformation estimated in steps S302 and S303, the intraoperative reference brain surface is transformed into the current brain surface, and the intersection of the imaging ray with the deformed current brain surface mesh is calculated. When the brain surface is represented by a triangular mesh, the intersection of the ray with each candidate triangle is calculated, and the effective intersection point closest to the imaging center along the ray direction is selected as the initial three-dimensional position. The physical basis for selecting the closest effective intersection point is that the contact point observed by the camera must be located at the first unobstructed surface position in the line of sight direction. For non-zero error depth cases, since the original depth reading itself is incorrect, the system will also ignore the erroneous reading and instead use the geometric position obtained by intersecting the ray with the deformed brain surface as a more reliable initial three-dimensional position estimate.
[0064] Step S403: Based on the local brain surface normal at the intersection or optimal proximity position, the electrode sheet thickness, and thickness or gap constraints, the position of the intersection or optimal proximity position is corrected.
[0065] The initial 3D position obtained by intersecting the ray with the brain surface mesh only represents the geometric position of the brain surface. The actual electrode contacts are located above the surface of the electrode pads, and there are physical thicknesses of the electrode pads themselves and potential gaps between the electrode pads and the brain surface. Therefore, a normal correction is needed for the initial 3D position. The expression for the position correction is: , in, The first position obtained after position correction The three-dimensional coordinates of each electrode contact The initial 3D position is obtained by intersecting the imaging ray with the deformed current brain surface mesh. This is the proportionality coefficient. For electrode sheet thickness, This is the estimated gap between the electrode pads and the current brain surface. The current normal to the brain surface at the intersection point.
[0066] In this expression, the proportionality coefficient The value of depends on the final definition of the contact position to be output. For example, when the output position is defined as the center in the thickness direction of the electrode sheet, Take 0.5; when the output position is defined as the outer surface of the electrode sheet, Take 1. Electrode sheet thickness The design values or manufacturer-provided nominal values for the electrode array can be used, such as a default thickness of 0.5 mm, but the specific values can be adjusted according to the actual electrode model used. Estimated gap. Used to compensate for the gap caused by the electrode pad not fully adhering to the brain surface when crossing the sulcus. Local brain surface normal. This ensures that the compensation direction for thickness and gap is always perpendicular to the local brain surface. Through this correction process, the geometric intersection located on the brain surface can be accurately shifted to the actual physical height of the electrode contact, thereby eliminating the systematic bias caused by neglecting the electrode thickness and brain sulcus gap.
[0067] Step S404: Based on array topology constraints and contact spacing constraints, the three-dimensional position of the target contact point is jointly optimized.
[0068] The 3D positions of the contact points obtained through the aforementioned steps, whether derived by substituting the median depth of the neighborhood or by ray intersection and thickness gap correction, are all independent initial estimates for a single contact point. To further improve the overall positioning accuracy and utilize the array's structural priors to correct residual errors at individual contact points, it is necessary to jointly optimize the 3D positions of all contact points based on array topology constraints and contact point spacing constraints. The energy function for the joint estimation is: , in, For joint estimation of the energy function; These are the weighting coefficients for the observation constraint terms; A set of touchpoints for direct observational evidence with depth or a two-dimensional center; For the first The observation confidence weight of each contact point; The first one to be optimized The three-dimensional coordinates of each contact point; For the first The initial 3D position estimate of each contact point is obtained based on the replaced depth backprojection or position correction. Sure; The array spacing constraint weights; This is a diagram showing the connection relationships between adjacent contacts in the array. The nominal spacing between adjacent contacts is known. Weights for local curvature smoothing constraints; This is a set of local neighborhood triplet indexes consisting of three consecutive touchpoints in the same row or column, where... As the central contact point, and Neighboring points; The weighting coefficients for the distance constraint between the contact point and the brain surface; This is the current brain surface model after thickness and gap correction.
[0069] The joint energy estimation function contains four constraint terms, which work together to achieve a globally optimal solution for the contact point location. The first term is an observation constraint term, which ensures that the contact point with direct observational evidence is as close as possible to its initial three-dimensional position estimate. .in, For the first The observation confidence weight of each contact point is used for contacts that are both reliable in depth and have a clear appearance. Take the higher value; for occluded or undetected touch points, assign a weight to their observation confidence level. Set it to zero to exclude it from this constraint. The second constraint is the array spacing constraint, which utilizes the connection relationship diagram of adjacent contacts in the array. and known nominal spacing The optimization constraint prevents abnormal stretching or compression between any two adjacent contacts, maintaining the physical structural integrity of the electrode array. The third term is a local curvature smoothing constraint, determined by traversing the local neighborhood triplet index set formed by three consecutive contacts in the same row or column. Punishment Center Touchpoint Deviation from adjacent points on both sides and The degree of connection at the midpoint is used to prevent unreasonable and severe bending of the electrode array in local areas. The fourth item is the constraint on the distance between the contact point and the brain surface, so that each optimized contact point... To the current brain surface model after thickness and gap correction The shortest distance approaches zero, ensuring that the contact point always adheres to the modified brain surface.
[0070] By minimizing this joint estimation energy function, the system achieves a balance between observational evidence, array physical structure, and brain-face geometry. Specifically, for occluded touchpoints, due to their observational confidence weights… When set to zero, the first observation constraint no longer exerts any influence, and its three-dimensional position is completely recovered by the combined constraints of the second (internal contact distance), the third (local curvature constraint), and the fourth (brain surface contact constraint). This approach allows for the estimation of the reasonable spatial location of occluded contacts even when some contacts are completely invisible, leveraging the overall topological relationship of the array. Furthermore, when outputting results, the system explicitly labels the evidence for contacts recovered through internal contact distance and array topological constraints as array topological estimation, rather than direct depth measurement or multi-observation combined recovery. This provides clinicians with a clear quantitative basis for assessing the reliability of contact localization results.
[0071] In one embodiment, the specific implementation process of steps S120 and S150 is described in detail. This embodiment focuses on explaining the basis for cross-modal registration between the intraoperative reference brain surface and the preoperative medical imaging model, as well as the uncertainty synthesis method for the final output of the three-dimensional coordinates of each electrode contact.
[0072] Step S501: Based on at least one of the following: the three-dimensional geometry, surface normal, curvature, sulcus structure, vascular landmarks, and surgical area prior of the intraoperative reference brain surface, the intraoperative reference brain surface is registered with the preoperative medical imaging model.
[0073] Intraoperative reference brain surface data includes the color, texture, and local 3D morphology of the actual exposed brain surface, while preoperative medical imaging models mainly contain individualized anatomical information such as brain surface geometry, sulci, curvature, and blood vessels. Due to the differences in the information formats, common physical and anatomical features are prioritized for matching during cross-modal registration. For example, the correspondence between the intraoperative reference brain surface and the preoperative medical imaging model in 3D geometry can be used, or the distribution characteristics of surface normals and curvature can be used to find areas with high consistency. Sulci and gyri, as relatively stable anatomical landmarks of the cerebral cortex, can also serve as important bases for registration. In addition, vascular landmarks can be extracted from the appearance observation data of the intraoperative reference brain surface, for example, by extracting superficial vascular networks through color gradients, brightness gradients, and tubular structure responses along the vascular direction, and matching them with superficial vascular structures extracted by preoperative magnetic resonance venography. Surgical area priors are used to limit the search range to a reasonable local area, such as the approximate center, observation direction, left and right hemispheres, lobes, or craniotomy area specified by the user in advance.
[0074] It should be noted that during the registration process, the colors, materials, or lighting effects artificially set in the preoperative medical imaging software are not used as the actual brain and facial colors for matching. This is because preoperative medical imaging software, in order to facilitate observation by doctors, usually assigns pseudo-colors to different tissues or adds virtual light sources. These rendering effects do not reflect the real optical characteristics of the brain and face. If these are compared with the actual appearance images acquired during surgery, it will introduce incorrect correspondences, causing the registration results to deviate from the actual anatomical positions.
[0075] As a specific implementation method, initial registration values can be obtained before fine registration by: rendering the preoperative cortical model as depth maps at several preset viewpoints, such as generating a series of synthetic depth maps around multiple azimuths of the craniotomy region normal; then coarsely matching, scoring, and sorting these synthetic depth maps with the depth map of the intraoperative reference brain surface; and using the candidate viewpoints with the highest scores as the initial transformation matrices for subsequent fine registration. This initialization method based on multi-view rendering and scoring can provide a reasonable starting point for fine registration in the absence of external tracking equipment, avoiding the optimization process from getting trapped in local minima. When both the intraoperative reference brain surface and the preoperative medical imaging model have the correct physical scale, rigid spatial relationships are calculated first; only when there is sufficient evidence to support local differences are low-degree-of-freedom and constrained non-rigid corrections added to prevent overfitting noise.
[0076] Step S502: Output the uncertainty of the three-dimensional coordinates of each electrode contact; the uncertainty is based on at least two of the following: calibration error, depth noise, cross-observation registration error, reference brain surface fusion error, preoperative image registration error, brain surface deformation error, contact detection error, array size or thickness error, and occlusion contact estimation error.
[0077] After the coordinate transformation in step S150, each electrode contact has obtained its three-dimensional coordinates in the preoperative medical imaging coordinate system. However, these coordinates are not absolutely accurate true values; their accuracy is affected by the cumulative errors in each stage of the data processing chain. To enable clinicians to quantitatively assess the reliability of each contact's positioning result, the system simultaneously outputs the uncertainty of each electrode contact's three-dimensional coordinates.
[0078] The synthesis of uncertainties encompasses multiple error sources from data acquisition to final coordinate output. These include: calibration error (resulting from residual deviations in the calibration of the imaging device's intrinsic and extrinsic parameters); depth noise (reflecting inherent measurement fluctuations of the depth camera at different working distances and incident angles); cross-observation registration error (resulting from residuals when aligning multiple sets of intraoperative surface observation data); reference brain surface fusion error (statistical deviation of surface geometry after weighted fusion of multiple sets of data); preoperative image registration error (deviation in surface distance after aligning the intraoperative reference brain surface with the preoperative medical image model); brain surface deformation error (uncertainty in estimating the displacement of the brain surface relative to the pre-placement reference brain surface); contact point detection error (pixel-level deviation in identifying the center of the contact point in a two-dimensional image); array size or thickness error (physical parameter deviation caused by electrode manufacturing tolerances); and occlusion contact point estimation error (additional uncertainty introduced when estimating the position of invisible contacts using array topology).
[0079] As a specific implementation method, the uncertainty can be calculated using a first-order variance propagation approach. In this method, the aforementioned error sources are treated as independent random variables, and the variance of each coordinate component of the contact point in the preoperative medical image coordinate system is approximately the sum of the variances of the corresponding components of each error source. Subsequently, the uncertainty is output as a multiple of the standard deviation, for example, as twice the standard deviation. This value corresponds to approximately a 95% confidence interval, indicating that the true position of the contact point has a high probability of falling within a spatial range centered on the output coordinates and with this uncertainty as its radius.
[0080] The composition of uncertainty varies depending on the source of evidence for the contact point. Sources of evidence include direct depth measurement, multi-observation joint recovery, or array topology estimation. When the three-dimensional position of the contact point is obtained by backprojecting reliable depth values directly, the source of evidence is labeled as direct depth measurement. In this case, the uncertainty is mainly composed of calibration error, depth noise, cross-observation registration error, reference brain surface fusion error, preoperative image registration error, and contact point detection error. When the contact point is obtained due to missing depth by combining ray intersection with multiple sets of observations, the source of evidence is labeled as multi-observation joint recovery. The uncertainty composition needs to additionally consider the consistency residuals between multiple observations. When the contact point is completely occluded, as in step S404 above, its three-dimensional position is recovered through the distance between adjacent contact points and array topology constraints. The source of evidence is labeled as array topology estimation. For this type of estimated contact point, an estimation error term needs to be added on top of the basic error sources mentioned above. The estimation error term can be adjusted by increasing variance compensation based on the topological distance between the estimated contact and the nearest directly observed contact; that is, the farther the estimated contact is from the known contact, the greater its accumulated estimation uncertainty. This method of distinguishing evidence sources and differentiating the combined uncertainty ensures that the final output not only includes spatial coordinates but also complete error source information, providing verifiable quantitative evidence for subsequent electrophysiological signal analysis and neural modulation planning.
[0081] like Figure 5 As shown, in one embodiment, an intraoperative electrode positioning system is provided. This system is used to execute the intraoperative electrode positioning methods described in the foregoing embodiments, mapping the aforementioned method steps to corresponding functional modules, so as to realize a complete conversion link from intraoperative two-dimensional observation to preoperative medical image three-dimensional coordinates.
[0082] The intraoperative electrode localization system includes an intraoperative observation and acquisition module, a brain-face region extraction and continuous confidence calculation module, a cross-observation appearance correspondence and 3D constraint establishment module, an initial spatial relationship estimation and fine registration module, a non-sequential global constraint map optimization module, a confidence-weighted fusion module, a B0-preoperative medical image registration module, a post-placement observation relocation module, a three-state joint estimation module, a contact point 3D recovery module, and a coordinate mapping and result output module. These modules logically constitute a complete data processing chain, where solid arrows represent the main pathways for data and control flow, and dashed arrows represent depth noise model input and degraded control feedback.
[0083] During the data flow in the main pathway, the intraoperative observation and acquisition module is responsible for acquiring multiple sets of intraoperative surface observation data before and after electrode placement. Each set of observation data includes appearance observation data and three-dimensional geometric observation data. The output of this module is divided into two paths: one path is used as the observation data before electrode placement and input to the brain-face region extraction and continuous confidence calculation module, and the other path is used as the observation data after electrode placement and input to the post-placement observation and relocation module.
[0084] The brain-surface region extraction and continuity confidence calculation module receives observation data before electrode placement, determines the region of interest on the brain surface, and calculates continuity confidence for at least one of the following: pixels, depth points, feature correspondences, surface regions, entire observation sets, or spatial constraint edges. During confidence calculation, this module receives depth noise model input from the calibration and depth noise modeling process via dashed arrows to evaluate depth effectiveness and reliability. The calculated continuity confidence and extracted brain-surface region data are then passed to the cross-observation appearance correspondence and 3D constraint establishment module.
[0085] The cross-observation appearance correspondence and 3D constraint establishment module searches for the same brain surface locations in the appearance images of different observations, establishes cross-observation appearance correspondences, and forms 3D or reprojection constraints. These constraints are fed into the initial spatial relationship estimation and fine registration module, which uses robust estimation methods to calculate the initial spatial relationship between the two sets of observations and performs fine registration by combining geometric and appearance error terms. Its functions correspond to the preprocessing in steps S201 and S202.
[0086] After fine registration, the data enters the non-sequential global constraint graph optimization module. This module treats each group of observations or the local surface formed by those observations as observation nodes, establishes spatial constraint edges between observation nodes with reliable overlap to form an observation constraint graph, and performs global optimization on the spatial relationship between each observation node and the unified reference coordinate system. This module does not accumulate observation nodes one by one according to the shooting order, but instead uses closed loops and multiple alternative paths to jointly constrain the data, thereby suppressing the generation of cumulative drift.
[0087] The globally optimized observation data and their spatial relationships are fed into the confidence-weighted fusion module. This module, based on the continuous confidence score output by the brain-surface region extraction and continuous confidence score calculation module, rejects data below a first threshold from fusion, reduces the fusion weight for data between the first and second thresholds, and allows data above the second threshold to participate in fusion with normal weights, ultimately generating an intraoperative reference brain surface B0. This generated intraoperative reference brain surface B0 serves as the core baseline data and is simultaneously connected to the B0-preoperative medical image registration module and the post-placement observation relocation module.
[0088] The B0-Preoperative Medical Image Registration Module receives the intraoperative reference brain surface B0 and the externally input preoperative medical image model. It performs registration based on at least one of the following: 3D geometry, surface normal, curvature, sulcus structure, vascular landmarks, and surgical area priors, obtaining the image spatial mapping relationship. This mapping relationship is then passed to the coordinate mapping and result output module for final coordinate system transformation.
[0089] In the post-placement processing branch, the post-placement observation relocation module receives post-placement observation data from the intraoperative observation acquisition module and the intraoperative reference brain surface B0 from the confidence-weighted fusion module. This module relocates each set of post-placement observation data relative to the intraoperative reference brain surface to determine the spatial relationship of the imaging device. During this process, if the direct evidence between the current observation data and the intraoperative reference brain surface or its preserved natural landmarks does not meet preset conditions, the system marks the current observation data as degraded, unreliable, requiring relocation, or requiring manual verification through a degradation control feedback loop triggered by the dashed arrow, preventing the propagation of erroneous localization results.
[0090] The spatial relationships of the imaging device determined by repositioning are sent to the three-state joint estimation module. This module represents and estimates the brain surface deformation relative to the intraoperative reference brain surface, as well as the overall pose and local morphology of the electrode array relative to the current brain surface. It then uses the observation data of the electrode contacts, brain surface deformation, electrode array state, and contact constraints for joint optimization to solve for the candidate three-dimensional positions of each electrode contact on the current brain surface. The functions performed by this module correspond to the processing procedures in steps S302 and S303.
[0091] When the depth data of the target contact area does not meet the depth reliability condition, the 3D contact recovery module intervenes. This module generates an imaging ray based on the 2D center of the target contact and calibration parameters, calculates the intersection of the imaging ray with the deformed current brain surface mesh, and performs position correction by combining the local brain surface normal, electrode thickness, and estimated gap. Subsequently, this module performs joint optimization of the 3D position of the target contact based on array topology constraints and contact spacing constraints. For occluded contacts, its 3D position is recovered through the spacing between adjacent contacts and array topology constraints. The function performed by this module corresponds to the processing steps S401 to S404.
[0092] Finally, the coordinate mapping and result output module receives the three-dimensional position of the contact points from the three-state joint estimation module or the contact point three-dimensional recovery module, the image spatial mapping relationship from the B0-preoperative medical image registration module, and the deformation correspondence between the current brain surface and the intraoperative reference brain surface. This module backmaps the three-dimensional position of each electrode contact point to the intraoperative reference brain surface and converts it to the preoperative medical image coordinate system, outputting the three-dimensional coordinates of each electrode contact point and its corresponding evidence source, confidence level, or processing status, while also outputting the uncertainty of the three-dimensional coordinates of each electrode contact point. The function performed by this module corresponds to the processing procedures of steps S150 and S502.
[0093] It should be noted that the above division of modules is only a description of logical functions and does not constitute a restriction on physical entities. Each module can be implemented by software, hardware, firmware, or a combination thereof. For example, each module can be written as computer program instructions, read and executed by a processor; it can also be implemented in hardware such as field-programmable gate arrays or application-specific integrated circuits. In terms of deployment, each module can be merged or split and deployed on the same intraoperative workstation for centralized processing, or it can be distributed and deployed on imaging equipment, edge computing devices, workstations, or servers to work collaboratively. The intraoperative electrode positioning system provided in this embodiment realizes all the method steps described in the foregoing embodiments through data flow and feedback control between the functional modules, providing system-level support for the devices or equipment topics that may be involved in the following.
[0094] In one embodiment, the application process of the intraoperative electrode positioning method provided by the present invention in a real surgical environment is described. This application process involves different operational stages in craniotomy, including the surface state of exposed brain tissue, the operational scenario of surgical instrument intervention, and the coverage state after electrode array placement.
[0095] Specifically, after craniotomy and dura mater treatment, but before electrode array placement, multiple sets of discrete intraoperative surface observation data need to be acquired from different locations using an imaging device. In the currently preferred embodiment, a composite camera that simultaneously acquires color images and depth information is used as the imaging device to acquire five to eight sets of discrete observation data from different viewing angles. Each set of observation data includes appearance observation data reflecting the texture of the brain surface and three-dimensional geometric observation data reflecting the spatial morphology of the brain surface. This number of acquisitions is only one implementation example; in other embodiments, the number of acquisition sets can be increased or decreased depending on the actual surgical field of view and equipment conditions.
[0096] To ensure that the constructed intraoperative reference brain surface accurately reflects the undisturbed brain surface state, the acquisition of the aforementioned multiple sets of observation data should be conducted within a relatively stable time window of the brain surface state. As a specific implementation method, the criteria for this stable window must simultaneously meet the following conditions: First, after craniotomy and dura mater treatment, disturbance operations such as irrigation and suction have ceased, and a preset waiting time has elapsed; second, at least two sets of brain surface observation data are acquired consecutively at the acquisition starting point, and after coarse registration, the median point-to-surface distance of the corresponding region does not exceed 1.5 mm, and the 95th percentile distance does not exceed 3 mm; specific thresholds must be calibrated based on measured data; third, the patient's vital signs are under control and stable, for example, arterial blood pressure and end-tidal carbon dioxide partial pressure are within the target range set by the surgical team. If any of the above conditions are not met, the system prompts for waiting or reassessment of the window state, and formal acquisition of the intraoperative reference brain surface does not begin. The aforementioned threshold parameters are a specific implementation method and can be adjusted according to the surgical procedure and equipment conditions.
[0097] After collecting multiple sets of observation data before electrode placement, the processing flow described in the aforementioned embodiment is followed sequentially to perform the following steps: cross-observation appearance correspondence and three-dimensional constraint establishment, non-sequential global constraint graph optimization, confidence-weighted fusion to generate intraoperative reference brain surface, registration of intraoperative reference brain surface with preoperative medical image model, observation relocation after electrode placement, three-state joint estimation, multi-constraint recovery of unreliable depth contacts, and coordinate back-mapping and result output. Finally, the three-dimensional coordinates of each electrode contact in the preoperative medical image coordinate system and its corresponding evidence source, confidence level, or processing status are obtained.
[0098] To objectively evaluate the performance of the present invention in practical applications, registration data between the intraoperative reference brain surface and the preoperative medical image surface were recorded in a set of completed tests. Specific values are as follows: the average surface distance before registration was 9.802 mm; the average surface distance after registration was 2.509 mm; the median distance after registration was 2.026 mm; the 95th percentile distance was 6.331 mm; and points with a distance of no more than 3 mm accounted for 66.69% of all evaluation points. These data reflect the degree of spatial alignment between the intraoperative reference brain surface established using the present invention and the preoperative medical image model.
[0099] In practical applications, to further evaluate the positioning accuracy and overall performance of this solution, the following verifications can be conducted: comparison of the final contact point coordinates after a complete end-to-end process with the gold standard of postoperative computed tomography; comparison of the contact point 3D positioning error between this solution and point-by-point recursion and manual image alignment; quantitative comparison of cross-observation registration residuals and cumulative drift; accuracy of judgment under reflection, occlusion, and non-zero error depth injection; and correlation analysis between continuous confidence and true error. These verifications will help comprehensively evaluate the reliability of this solution under various complex operating conditions.
[0100] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0101] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.
Claims
1. A method for intraoperative electrode positioning, characterized in that, include: Based on the calibration parameters of the acquired imaging device, the spatial constraint relationship between multiple sets of intraoperative surface observation data before electrode placement is determined, and an intraoperative reference brain surface is generated through global constraint optimization and confidence weighted fusion; wherein, each set of intraoperative surface observation data includes appearance observation data and three-dimensional geometric observation data; The intraoperative reference brain surface is registered with the acquired preoperative medical image model to obtain the image spatial mapping relationship; Multiple sets of intraoperative surface observation data after electrode placement are repositioned relative to the intraoperative reference brain surface to determine the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface, and the state of the electrode array. The spatial relationship of the imaging device, the brain surface deformation, and the state of the electrode array are jointly estimated to obtain the candidate three-dimensional position of the electrode contact. When the depth data of the target contact area does not meet the depth reliability condition, an imaging ray is generated based on the two-dimensional observation of the target contact and the calibration parameters. The three-dimensional position of the target contact is determined by combining the spatial relationship between the imaging ray and the current brain surface or electrode array support surface, array topology constraints, and thickness or gap constraints. Based on the deformation correspondence between the current brain surface and the intraoperative reference brain surface, the three-dimensional position of each electrode contact is back-mapped to the intraoperative reference brain surface, and converted to the preoperative medical image coordinate system based on the image space mapping relationship, outputting the three-dimensional coordinates of each electrode contact and its corresponding evidence source, confidence level or processing status.
2. The intraoperative electrode positioning method according to claim 1, characterized in that, The generation of an intraoperative reference brain surface through global constraint optimization and confidence-weighted fusion includes: Each set of intraoperative surface observation data before electrode placement, or the local surface formed by the observation data, is used as an observation node. Spatial constraint edges are established between observation nodes with reliable overlap to form an observation constraint map. The spatial constraint edges include constraint edges between observation nodes with adjacent shooting order, and constraint edges between observation nodes with non-adjacent shooting order but observing the same brain region. Based on the observation constraint diagram, the spatial relationship between each observation node and the unified reference coordinate system is globally optimized. The multiple sets of intraoperative surface observation data before electrode placement, which were globally optimized, were converted to the unified reference coordinate system, and weighted and fused based on the confidence level of each set of observation data to generate the intraoperative reference brain surface.
3. The intraoperative electrode positioning method according to claim 1, characterized in that, The multiple sets of intraoperative surface observation data after electrode placement are repositioned relative to the intraoperative reference brain surface to determine the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface, and the state of the electrode array, and are jointly estimated, including: For each set of intraoperative surface observation data after the electrode placement, the spatial relationship between the electrode and the intraoperative reference brain surface is calculated based on the intraoperative reference brain surface to determine the spatial relationship of the imaging device. Based on the spatial relationship of the imaging device, the brain surface deformation of the current brain surface relative to the intraoperative reference brain surface and the state of the electrode array are determined; wherein, the state of the electrode array includes the overall pose and local morphology of the electrode array relative to the current brain surface; Based on the observation data of the electrode contacts, the brain surface deformation, the state of the electrode array, and the contact constraints between the contacts and the deformed current brain surface, joint optimization is performed to solve the three-dimensional position of each electrode contact on the current brain surface, which is then used as the candidate three-dimensional position.
4. The intraoperative electrode positioning method according to claim 1, characterized in that, When the depth data of the target contact area does not meet the depth reliability condition, determining the three-dimensional position of the target contact includes: Based on the two-dimensional center of the target contact point and the calibration parameters, an imaging ray is generated that passes through the target contact point from the imaging center. Based on the brain surface deformation, determine the intersection point or optimal proximity position of the imaging ray with the current brain surface or the electrode array support surface; Based on the local brain surface normal at the intersection or optimal proximity position, the electrode sheet thickness, and the thickness or gap constraint, the position of the intersection or optimal proximity position is corrected. Based on the array topology constraints and contact point spacing constraints, the three-dimensional position of the target contact point is jointly optimized.
5. The intraoperative electrode positioning method according to claim 2, characterized in that, The global optimization of the spatial relationship between each observation node and the unified reference coordinate system based on the observation constraint diagram includes: The edge confidence weight of each of the spatial constraint edges is determined based on at least one of the following: matching interior point ratio, spatial coverage, and 3D registration residual. The translation and rotation residuals between the two sets of observations are normalized using an information matrix to construct a global optimization objective function; the expression of the global optimization objective function is as follows: , in, The objective function value for global pose optimization; The set of spatial constraint edges formed by pairs of observation nodes with reliable overlapping relationships in the observation constraint graph. The number of spatial constraint edges; This indicates traversing each pair of observation nodes in the set of spatial constraint edges; For the first Group and No. The edge confidence weights of spatial constraint edges between observation groups; This is an information matrix used to unify the dimensions of translation and rotation errors; The edge residual vector is calculated from the two sets of observation transformation relationships; A robust loss function; Fix a reference node to eliminate the overall degrees of freedom, and iteratively update the spatial relationship of each observation node until the convergence condition is met; The formula for calculating the edge credibility weight is as follows: , In the formula, For the proportion of interior points corresponding to appearance or depth, This refers to the spatial coverage index of the corresponding point. The three-dimensional registration residual between the two sets of observations, The scale parameters are determined based on the measured depth noise of the imaging device.
6. The intraoperative electrode positioning method according to claim 3, characterized in that, The brain-face deformation is represented by a control node displacement field, the electrode array state is represented by a global affine component and a non-rigid component, and the jointly estimated energy function includes: The expression for the brain-face deformation is: , in, For the spatial points on the current brain surface after deformation coordinates For the first basis functions This represents the three-dimensional displacement vector of the corresponding control node. To control the total number of nodes; The expression for the brain-face deformation smoothing constraint is: , in, The set that controls the adjacency relationships of nodes. For the first The three-dimensional displacement vector of each control node. These are the regularization weight coefficients; The deformation expression for the state of the electrode array is: , in, Initial position of the contact Position after deformation The three-dimensional coordinates of the contact points under the initial regular shape of the electrode array. Let be the matrix representing the overall affine transformation of the electrode array relative to its initial regular shape. It is a translation vector. The total number of control points. It is a non-rigid deformation coefficient. For the first The location of each control point are basis functions; The joint estimation energy function includes: , in, The surface registration residual between the deformed current brain surface and the observation after placement; These are the weighting coefficients for the brain-surface deformation smoothing constraint; These are the weighting coefficients for the array topology spacing constraints; This diagram illustrates the connection relationships between adjacent contacts in the array. and These are the first two electrodes in the initial regular shape of the electrode array. The and the first The three-dimensional coordinates of each contact point and The first The and the first The position of each contact point after deformation; The nominal spacing between adjacent contacts is known. The weighting coefficients for the distance constraint between the contact point and the brain surface; Contact after deformation To the deformed current brain surface The shortest distance; This is the thickness or gap compensation value of the contact center relative to the brain surface.
7. The intraoperative electrode positioning method according to claim 4, characterized in that, Determining the three-dimensional position of the target contact point further includes: When the center pixel depth of the target touch point is missing, the effective depth within the preset annular neighborhood is read, and the median of the effective depth is used to replace the center depth. When an effective depth cannot be obtained within the preset annular neighborhood, the intersection point of the imaging ray and the deformed current brain surface mesh is calculated, and the effective intersection point closest to the imaging center along the ray direction is selected as the initial three-dimensional position; the expression for the imaging ray is: , , in, For the first Spatial ray vectors of each contact point The three-dimensional coordinates of the optical center of the imaging device. The unit direction vector is determined by the intrinsic parameters of the imaging device and the center of the two-dimensional image of the contact point. This is the distance parameter along the ray direction; The expression for the position correction is: , in, The first position obtained after position correction The three-dimensional coordinates of each electrode contact The initial three-dimensional position is obtained by intersecting the imaging ray with the deformed current brain surface mesh. This is the proportionality coefficient. For electrode sheet thickness, This is the estimated gap between the electrode pads and the current brain surface. The current normal to the brain surface at the intersection point; The energy function of the joint estimation is: , in, For joint estimation of the energy function; These are the weighting coefficients for the observation constraint terms; A set of touchpoints for direct observational evidence with depth or a two-dimensional center; For the first The observation confidence weight of each contact point; The first one to be optimized The three-dimensional coordinates of each contact point; For the first The initial three-dimensional position estimate of each contact point is obtained based on the replaced depth back projection or the position correction. Sure; The array spacing constraint weights; This is a diagram showing the connection relationships between adjacent contacts in the array. The nominal spacing between adjacent contacts is known. Weights for local curvature smoothing constraints; This is a set of local neighborhood triplet indexes consisting of three consecutive touchpoints in the same row or column, where... As the central contact point, and Neighboring points; The weighting coefficients for the distance constraint between the contact point and the brain surface; This is the current brain surface model after thickness and gap correction; for occluded touch points, their observation confidence weights are... Set to zero, and recover its three-dimensional position by the spacing between adjacent contacts and the array topology constraints.
8. The intraoperative electrode positioning method according to claim 2, characterized in that, The confidence-weighted fusion includes: Calculate the continuity confidence for at least one of the following: pixel, depth point, feature correspondence, surface region, whole group of observations, or spatial constraint edge; The continuous confidence level is determined based on at least one of the following: depth validity, depth noise, multi-frame depth consistency, observation distance, observation angle, local surface orientation, overexposure, reflection, occlusion, and cross-observation registration residual. When the continuous confidence level is lower than the first threshold, the corresponding data is rejected from participating in the fusion; when the continuous confidence level is between the first threshold and the second threshold, the fusion weight of the corresponding data is reduced; when the continuous confidence level is higher than the second threshold, the data participates in the fusion with normal weight.
9. The intraoperative electrode positioning method according to claim 3, characterized in that, The intraoperative surface observation data after electrode placement for each group of data includes calculating the spatial relationship between the data and the intraoperative reference brain surface, respectively, based on the intraoperative reference brain surface, including: Based on direct evidence between the current observation data and the intraoperative reference brain surface or the natural landmarks preserved in the intraoperative reference brain surface, obtain or verify the spatial relationship of the current observation data; The direct evidence includes at least one of the following: corresponding quantity, spatial coverage, reprojection error, three-dimensional point-to-surface residual, and consistency with multipath constraints. When the direct evidence does not meet the preset conditions, the current observation data will be marked as downgraded, unreliable, relocated, or requiring manual verification. The positioning results of the observation data after the previous set of electrodes were placed are used as the initial values for calculating the spatial relationships of the current observation data.
10. The intraoperative electrode positioning method according to claim 1, characterized in that, The process of registering the intraoperative reference brain surface with the acquired preoperative medical image model includes: Based on at least one of the three-dimensional geometry, surface normal, curvature, sulcus structure, vascular landmarks, and surgical area prior of the intraoperative reference brain surface, the intraoperative reference brain surface is registered with the preoperative medical imaging model. The output of the three-dimensional coordinates of each electrode contact and its corresponding evidence source, confidence level, or processing status also includes: Output the uncertainty of the three-dimensional coordinates of each electrode contact; the uncertainty is based on at least two of the following: calibration error, depth noise, cross-observation registration error, reference brain surface fusion error, preoperative image registration error, brain surface deformation error, contact detection error, array size or thickness error, and occlusion contact estimation error. The sources of evidence include direct depth measurements, multi-observation joint reconstruction, or array topology estimation.
