Visual cortical prosthesis implantation navigation system and method based on multi-modal functional mapping

Abstract

The invention discloses a visual cortical prosthesis implantation navigation system and method based on multi-modal functional mapping, and the system comprises a multi-modal data fusion module which is used for receiving and fusing multi-modal image data of a patient to construct a functional voxel set; the implantation planning module is used for dynamically selecting an optimization strategy according to the configuration type of the electrode and generating a planning scheme including an optimal implantation pose and channel forbidding configuration based on an electronic fence mechanism; the virtual electrode simulation module is used for calculating a virtual receptive field between physical electrodes based on a planning scheme, and simulating and visualizing a visual perception effect generated by electrode array stimulation under the optimal implantation pose; and the intraoperative closed-loop navigation module is used for guiding and controlling an execution mechanism to complete the implantation operation of the electrode according to the target implantation pose provided by the planning scheme in the operation. The maximized visual function coverage is realized on the premise of ensuring the blood vessel safety, and the uniformity of the implantation precision and the postoperative visual reconstruction effect is improved.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a visual cortical prosthesis implantation navigation system and method based on multimodal functional mapping. Background Technology

[0002] Visual cortical prostheses, as a promising neural interface technology for restoring basic visual function in blind individuals, rely heavily on the precise implantation location and depth of the electrode array in the primary visual cortex for their clinical effectiveness. Current technologies typically rely on preoperative structural magnetic resonance imaging (MRI) and intraoperative optical observation for vascular obstacle avoidance planning, employing an "all or nothing" obstacle avoidance logic. This means that if any contact point in the electrode array has a spatial conflict with a blood vessel, the entire array is forcibly shifted to an avascular region. While this strategy ensures surgical safety, it often forces electrodes to deviate from areas with high visual function weight, resulting in a significant decline in postoperative visual perception quality. Furthermore, current implantation systems lack preoperative functional prediction capabilities regarding electrode stimulation effects. Surgeons cannot directly assess what visual images a patient will see after implantation at a specific location, nor can they quantify the impact of different implantation schemes on visual reconstruction performance. Intraoperative brain tissue deformation due to craniotomy and cerebrospinal fluid loss renders preoperative coordinate planning based on static images ineffective. Existing navigation systems generally lack real-time feedback and closed-loop control mechanisms at the cortical level, making it difficult to ensure that electrodes accurately remain at the target functional layer.

[0003] Therefore, there is an urgent need for an intelligent implantation navigation system that can integrate multimodal functions and anatomical information, maximize visual function coverage while ensuring safety, and support preoperative simulation and intraoperative dynamic correction, so as to break through the current bottlenecks in the functional, safety, and operational precision of visual cortical prosthesis implantation surgery. Summary of the Invention

[0004] In view of this, the present invention proposes a visual cortical prosthesis implantation navigation system and method based on multimodal functional mapping, which can maximize visual function coverage, improve implantation accuracy, and achieve a balance between postoperative visual reconstruction effect while ensuring vascular safety. The present invention provides the following technical solution: A visual cortical prosthesis implantation navigation system based on multimodal functional mapping, comprising: The multimodal data fusion module is used to receive and fuse the patient's multimodal image data to construct the patient's functional voxel set; The implantation planning module, connected to the multimodal data fusion module, is used to dynamically select optimization strategies based on the configuration type of the visual cortical prosthetic electrode, and generate a planning scheme that includes the optimal implantation pose and channel disabling configuration based on the electronic fence mechanism. The virtual electrode simulation module is connected to the implantation planning module and is used to simulate and calculate the virtual receptive field between physical electrodes based on the planning scheme through current guidance, and to simulate and visualize the visual perception effect produced by the stimulation of the electrode array under the optimal implantation pose. The intraoperative closed-loop navigation module, connected to the implantation planning module, is used to guide and control the actuator to complete the electrode implantation operation based on the target implantation pose provided by the planning scheme during the operation.

[0005] Optionally, the multimodal imaging data includes brain functional imaging data, vascular structure imaging data, and brain anatomical structure imaging data; The functional voxel set is a set of multidimensional feature vectors defined in the cerebral cortex space.

[0006] Optionally, the implantation planning module includes: The configuration recognition unit is used to read the configuration type of the visual cortical prosthetic electrode and trigger different pose optimization paths according to whether the configuration type is a rigid handle or a flexible conformal type. The pose search unit, connected to the configuration recognition unit, is used to perform configuration-adaptive pose optimization on the functional voxel set and generate multiple candidate poses. The electronic fence processing unit, connected to the pose search unit, is used to detect the spatial overlap relationship between the electrode contacts and the vascular structure for each candidate pose in order to identify conflicts. If the number of conflicting contacts is higher than a preset threshold, a channel disabling configuration table is generated to disable high-risk channels in the software, while retaining the pose of the entire array located in a high visual weight area. The comprehensive scoring unit, connected to the electronic fence processing unit, is used to calculate the comprehensive score for each candidate pose; The planning scheme output unit is connected to the comprehensive scoring unit and is used to output the candidate pose with the highest comprehensive score as the optimal implantation pose, and simultaneously output the corresponding channel disable configuration.

[0007] Optionally, the pose search unit is configured to perform the following operations on the functional voxel set, based on the configuration type, to generate multiple candidate poses: When the configuration type is rigid handle type, the tangent plane is calculated in the local flat area, and the electrode template is traversed in three degrees of freedom on the tangent plane to generate multiple candidate poses. Each candidate pose corresponds to a set of center coordinates and rotation angles, and the corresponding fitting error is calculated. When the configuration type is flexible conformal, the electrode template is applied to the skin surface using geodesic mapping, and the spatial distribution of the electrode contacts is adjusted within the allowable curvature constraint range to generate multiple candidate poses. Each candidate pose corresponds to a combination of surface fitting shape and insertion depth.

[0008] Optionally, the virtual electrode simulation module includes: The physical electrode mapping unit is used to map each physical electrode contact to the corresponding spatial position in the functional voxel set according to the optimal implantation pose in the planning scheme, and to extract the visual weight index and corresponding field polar coordinates of the voxel covered by each contact. The virtual receptive field calculation unit, connected to the physical electrode mapping unit, is used to calculate the virtual electrode formed between any two adjacent physical electrodes based on the current conduction model, and to generate the virtual receptive field between them through an interpolation algorithm. The super-resolution rendering unit, connected to the virtual receptive field computing unit, is used to render the original light points corresponding to the physical electrodes and the virtual receptive field together into a continuous visual perception heat map, thereby achieving super-resolution field reconstruction with a resolution higher than that of the physical electrodes. A bidirectional linkage visualization unit, connected to the super-resolution rendering unit, is used to update the visual perception heatmap in the functional view in real time when adjusting the electrode pose in the anatomical view, forming a real-time interactive feedback mechanism.

[0009] Optionally, the intraoperative closed-loop navigation module includes: The brain drift compensation unit is used to acquire the vascular texture map of the cortical surface during the operation and perform non-rigid registration with the preoperative vascular structure image data, and calculate the brain tissue deformation field to correct the spatial coordinates of the optimal implantation pose in real time. Impedance monitoring unit is used to acquire impedance spectrum data of the electrode tip at high frequency during electrode implantation; The layer recognition and braking control unit is connected to the impedance monitoring unit and is used to compare the impedance spectrum data with the preset visual cortex layer 4 feature threshold range. When the impedance value is detected to enter the feature threshold range and the absolute value of the impedance change rate is higher than the preset change rate threshold, the braking latching signal is triggered to control the actuator to stop the electrode insertion action, thereby achieving precise implantation of the target cortical layer.

[0010] Optionally, each functional voxel in the multidimensional feature vector set The eigenvectors are represented as: ,in, For the spatial physical coordinates of a voxel. This is a vascular risk index calculated based on vascular structure imaging data. A visual weighting index based on brain functional imaging data mapping. This is used to characterize the normal distance of the voxel from the central plane of the fourth layer of the visual cortex.

[0011] This invention further discloses a visual cortical prosthesis implantation navigation method based on multimodal functional mapping, comprising: Receive and fuse patients' multimodal imaging data to construct a functional voxel set for the patient; The optimization strategy is dynamically selected based on the configuration type of the visual cortex prosthetic electrode, and a planning scheme including the optimal implantation pose and channel disabling configuration is generated based on the electronic fence mechanism. Based on the planning scheme, the virtual receptive field between the physical electrodes is calculated by current-guided simulation, and the visual perception effect produced by the stimulation of the electrode array under the optimal implantation pose is simulated and visualized. During the procedure, the actuator is guided and controlled to complete the electrode implantation operation based on the target implantation pose provided by the planning scheme.

[0012] Optionally, the step of dynamically selecting an optimization strategy based on the configuration type of the visual cortical prosthetic electrode and generating a planning scheme containing the optimal implantation pose and channel disabling configuration based on an electronic fence mechanism includes: Read the configuration type of the visual cortical prosthetic electrode, and trigger different pose optimization paths according to whether the configuration type is rigid stalk-shaped or flexible conformal-shaped. Configuration-adaptive pose optimization is performed on the functional voxel set, and multiple candidate poses are generated; For each candidate pose, the spatial overlap between the electrode contacts and the vascular structure is detected to identify conflicts. If the number of conflicting contacts exceeds a preset threshold, a channel disabling configuration table is generated to disable high-risk channels in the software, while preserving the pose of the entire array located in a high visual weight region. Calculate the overall score for each candidate pose; The candidate pose with the highest overall score is output as the optimal implantation pose, and the corresponding channel disable configuration is output simultaneously.

[0013] Optionally, performing configuration-adaptive pose optimization on the functional voxel set and generating multiple candidate poses includes: When the configuration type is rigid handle type, the tangent plane is calculated in the local flat area, and the electrode template is traversed in three degrees of freedom on the tangent plane to generate multiple candidate poses. Each candidate pose corresponds to a set of center coordinates and rotation angles, and the corresponding fitting error is calculated. When the configuration type is flexible conformal, the electrode template is applied to the skin surface using geodesic mapping, and the spatial distribution of the electrode contacts is adjusted within the allowable curvature constraint range to generate multiple candidate poses. Each candidate pose corresponds to a combination of surface fitting shape and insertion depth.

[0014] According to the technical solution of the present invention, by constructing a functional voxel set that integrates multimodal image data, the implantation planning module can dynamically select optimization strategies based on the configuration type of the visual cortical prosthesis electrode. Based on the electronic fence mechanism, it generates a planning scheme that includes the optimal implantation pose and channel disabling configuration while preserving high visual weight areas, thereby effectively avoiding the problem of wasting functional areas. Furthermore, based on this planning scheme, the virtual electrode simulation module simulates and calculates the virtual receptive field between physical electrodes through current guidance, realizing real-time visualization of visual perception effects and significantly improving the accuracy and predictability of preoperative decision-making. In addition, by combining brain drift compensation and impedance feedback mechanisms, it ensures that the electrodes are accurately implanted in the target functional layer, thereby organically unifying functional maximization, vascular safety and operational precision, and realizing a paradigm upgrade of visual cortical prosthesis implantation from static obstacle avoidance to intelligent function guidance. Attached Figure Description

[0015] For illustrative and not limiting purposes, the present invention will now be described in conjunction with embodiments and accompanying drawings, wherein: Figure 1 This is a schematic diagram of the components of the visual cortical prosthesis implantation navigation system based on multimodal functional mapping in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the visual cortical prosthesis implantation navigation method based on multimodal functional mapping in an embodiment of the present invention. Figure 3 This is a schematic diagram of the architecture of the visual cortical prosthesis implantation navigation system based on multimodal functional mapping in an embodiment of the present invention. Figure 4 This is a flowchart illustrating the electronic fence cyclic evaluation mechanism in an embodiment of the present invention. Figure 5 These are schematic diagrams of the planning view and simulation view in the embodiments of the present invention; Figure 6 This is the impedance control timing diagram in an embodiment of the present invention. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0017] It should be noted that, where there is no conflict, the embodiments and features of the embodiments in this application can be combined with each other. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0018] refer to Figure 1 and Figure 3 This embodiment discloses a visual cortical prosthesis implantation navigation system based on multimodal functional mapping, comprising: The multimodal data fusion module 11 is used to receive and fuse the patient's multimodal image data to construct the patient's functional voxel set.

[0019] Specifically, firstly, the system receives three sets of multimodal image data from a 7T ultra-high field magnetic resonance imaging (MRI) device, including: Brain functional imaging data: acquired through population receptive field (pRF) functional magnetic resonance imaging, used to establish the topological mapping relationship between the retinal visual field and the primary visual cortex (V1 area); Vascular structure imaging data: acquired by time-of-flight magnetic resonance angiography (TOF-MRA) and used to construct a three-dimensional vascular tree model of the cortical surface and superficial veins / arteries; Brain anatomical imaging data: acquired through T1-weighted (T1w) and T2-weighted (T2w) structural imaging, used to accurately segment the gray matter structure of the cerebral cortex and locate the anatomical position of layer 4 (Gennari line) in area V1.

[0020] Subsequently, the three sets of image data were spatially standardized and registered, mapping them uniformly to the same anatomical coordinate system to ensure strict spatial alignment of the data across modalities. Based on this, the system divided the cerebral cortex into a series of tiny voxel units. This leads to the construction of a functional voxel set: ,in: The spatial physical coordinates of voxels are derived from brain anatomical imaging data. The vascular risk index is obtained by calculating the Euclidean distance from the voxel center to the nearest vessel wall and normalizing it to the [0,1] interval. The closer the distance, the higher the risk. The visual weight index is based on pRF analysis results. It assigns weights according to the eccentricity of the visual field region corresponding to the voxel. The region corresponding to the fovea is given a high weight (e.g., 1.0), while the peripheral visual field region is given a low weight (e.g., 0.2). This indicates the hierarchical matching degree, representing the normal distance of the voxel from the center plane of the 4th layer in the V1 region, and is used for subsequent intraoperative depth control.

[0021] This functional voxel set is stored in system memory as a three-dimensional matrix or graph structure, serving as a unified data carrier for subsequent implantation planning, virtual simulation, and intraoperative navigation. By integrating anatomical, functional, vascular, and hierarchical information into a single data structure, this module provides the data foundation for achieving a function-oriented, safe, controllable, and precisely executed implantation strategy.

[0022] Furthermore, the standardized configuration information of the visual cortical prosthetic electrodes is loaded and parsed to form a universal electrode parameter library. This parameter library adopts a mechanical-geometric dual-attribute structured definition to uniformly describe the physical constraints of different types of electrodes.

[0023] The system first identifies the electrode configuration: if it is a rigid, stem-like electrode (such as a 10×10 silicon-based Utah array), it is modeled as an indeformable rigid matrix, with all contacts having fixed relative positions and consistent normal vectors; if it is a flexible, conformal electrode (such as an independent flexible wire harness or thin-film electrode), it allows relative displacement of each contact in the vertical direction and the ability to bend along the skin curvature. Based on this, the system reads the electrode's geometric topology, including the spatial distribution matrix of the electrode contacts. The system incorporates substrate dimensions and loads constraint parameters, including the maximum bending curvature for flexible electrodes and the gradient insertion depth distribution for rigid arrays. Through this mechanism, this implementation achieves unified modeling and adaptation for both rigid and flexible electrodes, significantly improving the system's clinical versatility and scalability.

[0024] The implantation planning module 12, connected to the multimodal data fusion module, is used to dynamically select optimization strategies based on the configuration type of the visual cortical prosthetic electrode, and generate a planning scheme including the optimal implantation pose and channel disabling configuration based on an electronic fence mechanism. The implantation planning module 12 includes a configuration recognition unit 121, a pose search unit 122, an electronic fence processing unit 123, a comprehensive scoring unit 124, and a planning scheme output unit 125.

[0025] First, the configuration recognition unit 121 reads the user-preset visual cortical prosthetic electrode parameters, including configuration type (rigid stem-like or flexible conformal), geometric topology matrix Mtopo, substrate size, maximum bending curvature (for flexible electrodes), and depth distribution constraint parameters. Among these, the configuration type serves as a key branch criterion, triggering different pose optimization paths.

[0026] Specifically, the pose search unit 122 is connected to the configuration recognition unit 121 and performs configuration-adaptive pose search on the functional voxel set: when identified as a rigid handle-like electrode, it searches for a locally flat region in the functional voxel set, calculates the tangent plane of the region, and performs a three-degree-of-freedom (x, y, θ) traversal of the electrode template on the tangent plane to generate multiple candidate poses. Each candidate pose corresponds to a set of center coordinates and rotation angles, and the fitting error between the electrode substrate and the cortical surface is calculated simultaneously.

[0027] When the electrode is identified as a flexible conformal electrode, a geodesic mapping algorithm is used to simulate the application of the electrode template along the three-dimensional manifold surface of the skin. Within the allowable curvature constraint range, the spatial distribution of each electrode contact point is adjusted to generate multiple candidate poses. Each candidate pose corresponds to a combination of surface fitting shape and insertion depth.

[0028] Subsequently, the electronic fence processing unit 123 connects to the pose search unit 122 and performs electronic fence processing on each candidate pose: detecting the spatial overlap between all electrode contacts and vascular structures, and identifying conflicting contacts. If the number of conflicting contacts exceeds a preset threshold (e.g., 5% of the total number of channels), the pose is not directly discarded, but a channel disabling configuration table is generated, marking high-risk channels as disabling them post-operatively in a software manner, while retaining the pose of the entire array located in a high visual weight region. Further, refer to... Figure 4 The system initiates an electronic fence cyclic evaluation mechanism. This mechanism first makes minor adjustments to the electrode center coordinates or rotation angle within the neighborhood of the original candidate pose, along the gradient direction of the visual weight index in the functional voxel, to generate a new candidate pose. Then, it re-executes vascular conflict detection to determine if the new pose satisfies the condition that the number of conflicting contacts is less than a preset threshold. This iterative process is executed a maximum of a preset number of times. If a compliant pose is obtained within the iteration, it is included in the subsequent scoring process; if the constraint is still not met, the system automatically expands the search range and continues to generate candidate poses in nearby suboptimal high visual weight regions. Through this cyclic evaluation mechanism, the system can significantly improve the success rate of obtaining implantation solutions with high functional coverage and safety compliance in complex vascular distribution environments through local optimization and intelligent backoff strategies.

[0029] Based on this, each candidate pose is quantitatively evaluated using a comprehensive scoring unit 124. The scoring formula is as follows: ,in, Functional coverage, which is the sum of the visual weight indices of the covering voxels. For the number of vascular conflicts, To account for bonding error, it is only introduced in the case of rigid electrodes; , , These are the preset weighting coefficients.

[0030] Finally, the planning scheme output unit 125 connects to the comprehensive scoring unit 124, determining the candidate pose with the highest comprehensive score as the optimal implantation pose, and simultaneously outputting the corresponding channel disabling configuration through the planning scheme output unit, forming a complete planning scheme. This scheme not only includes spatial coordinates and attitude parameters, but also embeds a hardware channel mute command to ensure consistency between intraoperative execution and postoperative stimulation strategies. Through this mechanism, this module achieves a paradigm shift from physical obstacle avoidance to function-oriented electronic fences, maximizing the preservation of high-value visual functional areas while ensuring surgical safety, providing reliable input for subsequent virtual simulation and precise implantation. The virtual electrode simulation module 13, connected to the implantation planning module, is used to simulate and visualize the visual perception effect produced by the stimulation of the electrode array under the optimal implantation pose by calculating the virtual receptive field between the physical electrodes through current guidance based on the planning scheme. The virtual electrode simulation module 13 includes a physical electrode mapping unit 131, a virtual receptive field calculation unit 132, a super-resolution rendering unit 133, and a bidirectional linkage visualization unit 134.

[0031] First, the implantation planning module 12 receives a complete planning scheme, including an optimal implantation pose and a channel disabling configuration table. The optimal implantation pose includes center coordinates, rotation angle, or surface fit shape. Then, the physical electrode mapping unit 131 maps each physical electrode contact with excluded channels to its corresponding spatial position in the functional voxel set, and extracts the visual weight index of the voxel covered by each contact. and the corresponding polar coordinates of the field of view The coordinates are pre-calibrated by the receptive field analysis.

[0032] Next, the virtual receptive field computing unit 132 targets any pair of adjacent and simultaneously activated physical electrodes. and The system calculates the virtual electrodes formed between the physical electrodes based on a current-steering model, and simultaneously generates a virtual receptive field (pRF) located between them. Subsequently, the super-resolution rendering unit 133 renders the original light points corresponding to the physical electrodes and all virtual receptive fields together as a continuous visual perception heatmap. For example, if the distance between the physical electrodes corresponds to a 2° field of view, super-resolution reconstruction with an equivalent resolution of 0.5° can be achieved through virtual electrode technology, significantly improving image continuity and detail. Finally, the bidirectional interactive visualization unit 134 constructs a real-time interaction mechanism between the anatomical view and the functional view, referencing... Figure 5Specifically, when a physician fine-tunes the electrode pose (such as translation or rotation) in the anatomical view, the system immediately re-executes the aforementioned mapping, interpolation, and rendering process, and synchronously updates the visual perception heatmap in the functional view, forming a closed-loop interaction of adjustment-feedback-decision. This mechanism enables physicians to intuitively assess the impact of different implantation schemes on central visual field coverage, blind spot distribution, and overall visual quality, thereby selecting the optimal strategy.

[0033] Through the above implementation methods, the virtual electrode simulation module 13 not only achieves high-fidelity prediction of postoperative visual effects, but also releases system performance from physical hardware limitations through virtual electrode technology, providing a functional evaluation tool that surpasses physical resolution for preoperative planning, effectively solving the core problem of missing function and implantation mapping in the existing technology.

[0034] The intraoperative closed-loop navigation module 14, connected to the implantation planning module, is used to guide and control the actuator to complete the electrode implantation operation based on the target implantation pose provided by the planning scheme during the operation. Specifically, it includes a brain drift compensation unit 141, an impedance monitoring unit 142, and a hierarchical recognition and braking control unit 143.

[0035] First, after exposing the visual cortex during surgery, the brain drift compensation unit 141 initiates the intraoperative image acquisition process. Using a structured light camera or high-frequency ultrasound probe mounted on the end effector of the surgical robot, it scans the cortical surface in real time to acquire a high-resolution intraoperative vascular texture map. This image preserves the geometric morphology and spatial distribution characteristics of the cortical vessels during the intraoperative state.

[0036] Subsequently, the intraoperative vascular texture map was non-rigidly registered with the preoperative vascular structure image data constructed by TOF-MRA. The registration algorithm used vascular anatomical features as the matching benchmark and employed elastic registration to calculate the three-dimensional brain tissue deformation field, quantifying the brain tissue displacement caused by craniotomy, cerebrospinal fluid loss, and gravity.

[0037] Based on this deformation field, the spatial coordinates of the optimal implantation pose generated in the preoperative planning stage are corrected in real time to ensure that the target implantation point can still accurately correspond to the high visual weight region in the functional voxel set in the dynamic environment during surgery.

[0038] During the implantation procedure performed by the robot, the impedance monitoring unit 142 acquires impedance spectrum data from the electrode tip at a sampling frequency not exceeding 1 kHz. This data reflects the electrochemical characteristics of the microenvironment in which the electrode is located, and varies significantly with tissue type (cerebrospinal fluid, gray matter, white matter) and cortical level. (Reference) Figure 6The layer recognition and braking control unit 143 compares the real-time impedance spectrum with a preset threshold range for the fourth layer of the visual cortex. This threshold range, calibrated based on numerous in vitro and in vivo experiments, characterizes the low-frequency impedance amplitude range and high-frequency capacitive features unique to the fourth layer of V1 region (the layer containing the Gennari line). When an impedance value is detected to enter this threshold range, and the absolute value of the impedance change rate is higher than a preset change rate threshold (indicating that the electrode has stably entered the target layer rather than traversing the transition zone), a braking latching signal is immediately triggered. This signal is sent to the robot motion controller, driving the actuator to stop the electrode insertion action within milliseconds, achieving precise landing at the target cortical layer. If subsequent monitoring detects an abnormally high impedance value (indicating impending penetration into the white matter), the system will trigger a forced emergency stop to prevent irreversible damage caused by over-insertion of the electrode. During electrode implantation, a multi-stage speed control strategy is employed. Specifically, in the initial stage, a high-speed descent is used to quickly approach the cortical surface; after contact with the cortex, a low-speed, fine descent mode is switched to facilitate high-precision tissue recognition. During this stage, the impedance monitoring unit 142 continuously acquires impedance spectrum data Z(t) from the electrode tip. As the electrode insertion depth increases, the impedance value gradually rises from the low baseline of the cerebrospinal fluid environment, continues to rise after entering the gray matter, and forms a stable plateau when it reaches the fourth layer of the visual cortex (the layer where the Gennari line is located). This plateau is characterized by the impedance amplitude entering the preset characteristic threshold range, and the absolute value of its rate of change is lower than the preset rate of change threshold, indicating that the electrode has stabilized in the target functional layer rather than crossing the transition zone.

[0039] The hierarchy recognition and braking control unit operates in four states in real time: Monitoring mode: Detecting cortical boundaries during high-speed descent; Analysis mode: Continuously compare impedance characteristics during the low-speed downlink phase; Successful determination mode: When the impedance characteristics are confirmed to meet the fourth-level criterion, the braking latch signal is immediately triggered; Landing mode: Maintain position closed loop or mark mission completion.

[0040] Once the successful assessment mode is entered, the system sends an emergency stop command to the robot's motion controller within milliseconds, causing the longitudinal velocity Vz(t) of the actuator to instantly return to zero, achieving precise landing. The core innovation of this mechanism lies in the fact that it does not rely on preoperative static imaging, but dynamically identifies the target layer through real-time electrophysiological feedback, and triggers braking the instant the feature is identified. This avoids excessive implantation due to brain drift or individual anatomical differences, ensuring that the electrode is precisely positioned in the V1 area, layer 4, where visual signals are most sensitive.

[0041] As can be seen, the intraoperative closed-loop navigation module 14 deeply integrates preoperative functional planning with intraoperative physiological feedback, which not only compensates for the coordinate shift caused by brain tissue deformation, but also achieves precise control of longitudinal depth through impedance closed loop, ensuring that the electrode finally stays in the 4th layer of V1 area, which is most sensitive to visual signal reception, thereby ensuring the maximization of postoperative stimulation effect and long-term stability.

[0042] refer to Figure 2 This embodiment further discloses a visual cortical prosthesis implantation navigation method based on multimodal functional mapping, including the following steps: S100: Receives and fuses the patient's multimodal imaging data to construct a functional voxel set. Specifically, it receives the patient's multimodal imaging data, including brain functional imaging data, vascular structure imaging data, and brain anatomical structure imaging data. The three sets of data are spatially standardized and registered to a single anatomical coordinate system. Subsequently, the cerebral cortex is divided into multiple micro-units. This leads to the construction of a functional voxel set: ,in: The spatial physical coordinates of voxels are derived from brain anatomical imaging data. The vascular risk index is obtained by calculating the Euclidean distance from the voxel center to the nearest vessel wall and normalizing it to the [0,1] interval. The closer the distance, the higher the risk. The visual weight index is based on pRF analysis results. It assigns weights according to the eccentricity of the visual field region corresponding to the voxel. The region corresponding to the fovea is given a high weight (e.g., 1.0), while the peripheral visual field region is given a low weight (e.g., 0.2). This indicates the hierarchical matching degree, representing the normal distance of the voxel from the center plane of the 4th layer in the V1 region, and is used for subsequent intraoperative depth control.

[0043] S200: Based on the configuration type of the visual cortical prosthetic electrode, an optimization strategy is dynamically selected, and a planning scheme including the optimal implantation pose and channel disabling configuration is generated based on an electronic fence mechanism. Specifically, the configuration type of the visual cortical prosthetic electrode is read, and an optimization strategy is dynamically selected accordingly: If it is a rigid handle type, a tangent plane is calculated in a locally flat area, and the electrode template is traversed in three degrees of freedom (x, y, θ) pose on this tangent plane to generate multiple candidate poses; if it is a flexible conformal type, geodesic mapping is used to attach the electrode template to the cortical surface, and the spatial distribution of the electrode contacts is adjusted within the allowable curvature constraint range to generate multiple candidate poses. For each candidate pose, the spatial overlap relationship between the electrode contacts and the vascular structure is detected to identify conflicts. If the number of conflicting contacts exceeds a preset threshold, a channel disabling configuration table is generated, and high-risk channels are disabled in software, while retaining the pose of the entire array located in a high visual weight region. Subsequently, the system calculates the comprehensive score of each candidate pose: ,in, Functional coverage, which is the sum of the visual weight indices of the covering voxels. For the number of vascular conflicts, To account for bonding error, it is only introduced in the case of rigid electrodes; , , The weighting coefficients are preset. Finally, the candidate pose with the highest comprehensive score is output as the optimal implantation pose, and the corresponding channel disabling configuration is output simultaneously to form a complete planning scheme.

[0044] S300: Based on the planning scheme, the virtual receptive field between physical electrodes is calculated through current-guided simulation to simulate and visualize the visual perception effect produced by the electrode array stimulation under the optimal implantation pose. Specifically, based on the planning scheme, each physical electrode contact is mapped to its corresponding spatial position in the functional voxel set, and the visual weight index and corresponding field-of-view polar coordinates of the voxel covered by each contact are extracted. Subsequently, for any two adjacent physical electrodes, the virtual electrode formed between them is calculated based on the current-guided model, and a virtual receptive field located between them is generated. The original light spots corresponding to the physical electrodes and the virtual receptive field are jointly rendered into a continuous visual perception heatmap, achieving super-resolution field-of-view reconstruction higher than the spatial resolution of the physical electrodes. In the user interface, when the doctor adjusts the electrode pose in the anatomical view, the visual perception heatmap in the functional view is updated in real time, forming a real-time interactive feedback mechanism.

[0045] S400: During the procedure, the robot guides and controls the actuator to complete the electrode implantation operation based on the target implantation pose provided by the planning scheme. Specifically, during the procedure, a vascular texture map of the cortical surface is acquired and non-rigidly registered with preoperative vascular structure image data. The brain tissue deformation field is calculated to correct the spatial coordinates of the optimal implantation pose in real time. Subsequently, the robot guides the actuator to begin the implantation operation based on the corrected target implantation pose. During electrode insertion, the system acquires impedance spectrum data of the electrode tip at high frequency and compares it with the preset characteristic threshold range of the fourth layer of the visual cortex. When the impedance value is detected to enter the characteristic threshold range and the absolute value of the impedance change rate is higher than the preset change rate threshold, the system triggers a braking latching signal, controlling the actuator to stop the electrode insertion action, achieving precise implantation at the target cortical level.

[0046] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0047] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of each embodiment or some parts of the embodiments.

[0048] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A visual cortical prosthesis implantation navigation system based on multimodal functional mapping, characterized in that, include: The multimodal data fusion module is used to receive and fuse the patient's multimodal image data to construct the patient's functional voxel set; The implantation planning module, connected to the multimodal data fusion module, is used to dynamically select optimization strategies based on the configuration type of the visual cortical prosthetic electrode, and generate a planning scheme that includes the optimal implantation pose and channel disabling configuration based on the electronic fence mechanism. The virtual electrode simulation module is connected to the implantation planning module and is used to simulate and calculate the virtual receptive field between physical electrodes based on the planning scheme through current guidance, and to simulate and visualize the visual perception effect produced by the stimulation of the electrode array under the optimal implantation pose. The intraoperative closed-loop navigation module, connected to the implantation planning module, is used to guide and control the actuator to complete the electrode implantation operation based on the target implantation pose provided by the planning scheme during the operation.

2. The visual cortical prosthesis implantation navigation system according to claim 1, characterized in that, The multimodal imaging data includes brain functional imaging data, vascular structure imaging data, and brain anatomical structure imaging data. The functional voxel set is a set of multidimensional feature vectors defined in the cerebral cortex space.

3. The visual cortical prosthesis implantation navigation system according to claim 1, characterized in that, The implantation planning module includes: The configuration recognition unit is used to read the configuration type of the visual cortical prosthetic electrode and trigger different pose optimization paths according to whether the configuration type is a rigid handle or a flexible conformal type. The pose search unit, connected to the configuration recognition unit, is used to perform configuration-adaptive pose optimization on the functional voxel set and generate multiple candidate poses. The electronic fence processing unit, connected to the pose search unit, is used to detect the spatial overlap relationship between the electrode contacts and the vascular structure for each candidate pose in order to identify conflicts. If the number of conflicting contacts is higher than a preset threshold, a channel disabling configuration table is generated to disable high-risk channels in the software, while retaining the pose of the entire array located in a high visual weight area. The comprehensive scoring unit, connected to the electronic fence processing unit, is used to calculate the comprehensive score for each candidate pose; The planning scheme output unit is connected to the comprehensive scoring unit and is used to output the candidate pose with the highest comprehensive score as the optimal implantation pose, and simultaneously output the corresponding channel disable configuration.

4. The visual cortical prosthesis implantation navigation system according to claim 3, characterized in that, The pose search unit is configured to perform the following operations on the functional voxel set, based on the configuration type, to generate multiple candidate poses: When the configuration type is rigid handle type, the tangent plane is calculated in the local flat area, and the electrode template is traversed in three degrees of freedom on the tangent plane to generate multiple candidate poses. Each candidate pose corresponds to a set of center coordinates and rotation angles, and the corresponding fitting error is calculated. When the configuration type is flexible conformal, the electrode template is applied to the skin surface using geodesic mapping, and the spatial distribution of the electrode contacts is adjusted within the allowable curvature constraint range to generate multiple candidate poses. Each candidate pose corresponds to a combination of surface fitting shape and insertion depth.

5. The visual cortical prosthesis implantation navigation system according to claim 1, characterized in that, The virtual electrode simulation module includes: The physical electrode mapping unit is used to map each physical electrode contact to the corresponding spatial position in the functional voxel set according to the optimal implantation pose in the planning scheme, and to extract the visual weight index and corresponding field polar coordinates of the voxel covered by each contact. The virtual receptive field calculation unit, connected to the physical electrode mapping unit, is used to calculate the virtual electrode formed between any two adjacent physical electrodes based on the current conduction model, and to generate the virtual receptive field between them through an interpolation algorithm. The super-resolution rendering unit, connected to the virtual receptive field computing unit, is used to render the original light points corresponding to the physical electrodes and the virtual receptive field together into a continuous visual perception heat map, thereby achieving super-resolution field reconstruction with a resolution higher than that of the physical electrodes. A bidirectional linkage visualization unit, connected to the super-resolution rendering unit, is used to update the visual perception heatmap in the functional view in real time when adjusting the electrode pose in the anatomical view, forming a real-time interactive feedback mechanism.

6. The visual cortical prosthesis implantation navigation system according to claim 1, characterized in that, The intraoperative closed-loop navigation module includes: The brain drift compensation unit is used to acquire the vascular texture map of the cortical surface during the operation and perform non-rigid registration with the preoperative vascular structure image data, and calculate the brain tissue deformation field to correct the spatial coordinates of the optimal implantation pose in real time. Impedance monitoring unit is used to acquire impedance spectrum data of the electrode tip at high frequency during electrode implantation; The layer recognition and braking control unit is connected to the impedance monitoring unit and is used to compare the impedance spectrum data with the preset visual cortex layer 4 feature threshold range. When the impedance value is detected to enter the feature threshold range and the absolute value of the impedance change rate is higher than the preset change rate threshold, the braking latching signal is triggered to control the actuator to stop the electrode insertion action, thereby achieving precise implantation of the target cortical layer.

7. The visual cortical prosthesis implantation navigation system according to claim 2, characterized in that, Each functional voxel in the multidimensional feature vector set The eigenvectors are represented as: ,in, For the spatial physical coordinates of a voxel. This is a vascular risk index calculated based on vascular structure imaging data. A visual weighting index based on brain functional imaging data mapping. This is used to characterize the normal distance of the voxel from the central plane of the fourth layer of the visual cortex.

8. A visual cortical prosthesis implantation navigation method based on multimodal functional mapping, characterized in that, include: Receive and fuse patients' multimodal imaging data to construct a functional voxel set for the patient; The optimization strategy is dynamically selected based on the configuration type of the visual cortex prosthetic electrode, and a planning scheme including the optimal implantation pose and channel disabling configuration is generated based on the electronic fence mechanism. Based on the planning scheme, the virtual receptive field between the physical electrodes is calculated by current-guided simulation, and the visual perception effect produced by the stimulation of the electrode array under the optimal implantation pose is simulated and visualized. During the procedure, the actuator is guided and controlled to complete the electrode implantation operation based on the target implantation pose provided by the planning scheme.

9. The visual cortical prosthesis implantation navigation method according to claim 8, characterized in that, The process of dynamically selecting an optimization strategy based on the configuration type of the visual cortical prosthetic electrode and generating a planning scheme based on an electronic fence mechanism that includes the optimal implantation pose and channel disabling configuration includes: Read the configuration type of the visual cortical prosthetic electrode, and trigger different pose optimization paths according to whether the configuration type is rigid stalk-shaped or flexible conformal-shaped. Configuration-adaptive pose optimization is performed on the functional voxel set, and multiple candidate poses are generated; For each candidate pose, the spatial overlap between the electrode contacts and the vascular structure is detected to identify conflicts. If the number of conflicting contacts exceeds a preset threshold, a channel disabling configuration table is generated to disable high-risk channels in the software, while preserving the pose of the entire array located in a high visual weight region. Calculate the overall score for each candidate pose; The candidate pose with the highest overall score is output as the optimal implantation pose, and the corresponding channel disable configuration is output simultaneously.

10. The visual cortical prosthesis implantation navigation method according to claim 9, characterized in that, The step of performing configuration-adaptive pose optimization on the functional voxel set and generating multiple candidate poses includes: When the configuration type is rigid handle type, the tangent plane is calculated in the local flat area, and the electrode template is traversed in three degrees of freedom on the tangent plane to generate multiple candidate poses. Each candidate pose corresponds to a set of center coordinates and rotation angles, and the corresponding fitting error is calculated. When the configuration type is flexible conformal, the electrode template is applied to the skin surface using geodesic mapping, and the spatial distribution of the electrode contacts is adjusted within the allowable curvature constraint range to generate multiple candidate poses. Each candidate pose corresponds to a combination of surface fitting shape and insertion depth.

Images