Intelligent planning method for brain surgery path based on reinforcement learning

Abstract

The application discloses a brain surgery path intelligent planning method based on reinforcement learning, and aims to solve the problems of real-time re-planning and stable updating of the path under the condition of intraoperative brain shift. The application constructs a patient-specific neural implicit risk field and a confidence field through preoperative and intraoperative data calibration and registration, combines local residual errors in the shift mask with low-rank constant velocity field deformation and applies a zero displacement boundary, determines a default probability threshold and a minimum safety distance through order-preserving calibration and consistency calibration, generates a reference path by using a strategy and safety double-head network and a differentiable projection layer and a distributed critic, and then applies trust domain constraints and monotonic risk constraints by using a sliding window model prediction control and falls back to a conservative path when it is infeasible, so that the technical effect of real-time planning with continuous path, non-degraded risk and auditability and traceability under brain shift is achieved.

Description

Technical Field

[0001] This invention relates to the field of surgical navigation path planning, and more particularly to an intelligent planning method for neurosurgical surgical paths based on reinforcement learning. Background Technology

[0002] Intraoperative navigation and path planning in neurosurgery aim to avoid blood vessels, functional areas, and high-risk regions surrounding lesions within complex brain tissue, reaching the target along the safest and most accessible pathway. With the widespread use of preoperative MRI, diffusion tensor imaging, and intraoperative ultrasound or MRI, academia and industry have developed methods such as cost-grid-based graph search and sampling planning, continuous optimization, and curvature-constrained trajectory generation, combined with deformable registration to correct brain displacement. In recent years, learning-based risk assessment and reinforcement learning decision-making have been gradually introduced into medical robotics and navigation scenarios, and model predictive control has also been used for local replanning to improve real-time performance.

[0003] The existing technology still has the following shortcomings:

[0004] First, there is a lack of risk characterization and uncertainty management: most methods rely on a static risk field constructed preoperatively, which has a delayed response to intraoperative brain displacement. They lack a systematic calibration of risk and confidence, making it difficult to set a reliable default probability threshold and minimum safe distance, thus leading to misjudgment and an imbalance of conservatism.

[0005] Second, the enforceability and auditability of security constraints are insufficient: common practices involve penalties or ex-post hard constraints, lacking differentiable security projections and constraint margin feedback, making it difficult to output evidence that can be used for compliance audits; at the same time, the measurement of tail risks is insufficient, and strategy evaluation is mostly based on expected values, ignoring security in extreme situations.

[0006] Third, the stability of intraoperative replanning is insufficient: when brain displacement occurs, global replanning or frequent updates are often used, lacking sliding window and trust domain constraints centered on the affected area. The path is prone to large jumps in time and lacks monotonicity guarantees such as segmented risk non-deterioration. When optimization is not feasible, the rollback strategy is often achieved by relaxing safety constraints, which has the problem of risk rebound.

[0007] There is an urgent need for an intelligent planning method that can replan in real time under brain transfer conditions, ensure stable path updates, and provide auditable evidence within strict safety boundaries. Summary of the Invention

[0008] One objective of this invention is to propose an intelligent planning method for neurosurgical pathways based on reinforcement learning. Addressing the problems of distorted risk representation, unstable replanning, and unauditable safety compliance in existing technologies caused by brain displacement, this invention proposes a technical solution that combines patient-specific neural implicit risk and confidence modeling with displacement masking and low-rank steady velocity field deformation with zero-displacement boundaries. It also employs order-preserving and consistency calibration to determine the default probability threshold and minimum safe distance, a dual-headed network for strategy and safety, a differentiable projection layer, distributed commentator routing, and a sliding window model predictive control to apply trust domain increments and monotonic risk constraints, and to backtrack to a conservative path when infeasibility is not possible. This invention achieves the technical effects of real-time intraoperative replanning, continuous and stable pathways, no risk degradation, and auditable and traceable safety.

[0009] This invention provides a method for intelligent planning of neurosurgical surgical pathways based on reinforcement learning, comprising:

[0010] S1. Perform external parameter calibration and spatial registration on preoperative imaging data and intraoperative observation data, convert the entry point and target point to navigation coordinates, and parameterize the instrument kinematics and executability constraints into feasible domain parameters according to instrument specifications and navigation system parameters.

[0011] S2. Based on the aligned preoperative imaging data, the patient-specific neural implicit model is invoked to infer point by point, forming a baseline risk field and a baseline confidence field.

[0012] S3. Based on the consistency difference between the aligned intraoperative observation data and the baseline risk field and baseline confidence field, extract the displacement region mask according to the threshold, estimate the differential homeomorphic deformation field, and estimate the risk residual and confidence residual within the mask by local residual branch. Update the baseline risk field and baseline confidence field with the deformation and residual to obtain the updated risk field and updated confidence field.

[0013] S4. Perform order-preserving calibration and consistency calibration on the updated risk value field and the updated confidence field in sequence to obtain the calibrated risk value field and the calibrated confidence field, and determine the default probability threshold and the minimum safe distance threshold based on their quantile statistics and tail risk measurement.

[0014] S5. Using the entry point and the target point as boundary conditions, the original action sequence and the safety margin sequence are generated by the policy network with policy branches and safety margin branches. The original action sequence is projected onto the feasible region according to the threshold and feasible region parameters to obtain the projected action sequence and calculate the constraint residual sequence. The sequence is rolled out in the calibrated risk value field and the calibrated confidence field to form a candidate trajectory that satisfies the boundary conditions. The reference path is selected by the reviewer network score.

[0015] S6. Set up a sliding window around the mask of the shifted region, take the reference path as the initial value and the published path at the previous time as the trust domain benchmark, establish and solve the model prediction control problem containing trust domain incremental constraints and monotonic risk constraints. If it is feasible, the updated path is obtained. If it is not feasible, a conservative backup path that satisfies the monotonic risk constraints is generated as the updated path.

[0016] S7. For the updated path, combine the constraint residual sequence with the calibrated risk value field and the calibrated confidence field to calculate the minimum safe distance and segmented risk, and compare them with the corresponding indicators of the published path at the previous time. Check the satisfaction status of the trust domain incremental constraint and monotonic risk constraint, and generate audit records containing constraint residual statistics, minimum safe distance and segmented risk comparison.

[0017] Optionally, S1 includes:

[0018] After acquiring preoperative imaging data, intraoperative observation data, entry point, target point, instrument specifications and navigation system parameters, external parameter calibration is performed on the preoperative imaging data and intraoperative observation data according to the navigation system parameters to obtain the spatial transformation parameters from the image coordinate system to the navigation coordinate system.

[0019] The spatial transformation parameters are used to spatially register the intraoperative observation data, and the registration is confirmed to be effective when the registration quality reaches a preset threshold, thereby generating aligned preoperative image data and aligned intraoperative observation data.

[0020] Then, the spatial transformation parameters are used to transform the entry point and the target point to the navigation coordinate system, generating the entry point and the target point in the navigation coordinate system.

[0021] Simultaneously, based on the instrument specifications and navigation system parameters, the instrument kinematics and executability constraints are standardized into feasible domain parameters, forming a parameterized description of the instrument kinematics and executability constraints. The feasible domain parameters include at least the entry angle range, minimum radius of curvature, path length range, attitude change range, upper velocity limit, upper acceleration limit, joint angle range, and joint velocity range.

[0022] Output aligned preoperative imaging data, aligned intraoperative observation data, entry point in navigation coordinates, target point in navigation coordinates, and parametric descriptions of instrument kinematics and executability constraints.

[0023] Terminology definition:

[0024] The preoperative imaging data refers to three-dimensional medical imaging data acquired before surgery, including but not limited to MRI, CT, DTI, or vascular imaging, which are characterized by voxel grids and image coordinate systems and carry spatial reference information.

[0025] The intraoperative observation data refers to the observation data acquired during the operation to reflect the current anatomical state, including but not limited to intraoperative ultrasound, intraoperative MRI, stereo vision / endoscopic reconstruction point cloud or pose observation, which are characterized in their respective sensor coordinate systems.

[0026] The extrinsic parameter calibration is the process of estimating the external pose relationship between the image / sensor coordinate system and the navigation coordinate system, which is used to determine the spatial mapping relationship between the two coordinate systems to support subsequent coordinate transformations.

[0027] The spatial registration is a process of aligning intraoperative observation data with preoperative image data in space. The model used can be rigid, affine or non-rigid registration, and is based on similarity measurement or feature alignment.

[0028] The image coordinate system is the voxel Cartesian coordinate system that comes with the preoperative image data, and its origin, axis and voxel spacing are defined by the image metadata.

[0029] The navigation coordinate system is a global three-dimensional coordinate reference system for the surgical navigation / tracking system, used to uniformly represent the spatial position and orientation of the patient, instruments and images during the operation;

[0030] The navigation coordinates are numerical representations of spatial points or poses in a navigation coordinate system, used to perform coordinate-consistent planning and control calculations.

[0031] The entry point is the starting spatial location for entering the patient's intracranial passage, as designated by the surgeon based on preoperative planning. It can be located at anatomical locations such as the outer surface of the skull, the craniotomy / puncture entrance, or the surface of the dura mater.

[0032] The target point is the spatial location specified by the surgeon to reach the destination during the operation, including but not limited to the core of the lesion, the biopsy target point, or a safe point near the functional area.

[0033] The instrument specifications are a set of geometric and performance parameters of the surgical instruments to be used, including but not limited to outer diameter, effective length, joint / hinge structure, number of degrees of freedom, tip and cannula structure, and manufacturing tolerances.

[0034] The navigation system parameters are a set of performance and configuration parameters of the navigation / tracking system, including but not limited to spatial accuracy / repeatability, update frequency, delay, field of view, marker type and recognition algorithm configuration, time synchronization method, etc.

[0035] The device kinematics is a mathematical model that maps the spatial variables of the device joints to the pose of the device end in the navigation coordinate system, including forward / backward kinematic relationships and (if applicable) redundancy and constraint descriptions;

[0036] The executability constraints are a set of constraints that ensure the device operates within the physical and system capabilities, including but not limited to geometric, kinematic and dynamic limits, environmental interaction, and clinical safety-related restrictions.

[0037] The registration quality is a quantitative indicator used to measure the quality of spatial registration results, including but not limited to mutual information, normalized cross-correlation, feature matching residuals or target registration error, which is compared with a preset threshold to determine the registration validity.

[0038] The preset threshold is a numerical limit set in advance for registration quality or other judgment quantities. It can be derived from system calibration, empirical statistics or clinical settings, and can be fixed or adaptively adjusted according to the scenario.

[0039] The entry angle range is the set of posture angles allowed for the instrument trajectory at the entry point, and its measurement can be defined relative to the local anatomical surface normal, the operator-set reference direction, or the pre-planned reference system.

[0040] The minimum radius of curvature is the lower bound index of the trajectory curvature, defined as the value that the radius of curvature of the trajectory in three-dimensional space must not be less than, used to limit the degree of path bending to meet the requirements of instrument passability;

[0041] The path length range is the allowable range of trajectory arc length values ​​between the entry point and the target point, used to avoid the path being too short and unreachable or too long and resulting in excessive risk / delay.

[0042] The attitude change range is the upper limit of the allowable change amplitude or rate of change of the instrument attitude (direction / roll) along the trajectory or between discrete planning steps, in order to limit abrupt attitude changes and comply with the instrument's articulation capability.

[0043] The speed limit is the maximum permissible linear / angular velocity of the instrument tip or related joint during planned execution, used to meet the system's dynamic capabilities and safety constraints.

[0044] The acceleration limit is the maximum permissible linear / angular acceleration at the end of the instrument or related joint, used to limit dynamic impact and ensure controllability and tissue safety;

[0045] The joint angle range refers to the allowable position angle range of each joint of the device, which is usually determined by the mechanism's stroke and safety stop.

[0046] The joint velocity range refers to the permissible angular velocity range of each joint of the device, which is usually limited by the driving capability and control strategy.

[0047] Optionally, S2 includes:

[0048] Within the navigation coordinate system of the aligned preoperative image data, the volume of interest covering both and their surrounding safety margin is determined based on the entry point and target point in the navigation coordinate system.

[0049] Within the volume of interest, three-dimensional coordinate points are used as query input, and aligned preoperative image data are used as conditions to invoke patient-specific neural implicit models, and the corresponding baseline risk value and baseline confidence are obtained point by point.

[0050] The baseline risk values ​​are mapped to three-dimensional coordinate points to form a baseline risk value field, and the baseline confidence scores are mapped to three-dimensional coordinate points to form a baseline confidence field. The baseline risk value field and the baseline confidence field are aligned with the voxel grid of the aligned preoperative image data and cover the volume of interest.

[0051] Output the baseline risk value field and the baseline confidence field.

[0052] Terminology definition:

[0053] The aligned preoperative image data is the preoperative three-dimensional image volume data represented in the navigation coordinate system after step S1 extrinsic parameter calibration and spatial registration and confirmation of registration effectiveness.

[0054] The volume of interest is a three-dimensional region in the navigation coordinate system, determined by the entry point and the target point under the navigation coordinates and expanded outward according to a preset safety margin, which is used to limit the scope of subsequent queries and field construction.

[0055] The safety margin is a scale parameter used to spatially expand the volume of interest around the entry point and the target point. It can be an isotropic or anisotropic distance or voxel number, used to provide operational margin and safety buffer.

[0056] The query input is a three-dimensional spatial coordinate point selected within the volume of interest, which is used as input to a patient-specific neural implicit model to query the field value at that point.

[0057] The patient-specific neural implicit model is a conditional continuous field estimation model based on preoperative image data customized or aligned for the current patient. It takes query input and features extracted from aligned preoperative image data as input, and outputs at least the risk value and confidence level of the spatial point.

[0058] The point-by-point inference is a process of independently performing forward computation of the model for each query input to obtain the corresponding output;

[0059] The baseline risk value is a risk measure of a spatial point by a patient-specific neural implicit model under aligned preoperative imaging data. It is a scalar that characterizes the degree of surgical risk when crossing the point, and the larger the value, the higher the risk.

[0060] The baseline confidence is a measure of the reliability of the patient-specific neural implicit model for its baseline risk value, used to reflect uncertainty or coverage relationship, and its value can be normalized to [0,1].

[0061] The baseline risk value field is a three-dimensional scalar field formed by mapping the baseline risk values ​​corresponding to each query input according to their three-dimensional coordinates. It can be represented by a discrete raster or a continuous function.

[0062] The baseline confidence field is an uncertain three-dimensional field formed by mapping the baseline confidence of each query input according to its three-dimensional coordinates;

[0063] The alignment with the voxel grid is to ensure that the baseline risk field and the baseline confidence field are consistent with the voxel grid of the aligned preoperative image data in terms of resolution, voxel size, orientation, and origin, so as to index and calculate on the same grid.

[0064] The voxel grid is a discrete sampling structure for three-dimensional medical image volume data, defined by a set of regularly spaced voxels and their spatial orientation, origin, and voxel size parameters.

[0065] Optionally, S3 includes:

[0066] Within the navigation coordinate system, based on the consistency difference between the aligned intraoperative observation data and the baseline risk field and baseline confidence field, the consistency difference value is calculated by spatial point and the mask of the displaced region is extracted according to the preset threshold.

[0067] Brain tissue displacement is modeled using low-rank parameterization of a steady velocity field, which is represented by a linear combination of a finite number of basis functions. Zero-displacement boundary conditions are applied to the skull surface and dura mater surface determined by aligned preoperative image data. A topologically consistent differential homeomorphic deformation field is generated by time integration of the steady velocity field.

[0068] Risk residuals and confidence residuals are estimated by local residual branch within the domain of the shifted region mask;

[0069] The differential homeomorphic deformation field is applied to the baseline risk field and the baseline confidence field, and the corresponding risk residuals and confidence residuals are added at spatial points to obtain the updated risk field and the updated confidence field.

[0070] Output the differential homeomorphic deformation field, the shifted region mask, the updated risk field, and the updated confidence field.

[0071] Terminology definition:

[0072] The aligned intraoperative observation data is the intraoperative observation data represented in the navigation coordinate system after spatial registration in step S1 and confirmation of registration effectiveness;

[0073] The consistency difference is a measure of the degree of inconsistency between the current anatomical / tissue status reflected by aligned intraoperative observation data and the predictions of the baseline risk field and baseline confidence field, and can be calculated based on likelihood, normalized residuals, divergence or coverage bias, etc.

[0074] The consistency difference value is a scalar or vector index of the consistency difference calculated at spatial points, which is used for threshold segmentation and region determination.

[0075] The displaced region mask is a spatial binary (or multi-valued) marker obtained in the navigation coordinate system based on the consistency difference value and a preset threshold, indicating the region suspected of being affected by brain displacement. Optionally, it can be filtered by connected components and morphological processing to improve robustness.

[0076] The steady velocity field is a three-dimensional velocity vector field defined in the navigation coordinate system and which does not change with time. Its exponential mapping / time integration can generate smooth deformation.

[0077] The low-rank parameterization is a way of representing the steady velocity field with fewer degrees of freedom. The velocity field is expanded into a linear combination of a finite number of basis functions and controlled by a small number of coefficients, so as to reduce the computational burden and suppress overfitting.

[0078] The basis functions are spatial function bases that constitute a steady velocity field, including but not limited to B-spline basis, radial basis functions, finite element shape functions, or data-driven basis vector fields;

[0079] The skull surface and dura mater surface determined by the aligned preoperative image data are anatomical boundary surfaces obtained by segmentation and three-dimensional reconstruction based on the aligned preoperative image data, and are used as geometric constraints for deformation boundaries.

[0080] The zero-displacement boundary condition is a condition that fixes the displacement at the surface of the skull and the surface of the dura mater to zero, so as to keep the anatomical boundary from moving.

[0081] The time integration is the process of integrating the steady velocity field along the virtual time convection equation (or through scaling-square and other numerical exponential mappings) to generate a deformation map.

[0082] The differential homeomorphic deformation field is a deformation mapping obtained by time integration of a steady velocity field. It is differentiable and invertible everywhere in the domain, and its Jacobian determinant remains positive. It is used to describe the displacement of foldless tissue.

[0083] The local residual branch is a residual estimation module that operates within the shifted region mask. It outputs an additive correction to the baseline field, conditioned on aligned intraoperative observation data and spatial location.

[0084] The risk residual is an additive correction given by the local residual branch to the baseline risk value (or its deformed value) within the mask, used to compensate for local errors and reflect intraoperative changes;

[0085] The confidence residual is an additive correction given by the local residual branch to the baseline confidence level (or its deformed value) within the mask, used to update the uncertainty estimate;

[0086] The updated risk field is a three-dimensional risk field obtained by applying a differential homeomorphic deformation field to the baseline risk field and superimposing the risk residual within the shifted region mask.

[0087] The updated confidence field is a three-dimensional confidence field obtained by applying a differential homeomorphic transformation field to the baseline confidence field and superimposing the confidence residual within the shifted region mask.

[0088] Optionally, S4 includes:

[0089] Based on the updated risk value field and the updated confidence field, first perform order-preserving calibration to maintain the consistency of risk ranking among spatial points, and then perform consistency calibration to make the coverage relationship of confidence to actual error conform to the preset coverage level, thereby generating the calibrated risk value field and the calibrated confidence field.

[0090] On the calibrated risk value field and the calibrated confidence field, quantile statistics are performed according to the preset quantile level, and the safety margin is determined in combination with the tail risk measure, so that the minimum safety distance threshold matching the default probability is obtained under the condition that the default probability does not exceed the preset quantile level, and the preset quantile level is determined as the default probability threshold.

[0091] Output the calibrated risk value field, calibrated confidence field, default probability threshold, and minimum safe distance threshold.

[0092] Terminology definition:

[0093] The order-preserving calibration is a process of applying a monotonic calibration mapping to the updated risk value field to correct numerical biases without changing the relative risk ranking between spatial points.

[0094] The consistency calibration is the calibration of the updated confidence field, which is the process of matching the confidence value with the actual error or event coverage probability and achieving the target coverage requirement.

[0095] The coverage relationship is the correspondence between the confidence value and the empirical frequency of "actual error being included / event being covered", which is used to measure the reliability of the confidence after calibration.

[0096] The preset coverage level is the lower limit of the expected coverage probability, used to constrain the consistency target between the confidence level after consistency calibration and the actual coverage.

[0097] The calibrated risk value field is a three-dimensional risk field after order-preserving calibration, which is used for subsequent threshold setting and planning evaluation.

[0098] The calibrated confidence field is a three-dimensional confidence field after consistency calibration, which is used to characterize the uncertainty of risk estimation and participate in threshold setting;

[0099] The quantile statistics are quantile calculations performed on calibrated risk values ​​(which may be combined with corresponding confidence levels) at a given quantile level, and are used to extract representative statistics from the tail of the distribution.

[0100] The preset quantile level is the target quantile α used for quantile statistics, which is used to characterize the allowable tail probability quality and serves as a benchmark for subsequent safety threshold setting.

[0101] The tail risk metric is a weighted assessment of the tail of the risk distribution, used to introduce additional penalties or conservatism for extreme cases based on quantile statistics, including but not limited to extended metrics of Value at Risk (VaR) such as Conditional Value at Risk (CVaR) / Expected Shortage.

[0102] The safety margin is a non-negative margin introduced based on tail risk measurement and calibration uncertainty, used to resist extreme biases and model errors in threshold setting;

[0103] The default probability is the probability of the "risk exceeds the safety limit" event occurring under specified evaluation conditions, and is determined jointly by the calibrated risk value field and the calibrated confidence field.

[0104] The default probability threshold is the upper limit allowed for the default probability, and its value is equivalent to the preset quantile level, which is used as a risk control limit in subsequent planning and constraint projection.

[0105] The minimum safe distance threshold is the lower limit of the distance that the planned path must maintain from the high-risk area or risk equivalence boundary, provided that the probability of default does not exceed the default probability threshold and the safety margin is taken into account. It is used as a distance benchmark for safety constraints in subsequent steps.

[0106] Optionally, S5 includes:

[0107] In the environment characterized by the calibrated risk value field and the calibrated confidence field, the original action sequence and safety margin sequence are generated by the policy network with policy branch and safety margin branch, using the entry point and target point in navigation coordinates as boundary conditions.

[0108] Based on the default probability threshold, minimum safe distance threshold, and parameterized description of the kinematics and executability constraints of the device, the original action sequence is projected into the feasible region within the differentiable quadratic programming and barrier function projection layer to obtain the projected action sequence and calculate the constraint residual sequence.

[0109] The projected action sequence is rolled out in the calibrated risk value field and the calibrated confidence field to obtain a set of candidate trajectories that satisfy the entry point and the target point under the navigation coordinates. The critic network scores the set of candidate trajectories based on quantile distribution and conditional value at risk, and selects the trajectory with the highest score as the reference path.

[0110] Output the reference path and constraint residual sequence.

[0111] Terminology definition:

[0112] The policy network is a neural network structure used to generate planning control inputs, and its output includes actions for motion decisions and safety margin related quantities for safety constraint adjustment.

[0113] The policy branch is the branch in the policy network that outputs the action vector, and is used to generate unconstrained control inputs at each discrete planning step;

[0114] The safety margin branch is a branch in the policy network that outputs safety margin-related parameters, used to adjust the risk threshold, distance constraint, or feasible domain boundary by tightening or expanding.

[0115] The original action sequence is a sequence of action vectors output by the strategy branch in multiple discrete planning steps without constraint projection. Each action can be represented as a pose / displacement increment or joint space increment in the navigation coordinate system.

[0116] The safety margin sequence is a sequence of safety margin parameters output by the safety margin branch in multiple discrete planning steps, which is used to participate in the setting of safety boundaries when projecting the feasible region and generating the trajectory.

[0117] The feasible region is a set of actions or trajectories that satisfy safety conditions such as kinematic and executability constraints, default probability threshold, and minimum safe distance threshold.

[0118] The differentiable quadratic programming and barrier function projection layer is a projection operation module that performs the original action to the feasible region. It achieves the constraint correction of the input action through differentiable quadratic programming or barrier function method, and its input-to-output mapping is approximately differentiable with respect to the network parameters and the input so that the gradient can be backpropagated.

[0119] The quadratic programming problem is an optimization problem that measures the action correction amount with a quadratic objective and satisfies linear / quadratic constraints. It is used to find the projected solution that minimizes the deviation of the action and satisfies the constraints.

[0120] The barrier function is a continuous function that adds a penalty term to the objective function for approaching the constraint boundary, in order to prevent actions from going out of bounds and achieve smooth internalization of the feasible region during the optimization process.

[0121] The feasible region projection is the process of mapping the original action to the feasible region according to given metrics and constraints, and the result satisfies the boundary conditions related to equipment and safety.

[0122] The projection action sequence is an action vector sequence that satisfies the feasible region constraint after being processed by the differentiable projection layer.

[0123] The constraint residual sequence is a sequence index that characterizes the degree of violation of each constraint by the original action and the magnitude of the projection correction, including but not limited to the amount of violation of inequality constraints, the residual of equality constraints and the norm of action projection displacement, etc.

[0124] The rolling unfolding is a process of taking the projected action sequence as input, recursively obtaining the path state points through the device kinematics / state update model in chronological order, and evaluating the corresponding risks in the calibrated risk value field and the calibrated confidence field.

[0125] The candidate trajectory set is a set of feasible trajectories generated by different initial values, randomness, or diversified strategies and obtained by rolling expansion under the condition of satisfying the boundary between the entry point and the target point.

[0126] The critic network is a network module that evaluates and scores candidate trajectories. Its inputs include the trajectory and its risk assessment in the calibration field, and its output is a score for ranking and selection.

[0127] The quantile distribution is a distribution representation method that uses a number of quantile points to represent the trajectory risk / reward distribution, and is used to capture the tail characteristics of the distribution;

[0128] The conditional value at risk is a measure of the expected risk (CVaR / expected shortage) of the portion exceeding the value at risk under a given quantile level α, used to measure the tail risk of the trajectory;

[0129] The reference path is the target trajectory selected by the commentator network based on the quantile distribution and conditional risk value comprehensive score, and is used as the benchmark path for subsequent optimization and release.

[0130] Optionally, S6 includes:

[0131] A sliding window covering the shifted region mask is determined around the shifted region mask. The reference path is used as the initial value for optimization, the previously published path is used as the trust domain benchmark, and the segmented risk of the path is calculated on the calibrated risk value field and the calibrated confidence field.

[0132] Within the sliding window, a model predictive control optimization problem is established. The goal is to minimize the deviation and curvature increment relative to the reference path while satisfying the trust domain increment constraint and the monotonic risk constraint. The trust domain increment constraint limits the Hausdorff distance and curvature increment of the new path relative to the previously published path to not exceed a preset upper limit. The monotonic risk constraint limits the segment risk of the new path within the sliding window to not be higher than the risk of the corresponding segment of the previously published path plus a preset safety margin.

[0133] The optimization solution from the previous time step is used for hot-start solution. If the solution is feasible, the new path obtained is used as the update path. If the solution is not feasible, a conservative backup path that satisfies the monotonic risk constraint is generated and the conservative backup path is used as the update path.

[0134] During the initial execution, the previously published path is the initial path generated based on preoperative image data;

[0135] Output the update path.

[0136] Terminology definition:

[0137] The sliding window is a local optimization region defined by the shifted region mask, covering the mask and its nearby path segments in the navigation coordinate system, and is used to limit the calculation range of model prediction control;

[0138] The initial optimization value is the initial solution of the path variable to be optimized in the model predictive control optimization problem, which is taken from the discrete representation of the reference path within the sliding window;

[0139] The trust domain benchmark is a reference path used to measure path increment, which is a discrete representation of the published path in the previous time step within a sliding window.

[0140] The previously published path refers to the path trajectory that has been confirmed and made effective by the system in the previous planning cycle, and is used as a reference for the trust domain constraints in this cycle.

[0141] The model predictive control optimization problem is a mathematical programming problem that optimizes the discrete points or parameterized representation of a path within a sliding window with a finite field of view. Its objective and constraints comprehensively consider path smoothness and safety requirements.

[0142] The trust domain increment constraint is a constraint that limits the magnitude of the change of the new path relative to the trust domain baseline, and includes at least an upper bound constraint on the Hausdorff distance and curvature increment.

[0143] The monotonic risk constraint requires that the segment risk of a new path within the sliding window is not higher than the segment risk of the previously published path plus a preset safety margin.

[0144] The Hausdorff distance is a maximum-minimum distance metric between two paths, equal to the maximum distance from each point on one path to the nearest point on the other path, and is used to characterize geometric offset.

[0145] The curvature increment is the increase in curvature of the new path relative to the previously published path at the corresponding arc length position or sampling index, or its statistical value, used to limit curvature changes;

[0146] The preset upper limit is the maximum allowable value set for Hausdorff distance and curvature increment respectively, used to constrain the deviation of the path from the previously published path.

[0147] The segmented risk is a risk index obtained by dividing the path into several segments according to arc length or discrete step length within a sliding window, evaluating each segment in a calibrated risk value field and a calibrated confidence field, and then aggregating the results.

[0148] The preset safety margin is a non-negative tolerance parameter used in monotonic risk constraints to absorb minor fluctuations caused by measurement noise and modeling errors.

[0149] The deviation is a scalar objective used to measure the difference between the current optimized path and the reference path, and can be defined based on discrete point position difference, attitude difference, or a weighted sum thereof;

[0150] The hot start behavior is the process of using the optimization solution of the previous moment as the initial solution to start the optimization solution of the current cycle, which is used to accelerate convergence and improve stability.

[0151] The optimized solution at the previous moment is the solution result of the model predictive control optimization problem of the previous planning cycle, and is used for the hot start of this cycle;

[0152] The term "feasible solution" refers to the situation where the model predictive control optimization problem has a feasible solution that satisfies all constraints and is successfully found by the solver.

[0153] The term "unsolvable behavior" refers to a situation where there is no solution to the model predictive control optimization problem that satisfies all constraints, or where the solver determines that the problem is infeasible.

[0154] The conservative backup path is an alternative path generated based on the previously published path and security constraints when the solution is not feasible. It satisfies the monotonic risk constraint and ensures executability by tightening the variation range or increasing the safety margin.

[0155] The updated path is the target path determined in this cycle through a solution or rollback strategy and used to replace the published path in the previous time step.

[0156] Optionally, the S7 includes:

[0157] Within the navigation coordinate system, for the updated path, combining the constraint residual sequence and the calibrated risk value field and calibrated confidence field, the risk value and confidence of each sampling point are calculated by discrete sampling along the updated path, and the segmented risk is obtained accordingly.

[0158] Based on the default probability threshold and the minimum safe distance threshold, the safe distance at each sampling point is calculated so that the risk does not exceed the default probability threshold, and the minimum value of the whole path is taken as the minimum safe distance.

[0159] The constraint residual sequence is statistically analyzed to obtain the maximum value, mean, and proportion of non-zero constraint residuals, thereby reflecting the correction magnitude of the feasible region projection.

[0160] The segmented risk and minimum safe distance of the updated path are compared with the risk and minimum safe distance of the corresponding segment of the published path at the previous time step, and the satisfaction status of the trust domain incremental constraint and monotonic risk constraint is checked.

[0161] When the inspection passes, an audit record is generated that includes statistics on constraint residuals, minimum safe distance, and comparison results of segmented risks. When the inspection fails, the failed items are marked in the audit record and the corresponding indicator values ​​and thresholds are recorded.

[0162] Output the audit log and store it in association with the update path for subsequent iterations.

[0163] Terminology definition:

[0164] The discrete sampling is a process of selecting a series of sampling points along the update path according to arc length or index interval for evaluation.

[0165] The sampling points are spatial points obtained through discrete sampling and located on the update path, used to read values ​​from the calibrated risk value field and the calibrated confidence field;

[0166] The risk value of the sampling point is the query result of the sampling point location in the calibrated risk value field, and is a scalar characterizing the risk level of the point;

[0167] The confidence level of the sampling point is the query result of the sampling point location in the calibrated confidence field, and is a scalar that characterizes the reliability of the risk estimation of the point.

[0168] The risk isosurface is the set of isosurfaces in the calibrated risk field in which the risk value is equal to the default probability threshold;

[0169] The safety distance is the minimum Euclidean distance to the risk isosurface or a conservative approximation thereof required to ensure that the risk does not exceed the default probability threshold at a given sampling point.

[0170] The minimum safe distance is a scalar obtained by taking the global minimum value of the safe distance of each sampling point along the update path, and is used to characterize the lower bound of the safety margin of the path.

[0171] The constrained residual statistics are a set of statistics obtained by summarizing the constrained residual sequences, including at least the maximum value, the mean, and the proportion of non-zero values;

[0172] The non-zero percentage is the proportion of the number of samples in the constrained residual sequence that exceed a preset threshold to the total number of samples, which is used to reflect the frequency of feasible region projection correction.

[0173] The satisfied state is the result of determining whether the trust domain incremental constraint and the monotonic risk constraint are satisfied simultaneously.

[0174] The check is performed if the above conditions are met and the status is "yes";

[0175] The check failed when the above-mentioned condition was "No";

[0176] The audit records are structured records used for compliance audits and traceability, and at least include the statistics of constraint residuals, the comparison results of minimum safety distance and segmented risk, and the corresponding thresholds and indicator values;

[0177] The associated storage is the process of establishing an association between audit records and update paths through timestamps, unique identifiers, or version numbers and then persistently saving them.

[0178] Optionally, in step S5, the KKT multipliers and optimality residuals returned from the differentiable projection layer are aggregated into constraint margins and incorporated into the constraint residual sequence as evidence for subsequent audit records.

[0179] Optionally, in step S6, the upper limit of the Hausdorff distance and curvature increment of the trust domain incremental constraint adaptively tightens as the maximum value of the constraint residual sequence at the previous time step increases, and the larger the constraint residual at the previous time step, the smaller the corresponding upper limit at this time step.

[0180] The beneficial effects of this invention are:

[0181] 1. For brain displacement, deformation and risk correction are performed only in the affected area. Low-rank parameterization of steady velocity field is used and zero-displacement boundary is applied at the skull and dura mater. Combined with order-preserving calibration and consistency calibration, risk and confidence can be updated quickly and reliably during surgery, avoiding global distortion and sorting disorder, and reducing false alarms and false negatives.

[0182] 2. By combining a policy and security dual-head network with a differentiable projection layer, actions are projected onto the feasible region and constraint residuals and optimality evidence are fed back. Combined with evaluations based on quantiles and conditional value at risk, as well as default probability thresholds and minimum safe distance thresholds, security decisions with controllable violation probability, strong executability, and auditability and traceability are achieved.

[0183] 3. A sliding window model predictive control is adopted around the shifted region, with superimposed trust domain incremental constraints and monotonic risk constraints, and the upper limit is adaptively tightened based on the constraint residual of the previous time step; when it is infeasible, a conservative path is backed up without relaxing the monotonicity, thereby achieving stable updates with continuous path, low jitter and no risk deterioration. Attached Figure Description

[0184] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0185] Figure 1 This is a flowchart of an intelligent planning method for neurosurgical surgical pathways based on reinforcement learning proposed in this invention.

[0186] Figure 2 This is a schematic flowchart illustrating step S6 of the present invention, which involves stable replanning of the sliding window MPC around the shifted region mask. Detailed Implementation

[0187] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0188] refer to Figure 1 A method for intelligent planning of neurosurgical surgical pathways based on reinforcement learning, comprising:

[0189] S1. Perform external parameter calibration and spatial registration on preoperative imaging data and intraoperative observation data, convert the entry point and target point to navigation coordinates, and parameterize the instrument kinematics and executability constraints into feasible domain parameters according to instrument specifications and navigation system parameters.

[0190] S2. Based on the aligned preoperative imaging data, the patient-specific neural implicit model is invoked to infer point by point, forming a baseline risk field and a baseline confidence field.

[0191] S3. Based on the consistency difference between the aligned intraoperative observation data and the baseline risk field and baseline confidence field, extract the displacement region mask according to the threshold, estimate the differential homeomorphic deformation field, and estimate the risk residual and confidence residual within the mask by local residual branch. Update the baseline risk field and baseline confidence field with the deformation and residual to obtain the updated risk field and updated confidence field.

[0192] S4. Perform order-preserving calibration and consistency calibration on the updated risk value field and the updated confidence field in sequence to obtain the calibrated risk value field and the calibrated confidence field, and determine the default probability threshold and the minimum safe distance threshold based on their quantile statistics and tail risk measurement.

[0193] S5. Using the entry point and the target point as boundary conditions, the original action sequence and the safety margin sequence are generated by the policy network with policy branches and safety margin branches. The original action sequence is projected onto the feasible region according to the threshold and feasible region parameters to obtain the projected action sequence and calculate the constraint residual sequence. The sequence is rolled out in the calibrated risk value field and the calibrated confidence field to form a candidate trajectory that satisfies the boundary conditions. The reference path is selected by the reviewer network score.

[0194] S6. Set up a sliding window around the mask of the shifted region, take the reference path as the initial value and the published path at the previous time as the trust domain benchmark, establish and solve the model prediction control problem containing trust domain incremental constraints and monotonic risk constraints. If it is feasible, the updated path is obtained. If it is not feasible, a conservative backup path that satisfies the monotonic risk constraints is generated as the updated path.

[0195] S7. For the updated path, combine the constraint residual sequence with the calibrated risk value field and the calibrated confidence field to calculate the minimum safe distance and segmented risk, and compare them with the corresponding indicators of the published path at the previous time. Check the satisfaction status of the trust domain incremental constraint and monotonic risk constraint, and generate audit records containing constraint residual statistics, minimum safe distance and segmented risk comparison.

[0196] In this specific embodiment, S1 includes:

[0197] First, acquire preoperative imaging data and intraoperative observation data, collect entry point and target point, and read instrument specifications and navigation system parameters. Then, perform extrinsic parameter calibration based on navigation system parameters to establish spatial transformation parameters from image coordinate system to navigation coordinate system, with rigid model as the main model, which can be extended to affine or non-rigid model if necessary.

[0198] For consistent expression, the mapping of spatial points in any image coordinate system to the navigation coordinate system is expressed by the following formula:

[0199] ;

[0200] in, Representing the image coordinate system The three-dimensional point column vector, Representing the navigation coordinate system The corresponding three-dimensional point column vectors below Indicates from the image coordinate system To navigation coordinate system of Rotation matrix Indicates the pairing with Translation vector, the symbol "←" indicates mapping from the source coordinate system to the target coordinate system;

[0201] After obtaining the above spatial transformation parameters, spatial registration is performed on the intraoperative observation data and the registration quality index is used for evaluation. When the registration quality reaches the preset threshold, the registration is confirmed to be effective, thereby generating aligned preoperative image data and aligned intraoperative observation data.

[0202] Then, using the same spatial transformation parameters, the entry point and the target point are transformed to the navigation coordinate system to obtain the entry point and the target point in the navigation coordinate system.

[0203] Meanwhile, based on the instrument specifications and navigation system parameters, the instrument kinematics and executability constraints are standardized into feasible domain parameters, forming a parameterized description of the instrument kinematics and executability constraints. The feasible domain parameters include at least the entry angle range, minimum radius of curvature, path length range, attitude change range, upper velocity limit, upper acceleration limit, joint angle range, and joint velocity range.

[0204] Output aligned preoperative imaging data, aligned intraoperative observation data, entry point in navigation coordinates, target point in navigation coordinates, and parametric descriptions of instrument kinematics and executability constraints.

[0205] In this specific embodiment, S2 includes:

[0206] Within the navigation coordinate system, the volume of interest covering both the entry point and the target point in the navigation coordinate system and their surrounding safety margins is first determined. The entry point and target point are denoted as follows: and The volume of interest is denoted as ;

[0207] Three-dimensional coordinate points are selected within the volume of interest as query input, and a patient-specific implicit neural model is invoked point-by-point inference using aligned preoperative image data as conditions. The aligned preoperative image data is denoted as... ;

[0208] To standardize notation, the baseline risk value and baseline confidence at each query point are represented as ordered pairs, using the following relation:

[0209] ;

[0210] in This represents the queried 3D coordinate point located in the navigation coordinate system and satisfying the following conditions: ,symbol" "Indicates the membership relationship between sets, Represents the three-dimensional volume of preoperative image data aligned in the navigation coordinate system. This represents a patient-specific implicit neural model. This represents the parameter vector of the model. Indicates query point Baseline risk scalar value at the location Indicates query point The baseline confidence scalar is indicated by the semicolon ";". As conditional information participating in the same model, the left square brackets represent an ordered binary pair consisting of two scalars;

[0211] Then and Spatial mapping and resampling are performed based on 3D coordinate points to make them consistent with... The voxel mesh is aligned and covered in terms of origin, orientation, and voxel size. This forms the baseline risk value field and the baseline confidence field, respectively.

[0212] Output the baseline risk value field and the baseline confidence field.

[0213] In this specific embodiment, S3 includes:

[0214] Intraoperative observation data based on alignment within the navigation coordinate system are denoted as... The baseline risk field and the baseline confidence field are denoted as... and The consistency differences are evaluated point by point, the consistency difference value is calculated and the mask of the shifted region is extracted according to the preset threshold, and the stability of the mask is improved by connected component filtering and morphological processing.

[0215] Subsequently, low-rank parameterization of the steady velocity field was used to model brain tissue displacement. Zero-displacement boundary conditions were applied to the skull and dura mater surfaces obtained from aligned preoperative image data segmentation. A topologically preserving differential homeomorphic deformation field was obtained by time integration of the steady velocity field, and its forward mapping is denoted as... The reverse mapping is denoted as Within the masked region, the risk residuals and confidence residuals are estimated by local residual branching, denoted as follows: and ;

[0216] After completing deformation estimation and residual estimation, a mask indicator function is used. The modified region is confined within the shifted area, and the updated risk field and updated confidence field are generated according to the following relationship:

[0217] ;

[0218] in For querying 3D coordinate points located in the navigation coordinate system, Indicates query point The updated risk value scalar at the location, Indicates query point The updated confidence scalar at the location, and These represent the values ​​of the baseline hazard field and the baseline confidence field at the input location, respectively. Indicates the point The position mapped back from the deformed domain to the pre-deformed domain. The indicator function for the shifted region mask at point The value is 1 inside the mask and 0 outside the mask. and The square brackets represent the additive corrections to the baseline values ​​given by the local residual branches within the mask. Represents an ordered binary tuple consisting of two scalars;

[0219] Output differential homeomorphic deformation field , Displacement region mask Updated risk value field With the updated confidence field .

[0220] In this specific embodiment, S4 includes:

[0221] Using the updated risk field and the updated confidence field as inputs, denoted as follows: and ,in This indicates the queried 3D coordinate point located in the navigation coordinate system.

[0222] To ensure stable risk ranking and match the confidence level with the actual coverage, order-preserving calibration and consistency calibration are performed sequentially, and expressed by a unified formula:

[0223] ;

[0224] in Indicates the calibrated risk value scalar. Indicates the calibrated confidence scalar, The monotonically non-decreasing order-preserving calibration mapping function is used to correct numerical biases of risk while maintaining the relative order of spatial points. The square brackets indicate that the consistency calibration mapping function is used to statistically match the confidence level with the target coverage level. Represents an ordered binary tuple consisting of two scalars;

[0225] After obtaining the calibrated risk field and the calibrated confidence field, they are recorded according to the preset quantile levels as follows: right Quantile statistics are performed within the volume of interest, and the conditional value at risk is denoted as... As a measure of tail risk, a non-negative safety margin is introduced, denoted as... To mitigate extreme biases and model errors, based on this... Directly determined as the default probability threshold, and based on The isosurface location and numerical sensitivity (preferably obtained by using Lipschitz boundaries or numerical lookup tables) are compared with... and The minimum safe distance threshold for matching is denoted as ;

[0226] Output calibrated risk field Confidence field after calibration Default probability threshold With minimum safe distance threshold .

[0227] In this specific embodiment, S5 includes:

[0228] In the environments characterized by the calibrated risk field and the calibrated confidence field, respectively, they are denoted as... and The entry point and target point in the navigation coordinates are denoted as the boundary conditions respectively. and The policy network has policy branches and safety margin branches to generate the original action sequence and safety margin sequence, in discrete programming steps. The locations are respectively recorded as and ;

[0229] To ensure the motion is executable and meets the safety threshold, a differentiable projection layer is used to map the original motion to a feasible region. This feasible region is jointly determined by the parameterized description of the machine's kinematics and executability constraints, and the safety threshold. The parameterized description of the machine's kinematics and executability constraints is denoted as... The signs of the default probability threshold and the minimum safe distance threshold are: and The projection relationship is expressed in refined form as follows:

[0230] ;

[0231] in Indicates the first The projected motion vector of the step, Indicates the first Constraint residual indicator of step This represents a feasible region projection operator based on quadratic programming or a barrier function and possessing differentiability. This represents the parameters derived from the kinematics and executability constraints of the machine. and default probability threshold With minimum safe distance threshold Commonly defined action feasible region, square brackets This represents an ordered binary tuple consisting of two quantities;

[0232] Then, using the projection action sequence in and Scroll out and force compliance within the defined environment. and Boundary conditions are used to generate a set of candidate trajectories from different initial values ​​or random perturbations. These trajectories are then evaluated by a commentator network based on quantile distribution and conditional risk value. The candidate trajectories are comprehensively scored, and the trajectory with the highest score is selected as the reference path and denoted as . And denote the constrained residual sequence as ;

[0233] Output reference path With constrained residual sequence .

[0234] In this specific embodiment, S6 includes:

[0235] First, let the mask surrounding the shifted region be denoted as... The support range is determined by the sliding window that covers the range, and local optimization is performed only within the sliding window;

[0236] Using the reference path as the initial value for optimization, denoted as... The published path at the previous moment is used as the trust domain benchmark, denoted as... ,in Index for the current planning period;

[0237] Discrete sampling of the path within the window is performed on the calibrated risk value field and the calibrated confidence field to calculate the segmented risk. and ;

[0238] Based on this, a model predictive control optimization problem is constructed, which includes trust domain incremental constraints and monotonic risk constraints. The core solution form is written as follows:

[0239] subject to ;

[0240] in Indicates the update path for this period. This represents the candidate paths to be optimized within the sliding window. This indicates the feasible search domain defined by the sliding window. This represents the operator for finding the independent variable that minimizes the objective function. The weighted objective function representing the deviation relative to the reference path and the curvature increment. Represents the Hausdorff distance between two paths. This indicates the preset upper limit of the Hausdorff distance. A measure representing the increment of path curvature The preset upper limit for the curvature increment. Indicates that within the sliding window, based on and The calculated segment-aggregated risk vector and " Compare elements here. This indicates that the non-negative safety margin is used for monotonic risk constraints;

[0241] The solution is obtained by using the optimized solution from the previous time step for a hot start and... The internal numerical optimizer is called to search, and when a feasible solution that satisfies all constraints is found, it will... Published as an update path, and maintained when deemed infeasible. Based on the condition of monotonically non-increasing Generate a conservative alternative path and use it as The initial path generated based on preoperative imaging data will be used as the initial path during the first execution. use;

[0242] Output update path .

[0243] In this specific embodiment, S7 includes:

[0244] Within the navigation coordinate system, the updated path is denoted as... Combined with the constrained residual sequence, it is denoted as (in The number of planning steps, the calibrated risk field, and the calibrated confidence field are denoted as follows: and ,along The set of sampling points obtained by discrete sampling according to arc length is denoted as . (in (representing the number of sampling points), in each Inquiry and And aggregated by segment to obtain the segmented risk vector, denoted as ;

[0245] To assess the minimum safe distance, the risk isosurface set at each sampling point is denoted as . (Its definition is in) China satisfies The point set, Calculate the Euclidean distance (where the default probability threshold is used) and take the minimum value of the entire path. The definition is:

[0246] ;

[0247] in Represents the Euclidean distance function;

[0248] For constrained residual sequences The maximum value is denoted as The mean is denoted as The proportion of non-zero values ​​is denoted as To reflect the magnitude and frequency of feasible region projection correction;

[0249] Then The path published at the previous moment is recorded as Segmentation risk Compare segment by segment to check whether the monotonic risk constraint is met, that is, whether it is no higher than the standard at every point. Add non-negative safety margin Simultaneously, it calculates and checks whether the trust domain incremental constraint is satisfied, i.e., the Hausdorff distance is denoted as... Whether it does not exceed the upper limit is recorded as And the curvature increment measure is denoted as Whether it does not exceed the upper limit is recorded as and check Is it not less than the minimum safe distance threshold? ;

[0250] An audit record is generated when all checks pass. This includes constrained residual statistics. Minimum safe distance The conclusions of the segmented risk comparison and the status of the above constraints being met are then presented, using timestamps and path identifiers. and Associated storage;

[0251] If any check fails, the failed item should be marked in the audit log, and the corresponding indicator value and threshold should be recorded, including... , and To support traceability and compliance audits.

[0252] In this specific embodiment, it also includes:

[0253] For differentiable projection layers, in each discrete programming step (in When performing feasible region projection (indicating the number of planning steps), in addition to obtaining the constraint residual indicators... In addition, the KKT multiplier vector and the optimality residual vector are returned from the quadratic programming or barrier function solver, denoted as follows: and The two are then combined to form the constraint margin, denoted as... The constraint margin is extracted from dual information and optimality evidence through weighted norm aggregation, and the preferred relation is:

[0254] ;

[0255] in Indicates the first Constraint margin scalar of step Indicates the first Constraint residual indicator of step Indicates the discrete programming step index, Indicates the total number of planning steps, This represents the KKT multiplier vector (as the set of dual variables for the constraints, containing multiplier entries for inequalities and equality constraints) returned by the differentiable projection layer solver. This represents the optimality residual vector (which includes the evidence vector composed of stationarity residuals, original feasibility residuals, and complementary residuals). and Denotes the non-negative weighting coefficients of the KKT multipliers and the optimality residuals, respectively. and Representing vectors respectively Norm and Norm;

[0256] Then and Associated and merged into the constrained residual sequence using the same index. The stored entries are used to generate audit logs in subsequent steps and serve as traceable evidence of optimality and constraint satisfaction.

[0257] In this specific embodiment, it also includes:

[0258] For the incremental constraints in the trust domain, a strategy of adaptive tightening of the constraint residuals based on the previous time step is adopted: the index in the planning period is denoted as... Within this framework, we first summarize the sequence of constraint residuals generated by the projection of the feasible region at the previous time step:

[0259] ;

[0260] in Indicates the discrete programming step index, This represents the total number of planning steps, and its maximum value is denoted as . As a signal of tightening;

[0261] Subsequently, the upper bounds of Hausdorff distance and curvature increment are adaptively updated based on residual strength, using a refined relation:

[0262] ;

[0263] in This represents the upper limit scalar of the Hausdorff distance used for incremental constraints in the trust domain during this period. This represents the upper limit scalar of curvature increment used for incremental constraints in the trust domain during this period. and These represent the non-negative lower limit set to prevent over-tightening, respectively. and These represent the upper limit of the system's baseline without tightening, respectively. and Representing the nonnegative tightening coefficients for sensitivity to constraint residuals, respectively. Square brackets represent the maximum value of the constrained residual sequence at the previous time step. Represents an ordered binary tuple consisting of two scalars. This operator represents the larger of the two values;

[0264] The above updates make when Approaching zero time and Approaching their respective baseline upper limits, when As the value increases, both functions adaptively tighten according to a monotonic function and do not fall below their respective lower limits. Furthermore, in the numerical implementation, adjustments can be made to... Outlier suppression and moving averages are employed to improve robustness, and the updated model predictive control will ultimately be used in this cycle. and These are used to limit the Hausdorff distance and curvature increment between the published path in the previous time step and the current candidate path, respectively, thereby achieving adaptive trust domain management where "the larger the constraint residual in the previous time step, the smaller the corresponding upper limit in the current time step".

[0265] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0266] This invention directly addresses the technical problem of "real-time replanning and stable updating during intraoperative brain displacement" through a synergistic combination of "deformable neural field with residual confinement, safety reinforcement learning decision-making, and sliding window model predictive control." Patient-specific baseline risk and confidence are rapidly updated within the displacement mask via low-rank steady-state velocity field deformation and local residuals, and then undergo order-preserving and consistency calibration to generate reliable default probability thresholds and minimum safe distance thresholds. A policy and safety dual-head network is forced to satisfy instrument kinematics and safety constraints within a differentiable projection layer, outputting a reference path. Subsequently, incremental constraints of the trust domain and piecewise risk monotonic constraints are applied around the affected region using sliding window model predictive control; if infeasibility occurs, the path reverts to a conservative one without relaxing monotonicity. This linkage achieves continuous and smooth path updates under brain displacement conditions, controllable constraint satisfaction of default probability and safe distance, and auditable traceability including constraint residuals and optimality evidence.

[0267] In terms of algorithm structure, this invention specifically introduces three key improvements to enhance the technical effect: First, it adopts a restricted update of "baseline neural field plus masked local residuals" and applies zero-displacement boundaries to the skull and dura mater to avoid global distortion and stabilize risk ranking, providing a reliable foundation for subsequent threshold setting; Second, it uses a strategy-safety dual-head and differentiable projection layer in combination to back-propagate constraint residuals and dual information, and works with commentators on quantiles and conditional risk values ​​to strengthen tail risk management and improve executability and auditability; Third, the sliding window model predictive control stabilizes path evolution with internal and external dual constraints (incremental trust domain of Hausdorff distance and curvature, and piecewise risk monotonicity), and can adaptively tighten the upper limit based on the constraint residuals of the previous time step, improving the real-time performance and robustness of replanning while ensuring that safety does not deteriorate.

Claims

1. A method for intelligent planning of neurosurgical surgical pathways based on reinforcement learning, characterized in that, include: S1. Perform external parameter calibration and spatial registration on preoperative imaging data and intraoperative observation data, convert the entry point and target point to navigation coordinates, and parameterize the instrument kinematics and executability constraints into feasible domain parameters according to instrument specifications and navigation system parameters. S2. Based on the aligned preoperative imaging data, the patient-specific neural implicit model is invoked to infer point by point, forming a baseline risk field and a baseline confidence field. S3. Based on the consistency difference between the aligned intraoperative observation data and the baseline risk field and baseline confidence field, extract the displacement region mask according to the threshold, estimate the differential homeomorphic deformation field, and estimate the risk residual and confidence residual within the mask by local residual branch. Update the baseline risk field and baseline confidence field with the deformation and residual to obtain the updated risk field and updated confidence field. S4. Perform order-preserving calibration and consistency calibration on the updated risk value field and the updated confidence field in sequence to obtain the calibrated risk value field and the calibrated confidence field, and determine the default probability threshold and the minimum safe distance threshold based on their quantile statistics and tail risk measurement. S5. Using the entry point and the target point as boundary conditions, the original action sequence and the safety margin sequence are generated by the policy network with policy branches and safety margin branches. The original action sequence is projected onto the feasible region according to the threshold and feasible region parameters to obtain the projected action sequence and calculate the constraint residual sequence. The sequence is rolled out in the calibrated risk value field and the calibrated confidence field to form a candidate trajectory that satisfies the boundary conditions. The reference path is selected by the reviewer network score. S6. Set up a sliding window around the mask of the shifted region, take the reference path as the initial value and the published path at the previous time as the trust domain benchmark, establish and solve the model prediction control problem containing trust domain incremental constraints and monotonic risk constraints. If it is feasible, the updated path is obtained. If it is not feasible, a conservative backup path that satisfies the monotonic risk constraints is generated as the updated path. S7. For the updated path, combine the constraint residual sequence with the calibrated risk value field and the calibrated confidence field to calculate the minimum safe distance and segmented risk, and compare them with the corresponding indicators of the published path at the previous time. Check the satisfaction status of the trust domain incremental constraint and monotonic risk constraint, and generate audit records containing constraint residual statistics, minimum safe distance and segmented risk comparison.

2. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S1 includes: After acquiring preoperative imaging data, intraoperative observation data, entry point, target point, instrument specifications and navigation system parameters, external parameter calibration is performed on the preoperative imaging data and intraoperative observation data according to the navigation system parameters to obtain the spatial transformation parameters from the image coordinate system to the navigation coordinate system. The spatial transformation parameters are used to spatially register the intraoperative observation data, and the registration is confirmed to be effective when the registration quality reaches a preset threshold, thereby generating aligned preoperative image data and aligned intraoperative observation data. Then, the spatial transformation parameters are used to transform the entry point and the target point to the navigation coordinate system, generating the entry point and the target point in the navigation coordinate system. Simultaneously, based on the instrument specifications and navigation system parameters, the instrument kinematics and executability constraints are standardized into feasible domain parameters, forming a parameterized description of the instrument kinematics and executability constraints. The feasible domain parameters include at least the entry angle range, minimum radius of curvature, path length range, attitude change range, upper velocity limit, upper acceleration limit, joint angle range, and joint velocity range. Output aligned preoperative imaging data, aligned intraoperative observation data, entry point in navigation coordinates, target point in navigation coordinates, and parametric descriptions of instrument kinematics and executability constraints.

3. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S2 include: Within the navigation coordinate system of the aligned preoperative image data, the volume of interest covering both and their surrounding safety margin is determined based on the entry point and target point in the navigation coordinate system. Within the volume of interest, three-dimensional coordinate points are used as query input, and aligned preoperative image data are used as conditions to invoke patient-specific neural implicit models, and the corresponding baseline risk value and baseline confidence are obtained point by point. The baseline risk values ​​are mapped to three-dimensional coordinate points to form a baseline risk value field, and the baseline confidence scores are mapped to three-dimensional coordinate points to form a baseline confidence field. The baseline risk value field and the baseline confidence field are aligned with the voxel grid of the aligned preoperative image data and cover the volume of interest. Output the baseline risk value field and the baseline confidence field.

4. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S3 includes: Within the navigation coordinate system, based on the consistency difference between the aligned intraoperative observation data and the baseline risk field and baseline confidence field, the consistency difference value is calculated by spatial point and the mask of the displaced region is extracted according to the preset threshold. Brain tissue displacement is modeled using low-rank parameterization of a steady velocity field, which is represented by a linear combination of a finite number of basis functions. Zero-displacement boundary conditions are applied to the skull surface and dura mater surface determined by aligned preoperative image data. A topologically consistent differential homeomorphic deformation field is generated by time integration of the steady velocity field. Risk residuals and confidence residuals are estimated by local residual branch within the domain of the shifted region mask; The differential homeomorphic deformation field is applied to the baseline risk field and the baseline confidence field, and the corresponding risk residuals and confidence residuals are added at spatial points to obtain the updated risk field and the updated confidence field. Output the differential homeomorphic deformation field, the shifted region mask, the updated risk field, and the updated confidence field.

5. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S4 includes: Based on the updated risk value field and the updated confidence field, first perform order-preserving calibration to maintain the consistency of risk ranking among spatial points, and then perform consistency calibration to make the coverage relationship of confidence to actual error conform to the preset coverage level, thereby generating the calibrated risk value field and the calibrated confidence field. On the calibrated risk value field and the calibrated confidence field, quantile statistics are performed according to the preset quantile level, and the safety margin is determined in combination with the tail risk measure, so that the minimum safety distance threshold matching the default probability is obtained under the condition that the default probability does not exceed the preset quantile level, and the preset quantile level is determined as the default probability threshold. Output the calibrated risk value field, calibrated confidence field, default probability threshold, and minimum safe distance threshold.

6. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S5 include: In the environment characterized by the calibrated risk value field and the calibrated confidence field, the original action sequence and safety margin sequence are generated by the policy network with policy branch and safety margin branch, using the entry point and target point in navigation coordinates as boundary conditions. Based on the default probability threshold, minimum safe distance threshold, and parameterized description of the kinematics and executability constraints of the device, the original action sequence is projected into the feasible region within the differentiable quadratic programming and barrier function projection layer to obtain the projected action sequence and calculate the constraint residual sequence. The projected action sequence is rolled out in the calibrated risk value field and the calibrated confidence field to obtain a set of candidate trajectories that satisfy the entry point and the target point under the navigation coordinates. The critic network scores the set of candidate trajectories based on quantile distribution and conditional value at risk, and selects the trajectory with the highest score as the reference path. Output the reference path and constraint residual sequence.

7. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S6 include: A sliding window covering the shifted region mask is determined around the shifted region mask. The reference path is used as the initial value for optimization, the previously published path is used as the trust domain benchmark, and the segmented risk of the path is calculated on the calibrated risk value field and the calibrated confidence field. Within the sliding window, a model predictive control optimization problem is established. The goal is to minimize the deviation and curvature increment relative to the reference path while satisfying the trust domain increment constraint and the monotonic risk constraint. The trust domain increment constraint limits the Hausdorff distance and curvature increment of the new path relative to the previously published path to not exceed a preset upper limit. The monotonic risk constraint limits the segment risk of the new path within the sliding window to not be higher than the risk of the corresponding segment of the previously published path plus a preset safety margin. The optimization solution from the previous time step is used for hot-start solution. If the solution is feasible, the new path obtained is used as the update path. If the solution is not feasible, a conservative backup path that satisfies the monotonic risk constraint is generated and the conservative backup path is used as the update path. During the initial execution, the previously published path is the initial path generated based on preoperative image data; Output the update path.

8. The intelligent planning method for neurosurgical surgical pathways based on reinforcement learning according to claim 1, characterized in that, S7 includes: Within the navigation coordinate system, for the updated path, combining the constraint residual sequence and the calibrated risk value field and calibrated confidence field, the risk value and confidence of each sampling point are calculated by discrete sampling along the updated path, and the segmented risk is obtained accordingly. Based on the default probability threshold and the minimum safe distance threshold, the safe distance at each sampling point is calculated so that the risk does not exceed the default probability threshold, and the minimum value of the whole path is taken as the minimum safe distance. The constraint residual sequence is statistically analyzed to obtain the maximum value, mean, and proportion of non-zero constraint residuals, thereby reflecting the correction magnitude of the feasible region projection. The segmented risk and minimum safe distance of the updated path are compared with the risk and minimum safe distance of the corresponding segment of the published path at the previous time step, and the satisfaction status of the trust domain incremental constraint and monotonic risk constraint is checked. When the inspection passes, an audit record is generated that includes statistics on constraint residuals, minimum safe distance, and comparison results of segmented risks. When the inspection fails, the failed items are marked in the audit record and the corresponding indicator values ​​and thresholds are recorded. Output the audit log and store it in association with the update path for subsequent iterations.

9. The method according to claim 1, characterized in that, In step S5, the KKT multipliers and optimality residuals returned from the differentiable projection layer are aggregated into constraint margins and incorporated into the constraint residual sequence as evidence for subsequent audit records.

10. The method according to claim 1, characterized in that, In step S6, the upper limit of the Hausdorff distance and curvature increment of the trust domain incremental constraint adaptively tightens as the maximum value of the constraint residual sequence at the previous time step increases. The larger the constraint residual at the previous time step, the smaller the corresponding upper limit at this time step.

Images