A medical micro-robot navigation method for security corridor constraint

Abstract

The application relates to the technical field of intelligent medical navigation and neural network modeling, and discloses a medical micro-robot navigation method for safety corridor constraints. The method comprises the following steps: firstly, an environment model containing dynamic safety corridor parameters is constructed by fusing blood vessel structures, obstacle distribution and historical navigation feedback data; then, based on the model, a navigation state is constructed by fusing path progress, corridor deviation and obstacle perception, and a composite reward function is designed for reinforcement learning configuration; subsequently, interactive sampling and network training are carried out in a simulation environment to generate a navigation strategy; finally, control instructions and navigation trajectories are generated by using the strategy, and execution feedback is formed to optimize the model. The application effectively improves the safety, learning efficiency and adaptability of navigation, and reduces the collision risk.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical navigation and neural network modeling technology, and in particular to a navigation method for medical microrobots oriented towards safety corridor constraints. Background Technology

[0002] In the field of intelligent medical navigation and neural network modeling technology, existing solutions for intravascular navigation of medical microrobots typically construct vascular models and preset paths based on vascular structure map data, and then combine them with collision detection, threshold control, or deep learning navigation structures that do not explicitly introduce safety corridor constraints. These solutions suffer from limitations such as difficulty in accurately describing grid passage and boundary distances under vascular safety corridor constraints, difficulty in unifying obstacle distribution and navigation execution feedback into state representation and reward modeling, and difficulty in simultaneously considering navigation efficiency and clinical safety requirements. Existing methods mostly rely on static vascular structure map data and manually set safety rules, or on empirically adjusted reinforcement learning parameters. Under conditions of vascular safety corridor contraction, complex vascular geometry, and dynamic obstacle constraints, navigation trajectories are prone to deviating from the vascular centerline, and frequent collisions or boundary violations occur near the safety corridor boundaries. This makes it difficult to achieve stable generation of navigation control commands and navigation trajectories for medical microrobots within vascular safety corridors. For the joint processing of vascular structure map data with corridor constraint navigation state construction, composite reward modeling, and corridor constraint navigation control command generation, existing technologies generally suffer from fragmentation in several aspects, including information fusion between the vascular safety corridor constraint environment structure and robot pose data and target point configuration parameters, hierarchical utilization of corridor offset fields, boundary distance fields, and obstacle type labels in composite reward modeling, and targeted configuration of safety-related features in corridor constraint reinforcement learning hyperparameter assembly. This makes it difficult to form a continuous link from data acquisition, environment alignment, state determination, control command generation to navigation execution feedback recording in the application scenario of medical microrobots navigating along safe corridors within blood vessels. As a result, the generation of corridor constraint navigation control commands and navigation trajectory generation is insufficient in response to vascular safety corridor constraints and clinical safety requirements, which in turn adversely affects the safety, controllability, and workflow efficiency of interventional navigation operations. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a medical microrobot navigation method oriented towards safety corridor constraints, comprising: Acquire vascular structure map data, obstacle distribution data and navigation execution feedback structure, perform centerline extraction and passable area labeling processing, and generate vascular safety corridor constraint environment structure; Based on the constrained environment structure of the vascular safety corridor, we construct the corridor constrained navigation state, model the composite reward, and perform hyperparameter assembly processing for corridor constrained reinforcement learning to generate the corridor constrained reinforcement learning configuration structure. Based on the corridor-constrained reinforcement learning configuration structure, we perform corridor-constrained navigation interaction sampling, experience dataset construction, and policy value network parameter update processing to generate the corridor-constrained policy value network structure. Obtain the value network structure of the corridor constraint strategy, perform corridor constraint navigation control command generation, navigation trajectory generation and navigation execution feedback statistical processing to obtain the navigation execution feedback structure.

[0004] Furthermore, the process of centerline extraction also includes: The centerline extraction process includes applying distance transformation to calculate the distance from the blood vessel voxel to the background, using thinning and skeleton extraction algorithms to shrink the blood vessel cavity to obtain candidate skeleton points, performing path tracing and branch identification according to the blood vessel topology to generate candidate centerline curves, and combining historical successful trajectories, curvature, and failure events in the navigation execution feedback structure to select the main trunk centerline. Finally, a centerline point list with spatial coordinates, local tangential direction, curvature estimation, and cross-sectional radius estimation values ​​is generated.

[0005] Furthermore, the process of marking accessible areas also includes: The process of marking passable areas includes calculating the safe and recommended passable radii for each centerline point based on the minimum inner diameter of the blood vessel cross-section, the wall roughness, and the distance to obstacle elements. For each voxel, the distances to the centerline point, the blood vessel wall boundary, and obstacle elements are calculated and classified into passable areas, prohibited areas, or high-risk areas according to safe passage rules. At the same time, the failure event trajectory points in the navigation execution feedback structure are mapped to the coordinate space and expanded to form the affected area. The original passable area voxels within the area are marked as high-risk areas and the failure event type is recorded.

[0006] Furthermore, the process of constructing the corridor-constrained navigation state also includes: The corridor-constrained navigation state construction process includes mapping the robot pose to the local coordinate system of the safe corridor, calculating the path progress and lateral distance of the current position in the centerline point column to obtain the corridor offset field and boundary distance field, and retrieving the surrounding obstacle mesh cells to count the nearest obstacle distance and distribution direction to encode as local obstacle perception field, forming a structured corridor-constrained navigation state vector. At the same time, a short-term historical context is added to record historical poses and control commands through a rolling buffer, downsampling and pruning to obtain fixed-length historical state fragments, which are then concatenated with the current state to form an extended navigation state structure.

[0007] Furthermore, the process of modeling composite rewards also includes: The composite reward modeling process includes constructing a basic task reward based on the path progress difference between the robot's current position and the target point, generating a safety penalty based on the corridor offset field and the boundary distance field to classify the safety level, generating a smoothness penalty based on the movement distance and turning angle changes calculated based on historical pose changes and control commands, and generating a severe penalty and termination mark when the robot goes out of the safe corridor or enters the prohibited area by combining obstacle type markings. Finally, the basic task reward, safety penalty, smoothness penalty and termination reward are combined to form a composite reward value.

[0008] Furthermore, the process of assembling hyperparameters in corridor-constrained reinforcement learning also includes: The corridor-constrained reinforcement learning hyperparameter assembly process includes selecting the hidden layer size and number of layers of the policy network and value network based on the state vector dimension and action space range to generate network structure description fields; reading the learning rate, discount factor, target network update step size, experience replay batch size and buffer capacity; configuring the perturbation intensity, decay rate and minimum threshold of the exploration behavior; and the weight decay factor, gradient pruning threshold and soft update factor for network stability. Finally, these are uniformly organized into a set of corridor-constrained reinforcement learning hyperparameter fields, and the corridor-constrained reinforcement learning configuration structure is generated by encapsulating the state space definition, reward composition, network settings and training parameters.

[0009] Furthermore, the process of conducting corridor-constrained navigation interaction sampling also includes: The corridor-constrained navigation interactive sampling process includes parsing training round parameters into simulation rounds, resetting the environment based on the entrance and target positions in the simulated vascular environment instance and returning to the initial corridor-constrained navigation state, inputting the state vector into the policy network to output continuous action variables, configuring and mapping them to propulsion speed and steering adjustment through the robot control interface, driving the environment to update the robot's position, updating the pose and recalculating the corridor offset, boundary distance, and obstacle proximity if the robot points to a passable grid, calling the composite reward calculation logic to generate a single-step reward value and termination flag, clipping the position and setting the termination flag if the robot points to a prohibited area, and recording the interactive samples of state, action, next state, and termination flag.

[0010] Furthermore, the process of constructing the experience dataset also includes: The experience dataset construction process includes writing interaction samples into the experience replay buffer. Each sample records a state vector, action vector, reward scalar, next state vector, termination flag, and safety corridor auxiliary fields such as corridor number, boundary distance level, and obstacle type label. The sample priority is calculated based on the termination flag and auxiliary fields, and higher priority is assigned to high-risk samples. Before the training batch, training mini-batches are generated from the buffer according to the batch size, and the state vector, action vector, and reward are preprocessed.

[0011] Furthermore, the process of updating policy value network parameters also includes: The policy-value network parameter update process includes generating a target value estimate using the next state vector from the training mini-batch through the target policy network and the target value network; constructing the target reward by combining the single-step reward and termination flag; constructing the value network loss function by comparing the difference with the predicted value of the online value network; updating the online value network parameters through backpropagation and gradient pruning; then using the state vector to output the action through the online policy network and inputting it into the online value network to obtain the value estimate; constructing the policy network loss function with the negative mean of the value estimate; weighting the gradient contribution using the corridor offset field and the boundary distance field; updating the online policy network parameters; and finally performing a soft update on the target network. This process alternates between interactive sampling and parameter updates until the training termination condition is met, generating a corridor-constrained policy-value network structure containing policy network weights, value network weights, and target network weights.

[0012] Furthermore, the process of generating corridor-constrained navigation control commands also includes: The corridor constraint navigation control command generation process includes calling the scene instantiation interface to select the entrance grid and target grid in the passable corridor, setting the robot's virtual pose and generating an initial corridor constraint navigation state vector, inputting the state vector into the policy network for forward inference in each control cycle to obtain continuous motion vectors, performing amplitude clipping and numerical smoothing according to the motion range and clipping rules, configuring the drive motor control quantity and attitude adjustment quantity mapped to the control channel through the robot control interface, and forming a corridor constraint navigation control command sequence.

[0013] The key innovations of this invention include: (1) In the construction of the corridor constraint navigation state, the vascular mesh, center line and safety corridor parameters, mesh passage marker, boundary distance level and obstacle type marker in the vascular safety corridor constraint environment structure are aligned in time and mapped in space with the robot pose data and target point configuration parameters in a unified vascular principal coordinate system. The pose information, path progress, corridor offset field, boundary distance field and obstacle perception field are combined in a fixed order to form a structured corridor constraint navigation state vector and extended navigation state structure.

[0014] (2) In the composite reward modeling part, based on the corridor constraint navigation state and the safety corridor parameters in the vascular safety corridor constraint environment structure, a reward signal composed of multiple components is generated for each state transition. The path progress difference between the robot's current position and the target position described by the target point configuration parameters is formed into the basic task reward field. The safety level is divided according to the corridor offset field and the boundary distance field. The states of going out of the recommended passage range, approaching or contacting the boundary, and going out of the safety corridor or entering the prohibited area are mapped into graded safety penalties respectively. At the same time, the movement distance and turning angle changes are calculated from the historical pose changes and control commands to form a smoothness penalty field. When there is a collision, going out of the boundary or reaching the target area, a termination reward or termination penalty is added. The basic task reward, safety penalty, smoothness penalty and termination mark together constitute the composite reward modeling field set, and the single-step composite reward value is obtained by combining them according to the preset weight coefficients.

[0015] (3) In the hyperparameter assembly process of corridor-constrained reinforcement learning, control parameters related to deep reinforcement learning training are read from the algorithm configuration module and parameter tuning record. Based on the state vector dimension, action space range and geometric complexity of the safe corridor, the hidden layer size and network layer number are selected for the policy network and value network and encoded as network structure description fields. The learning rate, discount factor, target network update step size, experience replay batch size and experience buffer capacity are uniformly organized with the exploration perturbation intensity, perturbation decay rate, minimum perturbation threshold, weight decay factor, gradient pruning threshold and target network soft update factor of continuous action output, and written into the corridor-constrained reinforcement learning hyperparameter field set. It is then uniformly encapsulated with the state field definition and composite reward modeling field set into the corridor-constrained reinforcement learning configuration structure.

[0016] The following are its main beneficial effects: (1) By explicitly introducing vascular mesh, centerline and safety corridor parameters, mesh passage markers, boundary distance levels and obstacle type markers in the construction of corridor constraint navigation state, and jointly encoding them with robot pose data and target point configuration parameters into a structured corridor constraint navigation state vector and extended navigation state structure, the reinforcement learning model can simultaneously perceive path progress, corridor offset, boundary distance and local obstacle distribution in the decision at each moment. Compared with the existing scheme based only on simple pose or grid occupancy, it can more comprehensively reflect the relationship between medical microrobot and vascular safety corridor in scenarios with complex vascular geometry and shrinking safety corridor, thus providing a finer-grained state basis for subsequent composite reward modeling and corridor constraint navigation control command generation in continuous links.

[0017] (2) By forming the path progress difference into the basic task reward field in the composite reward modeling, and mapping the corridor offset field and boundary distance field into graded safety penalties, the state of going out of the safety corridor or entering the prohibited area is combined with the obstacle type label to form a severe penalty. At the same time, the smoothness penalty field is constructed using historical pose changes and control commands, and termination reward or termination penalty is introduced when there is a collision, boundary crossing or reaching the target area. The basic task reward, safety penalty, smoothness penalty and termination label are unified into a composite reward modeling field set and a single-step composite reward value is generated. This makes the corridor constraint reinforcement learning configuration structure trained no longer rely solely on the single indicator of reaching the target or avoiding collision when facing vascular safety corridor constraints and clinical safety needs. Instead, it provides targeted feedback on behaviors such as deviating from the safety corridor, approaching the boundary and drastic changes in trajectory around the key fields in the corridor constraint navigation state. This is beneficial to reduce the impact of trajectory deviation and boundary crossing behavior on the safety of interventional operations during actual intravascular navigation.

[0018] (3) In the hyperparameter assembly process of corridor constraint reinforcement learning, the network structure description field and control parameters such as learning rate, discount coefficient, target network update step size, experience replay batch size, experience buffer capacity, exploration perturbation intensity, perturbation decay rate, minimum perturbation threshold, weight decay coefficient, gradient pruning threshold, and target network soft update coefficient are uniformly organized into a set of hyperparameter fields for corridor constraint reinforcement learning. This is then encapsulated with the state field definition and composite reward modeling field set into a configuration structure for corridor constraint reinforcement learning. This allows the convergence characteristics and stability of the policy network and value network to be adjusted in a targeted manner when facing different vascular safety corridor geometries and different action space requirements. Compared with existing methods that rely on trial and error, a stable configuration foundation is formed in the complete link of corridor constraint navigation interaction sampling and policy value network parameter update, thereby reducing the adverse effects of parameter mismatch on the corridor constraint policy value network structure and subsequent corridor constraint navigation control command generation during the training process. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a medical microrobot navigation method oriented towards safety corridor constraints, provided as an embodiment of this application. Detailed Implementation

[0020] Example 1: Refer to Figure 1 This is a flowchart illustrating a medical microrobot navigation method oriented towards safety corridor constraints, provided by an embodiment of the present invention. The process may include at least steps S100-S400: S100: Acquire vascular structure map data, obstacle distribution data and navigation execution feedback structure, perform centerline extraction and passable area labeling processing, and generate vascular safety corridor constraint environment structure; S200: Based on the constrained environment structure of the vascular safety corridor, construct the corridor constrained navigation state, model the composite reward, and perform hyperparameter assembly processing of corridor constrained reinforcement learning to generate the corridor constrained reinforcement learning configuration structure. S300: Based on the corridor constraint reinforcement learning configuration structure, perform corridor constraint navigation interaction sampling, experience dataset construction and policy value network parameter update processing to generate the corridor constraint policy value network structure. S400: Obtain the value network structure of the corridor constraint strategy, perform corridor constraint navigation control command generation, navigation trajectory generation and navigation execution feedback statistical processing to obtain the navigation execution feedback structure.

[0021] In S100, vascular structure map data, obstacle distribution data and navigation execution feedback structure are acquired, centerline extraction and passable area labeling are performed, and vascular safety corridor constraint environment structure is generated. In this step, the system first reads the previously prepared vascular structure map data, obstacle distribution data, and navigation execution feedback structure in the data access layer, using these three types of data as a unified input source. The vascular structure map data originates from vascular-related medical images or 3D vascular models reconstructed from medical images, specifically including voxel-level or mesh-level density information, vascular cavity region labels, and background tissue labels. The obstacle distribution data comes from automatic analysis and manual annotation of vascular images, specifically including spatial location and extent descriptions of plaque areas prone to blood flow restriction or device collisions, stent and packing material areas, bifurcation and sharp bend areas, etc. The navigation execution feedback structure is compiled from the navigation logs of the microrobot in the in vitro model or simulation environment, containing a time-sequential sequence of robot poses, control command sequences, collision or boundary violation event markers, and corresponding locations. During the input phase, the system performs format standardization and coordinate alignment processing on the above data, establishing a vascular master coordinate system. Under this coordinate system, all spatial coordinates, voxel indices, and trajectory points use unified units and direction definitions, forming a data foundation that can be directly called upon in subsequent calculations in this step.

[0022] After standardizing the input data, the system performs vascular cavity extraction and topological analysis on the vascular structure map data. Specifically, the system separates voxels or mesh elements belonging to vascular cavities from the background region through steps such as intensity thresholding, connected component analysis, and morphological filtering, obtaining a set of voxels labeled as vascular cavities. For this set, the system further analyzes connectivity, dividing it into multiple continuous vascular segments, each corresponding to a trunk or branch in the actual anatomy. To facilitate subsequent centerline extraction and corridor description, the system assigns a unique segment number to each vascular segment and records the segment's start point, end point, and relationships with adjacent segments, forming a preliminary description of the vascular topological framework. The output of the above processing can be considered as a set of vascular cavity regions and their topological description; these remain within the internal structure of the vascular structure map data, providing the geometric basis for the centerline extraction step.

[0023] After obtaining the set of vascular cavity regions, the system performs a vascular centerline extraction operation. Specifically, the system applies a distance transformation operation to the vascular cavity region, calculating the distance from each vascular voxel to the surrounding background region, and considering voxels with larger distances as positions closer to the vascular central axis. Subsequently, the system applies a thinning algorithm and a skeleton extraction algorithm within the vascular cavity to gradually shrink the vascular cavity region, reducing the cross-section from the original multi-voxel region to one or a few thin lines, thus obtaining a set of candidate skeleton points. For the set of candidate skeleton points, the system performs path tracing and branch identification according to the connectivity of the vascular topology, connecting continuous skeleton points into several candidate centerline curves. For each candidate curve, the system combines the historical successful trajectories in the navigation execution feedback structure, comparing the routes successfully traversed by the robot with the candidate centerlines. When multiple candidate paths compete, the system prioritizes the path with higher overlap with historical successful trajectories, lower curvature, and no failure events as the main centerline. Finally, the system establishes an ordered point sequence for each determined centerline, with each centerline point carrying spatial coordinates, local tangential direction estimates, local curvature estimates, and cross-sectional radius estimates. The aforementioned centerline point list and its corresponding geometric attributes constitute the key intermediate results of the centerline extraction process in this step.

[0024] Based on the centerline generation, the system further combines obstacle distribution data with navigation execution feedback structure to mark the traversable area inside the blood vessel. In this invention, the traversable area is defined as the spatial region that a microrobot can occupy without contact with the blood vessel wall or collision with obstacles, given its external dimensions, turning ability, and control bandwidth. Specifically, for each centerline point, the system calculates the safe passage radius and recommended passage radius suitable for the microrobot based on the minimum inner diameter of the blood vessel cross-section at that location, the wall roughness, and the distance to the confirmed obstacle elements. For each voxel or grid point in the blood vessel structure map data, the system calculates its distance to the nearest centerline point, its distance to the blood vessel wall boundary, and its distance to the nearest obstacle element, and classifies them according to preset safe passage rules. When the voxel's distance from the centerline is less than the safe passage radius and its distance from the blood vessel wall and all types of obstacle elements are greater than the corresponding safety margin, the voxel is marked as a traversable area; when any distance does not meet the safety rules, the voxel is marked as a prohibited area or a high-risk area.

[0025] To fully utilize the real navigation risk information reflected in the navigation execution feedback structure, the system also corrects the areas near trajectory points that have experienced boundary crossings, collisions, or control instability during the traversable area labeling process. Specifically, the system retrieves all trajectory points marked as failure events in the navigation execution feedback structure, maps these trajectory points to the coordinate space of the vascular structure map, and expands them around a certain spatial range to form the failure event impact area. For voxels within these areas that were originally marked as traversable areas, the system adjusts their markings to high-risk areas, while recording the type of failure event and its relationship with the nearest obstacle for these voxels. This processing step ensures that traversable area labeling not only relies on geometric calculation results but also incorporates empirical information from historical navigation behavior, providing a foundation for subsequent parametric modeling of safe corridors that more closely approximates real navigation conditions.

[0026] After obtaining the centerline point sequence and the accessible area annotations, the system performs parametric modeling of the safety corridor. In this invention, the safety corridor is defined as a spatial strip-shaped area surrounding the centerline of the blood vessel. Its interior is the recommended path for robot navigation, the area near the corridor boundary is a high-risk area, and the area outside the corridor is a prohibited area. Specifically, the system constructs a local coordinate system at each centerline point, using the centerline point sequence as the main axis. This local coordinate system points tangentially to the navigation direction along the centerline, and defines normal and parallel normal directions in a plane perpendicular to the tangent, thus forming a set of local coordinate frames that move with the curve shape. After transforming the voxel coordinates in the accessible area annotations into the various local coordinate frames, the system detects the range of voxels belonging to the accessible area on each cross-section, determines the minimum safe radius, recommended accessible radius, and maximum accessible radius along the radial direction, and records these radius values ​​along with fields such as obstacle density, local curvature, and historical failure event density as the safety corridor parameters for that cross-section.

[0027] To obtain a continuously varying safe corridor model along the centerline, the system interpolates and smooths the safe corridor parameters at each cross-section. For areas with relatively gentle parameter changes, the system uses linear or spline interpolation to generate corridor radius and risk level functions continuously defined for the centerline point series, avoiding abrupt changes between adjacent cross-sections. For areas with bifurcations, sharp bends, or significantly increased obstacle density, the system uses more conservative rules, explicitly expressing local complexity in the parametric model by reducing the safe radius, increasing the risk level, and adding boundary weight fields. Furthermore, the system assigns a unique corridor number to each safe corridor, associating each point in the centerline point series with a specific corridor number and its positional parameters along the route, providing accurate corridor positioning information for subsequent navigation state coding and reward modeling. Through these operations, the system obtains a parametric model of the safe corridor extending around the centerline, containing various parameters such as corridor radius, risk level, obstacle proximity weights, and historical failure event statistics.

[0028] After completing the parametric modeling of the safety corridor, this step continues with the discretization of the constraint environment. In this invention, the constraint environment refers to the discrete spatial representation structure used for reinforcement learning and navigation inference. This structure divides the continuous vascular space into a regular grid, with each grid cell carrying fields related to drivability and safety. Specifically, the system constructs a three-dimensional grid structure of a specified resolution in the vascular principal coordinate system. For each grid cell, it calculates the distance between its center position and the nearest centerline point, the distance to the vascular wall, and the distance to the nearest obstacle element. Based on the safety corridor parametric model, it determines whether the cell is located inside the safety corridor, in a boundary region, or in a prohibited region. For grid cells located inside the safety corridor and marked as drivable, the system assigns a drivable marker and records the radial offset and position along the path of the cell relative to the safety corridor centerline. For grid cells near the safety corridor boundary, the system marks them as boundary regions and records detailed boundary distance fields and corresponding risk classification labels for use in the subsequent reward function as a basis for penalizing boundary-approaching behaviors. For grid cells located in prohibited regions or inside obstacles, the system sets a prohibited drivable marker and an obstacle type marker.

[0029] To support state transition computation in reinforcement learning, this step also constructs adjacency relationships and a multi-layer index structure for the discretized mesh. The system generates a unique index number for each mesh cell, records its coordinate index in the 3D mesh and its associated safe corridor number, and enumerates several allowed adjacent mesh cells for each cell based on the traversable area label. These adjacency relationships are recorded as a candidate set for state transitions. For adjacent cell pairs that cross vessel walls or barrier boundaries, the system directly eliminates them during adjacency relationship construction, prohibiting the formation of non-physical connections. For mesh cells located in vessel bifurcation regions, the system appends branch identifiers and a list of multiple candidate branches to the adjacency relationships. Simultaneously, it references the number of passes and failures of historical trajectories in the navigation execution feedback structure, encoding these statistical results into an auxiliary field for that region. This allows the subsequent policy network to distinguish the empirical risk levels of different branches during the state representation phase.

[0030] By uniformly organizing and encapsulating all the intermediate results mentioned above, this step ultimately constructs the vascular safety corridor constraint environment structure. This structure records vascular grid information, safety corridor centerline and radius parameters, passage markers for each grid cell, boundary distance levels, obstacle type markers, historical failure event statistics, and adjacency indexes in structured data form. It is a unified output representation of the comprehensive processing of vascular structure map data, obstacle distribution data, and navigation execution feedback structure. The output field is named "Vascular Safety Corridor Constraint Environment Structure." In subsequent steps, this field serves as the input to the environment description source for corridor constraint navigation state and composite reward modeling in S200, participating in the construction of the corridor constraint reinforcement learning configuration structure. Simultaneously, in S400, it is updated back through the navigation execution feedback structure, allowing the environment model to be continuously corrected and improved during multiple navigation and training processes.

[0031] The technical effect of this step can be summarized as follows: it transforms the original vascular images, obstacle distribution annotations, and navigation execution logs into a vascular safety corridor constraint environment structure that combines centerline semantics, safety corridor parameters, and discrete grid representation. This provides a unified, stable, and reusable environment foundation for subsequent navigation state modeling, reward design, and policy training based on deep reinforcement learning.

[0032] S200: Based on the constrained environment structure of the vascular safety corridor, construct the corridor constrained navigation state, model the composite reward, and perform hyperparameter assembly processing of corridor constrained reinforcement learning to generate the corridor constrained reinforcement learning configuration structure. In this step, the system uses the vascular safety corridor constraint environment structure, already generated in previous steps, as input along with robot pose data and target point configuration parameters. The vascular safety corridor constraint environment structure provides information such as the vascular mesh, centerline and safety corridor parameters, mesh access markers, boundary distance levels, and obstacle type markers. The robot pose data describes the microrobot's current position, orientation, and velocity in the vascular coordinate system. The target point configuration parameters indicate the vascular segments where the navigation start and end points are located, as well as the path position and termination conditions along the centerline. The system performs temporal alignment and spatial mapping on the three types of inputs in a unified vascular principal coordinate system, projecting the robot pose and target position onto the corresponding centerline and mesh index of the safety corridor, thus establishing a data foundation for subsequent corridor-constrained navigation state construction and composite reward modeling.

[0033] During the construction of the corridor-constrained navigation state, the system uses the centerline point array and safety corridor parameters in the vascular safety corridor constraint environment structure as spatial references to map the robot pose data at each moment to the local coordinate system of the safety corridor. Based on the robot's current position, the system finds the nearest centerline point in the centerline point array and calculates the path progress relative to the starting point along the centerline, using a progress field to reflect the robot's advancement along the overall path. Subsequently, the system calculates the lateral distance and direction between the robot's position and the centerline within the centerline cross-section, and combines this with the safety radius and boundary distance levels recorded in the safety corridor parameters to obtain the corridor offset field and boundary distance field, which describe the robot's relative position in the corridor's strip region. The system also retrieves obstacle mesh cells around the robot based on obstacle type labels and adjacency indexes in the vascular safety corridor constraint environment structure, calculates the nearest obstacle distance and obstacle distribution direction, and encodes these as local obstacle perception fields. Pose information, path progress, corridor offset field, boundary distance field, and obstacle perception field are combined in a fixed order to form a structured corridor-constrained navigation state vector.

[0034] To ensure the navigation state reflects recent motion trends, the system incorporates short-term historical context into the construction of the corridor-constrained navigation state. The system records robot pose data and executed control commands from the most recent few time steps using a rolling buffer. This historical data is downsampled and pruned to obtain fixed-length historical state segments. Fields such as historical pose changes, velocity changes, and angle changes are concatenated chronologically with the current corridor-constrained navigation state to form an extended navigation state structure. This allows the reinforcement learning model to reference recent motion trajectories when processing the current state. The constructed corridor-constrained navigation state and extended navigation state are written to an internal buffer and used in the same encoding method in subsequent interactive sampling and empirical datasets.

[0035] In the composite reward modeling part, the system generates a reward signal composed of multiple components for each state transition based on the corridor-constrained navigation state and the safety corridor parameters in the vascular safety corridor constraint environment structure. First, the system constructs a basic task reward field based on the path progress difference between the robot's current position and the target position described by the target point configuration parameters. When the robot approaches the target along the centerline direction, a positive reward higher than the background is given. Subsequently, the system classifies safety levels based on the corridor offset field and the boundary distance field. States where the offset is outside the recommended passage range but still within the corridor correspond to a mild safety penalty, while states approaching or touching the boundary correspond to a higher penalty. When the state indicates that the robot has crossed the safety corridor or entered a prohibited area, the system generates a severe penalty based on the obstacle type label and internally marks that time step as a termination candidate state. To constrain trajectory smoothness, the system calculates the movement distance and angle changes between adjacent time steps from historical pose changes and control commands, mapping excessive movement distances and sharp turns as smoothness penalty fields. When a step causes a collision, boundary crossing, or reaching the target area, that time step is marked as a termination state, and a termination reward or termination penalty component is added to the reward. The basic task reward, safety penalty, smoothness penalty, and termination flag together constitute the composite reward modeling field set. The system obtains the single-step composite reward value by combining them according to preset weight coefficients.

[0036] In the corridor-constrained reinforcement learning hyperparameter assembly process, the system reads control parameters related to deep reinforcement learning training from the algorithm configuration module and parameter tuning records, and matches them with the features of corridor-constrained navigation state and composite reward modeling. Based on the state vector dimension, action space range, and geometric complexity of the safe corridor, the system selects the hidden layer size and number of network layers for the policy network and value network, and encodes the network structure as a network structure description field. Subsequently, the system reads training control parameters such as learning rate, discount factor, target network update step size, experience replay batch size, and experience buffer capacity, adjusting parameters sensitive to state dimension and action amplitude according to preset rules. For exploration-related parts, the system configures the exploration perturbation intensity, perturbation decay rate, and minimum perturbation threshold fields for continuous action outputs; for network stability-related parts, the system configures the weight decay factor, gradient pruning threshold, and target network soft update factor. All reinforcement learning-related control parameters are uniformly organized in this step and written into the corridor-constrained reinforcement learning hyperparameter field set, providing a parameter basis for subsequent policy learning.

[0037] This step generates a corridor-constrained reinforcement learning configuration structure within the system by uniformly encapsulating the state field definitions obtained from corridor-constrained navigation state construction, the composite reward modeling field set obtained from composite reward modeling, and the corridor-constrained reinforcement learning hyperparameter field set. This structure, in its data description, simultaneously includes state space definitions, reward composition and weights, network structure settings, training control parameters, and reference indexes to the vascular safety corridor constraint environment structure. This allows subsequent training modules to directly reference it when conducting corridor-constrained navigation interaction sampling and policy value network parameter updates. The output field, named "Corridor-Constrained Reinforcement Learning Configuration Structure," is passed to the corridor-constrained navigation interaction sampling and experience dataset construction processes in subsequent steps. It serves as the primary parameter source for generating corridor-constrained navigation interaction sampling configurations and training plans, and provides a unified configuration for policy loading during the navigation execution phase after policy convergence, thus forming a data link between steps from environment modeling to policy inference.

[0038] The technical effect of this step can be summarized as follows: based on the constrained environment structure of the vascular safety corridor, the environmental description, robot pose data and target point configuration parameters are transformed into a structured corridor constrained navigation state, a well-defined composite reward model and a set of reinforcement learning hyperparameters that match task features, and are uniformly encapsulated into a corridor constrained reinforcement learning configuration structure, providing a configuration basis for subsequent training and execution.

[0039] S300: Based on the corridor constraint reinforcement learning configuration structure, perform corridor constraint navigation interaction sampling, experience dataset construction and policy value network parameter update processing to generate the corridor constraint policy value network structure. In this step, the system first obtains the corridor-constrained reinforcement learning configuration structure from the previous steps. This structure already defines the corridor-constrained navigation state space, the composite reward modeling fields, the hierarchical settings of the policy network and value network, the training batch size, the learning rate, the discount factor, the target network soft update coefficient, and the experience replay-related control parameters. Simultaneously, the system loads a simulated vascular environment instance from the environment management module. This simulated vascular environment instance is generated by the vascular safety corridor constraint environment structure and includes information such as discrete grid, centerline index, passage markers, obstacle distribution, boundary distance levels, and bifurcation connections. It also implements interfaces for state reset, step update, and termination determination. The robot control interface configuration is pre-provided by the robot's underlying control system, describing the mapping relationship between the continuous motion output of the policy network and control channels such as the microrobot's longitudinal propulsion speed and rotational angular velocity. It also includes motion amplitude limits, acceleration limits, and safety pruning rules. At the beginning of this step, the system creates a policy network and a value network according to the network structure description in the corridor constraint reinforcement learning configuration structure, and randomly initializes the network parameters or loads them from an existing model. Based on this, an experience playback buffer is built and the association with the simulated vascular environment instance and robot control interface configuration is completed.

[0040] Specifically, during the corridor-constrained navigation interactive sampling phase, the system parses each set of training round parameters in the corridor-constrained reinforcement learning configuration structure into several simulation rounds, with each simulation round starting from an environment reset. Upon receiving the round start command, the simulated vascular environment instance selects a starting grid from the passable corridor based on the entrance position, target point position, and branch selection strategy given in the vascular safety corridor constraint environment structure. It then sets the virtual pose of the microrobot near the entrance and returns to the initial corridor-constrained navigation state. The system packages this state according to the state encoding rules in the corridor-constrained reinforcement learning configuration structure, forming an input vector that is fed into the policy network. The policy network outputs a set of continuous action variables, representing the microrobot's propulsion speed and steering adjustment amount within the blood vessel at the current time step. Upon receiving this action vector, the robot control interface configuration converts it into a sequence of drive instructions adapted to the simulation control cycle based on the channel mapping relationship, action amplitude limitations, and safety pruning rules, and calls the step update interface in the simulated vascular environment instance.

[0041] Upon receiving a drive command, the simulated vascular environment instance advances the microrobot's virtual position based on its current grid location, adjacency relationships, and safe corridor parameters. If the advancement direction points to a passable grid cell, the robot's position and attitude in the vascular coordinate system are updated, corridor offset, boundary distance, and obstacle proximity are recalculated, and a composite reward calculation logic is invoked to generate a single-step reward value and a termination flag. This composite reward calculation logic uses weight configurations recorded in the corridor constraint reinforcement learning configuration structure to provide numerical feedback on various factors such as path advancement, corridor offset, boundary contact, obstacle collision, trajectory smoothness, and termination events. If the advancement direction touches a prohibited area or an obstacle collision occurs, the simulated vascular environment instance trims its current position to the nearest safe position or remains in its original position, while simultaneously setting a termination flag and outputting the corresponding penalty component. At each time step, the system receives the next-moment corridor constraint navigation state, single-step reward value, and termination flag from the simulated vascular environment instance, and records an interaction sample consisting of the current state, executed action, next state, and termination flag.

[0042] Furthermore, during the construction of the experience dataset, the system writes the aforementioned interaction samples into the experience replay buffer. The experience replay buffer is defined as a ring-shaped storage structure storing finite-length temporal difference samples. Each sample records the current corridor constraint navigation state vector, the continuous action vector output by the policy network, the single-step reward scalar generated by the composite reward calculation logic, the next corridor constraint navigation state vector, the termination flag, and auxiliary fields related to the safe corridor. The auxiliary fields include information such as the corridor number to which the current state belongs, the boundary distance level, the obstacle type label, and whether a boundary crossing or collision event has been triggered. When writing samples, the system overwrites earlier samples according to the buffer capacity and calculates sample priority based on the termination flag and the safe corridor auxiliary fields in the sample. This priority is used to control the frequency of sample selection during subsequent sampling. For samples located near the safe corridor boundary or where risk events have occurred, the system can assign higher priority, prompting the policy value network to pay more attention to these key state areas during training, thereby enabling the network to form clearer value judgments in high-risk areas.

[0043] Before the training batch begins, the system sets the number of experience samples to be extracted for each round of parameter updates based on the batch size and number of training rounds in the corridor-constrained reinforcement learning configuration structure. Upon receiving a sampling request, the experience replay buffer selects a set of sample indices based on the current sample quantity and priority distribution, and returns the corresponding set of states, actions, rewards, next states, and termination flags, constituting a training mini-batch. The system performs preprocessing operations on the training mini-batch, organizing the state vector and next state vector according to the input structure of the policy network and value network, splitting the action vector into tensors of a unified format according to the various control channels of the action space, organizing the rewards and termination flags into scalar sequences, and internally recording the safe corridor number and risk level of each sample, providing additional reference for subsequent loss function construction and gradient clipping.

[0044] In the parameter update stage of the policy-value network, the system first uses the next-state vector from the training mini-batch to generate a target value estimate through the target policy network and the target value network. The structure of the target policy network and the target value network is consistent with that of the online policy network and the online value network, but the parameter update adopts a soft update method. For each sample, the system constructs the target reward based on the action estimated by the target policy in the next state and the value estimate given by the target value network. Then, it combines the single-step reward and termination flag of the current sample to generate the target value of the sample internally. Subsequently, the system inputs the current state vector and the current action vector into the online value network to calculate the predicted value, and constructs the value network loss function based on the difference between the target value and the predicted value. The loss function updates the parameters of each layer inside the online value network through backpropagation. The update magnitude is controlled by the learning rate and weight decay coefficient in the corridor-constrained reinforcement learning configuration structure. At the same time, when the gradient is too large, gradient pruning logic is triggered to prevent excessive oscillation of network parameters.

[0045] After updating the value network parameters, the system uses the same batch of state vectors to generate the current policy output action through the online policy network, and inputs these actions into the online value network to obtain the corresponding value estimate. The policy network loss function is defined as the negative mean of the value estimate. Minimizing this loss function guides the policy network to output continuous actions that are better under the current value estimate. When calculating the gradient of the policy network, the system uses the corridor offset field, boundary distance field, and obstacle auxiliary field to weight the gradient contribution, making the policy network more sensitive to states near corridor boundaries and obstacles during the optimization process, reducing the tendency to output over-excited actions in high-risk areas. The policy network parameter update is also subject to control parameters such as learning rate and gradient pruning. After each round of online network parameter update, the system performs slow parameter interpolation on the target policy network and the target value network according to the soft update coefficient in the corridor-constrained reinforcement learning configuration structure, transferring some parameter increments from the online network to the target network to stabilize the target value estimate.

[0046] Throughout the training process, corridor-constrained navigation interaction sampling and policy-value network parameter updates are performed alternately. The system follows the number of training rounds, interaction steps per round, and update batches per round set in the corridor-constrained reinforcement learning configuration structure. In each round, several simulated interaction steps are executed first, and the generated samples are written to the experience replay buffer. Then, small training batches are sampled from the buffer to perform multiple network parameter updates. In the event of a termination event during a simulation round, the system decides whether to immediately reset the simulated vascular environment instance and start a new round based on the termination type, or to record the termination reason and trajectory statistics before resetting, for use in generating navigation execution feedback structures in subsequent steps. As training iterates, the internal parameters of the policy network and value network gradually converge. The system records changes in network loss, action distribution, and safety-related indicators in real time in the training monitoring module, providing a reference for determining the timing of training termination.

[0047] Once the preset number of training rounds, the range of loss function variation, and the trend of safety indicator changes meet the termination conditions, the system derives the final parameter set and structure description from the online policy network, online value network, target policy network, and target value network. These are then integrated with the network structure definition and hyperparameter records in the corridor-constrained reinforcement learning configuration structure to form the corridor-constrained policy value network structure. This structure includes policy network weights for navigation inference, value network weights for safety assessment, and target network weights for subsequent incremental training. It also includes reference indexes to the vascular safety corridor constraint environment structure and the corridor-constrained reinforcement learning configuration structure. This output field, named "Corridor-Constrained Policy Value Network Structure," is passed as input to the navigation control module in subsequent steps. It is invoked in step S400, the corridor-constrained navigation policy inference and trajectory generation stage, to generate corridor-constrained navigation control commands and navigation trajectories. Furthermore, it influences the new interactive sampling process through the navigation execution feedback structure.

[0048] The technical effect of this step can be summarized as follows: by performing corridor constraint navigation interactive sampling and constructing an experience dataset in a simulated vascular environment instance, and then combining the joint parameter update of the policy network and the value network, the corridor constraint reinforcement learning configuration structure is specifically implemented as a corridor constraint policy value network structure with trained parameters, providing a policy model foundation that can be directly invoked for the subsequent deep neural network-based safe corridor constraint navigation inference process.

[0049] S400: Obtain the corridor constraint policy value network structure, perform corridor constraint navigation control command generation, navigation trajectory generation, and navigation execution feedback statistical processing to obtain the navigation execution feedback structure. In this step, the system first obtains the corridor constraint policy value network structure from the previous main step. This structure stores the final parameters of the policy network and value network, the network topology description, and reference indexes to the corridor constraint reinforcement learning configuration structure. Simultaneously, it reads environmental description fields such as vascular mesh, centerline point sequence, safety corridor parameters, passage markers, obstacle distribution, and boundary distance levels from the vascular safety corridor constraint environment structure. It also reads navigation start and end configuration parameters from the surgical planning module or navigation task management module. These parameters include task constraints such as navigation entry position, target position, starting orientation, allowed navigation duration, and maximum number of steps. The system performs spatial mapping and task matching on these three types of inputs under a unified vascular coordinate system. An interactive channel is established between the policy inference module and the vascular safety corridor constraint environment structure through an environmental interface, preparing foundational data for subsequent corridor constraint navigation control command generation and navigation trajectory generation.

[0050] Specifically, the system calls the scene instantiation interface in the vascular safety corridor constraint environment structure. Based on the entrance and target positions given in the navigation start-end configuration parameters, it selects the corresponding entrance and target grids from the set of passable corridors to construct the navigation start state. The environment module places the virtual pose of the microrobot in a safety grid cell near the entrance, assigning the grid cell a centerline index, corridor number, and initial corridor offset fields, and setting the initial boundary distance and obstacle proximity fields according to the safety corridor parameters. Subsequently, the environment module generates the first corridor constraint navigation state vector based on the current grid cell and adjacency relationship. The state vector is consistent with the state encoding rules in the corridor constraint reinforcement learning configuration structure and serves as the initial input to the policy network in the corridor constraint policy value network structure. During the initialization phase, the system also sets a navigation step counter, current navigation round identifier, and termination reason placeholder fields according to the navigation start-end configuration parameters, providing an index benchmark for subsequent navigation execution feedback statistics.

[0051] During the generation of corridor constraint navigation control commands, the system retrieves the corridor constraint navigation state vector returned from the previous time step from the environment module in each control cycle. This state vector is then input into the forward inference path of the policy network in the corridor constraint policy value network structure to obtain a continuous action vector. The continuous action vector contains a longitudinal propulsion velocity component and a steering adjustment component, corresponding to the microrobot's forward movement along the blood vessel centerline and its rotation amplitude in the cross-sectional plane. Based on the action range and action pruning rules recorded in the corridor constraint reinforcement learning configuration structure, the system performs amplitude pruning and numerical smoothing on the continuous action vector to prevent abrupt commands exceeding the allowable range of the safe corridor. Subsequently, the system calls the robot control interface configuration to map the pruned and smoothed continuous action vector to specific control channels, corresponding to drive motor control quantities and attitude adjustment control quantities, forming a corridor constraint navigation control command sequence that conforms to the control cycle rhythm, and sends it to the motion update module in the blood vessel safety corridor constraint environment structure.

[0052] Upon receiving the corridor constraint navigation control command sequence, the motion update module in the vascular safety corridor constraint environment structure updates the virtual position and posture of the microrobot step by step based on the current grid position, centerline direction, and adjacent grid set. If the target grid cell corresponding to the control command is in a passable state on the pass marker and is still within the radius of the corresponding safety corridor, the environment module moves the robot position to the center of that grid and updates state fields such as centerline progress, corridor offset, boundary distance, and obstacle proximity. If the target grid cell pointed to by the control command is located in a high-risk area or marked as a prohibited area, the environment module executes a pruning strategy based on the safety corridor parameters and obstacle distribution fields, projecting the target position to the nearest safe grid cell, or keeping the robot position unchanged, while recording the risk event type and setting a collision flag or boundary crossing flag. After completing the position update or pruning operation, the environment module calculates the single-step reward value and termination flag based on the latest state and the composite reward rule described in the corridor constraint reinforcement learning configuration structure, and returns them to the control command generation module along with the current corridor constraint navigation state vector, forming a closed-loop interaction.

[0053] During navigation trajectory generation, at each time step, the system records the current position's 3D coordinates in the vascular coordinate system, centerline index, corridor number, corridor offset, boundary distance level, obstacle proximity, and the continuous motion vector corresponding to the control command as navigation trajectory nodes, based on the robot's position and attitude information returned by the environment module. The system adds a timestamp, current round number, and termination flag status to each trajectory node, and concatenates all trajectory nodes in the generation order to form one or more complete navigation trajectories. For situations involving multiple pauses, restarts, or external interventions, the system reads relevant markers from the navigation start and end configuration parameters, divides the navigation process into several trajectory segments, and records the internal indices of the start and end points between each segment for subsequent statistical analysis to distinguish performance at different stages. Throughout the navigation process, the system continuously accumulates statistics such as path length, average corridor offset, number of boundary contacts, number of obstacle contact events, and termination event type distribution, and updates the temporary counters maintained during navigation execution based on these statistics.

[0054] When any of the preset termination conditions in the navigation start and end configuration parameters are met, the environment module returns a termination flag to the control command generation module. Termination conditions include reaching the target area, the occurrence of consecutive collision events, reaching the maximum allowed number of steps, exceeding the navigation time limit, or a manual termination command being issued. Upon receiving the termination flag, the control command generation module stops issuing new corridor-constrained navigation control commands and triggers the navigation execution feedback statistical processing flow. This flow reads all trajectory nodes and corresponding statistical counters from the navigation trajectory, organizes information such as path length distribution, corridor offset distribution, boundary contact position, obstacle contact position, termination reason, and time consumption, and writes key indicators related to the safety corridor constraints into the navigation execution feedback field set. For areas where boundary violations or concentrated risk events occur, the system appends the corridor number and centerline progress label of that area to the navigation execution feedback field as a reference for subsequent updates to the vascular safety corridor constraint environment structure. Thus, the navigation execution process outputs control commands from the corridor constraint strategy value network structure, completes motion updates through the vascular safety corridor constraint environment structure, and is summarized and organized by the statistical module, corresponding to one complete path generation process.

[0055] After completing the above statistical processing, the system encapsulates the navigation execution feedback field set, original navigation trajectory, risk event markers, task start and end information, and training round information into a unified navigation execution feedback structure. This structure serves as the output field name for this step, corresponding to the navigation process summary data object in this invention. It includes a subset of trajectory data for strategy training, a subset of risk area markers for environment correction, and a subset of path summaries for clinical presentation. In the cross-step data flow, the navigation execution feedback structure serves both as input to the next round of environment modeling and as a supplementary source of experience for subsequent incremental training phases. During the next execution of S100, the system reads the latest trajectory information and risk statistics from the navigation execution feedback structure to correct the traversable area and safety corridor parameters in the vascular safety corridor constraint environment structure. Simultaneously, during the next execution of S300, some trajectory fragments conforming to the sampling rules in the navigation execution feedback structure are written into a new experience dataset, thus forming a closed-loop link from environment modeling, configuration generation, strategy training to navigation execution and subsequent environment revision.

[0056] The technical effect of this step can be summarized as follows: Utilizing the corridor constraint policy value network structure trained in the previous stage, navigation control commands and navigation trajectories are generated under the joint constraints of the vascular safety corridor constraint environment structure and navigation start and end configuration parameters. A structured navigation execution feedback structure is formed, enabling the entire medical microrobot navigation process to form a unified closed loop with the aforementioned deep learning model and safety corridor modeling logic during actual operation. This step, through the systematic recording and organization of control commands, trajectory nodes, and risk statistics, provides a directly usable data foundation for subsequent environment modeling and policy training stages.

Claims

1. A navigation method for a medical microrobot oriented towards safety corridor constraints, characterized in that, include: The process involves acquiring vascular structure data, obstacle distribution data, and navigation execution feedback structure; extracting the centerline and labeling passable areas to generate a constrained environment structure for the vascular safety corridor; specifically including: Centerline extraction of the vascular structure map data includes: applying distance transformation to calculate the distance from the vascular voxel to the background; using a thinning algorithm and a skeleton extraction algorithm to shrink the vascular cavity to obtain candidate skeleton points; performing path tracing and branch identification according to the vascular topology to generate candidate centerline curves; and combining the historical successful trajectories, curvature, and failure events in the navigation execution feedback structure to select the main trunk centerline, generating a centerline point list with spatial coordinates, local tangential direction, curvature estimation, and cross-sectional radius estimation values. The process of marking passable areas based on the centerline point array includes: calculating the safe and recommended passable radius for each centerline point based on the minimum inner diameter of the blood vessel cross-section, the wall roughness, and the distance to the obstacle element; calculating the distance to the centerline point, the blood vessel wall boundary, and the obstacle element for each voxel and classifying them into passable areas, prohibited areas, or high-risk areas according to the safe passable rules; mapping the failure event trajectory points in the navigation execution feedback structure to the coordinate space and expanding them to form an affected area; and marking the original passable area voxels within the area as high-risk areas. Based on the centerline point sequence and the labeled passable area, parametric modeling and environmental discretization of the safety corridor are performed, including: constructing a local coordinate system at the centerline point with the centerline point sequence as the main axis; detecting the voxel range of the passable area on each cross section to determine the radial safety corridor parameters; interpolating and smoothing the safety corridor parameters on each cross section to generate a safety corridor model that continuously varies along the centerline; constructing a three-dimensional mesh structure in the blood vessel principal coordinate system; determining whether each mesh cell is located inside the safety corridor, in the boundary area, or in the prohibited area according to the safety corridor model; assigning a passable mark to mesh cells located inside the safety corridor and labeled as passable areas and recording the radial offset and position along the path; labeling the boundary area and recording the boundary distance field and graded risk label for mesh cells located near the boundary of the safety corridor; setting a prohibited passable mark and obstacle type mark for mesh cells located in the prohibited area or inside the obstacle; and constructing an adjacency relationship and a multi-level index structure for the discretized mesh to generate the constrained environment structure of the blood vessel safety corridor. Based on the constrained environment structure of the vascular safety corridor, this study constructs the corridor constrained navigation state, models composite rewards, and assembles hyperparameters for corridor constrained reinforcement learning to generate a corridor constrained reinforcement learning configuration structure. Specifically, this includes: The corridor-constrained navigation state construction includes: mapping the robot pose to the local coordinate system of the safe corridor; calculating the path progress and lateral distance from the centerline point column of the current position to obtain the corridor offset field and boundary distance field; and retrieving the surrounding obstacle mesh cells to count the nearest obstacle distance and distribution direction to encode as local obstacle perception fields, forming a structured corridor-constrained navigation state vector; adding a short-term historical context to record historical poses and control commands through a rolling buffer, and concatenating it with the current state to form an extended navigation state structure. The composite reward modeling includes: constructing a basic task reward based on the path progress difference between the robot's current position and the target point; generating safety penalties based on the corridor offset field and the boundary distance field to classify safety levels—corresponding to a mild safety penalty for a state where the offset is outside the recommended passage range but still inside the corridor; corresponding to a higher penalty for a state that is close to or touches the boundary; generating a severe penalty and marking a termination candidate state when the state indicates that the robot has crossed the safe corridor or entered a prohibited area, combined with obstacle type marking; calculating the movement distance and turning angle changes based on historical pose changes and control commands to generate a smoothness penalty; and combining the basic task reward, safety penalty, smoothness penalty, and termination reward to form a composite reward value. Based on the corridor-constrained reinforcement learning configuration structure, we perform corridor-constrained navigation interaction sampling, experience dataset construction, and policy value network parameter update processing to generate the corridor-constrained policy value network structure. Obtain the value network structure of the corridor constraint strategy, perform corridor constraint navigation control command generation, navigation trajectory generation and navigation execution feedback statistical processing to obtain the navigation execution feedback structure.

2. The method according to claim 1, characterized in that, The process of modeling composite rewards also includes: The composite reward modeling process includes constructing a basic task reward based on the path progress difference between the robot's current position and the target point, generating a safety penalty based on the corridor offset field and the boundary distance field to classify the safety level, generating a smoothness penalty based on the movement distance and turning angle changes calculated based on historical pose changes and control commands, and generating a severe penalty and termination mark when the robot goes out of the safe corridor or enters the prohibited area by combining obstacle type markings. Finally, the basic task reward, safety penalty, smoothness penalty and termination reward are combined to form a composite reward value.

3. The method according to claim 1, characterized in that, The process of hyperparameter assembly in corridor-constrained reinforcement learning also includes: The corridor-constrained reinforcement learning hyperparameter assembly process includes selecting the hidden layer size and number of layers of the policy network and value network based on the state vector dimension and action space range to generate network structure description fields; reading the learning rate, discount factor, target network update step size, experience replay batch size and buffer capacity; configuring the perturbation intensity, decay rate and minimum threshold of the exploration behavior; and the weight decay factor, gradient pruning threshold and soft update factor for network stability. Finally, these are uniformly organized into a set of corridor-constrained reinforcement learning hyperparameter fields, and the corridor-constrained reinforcement learning configuration structure is generated by encapsulating the state space definition, reward composition, network settings and training parameters.

4. The method according to claim 1, characterized in that, The process of performing corridor-constrained navigation interaction sampling also includes: The corridor-constrained navigation interactive sampling process includes parsing training round parameters into simulation rounds, resetting the environment based on the entrance and target positions in the simulated vascular environment instance and returning to the initial corridor-constrained navigation state, inputting the state vector into the policy network to output continuous action variables, configuring and mapping them to propulsion speed and steering adjustment through the robot control interface, driving the environment to update the robot's position, updating the pose and recalculating the corridor offset, boundary distance, and obstacle proximity if the robot points to a passable grid, calling the composite reward calculation logic to generate a single-step reward value and termination flag, clipping the position and setting the termination flag if the robot points to a prohibited area, and recording the interactive samples of state, action, next state, and termination flag.

5. The method according to claim 1, characterized in that, The process of constructing the experience dataset also includes: The experience dataset construction process includes writing interaction samples into the experience replay buffer. Each sample records a state vector, action vector, reward scalar, next state vector, termination flag, and safety corridor auxiliary field. The sample priority is calculated based on the termination flag and auxiliary field, and higher priority is assigned to high-risk samples. Before the training batch, training mini-batches are generated by sampling from the buffer according to the batch size, and the state vector, action vector, and reward are preprocessed.

6. The method according to claim 1, characterized in that, The process of updating policy value network parameters also includes: The policy-value network parameter update process includes generating a target value estimate using the next state vector from the training mini-batch through the target policy network and the target value network; constructing the target reward by combining the single-step reward and termination flag; constructing the value network loss function by comparing the difference with the predicted value of the online value network; updating the online value network parameters through backpropagation and gradient pruning; then using the state vector to output the action through the online policy network and inputting it into the online value network to obtain the value estimate; constructing the policy network loss function with the negative mean of the value estimate; weighting the gradient contribution using the corridor offset field and the boundary distance field; updating the online policy network parameters; and finally performing a soft update on the target network. This process alternates between interactive sampling and parameter updates until the training termination condition is met, generating a corridor-constrained policy-value network structure containing policy network weights, value network weights, and target network weights.

7. The method according to claim 1, characterized in that, The process of generating corridor-constrained navigation control commands also includes: The corridor constraint navigation control command generation process includes calling the scene instantiation interface to select the entrance grid and target grid in the passable corridor, setting the robot's virtual pose and generating an initial corridor constraint navigation state vector, inputting the state vector into the policy network for forward inference in each control cycle to obtain continuous motion vectors, performing amplitude clipping and numerical smoothing according to the motion range and clipping rules, configuring the drive motor control quantity and attitude adjustment quantity mapped to the control channel through the robot control interface, and forming a corridor constraint navigation control command sequence.

Images