Medical assistant robot and adaptive scan view planning method and system thereof

Abstract

The application provides a kind of adaptive scanning visual angle planning method of medical auxiliary robot, mainly includes four steps: first, the three-dimensional grid model of patient is normalized to simulation environment, and the operating space of camera is limited;Second, according to three-dimensional grid model, target site identification and historical scanning information, construct composite state vector;Then, input the vector through strategy network, generate three-dimensional action vector representing the next optimal viewpoint coordinates;Finally, generate a series of viewpoints composed of multiple viewpoints, based on this sequence, use path planning algorithm to generate collision-free trajectory, and collect data at each viewpoint, finally obtain the three-dimensional point cloud of target site.The application has the beneficial effects: through conditional learning, a model with strong generalization ability for multiple parts is realized, and multiple targets such as scanning integrity, quality and safety can be automatically weighed, and finally a safe, efficient and high-precision automatic scanning scheme far beyond traditional methods is generated.

Description

Technical Field

[0001] This invention belongs to the field of visual recognition, and in particular relates to medical assistive robots and their adaptive scanning perspective planning methods, systems and devices. Background Technology

[0002] In 3D reconstruction using medical assistive robots, traditional camera view planning methods mainly rely on predefined trajectories or simple heuristic rules. These methods suffer from poor adaptability and low efficiency, making it difficult to simultaneously consider multiple optimization objectives such as scan coverage and point cloud accuracy while ensuring safety. Although advanced technologies such as reinforcement learning have shown potential in this field, existing methods generally cannot effectively address the challenges of high-dimensional state spaces, continuous action spaces, and diverse task objectives. In particular, they lack a clear conditionalization mechanism that enables agents to universally adapt to different body parts, thus limiting the application of automated scanning technology in clinical settings. Summary of the Invention

[0003] In view of this, the present invention aims to propose a medical assistive robot and its adaptive scanning perspective planning method and system, in order to solve at least one of the above-mentioned technical problems.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0005] The first aspect of this invention proposes an adaptive scanning perspective planning method for medical assistive robots, comprising:

[0006] S1. Normalize the patient's 3D mesh model to the simulation environment coordinate system, and constrain the camera's operable space to the preset 3D workspace.

[0007] S2. Construct a composite state vector based on the target scanning part identifiers and their historical scanning information in the 3D mesh model;

[0008] S3. Input the composite state vector into the policy network to obtain a three-dimensional motion vector representing the camera's next optimal viewpoint coordinates in the workspace.

[0009] S4. Repeat steps S2 to S3 to generate a viewpoint sequence consisting of multiple optimal viewpoint coordinates. Based on the viewpoint sequence, generate an executable, collision-free, smooth trajectory using a path planning algorithm.

[0010] The medical assistive robot executes the collision-free smooth trajectory obtained in step S4, and collects data at each viewpoint, and finally integrates them to obtain a three-dimensional point cloud of the target area.

[0011] Furthermore, the composite state vector described in step S2 is composed of human geometric state, scan integrity mask, target part identifier, and random noise vector concatenation, wherein:

[0012] The human body geometry is constituted by the three-dimensional coordinates of all vertices of the patient's three-dimensional mesh model; the scan integrity mask is used to identify whether the vertices of each patient's three-dimensional mesh model have been scanned from a historical perspective; the target part identifier is generated from the target part name, which serves as a unique identification code vector to conditionally specify the current scanning task; the random noise vector is used to introduce exploratory behavior during training and decision-making.

[0013] Furthermore, the policy network operation process includes:

[0014] The policy network extracts geometric and coverage features, concatenates the extracted features with target location identifiers and random noise, and processes the concatenation result using a near-end policy optimization algorithm to obtain the optimal viewpoint coordinates; the near-end policy optimization algorithm guides the optimization strategy through a reward function.

[0015] Furthermore, the reward function is obtained by combining sparse reward and reward shaping, and the reward function includes at least:

[0016] A collision penalty term is used to apply a negative reward when the distance between the camera and the human mesh model is less than a safety threshold;

[0017] The perspective dispersion reward is designed to encourage a dispersed distribution of viewpoints in space to facilitate a complete exploration of the target surface.

[0018] The scan integrity bonus is used to give a positive reward when the number of scanned vertices reaches a predetermined percentage of the total number of vertices in the target area;

[0019] The single-scan quality bonus is used to reward the quantity and quality of unscanned surfaces captured in a single scan.

[0020] Furthermore, the single-scan quality reward item consists of multiple sub-items, including:

[0021] The distance reward sub-item is used to encourage the camera to maintain a preset ideal scanning distance from the surface of the target area;

[0022] The orientation alignment reward sub-item is used to encourage alignment of the camera's optical axis with the average normal vector direction of the target area's surface.

[0023] The quantity reward sub-item is used to reward the number of unscanned vertices captured in a single scan.

[0024] Furthermore, the calculation method for the direction alignment reward sub-item is as follows:

[0025] Principal component analysis is performed on the set of vertices captured in the current scanning cycle. The eigenvector corresponding to the smallest eigenvalue is taken as the normal vector estimate of the local surface. Then, the cosine value of the angle between the normal vector and the camera optical axis direction vector is calculated as the reward.

[0026] Furthermore, the training process of the policy network is completed in a high-fidelity virtual simulation environment. The simulation environment simulates the camera observation process through ray tracing technology. After obtaining the optimal viewpoint coordinates, the camera optical axis is directed to the geometric center of the target part's vertex, and a fixed upward direction vector is preset to determine the camera attitude.

[0027] Furthermore, the aforementioned path planning algorithm enables rapid exploration. An algorithm for generating optimal collision-free trajectories connecting the viewpoint sequence.

[0028] A second aspect of the present invention provides an adaptive scanning perspective planning system for a medical assistive robot, used to perform the method described in the first aspect, the system comprising:

[0029] The simulation training module is used to construct a high-fidelity virtual simulation environment and train the policy network based on the conditional deep reinforcement learning algorithm.

[0030] The state observation module is used to acquire or construct the composite state observation vector in real time;

[0031] The decision control module, which has the trained policy network embedded in it, is used to receive the state observation vector and output the optimal viewpoint coordinates;

[0032] The trajectory planning and execution module is used to plan the viewpoint sequence output by the decision module into the robot's motion trajectory and control the robot to perform scanning tasks.

[0033] A third aspect of the present invention proposes a medical assistive robot, including a robotic arm, an RGB-D camera mounted on the end of the robotic arm, and a control system, characterized in that the control system is configured to run the method described in the first aspect to achieve adaptive, refined three-dimensional scanning of patient body parts.

[0034] Compared with existing technologies, the medical assistive robot and its adaptive scanning perspective planning method and system described in this invention have the following beneficial effects: through conditional learning, it achieves a powerful generalization ability of "one model, applicable to multiple parts", and can automatically balance multiple objectives such as scanning integrity, quality and safety, and finally generate an automated scanning scheme that is far safe, efficient and high-precision than traditional methods. Attached Figure Description

[0035] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0036] Figure 1 This is a schematic diagram of the workflow of the medical assistive robot and its adaptive scanning perspective planning method according to Embodiment 1 of the present invention;

[0037] Figure 2 This is a schematic diagram of the workflow of the medical assistive robot and its adaptive scanning perspective planning method described in Embodiment 2 of the present invention. Detailed Implementation

[0038] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0039] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0040] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0041] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0042] The primary objective of this invention is to overcome the shortcomings of the prior art and provide a scanning perspective planning method and system that can adapt to the patient's body shape, posture and target area. Its core is to solve the challenges of three-dimensional perception in highly diverse scenarios and achieve full automation and optimization of the scanning process.

[0043] Therefore, this invention proposes a medical robot scanning perspective planning method based on conditional deep reinforcement learning. The core of this method is to transform the complex perspective planning problem into a partially observable Markov decision process guided by conditional signals, and to use deep reinforcement learning to train a general decision strategy in a simulation environment. This strategy can adaptively generate the optimal scanning perspective sequence according to different input conditions (i.e., different target parts).

[0044] like Figure 1 As shown in Example 1, the adaptive scanning perspective planning method for a medical assistive robot includes:

[0045] S1. Normalize the patient's 3D mesh model to the simulation environment coordinate system, and constrain the camera's operable space to the preset 3D workspace.

[0046] S2. Construct a composite state vector based on the target scanning part identifier and its historical scanning information of the 3D mesh model.

[0047] The composite state vector described in step S2 is composed of human geometric state, scan integrity mask, target part identifier, and random noise vector concatenation, wherein:

[0048] The human body geometry is constituted by the three-dimensional coordinates of all vertices of the patient's three-dimensional mesh model; the scan integrity mask is used to identify whether the vertices of each patient's three-dimensional mesh model have been scanned from a historical perspective; the target part identifier is generated from the target part name, which serves as a unique identification code vector to conditionally specify the current scanning task; the random noise vector is used to introduce exploratory behavior during training and decision-making.

[0049] S3. Input the composite state vector into the policy network to obtain a 3D motion vector representing the next optimal viewpoint coordinates of the camera in the workspace.

[0050] The policy network operation process described in step S3 includes:

[0051] The policy network extracts geometric and coverage features, concatenates the extracted features with target location identifiers and random noise, and processes the concatenation result using a near-end policy optimization algorithm to obtain the optimal viewpoint coordinates; the near-end policy optimization algorithm guides the optimization strategy through a reward function.

[0052] The reward function is obtained by combining sparse reward and reward shaping, and the reward function includes at least the following:

[0053] A collision penalty term is used to apply a negative reward when the distance between the camera and the human mesh model is less than a safety threshold;

[0054] The perspective dispersion reward is designed to encourage a dispersed distribution of viewpoints in space to facilitate a complete exploration of the target surface.

[0055] The scan integrity bonus is used to give a positive reward when the number of scanned vertices reaches a predetermined percentage of the total number of vertices in the target area;

[0056] The single-scan quality bonus is used to reward the quantity and quality of unscanned surfaces captured in a single scan.

[0057] The single-scan quality bonus item consists of multiple sub-items, which include:

[0058] The distance reward sub-item is used to encourage the camera to maintain a preset ideal scanning distance from the surface of the target area;

[0059] The orientation alignment reward sub-item is used to encourage alignment of the camera's optical axis with the average normal vector direction of the target area's surface.

[0060] The quantity reward sub-item is used to reward the number of unscanned vertices captured in a single scan.

[0061] The calculation method for the orientation alignment reward sub-item is as follows:

[0062] Principal component analysis is performed on the set of vertices captured in the current scanning cycle. The eigenvector corresponding to the smallest eigenvalue is taken as the normal vector estimate of the local surface. Then, the cosine value of the angle between the normal vector and the camera optical axis direction vector is calculated as the reward.

[0063] S4. Repeat steps S2 to S3 to generate a viewpoint sequence consisting of multiple optimal viewpoint coordinates. Based on the viewpoint sequence, generate an executable, collision-free, smooth trajectory using a path planning algorithm.

[0064] The medical assistive robot executes the collision-free smooth trajectory obtained in step S4, and collects data at each viewpoint, and finally integrates them to obtain a three-dimensional point cloud of the target area.

[0065] like Figure 2 As shown in Example 2, the adaptive scanning perspective planning method for a medical assistive robot includes:

[0066] S1. Environment modeling and initialization;

[0067] S2. Construct multimodal state observation vectors ;

[0068] S3, Decision making based on the perspective of conditional deep reinforcement learning;

[0069] S4. Reward function design;

[0070] S5, Trajectory Generation and Scan Execution.

[0071] Step S1 aims to provide a standardized, securely constrained simulation environment for subsequent reinforcement learning decisions, including:

[0072] S11. Personalized Human Model Acquisition and Normalization:

[0073] A personalized 3D human body mesh model of the patient to be scanned is obtained through a pre-processed 3D vision algorithm. The model contains vertex coordinates. And surface information. Model normalization process: Transform model M from its original coordinate system to a standardized simulation environment coordinate system;

[0074] Specifically, rigid body transformations (rotation and translation) are used to place the pelvic center or the center of the overall bounding box of the model at the origin of the coordinate system. And make its main body roughly aligned with the positive Z-axis of the coordinate system;

[0075] The key role of this step is to eliminate interference from irrelevant variables caused by different global positions and orientations of the patient, enabling the reinforcement learning agent to focus on learning the universal geometric relationship between the viewpoint and the human body shape, which greatly improves the generalization ability of the strategy.

[0076] S12, Sensor working space constraints:

[0077] Based on the workspace limitations of real robots and patient safety requirements, a movable three-dimensional space for a camera (sensor) is defined in the simulation environment. This space is typically defined as having a size of A cuboid, the center of which is located at This space ensures that the camera always moves within a safe and effective range.

[0078] In step S2, at each decision-making moment The agent needs to receive a comprehensive observation vector describing the current environmental state and information. The careful design of this vector is the foundation for the agent to make effective decisions, and its specific components include:

[0079] S21, Human Geometric State :

[0080] We can directly use the 3D coordinates of all N (e.g., 6890) vertices of the normalized human body mesh model, i.e. This section provides complete geometric information about the patient's body shape and posture.

[0081] S22, Scan Integrity Mask :

[0082] This is dynamically updated. 2D binary vector ,in Its initial value is all 0, in each step Using the ray tracing capabilities of libraries such as Open3D, the simulation is performed from the current viewpoint. When a ray is emitted onto the model surface, if vertex i is successfully hit by at least one ray (i.e., scanned), then its corresponding mask value is... Set to 1, otherwise keep the original value. This mask is crucial because it links single scan decisions into a sequential decision-making process, enabling the agent to clearly know which areas have been covered, thereby proactively planning subsequent views to fill the gaps and avoid repeated scans.

[0083] S23, Target Location Marking :

[0084] To uniquely identify the body part to be scanned (such as "right knee" or "left shoulder"), this invention uses a 64-dimensional universally unique identification code vector as a conditional signal. This vector is generated by hashing the body part name string. This is the core design for realizing "conditional" decision-making. By using this identifier as an additional input to the network, a trained policy can understand the current task objective and generate exclusive perspective planning for different body parts, thus achieving universality.

[0085] S24, Random Noise Vector :

[0086] Introduce a 64-dimensional random vector Randomly generating each step and injecting random noise into a deterministic strategy is an effective technique to promote exploration and prevent the strategy from getting stuck in local optima.

[0087] Finally, the complete observation vector at time t is the concatenation of the above four parts: .

[0088] Step S3, as the core step of Embodiment 2, has the following specific working process:

[0089] S31. Network forward propagation and action generation:

[0090] because and Due to the high dimensionality, directly inputting into a fully connected network is inefficient. Therefore, a dedicated feature embedding module is first used for processing. This module typically consists of one-dimensional convolutional layers (with kernel sizes in the range of [128, 256, 512]) and global max-pooling layers, used to extract representative geometric and coverage features from the vertex sequence. ;

[0091] Geometric features Marking the target area Random noise The data are concatenated to form a low-dimensional, information-rich comprehensive state vector. ;

[0092] Will Input to policy network (A multilayer perceptron (MLP) with a layer structure of [1024, 512, 256]) The network outputs actions. The distribution parameters, and the specific actions obtained from the final sampling. It is a three-dimensional continuous vector representing the next viewpoint in the normalized workspace. coordinates within ;

[0093] The position of the camera is determined by To simplify the problem and ensure observation quality, the camera attitude is determined using a "gazing" model, with the camera optical axis direction... Set from Geometric center pointing to the vertex of the target part The direction vector, i.e. The camera's up direction vector is usually fixed as .

[0094] S32, Training the Policy Network:

[0095] The training in step S32 is performed offline;

[0096] The proximal policy optimization algorithm is used, which is an advanced Actor-Critic type reinforcement learning algorithm that is widely used due to its high training stability and sample efficiency.

[0097] The network parameter θ is updated by the complex reward function in S4. Guidance. The design of the reward function is crucial for successful training. It shapes the agent's behavior through rewards and punishments, enabling it to ultimately learn to balance multiple objectives such as safety, integrity, and quality.

[0098] The training process takes place in a high-fidelity simulation environment built by S1. The agent learns the optimal policy through a large number of trial and error. The advantage of simulation is that it can obtain global real information (such as accurate normal vectors) as supervision signals without risk and at low cost, which is difficult to provide in the real environment.

[0099] The reward function proposed in step S4 The design directly affects the performance of the final strategy, which employs a combination of sparse and dense reward shaping, specifically including:

[0100] S41, Collision Penalty :

[0101] If the camera position If the minimum distance to any vertex of the human body grid is less than the safety threshold, a large negative reward is given; otherwise, it is 0. This is the bottom line for safety assurance.

[0102] S42, Perspective Dispersion Reward :

[0103] calculate Minimum distance to the set of all historical viewpoints P ,like Reward +1, otherwise punish -1, to encourage the camera to explore different areas and avoid lingering in one place.

[0104] S43, Scan Integrity Bonus :

[0105] When the proportion of the number of scanned vertices to the total number of vertices in the target area exceeds a threshold, a large one-time reward is given to incentivize the agent to cover most of the area as the ultimate goal.

[0106] S44, Single Scan Quality Bonus :

[0107] This is a compound reward, calculated only when a new vertex is scanned, and consists of three parts:

[0108] Distance Rewards = This encourages the camera to maintain an ideal distance from the scanned local surface. ;

[0109] Orientation Alignment Rewards Principal component analysis was performed on the newly captured vertex set in this scan, and the eigenvector corresponding to the smallest eigenvalue was selected. As an estimate of the local surface normal vector, the reward value is That is, the direction of the camera's optical axis The absolute value of the cosine of the angle between the point cloud and the normal vector is the reward that directly drives the camera to find the vertical observation angle and maximize the point cloud quality.

[0110] Quantity Rewards = ,in, This is the number of new vertices in this scan. It represents the number of currently active unscanned vertices in the workspace, encouraging the capture of as much new data as possible.

[0111] The working process of step S5 is as follows:

[0112] S51. By repeating S2-S3, a discrete optimal viewpoint sequence is obtained. .

[0113] S52, Sequence The input is fed into a path planning algorithm, which, taking into account the constraints of robot links and joints, searches within the robot's configuration space to generate a continuous and collision-free joint space trajectory connecting all viewpoints. .

[0114] S53, Track The data is sent to the actual medical robot controller, and the robotic arm moves strictly according to the trajectory, and at each viewpoint... After a brief pause, a frame of point cloud data is acquired using the RGB-D camera mounted on the terminal. Finally, a point cloud registration algorithm is used to fuse multiple frame point clouds to generate a complete, high-precision 3D model of the target area.

[0115] The training process of the policy network is completed in a high-fidelity virtual simulation environment. The simulation environment simulates the camera observation process through ray tracing technology. After obtaining the optimal viewpoint coordinates, the camera optical axis is directed to the geometric center of the vertex of the target part, and a fixed upward direction vector is preset to determine the camera attitude.

[0116] The path planning algorithm described is for rapid exploration. An algorithm for generating optimal collision-free trajectories connecting the viewpoint sequence.

[0117] Example 3, an automated 3D scanning system for the knee joint based on conditional deep reinforcement learning, further demonstrates the workflow of the adaptive scanning perspective planning method for medical assistive robots, and provides a complete system for automated, high-precision 3D scanning of patients' knee joints.

[0118] I. System Hardware and Software Configuration:

[0119] Hardware platform:

[0120] Medical robots: Employ a collaborative robot with six or seven degrees of freedom, such as Universal Robots UR5 or KUKALBRiiwa.

[0121] 3D sensor: A Microsoft Azure Kinect DK camera is mounted on the robot's end flange to acquire color images and depth information.

[0122] Computing Unit: Equipped with a high-performance workstation featuring an NVIDIA RTX series GPU for running deep reinforcement learning inference, real-time path planning, and point cloud processing algorithms.

[0123] Software environment:

[0124] Operating system: Ubuntu 20.04LTS or later.

[0125] Robot operating system: ROS2. Core algorithm libraries: PyTorch, Open3D, OMPL.

[0126] II. Implementation steps:

[0127] Step 1: Personalized Human Model Acquisition and Environment Initialization

[0128] Model Acquisition: The patient is positioned in a seated or standing position within the robot's workspace. Multiple pre-positioned Azure Kinect DK cameras acquire multi-view images and point clouds of the patient from different angles. Using a neural network-based SMPL model fitting algorithm, these multi-view data are input to reconstruct a personalized SMPL human mesh model M for the patient in real time. This model contains... There are 13,776 vertices and 13,776 triangular faces.

[0129] Model normalization: computational model Calculate the center point of the bounding box of all vertices along their axes. Perform a rigid body transformation on the model, subtracting the center point from the coordinates of all vertices. This ensures that the center of the new bounding box is located at the origin of the world coordinate system. Furthermore, principal component analysis is used to determine the principal orientation of the model, and a rotation matrix is ​​applied to make the first principal component of the model (usually corresponding to the height direction of the human body) roughly aligned with the positive Z-axis of the world coordinate system. The key to this step is to eliminate the randomness of the patient's standing position and orientation, and to provide a standardized input for the reinforcement learning agent.

[0130] Workspace definition: Defines the accessible workspace for the robot's end effector (camera). It is a cuboid with dimensions of The space center is located All planned viewpoints must be located within this space.

[0131] Step 2: Construct the multimodal state observation vector:

[0132] At every decision moment The state observation vector received by the agent Build it as follows:

[0133] Human geometric state : Directly use the 3D coordinates of the 6890 vertices of the normalized SMPL model and flatten it into a single... The dimensional vector is standardized before being input into the actual network, so that its mean is 0 and its standard deviation is 1.

[0134] Scan integrity mask Before the scan begins ( ), It is a vector of all zeros; at time... In the simulation environment, the virtual camera is placed at the position determined by the previous step. The optical axis points to the geometric center of the target area (knee joint). Using Open3D's ray_cast function, a ray is emitted from the camera's optical center towards the SMPL model in the scene. If the ray intersects a triangle, the index of the nearest vertex at the intersection point is recorded, and the mask values ​​corresponding to all hit vertices are generated. Set to 1.

[0135] Target area marking The target region is specified as "right_knee". Using Python's UUID library, a UUID based on this string is generated. This UUID string is then converted into a 128-bit integer, and further quantized into a 64-dimensional floating-point vector, which is used as the target region. .

[0136] random noise vector Each step uses numpy.random.rand(64) to generate a 64-dimensional random vector, where each element is in... Within the range.

[0137] final, It is the concatenation of the above four vectors, with a total dimension of dimension.

[0138] Step 3: Conditional Deep Reinforcement Learning Decision Making

[0139] because and Due to the extremely high dimensionality, an encoder is first used for dimensionality reduction. This encoder consists of three one-dimensional convolutional layers (with 128, 256, and 512 kernels respectively, a kernel size of 1, and a stride of 1). Each convolutional layer is followed by a ReLU activation function, and finally a global max pooling layer is used to compress the variable-length vertex sequence features into a 512-dimensional global feature vector. .

[0140] 512-dimensional 64-dimensional And 64-dimensional The concatenation results in a 640-dimensional comprehensive feature vector. .

[0141] The policy network (Actor) and value network (Critic) share the aforementioned 640-dimensional features as input, and are then each connected to a multilayer perceptron. The MLP structure of the policy network is: Linear(640,1024)->ReLU->Linear(1024,512)->ReLU->Linear(512,256)->ReLU->Linear(256,3); finally outputting a 3-dimensional action. mean ;

[0142] Additionally, the network has a separate branch that outputs a logarithmic standard deviation. Actions follow a normal distribution The value network obtained from sampling is similar in structure, but the last layer is Linear(256,1), which outputs a scalar representing the value of the current state.

[0143] The built The input is fed into a pre-trained policy network, and after forward propagation, the network outputs a three-dimensional action. This action is a relative displacement with respect to the previous viewpoint, achieved through... The next absolute viewpoint coordinates are calculated and cropped to prevent excessive single-step movement. The camera's optical axis always points to the geometric center of the vertex of the knee joint region.

[0144] Step 4: Calculate the reward function:

[0145] Perform actions in a simulation environment Then, calculate the reward immediately. The specific quantities are as follows:

[0146] :calculate Minimum Euclidean distance to all vertices of the SMPL model ,like ,but Otherwise, it is 0.

[0147] : Calculating et and the set of historical viewpoints minimum distance ,like Reward +1; if Penalty -1; otherwise 0.

[0148] : Count the total number of apex points of the target area (knee joint) and the number of vertices that have been scanned ,like A one-time reward will be given. .

[0149] : Calculated only when a new vertex is captured in this scan. ,in It is the average distance from the camera to the geometric center of the vertex scanned in this scan, and 0.5 meters is the preset ideal distance.

[0150] Perform PCA on the newly captured vertex set in this scan, and take the eigenvector corresponding to the smallest eigenvalue. As a normal vector.

[0151] calculate ,in The direction of the camera's optical axis (from the camera towards the geometric center of the vertex).

[0152] ,in This is the number of vertices scanned this time. It is the number of vertices in the target area that have not yet been scanned.

[0153] Total Rewards .

[0154] Step 5: Trajectory Generation and Real-World Scanning

[0155] Repeat steps two through four until no new vertex is detected for 10 consecutive steps or the total number of steps exceeds 50. Then terminate the decision loop to obtain the viewpoint sequence. .

[0156] Viewpoint sequence Input into the OMPL library In the path planner, the planner searches within the robot's configuration space, taking into account the robot's own link collision detection, to generate a smooth, collision-free joint space trajectory connecting all viewpoints. .

[0157] Trajectory via ROS2 The message is sent to the robot controller, and the robot moves sequentially to each trajectory point, at each viewpoint. The robot briefly pauses, triggering Azure KinectDK to capture a frame of depth and color images, which are then converted into point clouds using the Open3D library. Using the generalized iterative nearest-point algorithm, the current frame point cloud is... Register it with the already merged global point cloud and merge it into the global point cloud.

[0158] After the scan is completed, a high-precision, complete 3D point cloud model of the knee joint is output, which can be used for subsequent medical analysis.

[0159] III. Offline Training Process:

[0160] The policy network of this invention requires extensive offline training in a simulation environment. The training process is as follows:

[0161] Simulation environment setup:

[0162] The training environment is built based on the OpenAIGym interface, and Open3D is used for ray tracing and physics simulation. The environment contains a large number of randomly generated SMPL models of different sizes and poses as training samples.

[0163] Training cycle:

[0164] Using a near-end strategy optimization algorithm, set a discount factor. Generalized dominance estimation parameters .

[0165] Conditioned training:

[0166] At the start of each training round, an SMPL model, a random initial pose, and a target body part are randomly selected. This is key to achieving generality by subjecting the agent to diverse conditions and tasks, forcing it to understand the "target body part identifier." The semantics of this knowledge allow us to learn how to plan for different parts.

[0167] Training objectives:

[0168] The goal of training a policy network is to maximize the expected cumulative reward. Training continues until the average reward of the policy on the validation set converges.

[0169] Through the detailed steps described above, those skilled in the art can fully implement the medical robot scanning perspective planning method based on conditional deep reinforcement learning described in this invention. This method demonstrates a high degree of adaptability to patient differences and target sites, as well as excellent scanning quality and automation level.

[0170] Example 4: An adaptive scanning perspective planning system for a medical assistive robot, used to execute the aforementioned adaptive scanning perspective planning method for a medical assistive robot, the system comprising:

[0171] The simulation training module is used to construct a high-fidelity virtual simulation environment and train the policy network based on the conditional deep reinforcement learning algorithm.

[0172] The state observation module is used to acquire or construct the composite state observation vector in real time;

[0173] The decision control module, which has the trained policy network embedded in it, is used to receive the state observation vector and output the optimal viewpoint coordinates;

[0174] The trajectory planning and execution module is used to plan the viewpoint sequence output by the decision module into the robot's motion trajectory and control the robot to perform scanning tasks.

[0175] Example 5: A medical assistive robot, including a robotic arm, an RGB-D camera mounted on the end of the robotic arm, and a control system. The control system is configured to run an adaptive scanning perspective planning method for the medical assistive robot to achieve adaptive and refined three-dimensional scanning of patient body parts.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

[0177] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An adaptive scanning perspective planning method for medical assistive robots, characterized in that, include: S1. Normalize the patient's 3D mesh model to the simulation environment coordinate system, and constrain the camera's operable space to the preset 3D workspace. S2. Construct a composite state vector based on the target scanning part identifiers and their historical scanning information in the 3D mesh model; S3. Input the composite state vector into the policy network to obtain a three-dimensional motion vector representing the camera's next optimal viewpoint coordinates in the workspace. S4. Repeat steps S2 to S3 to generate a viewpoint sequence consisting of multiple optimal viewpoint coordinates. Based on the viewpoint sequence, generate an executable, collision-free, smooth trajectory using a path planning algorithm. The medical assistive robot executes the collision-free smooth trajectory obtained in step S4, and collects data at each viewpoint, and finally integrates them to obtain a three-dimensional point cloud of the target area.

2. The adaptive scanning perspective planning method for a medical assistive robot according to claim 1, characterized in that, The composite state vector described in step S2 is composed of human geometric state, scan integrity mask, target part identifier, and random noise vector concatenation, wherein: The human body geometry is constituted by the three-dimensional coordinates of all vertices of the patient's three-dimensional mesh model; the scan integrity mask is used to identify whether the vertices of each patient's three-dimensional mesh model have been scanned from a historical perspective; the target part identifier is generated from the target part name, which serves as a unique identification code vector to conditionally specify the current scanning task; the random noise vector is used to introduce exploratory behavior during training and decision-making.

3. The adaptive scanning perspective planning method for a medical assistive robot according to claim 1, characterized in that, The working process of the policy network includes: The policy network extracts geometric and coverage features, concatenates the extracted features with target location identifiers and random noise, and processes the concatenation result using a near-end policy optimization algorithm to obtain the optimal viewpoint coordinates; the near-end policy optimization algorithm guides the optimization strategy through a reward function.

4. The adaptive scanning perspective planning method for a medical assistive robot according to claim 3, characterized in that, The reward function is obtained by combining sparse reward and reward shaping, and the reward function includes at least the following: A collision penalty term is used to apply a negative reward when the distance between the camera and the human mesh model is less than a safety threshold; The perspective dispersion reward is designed to encourage a dispersed distribution of viewpoints in space to facilitate a complete exploration of the target surface. The scan integrity bonus is used to give a positive reward when the number of scanned vertices reaches a predetermined percentage of the total number of vertices in the target area; The single-scan quality bonus is used to reward the quantity and quality of unscanned surfaces captured in a single scan.

5. The adaptive scanning perspective planning method for a medical assistive robot according to claim 4, characterized in that, The single-scan quality bonus item consists of multiple sub-items, which include: The distance reward sub-item is used to encourage the camera to maintain a preset ideal scanning distance from the surface of the target area; The orientation alignment reward sub-item is used to encourage alignment of the camera's optical axis with the average normal vector direction of the target area's surface. The quantity reward sub-item is used to reward the number of unscanned vertices captured in a single scan.

6. The adaptive scanning perspective planning method for a medical assistive robot according to claim 5, characterized in that, The calculation method for the orientation alignment reward sub-item is as follows: Principal component analysis is performed on the set of vertices captured in the current scanning cycle. The eigenvector corresponding to the smallest eigenvalue is taken as the normal vector estimate of the local surface. Then, the cosine value of the angle between the normal vector and the camera optical axis direction vector is calculated as the reward.

7. The adaptive scanning perspective planning method for a medical assistive robot according to claim 1, characterized in that: The training process of the policy network is completed in a high-fidelity virtual simulation environment. The simulation environment simulates the camera observation process through ray tracing technology. After obtaining the optimal viewpoint coordinates, the camera optical axis is directed to the geometric center of the vertex of the target part, and a fixed upward direction vector is preset to determine the camera attitude.

8. The adaptive scanning perspective planning method for a medical assistive robot according to claim 1, characterized in that: The path planning algorithm described is for rapid exploration. An algorithm for generating optimal collision-free trajectories connecting the viewpoint sequence.

9. An adaptive scanning perspective planning system for a medical assistive robot, used to execute the method according to any one of claims 1 to 8, characterized in that, The system includes: The simulation training module is used to construct a high-fidelity virtual simulation environment and train the policy network based on the conditional deep reinforcement learning algorithm. The state observation module is used to acquire or construct the composite state observation vector in real time; The decision control module, which has the trained policy network embedded in it, is used to receive the state observation vector and output the optimal viewpoint coordinates; The trajectory planning and execution module is used to plan the viewpoint sequence output by the decision module into the robot's motion trajectory and control the robot to perform scanning tasks.

10. A medical assistive robot, comprising a robotic arm, an RGB-D camera mounted at the end effector of the robotic arm, and a control system, characterized in that, The control system is configured to operate the method of any one of claims 1 to 8 to achieve adaptive, refined three-dimensional scanning of patient body parts.

Images