Robot scanning path adjusting method and device, equipment and medium

Abstract

The invention relates to the technical field of robot control, and discloses a robot scanning path adjusting method, device, equipment and medium, and the method comprises the steps: controlling a robot equipped with a sensor to move along a preliminary path, collecting a preliminary two-dimensional image of a target object, processing the image, extracting the feature points of the target object, and combining the spatial information of the sensor when the image is collected; the method comprises the steps of generating a primary three-dimensional point cloud, determining a plurality of ideal sensor postures based on the primary three-dimensional point cloud, determining a new sensor position of each posture on a contactable surface to generate a re-planned scanning path, controlling a robot to move along the re-planned scanning path, adjusting the sensor postures, and collecting a final two-dimensional image of a target object. According to the method, the scanning path is optimized in combination with the initial three-dimensional point cloud local surface normal direction, so that the attitude of the sensor is matched with the geometrical characteristics of the target surface, the problem of three-dimensional reconstruction surface breakage caused by inter-frame pose mismatching under free breathing is solved, and the accuracy of autonomous ultrasonic scanning of the robot is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a robot scanning path adjustment method, device, equipment and storage medium. BACKGROUND

[0002] The existing robot autonomous ultrasound system usually relies on a resting state or breath-holding cooperation to reduce imaging errors caused by respiratory motion when scanning abdominal large vessels. However, in actual clinical environments, the uncontrollability of patient's autonomous breathing can cause dynamic changes in the relative pose between the ultrasound probe and the anatomical structure, thereby causing obvious spatial mismatches between the continuously collected two-dimensional image frames. Such mismatches can directly affect the three-dimensional reconstruction based on the sequence images, resulting in discontinuous and fragmented defects of the reconstructed surface, which severely limits the application of robot autonomous ultrasound under free breathing conditions.

[0003] To address such problems, the prior art often relies on external optical tracking systems or requires patients to actively hold their breath to reduce motion effects, but external tracking devices are expensive, complex to operate, and have high requirements for clinical operating environments, and the breath-holding requirement makes the scanning process highly dependent on patients, significantly reducing clinical applicability and operational convenience.

[0004] In addition, although the existing image segmentation methods based on deep learning can achieve good results under the condition of sufficient labeled data, they are difficult to guarantee semantic consistency and accuracy of high-frequency details in the scene of clinical unlabeled data or pseudo-labeled data containing noise. This problem exacerbates the blurring of blood vessel contours and tissue edges in the image, further affecting the accurate extraction of feature points and the recovery of spatial information in the subsequent reconstruction process, and cannot meet the requirements of robot autonomous ultrasound for three-dimensional reconstruction accuracy under dynamic scanning conditions. SUMMARY

[0005] The main purpose of the present application is to provide a robot scanning path adjustment method, device, equipment and storage medium, which aims to solve the technical problems that the prior art cannot accurately obtain a continuous registration of high-quality two-dimensional image sequences under the condition of patient's free breathing in autonomous ultrasound abdominal scanning, resulting in a fragmented three-dimensional reconstruction surface and poor clinical applicability.

[0006] To achieve the above-mentioned purpose, the present application provides a robot scanning path adjustment method, comprising: controlling a robot with a sensor mounted at the end to move the sensor along a preliminary path and collect multiple preliminary two-dimensional images of a target object; processing the multiple preliminary two-dimensional images to extract feature points of the target object, and combining the spatial information of the sensor when each of the preliminary two-dimensional images is collected to generate a preliminary three-dimensional point cloud representing the surface of the target object; determine a plurality of ideal sensor poses based on the preliminary three-dimensional point cloud, wherein each ideal sensor pose corresponds to a normal direction of a local surface on the preliminary three-dimensional point cloud; generate a re-planned scanning path by determining for each of the ideal sensor poses a corresponding new sensor position on the touchable surface; control the robot to move the sensor along the re-planned scanning path and adjust the pose of the sensor according to the re-planned scanning path, and collect a plurality of final two-dimensional images of the target object.

[0007] Beneficial effects: The present application relates to the technical field of robot control, and discloses a robot scanning path adjustment method, device, equipment and medium, which comprises the following steps: controlling a robot with a sensor at the end to move along a preliminary path, collecting a plurality of preliminary two-dimensional images of a target object, processing the plurality of preliminary two-dimensional images to extract feature points of the target object and combining spatial information of the sensor when each preliminary two-dimensional image is collected to generate a preliminary three-dimensional point cloud representing a surface of the target object, determining a plurality of ideal sensor poses based on the preliminary three-dimensional point cloud, wherein each ideal sensor pose corresponds to a normal direction of a local surface on the preliminary three-dimensional point cloud, generating a re-planned scanning path by determining for each of the ideal sensor poses a corresponding new sensor position on the touchable surface, controlling the robot to move the sensor along the re-planned scanning path and adjust the pose of the sensor, and collecting a plurality of final two-dimensional images of the target object. The present application optimizes the robot scanning path by combining the normal direction of the local surface of the preliminary three-dimensional point cloud, ensures that the pose of the sensor matches the geometric features of the surface of the target object, solves the problem of surface fragmentation in three-dimensional reconstruction caused by the mismatch of the poses between two-dimensional image frames under the condition of free breathing in the traditional method, and improves the accuracy and clinical applicability of autonomous ultrasonic scanning of the robot. BRIEF DESCRIPTION OF DRAWINGS

[0008] The present application will be further described below in conjunction with the drawings and embodiments. In the drawings: Figure 1 An application environment schematic diagram of the robot scanning path adjustment method in an embodiment of the present application; Figure 2 A flowchart schematic diagram of the robot scanning path adjustment method in an embodiment of the present application; Figure 3 A functional module schematic diagram of the robot scanning path adjustment device in a preferred embodiment of the present application; Figure 4 A structure schematic diagram of the computer equipment in an embodiment of the present application; Figure 5 Another structure schematic diagram of the computer equipment in an embodiment of the present application. DETAILED DESCRIPTION

[0009] It is to be understood that the specific embodiments described herein are merely illustrative of the present application and do not limit the scope of the application.

[0010] The robot scanning path adjustment method provided by the embodiments of the present application can be applied in the application environment as shown in Figure 1 , wherein a user terminal communicates with a server through a network. The server can control a robot with a sensor at the end to move along a preliminary path, collect multiple preliminary two-dimensional images of a target object, process the multiple preliminary two-dimensional images to extract feature points of the target object and combine the spatial information of the sensor when collecting each preliminary two-dimensional image to generate a preliminary three-dimensional point cloud representing the surface of the target object, determine multiple ideal sensor poses based on the preliminary three-dimensional point cloud, each ideal sensor pose corresponding to the normal direction of a local surface on the preliminary three-dimensional point cloud, determine a corresponding new sensor position on the touchable surface for each ideal sensor pose to generate a re-planned scanning path, control the robot to move the sensor along the re-planned scanning path and adjust the sensor pose, and collect multiple final two-dimensional images of the target object. The present application optimizes the robot scanning path by combining the normal direction of the local surface of the preliminary three-dimensional point cloud, ensures that the sensor pose matches the geometric features of the surface of the target object, solves the problem of surface fragmentation in three-dimensional reconstruction caused by the mismatch of the poses between two-dimensional image frames under the condition of free breathing in the traditional method, and improves the accuracy and clinical applicability of the autonomous ultrasonic scanning of the robot. The user terminal can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The present application will be described in detail through specific embodiments.

[0011] Please refer to Figure 2 , Figure 2 for the flowchart of an embodiment of the robot scanning path adjustment method provided by the present application. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in an order different from that described herein.

[0012] As shown in Figure 2 , the robot scanning path adjustment method proposed by the present application includes the following steps: S10, control a robot with a sensor at the end to move the sensor along a preliminary path and collect multiple preliminary two-dimensional images of a target object; In this embodiment, when the robot performs scanning operations in a free-breathing environment, it must accurately control the movement path and spatial posture of the sensors to ensure that the acquired two-dimensional image data meets the requirements of three-dimensional reconstruction. The robot drives the robotic arm equipped with sensors at its end effector through a control module. This movement follows a pre-determined path, which is determined to ensure that the sensors can cover the scanned area of ​​the target object, taking into account its geometric surface and positional characteristics. The initial path definition can be combined with the anatomical structure and spatial geometry of the target object to ensure that the distance and incident angle between the sensor and the surface are suitable when the sensor covers the required area. The movement along the path is achieved by the motion control system coordinating multiple joint drive units of the robot. The motion control system calculates the position and orientation of the end effector at each time point in real time by analyzing the coordinates and posture information of the path points, and drives the robot joints to perform synchronous movements accordingly. This movement needs to be combined with the robot's dynamic characteristics and the task space inverse kinematics algorithm to achieve smooth control, so as to ensure continuity and dynamic stability.

[0013] During motion, the sensor is responsible for acquiring images of the target object. Its image acquisition unit works continuously, receiving signals reflected from the target object through an array of data acquisition channels and converting them into two-dimensional grayscale images. The number of images and the time interval can be set according to the sampling frequency, and are synchronized with the robot's motion state through a high-speed data acquisition module. The acquired images have spatial correlation attributes, which are determined by the position of the robot's end effector and the sensor's attitude parameters at the time of acquisition. Therefore, the spatial information includes position data and a set of attitude angles (pitch, yaw, roll) in the robot coordinate system, used for subsequent generation of 3D point clouds. The recording of spatial information is achieved through the calibration of the sensor mounting points and integration with the robot's state feedback. Calibration involves spatial transformation matrices and coordinate system alignment, so that the sensor's workspace can be described in the same way as the robot's base coordinate system.

[0014] The binding of spatial information with 2D image data requires a data synchronization mechanism. This ensures that each image is stored along with its corresponding spatial position and orientation data, enabling subsequent spatial reconstruction and restoration. Sensor control involves more than just path tracking; it also includes real-time attitude adjustment to ensure that the sensor's emission direction maintains the smallest possible angle with the local surface normal of the target object during movement, thus obtaining the optimal imaging signal. This attitude adjustment relies on the robot's multi-degree-of-freedom motion capabilities and dynamic adaptation to the geometry of the target surface. It can correct the motion trajectory and spatial orientation in real time during movement, tightly coupling the imaging process with the motion path.

[0015] Achieving robot movement along a path can be achieved by combining various path planning methods. For example, by pre-establishing a 3D model of the target object's surface, a feasible curved path can be planned, allowing the robot's movement to better conform to the object's surface geometry. Alternatively, path parameters can be adjusted based on visual perception feedback to correct the distance to the target surface in real time. Sensors can be installed using either a rigid end-effector fixation or a flexible coupling mounting structure. Flexible mounting helps adapt to changes in surface curvature, ensuring stable contact conditions. The motion control algorithm can employ nonlinear model predictive control with velocity feedforward compensation, smoothly adjusting joint speeds during robot acceleration, deceleration, and path transitions to avoid the impact of motion inertia on image acquisition accuracy.

[0016] The imaging module for acquiring preliminary two-dimensional images can be an ultrasonic array probe with broadband detection capabilities, providing high longitudinal and lateral resolution. Its imaging parameters, such as center frequency and pulse width, can be selected and optimized according to the tissue characteristics of different target objects. The spatial information acquisition module can use a combination of coded absolute position sensors and high-resolution joint encoders to improve the accuracy of robot end-effector position and attitude recording. Data synchronization adopts a hardware triggering method to ensure complete consistency between imaging and position acquisition, reducing data mismatch caused by time delay.

[0017] Example Description: A robot carrying an ultrasound sensor moves along a planned curved path covering the major blood vessels in the abdomen. During the robot's movement, the motion control unit calculates the end effector's position and orientation in real time, driving the robotic arm to move smoothly along the path. The sensor's imaging module acquires two-dimensional grayscale images at uniform intervals along the path, while the spatial information acquisition module records the spatial coordinates and three-axis attitude angles of each acquisition location. These data are correlated in real time by the data binding module and written to a storage device, providing consistent and high-precision input data for subsequent 3D reconstruction. Throughout the process, the robot maintains stable movement, avoiding speed fluctuations to ensure imaging signal quality and reduce imaging artifacts caused by respiratory movements.

[0018] This embodiment controls the robot to move stably along a defined path, and combines this with synchronous image acquisition during the movement. This ensures the integrity and geometric continuity of the target object's surface, and significantly reduces spatial mismatch problems caused by target movement, especially in a free-breathing environment, thereby improving the spatial consistency and imaging quality of the 3D reconstruction input data.

[0019] S20, process the multiple preliminary two-dimensional images to extract feature points of the target object, and combine the spatial information of the sensor when acquiring each preliminary two-dimensional image to generate a preliminary three-dimensional point cloud representing the surface of the target object. In this embodiment, during the continuous acquisition of two-dimensional images, segmentation and feature extraction operations need to be performed on these images. Feature point extraction refers to identifying a set of pixels related to the surface boundary of the target object from the two-dimensional image, typically including pixels with strong echo edges or regions with significant tissue-background intensity changes. Feature point extraction can rely on deep learning networks, such as semantic segmentation networks, which, through training, can identify the edges of the target object and output corresponding pixel masks, thereby accurately distinguishing the region of interest from the background region. The edge pixels extracted in this way are represented as a set of two-dimensional coordinates in the image coordinate system, preserving the planar position and contour information of the target object in each frame of the image.

[0020] To map a set of two-dimensional coordinates to three-dimensional space, it is necessary to incorporate the spatial information acquired at the time of each two-dimensional image acquisition. This spatial information includes the spatial position coordinates of the sensors mounted on the robot's end effector and the three-axis attitude angles at the moment of image acquisition. This data is analyzed using motion state feedback and coordinate transformation matrices provided by the robot's control system, enabling image coordinate points to be accurately mapped to the robot's three-dimensional coordinate system through projection transformation and coordinate system rotation. The use of spatial information ensures that feature points extracted from each two-dimensional image can be uniformly transformed to the same spatial reference, forming a three-dimensional point cloud with global spatial consistency.

[0021] Point cloud data is generated by performing coordinate transformation operations on the feature points extracted from each frame of the image, and then fusing the 3D coordinate point sets of all frames in temporal order. Temporal alignment is necessary to ensure the spatial continuity and motion consistency of the point cloud, especially important when there is periodic motion caused by breathing. The fused data forms a spatial point set, providing input for subsequent 3D modeling. The final point cloud is called a preliminary 3D point cloud. Its sparsity is reflected in the fact that the spatial distribution of points covers the main contours rather than completely filling the object surface. The density of the point cloud depends on the image sampling frequency and the accuracy of feature point extraction.

[0022] Feature point extraction can be achieved by training convolutional neural networks, such as the U-Net architecture, which uses an encoder-decoder module to parse the context of an image and output a binary mask. During network training, manually labeled bone structure edge data can be used as supervisory labels, or a weak supervision and pseudo-label strategy can be employed to reduce the need for manual data. The combination of two-dimensional coordinate sets and spatial information can utilize robot coordinate system calibration data. For example, a spatial calibration matrix can be used to align the sensor's local coordinate system with the robot's global coordinate system, allowing the spatial position of each pixel to be solved using a set of homogeneous transformation formulas.

[0023] Time sequence alignment can be achieved through timestamp matching, ensuring that the time labels of image frames are completely consistent with the robot's motion state sampling data, avoiding spatial misalignment caused by data delays and asynchrony. Fusion operations can be performed by simply merging sets, accumulating the 3D points mapped from all frames into a global point cloud. Alternatively, a spatial distance-based filtering strategy can be added to remove duplicate points between adjacent frames, improving the sparsity and coverage efficiency of the point cloud data.

[0024] Example Description: As the robot moves along a defined scanning path, the imaging module acquires 2D images at different time points. The image segmentation module uses a deep convolutional network to identify the high-contrast boundaries of large blood vessels in each frame and extracts pixel coordinates. The spatial position and attitude acquisition module simultaneously records the position and three-axis attitude angles of the robot's end effector, maps the extracted 2D boundary pixels to the robot's 3D coordinate system, and accumulates a point cloud set covering the outline of the large blood vessel frame by frame. The point cloud data is then fused after temporal alignment to form a preliminary 3D point cloud. This sparse point cloud accurately reflects the position and geometric orientation of the target large blood vessel surface, providing the input basis for subsequent path replanning and scanning attitude calculation.

[0025] This embodiment extracts the edge feature points of the target object from each image and combines them with the spatial position and pose information at the corresponding time. This can unify the scattered two-dimensional image data into a global three-dimensional coordinate space, forming a sparse three-dimensional point cloud with consistent structure and spatial alignment. This provides an accurate geometric basis for subsequent path replanning and three-dimensional model generation, while avoiding the inter-frame misalignment problem caused by breathing motion, thus improving the continuity and spatial accuracy of three-dimensional reconstruction.

[0026] S30, based on the preliminary three-dimensional point cloud, determine multiple ideal sensor poses, wherein each ideal sensor pose corresponds to the normal direction of a local surface on the preliminary three-dimensional point cloud; In this embodiment, the initial 3D point cloud is generated using the spatial position and attitude information of multiple 2D images. Each point in the point cloud data contains 3D spatial coordinates, representing the local geometric features of the target object's surface. To analyze and determine the sensor's attitude, the point cloud is first projected onto the robot's working plane along the major axis of the target object, reducing spatial dimensions and making the data distribution easier to process. The robot's working plane serves as a 2D reference for motion control and attitude calculation, used to express the positional relationship of the point cloud in the robot's task space.

[0027] In the projected two-dimensional data, the upper envelope point sequence is extracted by scanning the vertical direction to form a point set representing the geometric shape of the target object's surface. This point sequence is smoothed using a moving average algorithm to eliminate local data fluctuations, and then fitted with a cubic spline function to obtain a continuous, smooth curve. Equally spaced sampling points selected on the curve are used for subsequent geometric calculations, with each sampling point serving as a discrete data reference position.

[0028] At each sampling point, the tangent direction is calculated to describe the local variation trend of the curve. The tangent direction is obtained by calculating the coordinate difference between the points before and after the sampling point. Based on this tangent direction, its perpendicular direction in the curve plane is further solved as the normal direction of the local surface. This normal direction defines the direction in which the sensor should be aligned at that location, ensuring that the sound beam is orthogonal to the surface of the target object and improving imaging quality.

[0029] Each ideal sensor attitude is described by the spatial position of that sampling point and its three-axis attitude angles. The three-axis attitude angles include pitch, yaw, and roll, representing the angular relationship between the sensor coordinate system and the robot's working coordinate system. They are calculated by comparing the normal direction vector with the reference direction of the working plane coordinate system. The spatial positions of all sampling points and their corresponding three-axis attitude angles form a sequential record, which serves as the basis for subsequent control of the robot's scanning motion.

[0030] Projecting the initial 3D point cloud onto the robot's working plane can be achieved by defining a global plane equation and using a projection matrix. When extracting the upper envelope point sequence, the outermost points along the vertical direction on the projection plane are selected to form the contour point set. For smoothing with a moving average, a three-point window averaging method can be used to make the local curve smoother. When fitting the curve with cubic splines, each upper envelope point is used as a control point for the fitting, and natural boundary conditions are set so that the first derivative of the fitted curve is zero at the boundaries. During the fitting process, the continuity and smoothness of the curve are maintained by minimizing the error.

[0031] On the fitted curve, a series of sampling points are selected at fixed intervals. At each sampling point, the tangent direction is calculated based on the coordinates of adjacent sampling points. The normal direction is determined by calculating the direction perpendicular to the tangent. The ideal three-axis attitude angles of each sampling point are determined by the spatial angular relationship between the normal direction and the robot coordinate system, and can be calculated using the Euler angle formula. The position and attitude data of the sampling points are recorded in the storage module for use by the robot control system.

[0032] Example Description: In an abdominal large vessel scanning application, a sparse preliminary 3D point cloud is projected onto the robot's working plane along the longitudinal direction of the spine. The outermost points along the Z-axis are extracted in the 2D plane as the upper envelope point sequence. A three-point moving average is used to smooth the envelope point sequence, and a continuous contour curve is generated using cubic spline fitting. Sampling points are selected along the curve at 2 mm intervals. At each sampling point, the difference in coordinates between adjacent points is calculated as the tangent direction. The direction perpendicular to the tangent direction within the curve plane is further determined as the normal direction at that point. By comparing the normal direction with the robot's coordinate system reference direction, the ideal three-axis attitude angle for each sampling point is calculated. The sampling point positions and attitude angles are recorded sequentially to guide the robot in subsequent high-precision autonomous scanning along the target large vessel surface trajectory.

[0033] This embodiment extracts curve features consistent with the surface morphology of the target object through geometric analysis and processing of the preliminary 3D point cloud, and calculates the orientation of the sensor at each position on the curve. This enables the robot's motion and sensor orientation to match the geometric characteristics of the target object's surface during subsequent scanning, improving the consistency and accuracy of scanning and imaging. This reduces image distortion caused by orientation deviations and enhances the adaptability and imaging quality of the robot's autonomous scanning.

[0034] S40, by determining the corresponding new sensor position on the accessible surface for each of the ideal sensor orientations, a replanned scan path is generated; In this embodiment, the ideal sensor pose is defined by the position of the sampling point and its corresponding triaxial pose angle, which represents the desired spatial orientation at that position. The accessible surface refers to the external surface of the target object that can directly and physically contact the sensor; its geometry is predefined by 3D reconstruction or anatomical modeling before the scanning task begins. To map the ideal sensor pose to the accessible surface, a spatial ray needs to be constructed outward from the sampling point position corresponding to each ideal sensor pose, based on the normal direction of that pose. The starting point of the ray is the sampling point coordinate, and its direction is consistent with the normal direction of that point, indicating the direction in which the ideal sensor should face the surface of the target object.

[0035] By calculating the spatial geometric intersection points, it is determined whether there is an intersection point between the ray and the accessible surface. If so, the first intersection point is selected as the new sensor position corresponding to the ideal sensor posture. This new sensor position serves as the target position for controlling the movement of the end effector during robot scanning. To ensure the stability of the robot's movement and that the sensor's incident angle remains within a safe range, the tilt angle of the ideal sensor posture at the new sensor position also needs to be calculated. The tilt angle is the angle between the ideal three-axis posture angle at the new sensor position and the normal to the accessible surface. If the tilt angle exceeds a preset safety threshold, such as 20 degrees, the original normal direction needs to be adjusted, typically by gradually increasing or decreasing the angle in 5-degree increments, and the intersection point calculation between the ray and the accessible surface is repeated until a new sensor position and its ideal posture that meet the tilt angle requirements are obtained.

[0036] All new sensor positions are bound to their corresponding ideal sensor poses to form path nodes. These path nodes are connected in spatial order according to the original ideal sensor poses, forming a preliminary path point sequence. This preliminary path point sequence is divided into equidistant segments. Within each segment, tilt angle changes are analyzed, and the positions with the smallest and largest tilt angles are selected as key points. All key points are sequentially connected to form a smooth and continuous replanned scanning path. This replanned scanning path not only spatially closely matches the geometry of the target object's surface but also conforms to robot dynamics and force control requirements in terms of pose adjustment, ensuring robot motion and imaging stability.

[0037] Computational geometry software tools can be used to take the sampling points of each ideal sensor posture as the starting point of a ray, and use the normal direction as the ray direction parameter to solve for the coordinates of the first intersection point between the ray and the geometric model of the accessible surface, which is then used as the new sensor position. If the angle between the ideal sensor posture at this position and the normal of the accessible surface exceeds a safety threshold, the direction vector is gradually rotated and recalculated along the local normal direction. During the adjustment process, a 5-degree step size provides a balance between accuracy and computational cost, ensuring that the calculation is feasible and efficient. The initial path point sequence can be divided by setting fixed-length intervals along the path length, calculating the tilt angle of each path node within each interval, and selecting the two nodes with the largest and smallest tilt angles as the key points of that segment. Finally, all key points are connected by spline curve fitting or polyline connection to generate a replanning scan path with no bends and continuous curvature.

[0038] Example Description: In an abdominal large blood vessel scanning task, the robotic system first uses each ideal sensor pose sampling point on the preliminary 3D point cloud as a starting point. A ray is constructed along the normal direction of the sampling point, and the intersection of this ray with the human skin model is determined as the new sensor position. For each intersection point, the angle between the ideal pose and the skin surface is calculated. When the angle exceeds 20 degrees, the normal direction is automatically adjusted by ±5 degrees and recalculated. All obtained new sensor positions that meet the safety angle conditions and their ideal poses are recorded as path nodes and connected in spatial order to form a path. The path is divided into 10 mm segments. Within each segment, the positions with the smallest and largest tilt angles are selected as key points. Spline curves are used to fit all key points, ultimately generating a smooth, continuous, and pose-constrained replanning scanning path to guide the robotic system in autonomously completing the scanning operation conforming to the surface geometry.

[0039] This embodiment achieves optimal imaging incidence by performing spatial ray mapping on each ideal sensor posture and combining the intersection point with the geometric model of the accessible surface. This converts the optimal imaging incident direction into executable path points and posture control parameters for robot control, enabling the robot to obtain a path trajectory conforming to the geometric features of the target surface during scanning tasks without the need for external optical tracking devices. The replanned scanning path ensures that the sensor tilt angle at each scanning position meets safety requirements, improves the stability of the robot's end effector motion, reduces the risk of poor contact with the accessible surface, and simultaneously enhances the coverage and imaging clarity of the target object.

[0040] S50, control the robot to move the sensor along the replanned scanning path, adjust the sensor's posture according to the replanned scanning path, and acquire multiple final two-dimensional images of the target object.

[0041] In this embodiment, the robot drives a sensor-equipped robotic arm via a motion control unit to plan its motion trajectory based on a pre-generated replanning scan path. The replanning scan path defines a set of new sensor positions arranged in spatial order and their associated ideal three-axis attitude angles. During robot control, the trajectory analysis module analyzes the position and attitude data of the path nodes in real time, treating them as continuous spatial targets. As the end-effectors move, the robot gradually adjusts the motion commands of each joint of the robotic arm to approximate the target trajectory.

[0042] Between adjacent path nodes, attitude adjustment is not accomplished through discrete jumps, but rather by interpolation to generate a smooth attitude transition sequence across the entire path segment. Attitude adjustment involves three degrees of freedom: pitch, yaw, and roll. Interpolation methods can be linear interpolation, spherical linear interpolation, etc., ensuring continuous changes in spatial attitude and minimizing the angle between the sensor's incident direction and the local surface normal of the target object throughout the entire movement process.

[0043] During movement, the robot continuously collects real-time contact force between the end effector and the accessible surface through a force feedback control module, and dynamically fine-tunes its trajectory based on the deviation between the current contact force and the preset target contact force value. Fine-tuning includes minor displacement corrections along the normal direction to maintain stable contact conditions and prevent poor contact or detachment caused by breathing motion or surface irregularities.

[0044] While the sensor moves along the replanned scanning path and completes smooth attitude adjustment and dynamic adjustment of contact force, the imaging module acquires two-dimensional ultrasound images of the target object in real time at preset anatomical positions, ensuring that reliable imaging data is obtained at every anatomically significant location during movement. Each final two-dimensional image is bound to its spatial position and three-axis attitude angle data at the time of acquisition and written to the storage medium, providing complete data chain support for subsequent reconstruction and analysis.

[0045] The robot motion control software can be used to input the replanned scanning path. A continuous path tracking mode can be set in the robot motion plan, and the spatial coordinates of the path nodes are used to define the displacement trajectory of the robotic arm's end effector. The ideal three-axis attitude angles of the path nodes serve as the target attitude constraints. Spherical linear interpolation can be used to calculate the attitude change sequence between adjacent nodes, ensuring smooth attitude adjustment. A six-axis force sensor is installed between the robot's end effector and the contactable surface to collect contact force data in real time. The contact force error is input to a PID controller for feedback adjustment, dynamically adjusting the robot's end effector displacement in the local normal direction to ensure that the contact force remains constant throughout the scanning process. For example, if the target contact force value is set to 5 Newtons, when the real-time measured contact force deviates from the target value, the robot immediately compensates by increasing or decreasing the end effector's small displacement along the normal direction. The imaging module is configured to trigger image acquisition based on the robot's spatial position when it passes through path nodes, and can also acquire images at a higher sampling frequency when passing through key anatomical locations to ensure the integrity of imaging information in key areas.

[0046] Example Description: When scanning large abdominal blood vessels, the robot first moves its end effector, equipped with an ultrasound sensor, to accurately position the sensor at the first node of the replanned scanning path and adjusts its posture to the ideal three-axis posture angle corresponding to that node. It then moves along the path at a constant speed, analyzing the spatial coordinates and posture information of the path nodes in real time. An interpolation algorithm is used to calculate the posture at each instant and adjust the motion commands of the robot's joints accordingly. Throughout the movement, the robot monitors the contact force between the end effector and the skin surface in real time using a force sensor. The current contact force is compared to a preset target value of 5 Newtons. If a deviation occurs, the robot immediately adjusts the small displacement of the end effector in the local normal direction to compensate. The imaging module is configured to automatically acquire two-dimensional ultrasound images when passing through path nodes, increasing the sampling frequency at key anatomical locations such as the junction of main and branch vessels. Each acquired two-dimensional image is bound to the spatial position and three-axis posture angle data at the current acquisition moment and synchronously written to the data storage module, forming a complete two-dimensional image and pose information dataset.

[0047] This embodiment guides robot movement by replanning the scanning path, tracking the positions of path nodes and ideal three-axis attitude angles as continuous motion targets. This allows the robot to autonomously adjust the position and spatial attitude of its end effector during scanning, ensuring the sensor maintains the optimal incident angle throughout its movement. Simultaneously, real-time force feedback adjusts the trajectory, effectively addressing contact force instability caused by geometric changes on the target surface or breathing motion, thus improving the reliability of robot operation. The resulting 2D images not only have complete spatial coverage but also each image carries accurate spatial position and attitude labels, guaranteeing the accuracy and stability of subsequent 3D reconstruction.

[0048] This invention relates to the field of robot control technology and discloses a method, apparatus, device, and medium for adjusting a robot scanning path. The method includes: controlling a robot equipped with sensors at its end effector to move along a preliminary path; acquiring multiple preliminary two-dimensional images of a target object; processing the multiple preliminary two-dimensional images to extract feature points of the target object and combining this with the spatial information of the sensors acquired during the acquisition of each preliminary two-dimensional image to generate a preliminary three-dimensional point cloud representing the surface of the target object; determining multiple ideal sensor postures based on the preliminary three-dimensional point cloud, each ideal sensor posture corresponding to the normal direction of a local surface on the preliminary three-dimensional point cloud; determining a corresponding new sensor position on an accessible surface for each ideal sensor posture to generate a replanned scanning path; controlling the robot to move the sensors along the replanned scanning path and adjusting the sensor postures; and acquiring multiple final two-dimensional images of the target object. This invention optimizes the robot scanning path by combining the local surface normal direction of the preliminary three-dimensional point cloud, ensuring that the sensor posture matches the geometric features of the target object's surface. This solves the problem of surface fragmentation in three-dimensional reconstruction caused by inter-frame pose mismatch in two-dimensional images under free breathing conditions, thus improving the accuracy and clinical applicability of autonomous robot ultrasound scanning.

[0049] In one embodiment, step S10 includes: S101, the gesture guidance module marks the start and end points on the accessible surface of the target object, and the path planning module connects the start and end points to generate preliminary path data to define the motion range of the robot with the end sensor. S102, at the starting position of the preliminary path data, the robot control system calculates the local normal direction based on the three-dimensional reconstruction model of the accessible surface, and adjusts the robot joint angle to make the emission direction of the sensor consistent with the local normal direction, so as to initialize the posture of the robot with the sensor at the end. S103, the robot drive module controls the robot with sensors at the end to move along the preliminary path data at a constant translation speed, and the pose acquisition module synchronously records the spatial position and three-axis attitude angle data of the sensors in real time. S104, during the process of the sensor moving along the preliminary path data, the imaging module of the sensor emits a sound beam toward the bone structure of the target object and receives the echo signal. The imaging signal processing module converts the received echo signal into multiple preliminary two-dimensional images of the target object and generates a two-dimensional image sequence. S105, the data fusion module binds each preliminary two-dimensional image with the spatial position and three-axis attitude angle data of the sensor at the corresponding acquisition time, and writes it into the non-volatile memory array through the storage controller to form the acquisition result with spatial position and three-axis attitude angle labels. S106, the image quality analysis module calculates the signal-to-noise ratio of the bone structure edge pixels of each preliminary two-dimensional image. If the signal-to-noise ratio is lower than a preset threshold, the preliminary two-dimensional image is marked as invalid data. The path replanning module generates a local supplementary path based on the location of the invalid data and triggers the robot to re-acquire the preliminary two-dimensional image at the corresponding location.

[0050] In this embodiment, during the process of controlling the robot to move along a preliminary path and acquire multiple preliminary two-dimensional images of the target object, it is first necessary to determine the spatial range of the scanning path. Through the gesture guidance module, the human-machine interface captures the operator's movement trajectory, accurately identifying and recording two spatial position points on the accessible surface of the target object, serving as the starting and ending position points of the path, respectively. This marking operation not only defines the spatial boundaries of the scanning area but also provides input parameters for the path planning module. After receiving the starting and ending position points, the path planning module automatically generates a spatial curve between the two points based on the surface shape characteristics of the target object, forming preliminary path data. The preliminary path data is essentially a continuous set of spatial coordinates, which defines the range of motion of the robot with sensors at its end effector and provides spatial reference for subsequent movement.

[0051] At the starting point of the initial path data, the robot control system calls upon the 3D reconstructed model of the accessible surface and extracts the local normal direction at the starting point using the local geometric information of this model. This normal direction reflects the spatial geometry of the target object's surface and is crucial in determining the incident direction of the sensor's acoustic beam. Through inverse kinematics calculations, the robot control system solves for the corresponding joint angle configuration, ensuring precise alignment of the position and orientation of the sensor mounted on the robot's end effector. This ensures that the sensor's emission direction is completely consistent with the local normal direction, achieving the initialization of the robot's initial posture and guaranteeing that subsequent scanning processes begin from the optimal posture state.

[0052] After acquiring path data and initial posture, the robot drive module controls the robot to move along the initial path data at a constant translational speed. During movement, the pose acquisition module samples and records the spatial position and three-axis attitude angles of the sensors mounted on the robot's end effector at fixed time intervals within the system's real-time control loop. The spatial position data includes the x, y, and z coordinates in three-dimensional space, while the three-axis attitude angle data represents the sensor's spatial orientation at pitch, yaw, and roll angles. This real-time data allows for complete trajectory reconstruction of the robot's movement and provides pose information support for subsequent data processing.

[0053] During the movement, the imaging module operates at a fixed frame rate, transmitting a sound beam towards the bone structure of the target object according to the synchronization control signal and receiving echo signals returned from different tissue interfaces. The imaging signal processing module converts the echo signals into a two-dimensional grayscale image sequence. Each image records the two-dimensional slice information of the sound beam passing through the target object along the incident direction at that moment, forming a series of preliminary two-dimensional image data.

[0054] The data fusion module associates and binds each preliminary 2D image with its corresponding spatial location and three-axis attitude angle data at the time of acquisition, ensuring that each image can be traced back to its complete spatial location information at the time of acquisition. After binding, the acquisition results are written to a non-volatile memory array by the storage controller, ensuring long-term traceability and integrity of the data. Each image data is stored as a data unit along with its corresponding spatial location and three-axis attitude angle, forming a spatially labeled 2D image sequence dataset.

[0055] The image quality analysis module performs real-time quality assessment on each image during data acquisition, particularly calculating the signal-to-noise ratio (SNR) of pixels at the edges of the bone structure of the target object. This calculation is based on predefined spatial filtering and local contrast analysis algorithms to ensure the accuracy of the assessment results. If the SNR is lower than a preset threshold, it means that the image data may be affected by motion artifacts, poor acoustic beam coupling, or abnormal tissue acoustic properties, resulting in substandard image quality. In this case, the image quality analysis module marks the image as invalid data, and the path replanning module locally generates a supplementary path on the initial path data based on the location of the invalid data. The robot is then re-instructed to re-execute the acquisition operation at the location of the supplementary path, automatically completing the local data repair. This process improves the effective data ratio and spatial coverage of the final dataset, ensuring the integrity and accuracy of the input data during subsequent 3D point cloud reconstruction.

[0056] Each stage is tightly linked through data dependencies and real-time feedback: initial posture initialization establishes a benchmark for subsequent trajectory control and posture recording; robot translation and pose acquisition ensure strict spatial and directional correspondence between the 2D images at each moment; imaging and data binding ensure a closed-loop data structure; and quality analysis and path replanning correct abnormal data points in real time during acquisition. The overall process forms a composite mechanism of spatial path, pose adjustment, imaging, data annotation, quality self-checking, and local re-acquisition, improving spatial coverage accuracy and data consistency, and reducing imaging data errors caused by irregular target surfaces or motion artifacts.

[0057] This embodiment ensures high traceability and spatial consistency of image data by precisely initializing and dynamically controlling the robot's spatial position and posture during the initial path movement and data acquisition process, combined with binding imaging data to real-time pose. Utilizing a real-time image quality analysis mechanism, low-quality data is immediately reacquired and supplemented locally, ensuring the final dataset possesses advantages such as comprehensive spatial coverage, reliable quality, and a high signal-to-noise ratio. This closed-loop process of continuous control, recording, analysis, and supplementary acquisition overcomes trajectory deviations, posture inconsistencies, and imaging artifacts caused by the geometric complexity of the target surface and breathing motion, ensuring the accuracy and completeness of the 2D imaging data for subsequent spatial reconstruction.

[0058] In one embodiment, step S20 above includes: S201, read multiple preliminary two-dimensional images and the spatial position and three-axis attitude angle data of the sensor corresponding to each preliminary two-dimensional image as input data; S202, apply a deep learning segmentation model to process the multiple preliminary two-dimensional images respectively, and extract the bone structure edge pixels of the target object in each preliminary two-dimensional image as feature points; S203, convert the bone structure edge pixels extracted from each preliminary two-dimensional image into a two-dimensional coordinate set in the image coordinate system to form a feature point coordinate set; S204, Based on the spatial position and three-axis attitude angle data of the sensor corresponding to each preliminary two-dimensional image, the set of feature point coordinates is transformed into a unified robot three-dimensional coordinate system through a spatial calibration matrix to generate the transformed three-dimensional coordinate data; S205: Align all the transformed 3D coordinate data according to the corresponding acquisition timestamps and fuse them to form a spatial point set; S206, Execute a statistical outlier removal algorithm on the set of spatial points to remove isolated points and output a sparse preliminary three-dimensional point cloud representing the surface of the target object.

[0059] In this embodiment, multiple preliminary 2D images are processed to extract feature points of the target object, and combined with the sensor spatial information acquired at the time of each image acquisition, aiming to construct a sparse preliminary 3D point cloud consistent with the physical world. First, reading multiple preliminary 2D images and their corresponding sensor spatial positions and three-axis attitude angle data as input data is a multimodal data integration process. The 2D images provide visual information, while the spatial position and attitude angle data provide geometric references at the time of imaging; their integration is a prerequisite for subsequent image space mapping to the robot's 3D coordinate system. The reading operation requires the data management module to retrieve data records from the memory in timestamp order and pair the image data with the spatial data to form standardized input.

[0060] Deep learning segmentation models process multiple preliminary 2D images separately to automatically extract bone structure edge pixels from complex images. This model can employ convolutional neural networks with architectures such as U-Net or DeepLab, and achieves strong bone tissue edge recognition capabilities after pre-training on large-scale labeled datasets. The segmentation model outputs a binary mask that clearly marks the pixel locations of bone tissue boundaries in each 2D image; these pixels describe the local geometry of the anatomical structure on the image plane.

[0061] The key step in coordinateization is converting the extracted bone structure edge pixels into a two-dimensional coordinate set in the image coordinate system. Each image itself contains a fixed pixel spacing and an image center origin definition. The pixel indices are converted into actual physical size coordinates (e.g., millimeter level) using image geometric information. The resulting set of feature point coordinates accurately describes the two-dimensional spatial distribution of the target object's edge within the image plane.

[0062] Based on the sensor spatial position and three-axis attitude angle data corresponding to each image, the core operation of restoring local two-dimensional observation data to the physical world is to transform the set of feature point coordinates into a unified robot three-dimensional coordinate system through a spatial calibration matrix. The spatial calibration matrix defines the spatial relationship between the image plane coordinates and the robot coordinate system, taking into account the camera's extrinsic parameters (position and orientation) and intrinsic parameters (focal length, principal point offset). This transformation ensures that feature points of two-dimensional images acquired at different times and locations can be accurately superimposed under the same three-dimensional spatial reference frame, generating a three-dimensional description consistent with the real target object.

[0063] Aligning all transformed 3D coordinate data according to their corresponding acquisition timestamps is a crucial step in the fusion process, ensuring both temporal continuity and spatial consistency. This alignment operation integrates multiple 3D coordinate sets in chronological order, reducing spatial misalignment caused by acquisition sequence or synchronization deviations, and forming a continuous and comprehensive set of spatial points.

[0064] Performing statistical outlier removal algorithms on a spatial point set to remove isolated points is a crucial step in improving data quality. These algorithms calculate the degree of outlier status based on the density distribution of points within the local neighborhood of the point cloud, discarding points whose distance from neighboring points exceeds a set multiple of the standard deviation. This process eliminates spatial noise points caused by image segmentation errors, spatial transformation errors, or acquisition anomalies, preserving representative spatial points on the surface of the target object.

[0065] The final output, a sparse preliminary 3D point cloud representing the surface of the target object, is the entire process and forms the basic input for subsequent steps such as attitude calculation and scan path replanning. This point cloud accurately records the spatial contour of the target object's surface. Although the point cloud density is low, it can completely cover the key structural contours, meeting the requirements for geometric reconstruction and data quality.

[0066] This embodiment combines the skeletal edge pixels of multiple 2D images with the sensor's spatial position and three-axis attitude angle data at the time of image acquisition. After standardized spatial calibration and transformation, it ensures that data acquired at different times and spaces are unified into the robot's global 3D coordinate system. Timestamp alignment and statistical outlier removal algorithms are used to further clean up noisy data, resulting in a sparse and accurate 3D point cloud. This process solves the positional mismatch problem between 2D images caused by sensor movement and attitude changes, ensuring that the true spatial surface of the target object can still be recovered through standardized geometric calculations even under free-breathing conditions, thus ensuring the accuracy and data quality of subsequent reconstruction and path planning processes.

[0067] In one embodiment, step 30 above includes: S301, Project the preliminary three-dimensional point cloud onto the robot's working plane along the long axis of the target object to obtain the two-dimensional distribution of the preliminary three-dimensional point cloud; S302, Extract the upper envelope point sequence from the two-dimensional distribution of the preliminary three-dimensional point cloud; S303, The upper envelope point sequence is processed using a moving average algorithm, and a continuous bone surface contour curve is generated by fitting a cubic spline function. S304, Calculate the tangent direction of each sampling point of the bone surface contour curve; S305, at each sampling point, the perpendicular direction of the tangent direction in the plane of the bone surface contour curve is calculated based on the tangent direction as the normal direction of the local surface, and the normal direction is used to define the ideal sensor posture at the sampling point position, so that the ideal sensor posture includes the sampling point position and the ideal three-axis posture angle. S306, Record and store the ideal sensor attitude corresponding to all sampling points on the preliminary three-dimensional point cloud.

[0068] In this embodiment, the process of determining the poses of multiple ideal sensors based on a preliminary 3D point cloud begins with the geometric analysis and projection of the preliminary 3D point cloud. The preliminary 3D point cloud records the sparse distribution information of the target object's surface in space. Projecting along the major axis of the target object compresses the spatial 3D information into a 2D distribution within the robot's working plane, clearly representing the geometric features of the point cloud data along the main anatomical directions. The major axis direction can be determined based on the spatial orientation of the target object (e.g., spine or blood vessels), typically using the anatomically dominant orientation as the major axis reference. The robot's working plane is a coordinate plane consistent with the robot's motion trajectory planning, usually a fixed plane related to the robot's base coordinate system.

[0069] In the projected two-dimensional distribution, the purpose of extracting the upper envelope point sequence is to select the point with the maximum vertical height from the two-dimensional point cloud, forming a sequence describing the variation of the highest point on the target object's surface. This process involves traversing the projection results horizontally and selecting the point with the maximum vertical height to ensure that the outline of the target object's surface is captured in the vertical direction.

[0070] The moving average algorithm is applied to process the upper envelope point sequence to eliminate high jitter caused by data noise or local anomalies through local mean smoothing. The moving average algorithm effectively reduces local fluctuations by calculating the mean within the neighborhood of each point. After smoothing, a continuous bone surface contour curve is generated by fitting a cubic spline function. The cubic spline function has continuous first and second derivatives at each node, thus accurately fitting a continuous smooth curve, preserving the overall trend of bone surface shape change, while avoiding local oscillations caused by noise.

[0071] Calculating the tangent direction at each sampling point of the bone surface contour curve depends on the curve's parametric representation. At each point on the curve, the tangent direction at that point can be obtained by calculating the curve's derivative vector. The tangent direction describes the local surface orientation within the curve plane, providing the geometric basis for subsequent calculations of the normal direction.

[0072] At each sampling point, the perpendicular direction of the tangent direction within the bone surface contour curve plane is calculated based on the tangent direction. This direction serves as the normal direction of the local surface, achieved by rotating the tangent direction by 90 degrees within the curve plane. The normal direction describes the direction of the outward normal vector of the local surface. The ideal sensor posture is defined based on this direction, ensuring that the sensor's posture at that sampling point is orthogonal to the local surface, thereby improving signal reception quality. The ideal sensor posture includes the spatial position of the sampling point and the three-axis posture angles aligned with the normal direction. These three-axis posture angles can be expressed by a rotation matrix or Euler angles, describing the sensor's spatial orientation relative to the robot's base coordinate system.

[0073] Recording and storing the ideal sensor poses corresponding to all sampling points on the initial 3D point cloud is to provide a complete data foundation for subsequent scanning path planning. The recorded data must include spatial 3D position and complete pose information so that the robot can perform trajectory reconstruction and pose adjustment operations later.

[0074] This embodiment compresses preliminary 3D point cloud data into the robot's working plane and extracts the envelope point sequence on the surface. Combining moving average and cubic spline fitting methods, a continuous and smooth bone surface contour curve is obtained, effectively extracting the continuous geometric shape of the target object from the sparse point cloud. Further calculation of the tangent direction and local perpendicular direction at the contour curve sampling points accurately determines the ideal sensor posture orthogonal to the local surface at each sampling point, ensuring optimized incident angle between the sensor and the target surface and improving the quality and consistency of the final 2D image. This process significantly reduces posture errors caused by data discontinuities or geometric noise, laying a high-quality data foundation for accurate generation of the subsequent scanning path and robot posture adjustment.

[0075] In one embodiment, step S40 above includes: S401, starting from the sampling point position corresponding to each ideal sensor posture, construct a ray along the normal direction corresponding to the ideal sensor posture; S402, Calculate the first intersection point between the ray and the accessible surface, and determine the intersection point as the new sensor position corresponding to the ideal sensor posture; S403, calculate the tilt angle of the ideal sensor orientation for each new sensor position. When the tilt angle exceeds the preset safety threshold, adjust the normal direction and recalculate the new sensor position. S404, bind each new sensor position to the ideal sensor attitude corresponding to that position to form a path node; S405, connect all path nodes in the order of the ideal sensor orientation to generate a preliminary path point sequence; S406, the preliminary path point sequence is divided into equidistant segments, and the path nodes with the smallest and largest tilt angles in each segment are selected as key points. S407 connects all key points to generate a smooth and continuous replanned scan path.

[0076] In this embodiment, a replanned scan path is generated by determining the corresponding new sensor position on the accessible surface for each ideal sensor pose. This begins with the sampling point positions corresponding to the ideal sensor pose. These sampling point positions are recorded 3D spatial points that reflect the spatial coordinates of the initial 3D point cloud related to the local surface normal direction. The normal direction corresponding to the ideal sensor pose is obtained by rotating the tangent direction of the curve 90 degrees in the plane, describing the direction of the ideal incident angle. Constructing a ray along the normal direction corresponding to the ideal sensor pose means defining a linear geometric trajectory in space from each sampling point position, based on the normal direction, for subsequent calculation of the intersection with the accessible surface.

[0077] Calculating the first intersection point between the ray and the accessible surface involves performing intersection point calculations based on a spatial geometric model. The accessible surface is a model in space representing a scannable external surface such as skin, typically defined using 3D reconstructed data. This intersection point is determined as the new sensor position corresponding to the ideal sensor posture, serving as the target point for future robot trajectory adjustments.

[0078] For each new sensor location, the tilt angle of the ideal sensor orientation needs to be calculated. This tilt angle is the angle between the ideal sensor orientation and the normal direction of the accessible surface, used to assess whether the current orientation is suitable for safe scanning. When this tilt angle exceeds a preset safety threshold, the current normal direction needs to be adjusted and the intersection point recalculated. Adjusting the normal direction typically involves iteratively rotating in positive and negative directions with small angular steps (e.g., 5 degrees) from the original orientation until an intersection point that satisfies the safety angle is found. This ensures contact stability and signal quality between the sensor and the accessible surface during scanning.

[0079] Each new sensor position is bound to its corresponding ideal sensor attitude, forming a path node. The path node includes spatial position and three-axis attitude angle data, serving as the unit point for subsequent path execution. All path nodes are connected in the order of the ideal sensor attitude to generate a preliminary path point sequence. This order is consistent with the spatial arrangement of sampling points in the preliminary 3D point cloud, ensuring consistency between the scan coverage and the preliminary analysis results.

[0080] The initial path point sequence is divided into segments with equal spatial length. Within each segment, the path nodes with the smallest and largest tilt angles are selected as key points to simplify the path while retaining the extreme points of tilt angle changes in the path, thereby reducing redundant path points.

[0081] Finally, connecting all key points generates a smooth and continuous replanning scan path, ensuring that the robot's movement path is smooth and without abrupt changes in space, and that attitude adjustment is continuous and feasible, making it suitable for actual execution.

[0082] This embodiment utilizes spatial geometric calculations based on the ideal sensor posture and normal direction, combined with a contactable surface model to solve for intersection points, and introduces tilt angle threshold constraints and angle adjustment mechanisms. This enables the generation of a replanned scanning path consistent with the surface morphology of the target object while ensuring safe incidence between the sensor and the scanning surface. Furthermore, by optimizing path nodes and selecting key points, path complexity is reduced and path smoothness and posture continuity are improved. This significantly enhances the stability, scanning efficiency, and image acquisition quality of the robot when executing the scanning path, ensuring strong applicability in clinical or complex environments.

[0083] In one embodiment, step S50 above includes: S501, control the robot to move the sensor to the new sensor position corresponding to the starting path node of the replanned scan path, and adjust the sensor's posture to the ideal sensor posture corresponding to the starting path node. S502, control the robot to move the sensor along the replanned scanning path at a constant translation speed, and continuously interpolate the three-axis attitude angles of the sensor in real time between adjacent path nodes so that the attitude of the sensor transitions smoothly during the movement. S503, during the process of the sensor moving along the replanned scanning path, the force feedback control module monitors the contact force between the sensor and the contactable surface in real time, and dynamically adjusts the position of the sensor based on the real-time contact force to maintain the preset contact force value. S504, During the movement, the imaging unit is triggered at a preset anatomical position to acquire the final two-dimensional image of the target object; S505, bind each final two-dimensional image to the position of the sensor and the three-axis attitude angle at the time of acquisition, and write it to the storage medium.

[0084] In this embodiment, firstly, when controlling the robot to move the sensor to the new sensor position corresponding to the starting path node of the replanned scanning path, the starting path node represents the first valid scanning point defined in the previous replanned path, containing spatial coordinates and attitude angle data. The new sensor position is the actual spatial position on the accessible surface corresponding to the starting node, which is the target position for robot motion control. Adjusting the sensor's attitude to the ideal sensor attitude corresponding to the starting path node means that the robot adjusts the mechanical joint angles and the end effector attitude to align the sensor with the ideal three-axis attitude angles, achieving attitude registration with consistent angles and directions in space, and ensuring that the scanning emission axis is consistent with the normal direction of the target surface.

[0085] Controlling a robot to move sensors along a replanned scanning path at a constant translational speed means that the motion control system moves along a set of path points in space at a fixed linear velocity through velocity planning and path tracking algorithms, while requiring smooth speed to reduce vibrations caused by dynamic acceleration and deceleration. Real-time continuous interpolation of three-axis attitude angles between adjacent path nodes refers to calculating a continuous sequence of attitude angles between two discrete path nodes using interpolation algorithms (such as linear interpolation or spherical linear interpolation), enabling smooth changes in the robot's end effector attitude in continuous space and avoiding abrupt changes between adjacent nodes.

[0086] During movement, the force feedback control module monitors the contact force between the sensor and the accessible surface in real time, and the force-sensitive element embedded in the sensor collects the force data in real time. Based on this real-time contact force, the position of the sensor is dynamically adjusted. That is, when the detected contact force deviates from the preset contact force value, the robot control system issues a correction command to adjust the displacement along the normal direction in real time to maintain stable contact pressure, thereby optimizing the sound beam coupling conditions and reducing signal attenuation and image artifacts.

[0087] During movement, the imaging unit is triggered at a preset anatomical location to acquire the final two-dimensional image of the target object. The preset anatomical location is a spatial region selected in advance based on the importance of the target anatomical structure, such as the intersection of important blood vessels or key surface curvature change points, ensuring that the sampling density matches the importance. Triggered acquisition occurs when the robot moves to this spatial location, and the control logic automatically calls the imaging hardware to perform real-time data acquisition.

[0088] Each final 2D image is bound to the sensor position and three-axis attitude angles at the time of acquisition and written to a storage medium. This means that each time an image is acquired, the current spatial coordinates and attitude angle data are packaged together to form a complete data record, which is then stored in a non-volatile storage medium to support subsequent 3D reconstruction and data traceability. The binding process ensures a strict one-to-one correspondence between the 2D image data and the corresponding acquisition time and spatial pose, forming a complete spatial data chain.

[0089] This embodiment, through motion control, posture adjustment, real-time force control, precise acquisition, and complete data recording of the robot and sensors, ensures smooth movement of the robot along a replanned path on accessible surfaces with complex topography without relying on external positioning devices. Simultaneously, it ensures that the sensors maintain normal contact with the target surface and continuously adjust their posture angles. This process achieves high-precision dynamic coordination of posture and position, significantly improving the spatial consistency, image quality, and anatomical coverage of the final 2D image. Ultimately, it provides high-quality, spatiotemporally consistent input data for subsequent reconstruction, ensuring stable and reliable scanning efficiency and imaging accuracy in clinical or complex operating environments.

[0090] In one embodiment, after step S50 above, the method further includes: S601, merge the sparse preliminary three-dimensional point cloud representing the surface of the target object with the three-dimensional point cloud data generated based on the multiple final two-dimensional images; S602, performs a statistical outlier removal algorithm on the merged 3D point cloud data to remove isolated points and generate a high-density 3D point cloud model representing the surface of the target object. S603, the iterative nearest point algorithm is applied to register the high-density 3D point cloud model with the reference model, and the surface coverage, reconstruction accuracy and comprehensive F value of the high-density 3D point cloud model are calculated based on the registration results to generate a 3D reconstruction quality assessment report to characterize the reconstruction effect.

[0091] In this embodiment, merging the sparse preliminary 3D point cloud representing the surface of the target object with the 3D point cloud data generated from multiple final 2D images refers to unifying the coordinate systems and aligning the timestamps of two sets of 3D coordinate data from different sources to form a unified dataset. The sparse preliminary 3D point cloud represents the point cloud obtained by processing preliminary 2D images and sensor spatial position and attitude information; it typically has a wide coverage but low density. The 3D point cloud data generated from multiple final 2D images is a higher-resolution 3D point cloud obtained through a similar spatial transformation and reprojection method after the robot has completed its movement along the replanned scanning path, completed attitude adjustment, and acquired the final 2D images. Merging the two achieves spatial fusion of scanning data from different stages.

[0092] Performing a statistical outlier removal algorithm on the merged 3D point cloud data to remove isolated points involves using statistical filtering methods (such as setting neighborhood distance and standard deviation thresholds) to identify abnormally sparse isolated points or noise points in the 3D point cloud space and eliminating them to improve the overall spatial consistency and smoothness of the point cloud. This process can reduce scattered data introduced by imaging errors, motion jitter, and abnormal photoacoustic coupling, thereby improving data quality.

[0093] Generating a high-density 3D point cloud model representing the surface of a target object involves further interpolation and spatial filling of the point cloud data after outlier removal. This expands the low-density data into a surface representation with high point density, thus meeting the needs of subsequent surface reconstruction and precise measurement, and improving the completeness of model details.

[0094] Applying the iterative nearest-point algorithm to register a high-density 3D point cloud model with a reference model means using an optimization algorithm that progressively minimizes the spatial distance between the two point cloud sets, gradually adjusting the rigid transformation parameters (translation and rotation) of the point cloud model to match the reference model. The reference model can be a pre-acquired high-precision gold standard model or a standard template of the target anatomical region, used as a spatial benchmark for the reconstruction results. After registration, the two achieve optimal overlap in 3D space.

[0095] Calculating the surface coverage, reconstruction accuracy, and overall F-score of a high-density 3D point cloud model based on registration results involves analyzing indicators such as the overlap area and distance error distribution of the two registered models to obtain coverage indicators (i.e., the percentage of the reconstructed model covering the surface of the reference model), reconstruction accuracy (i.e., the root mean square error between the two models), and an overall F-score (which may be a weighted combination indicator used to reflect the overall quality). These indicators quantitatively evaluate the accuracy, completeness, and spatial consistency of the scanning and reconstruction results.

[0096] Generating a three-dimensional reconstruction quality assessment report to characterize the reconstruction effect means integrating the above-mentioned indicators into a document or data record for subsequent verification, archiving, or as a basis for quality inspection before clinical use.

[0097] In one specific implementation, the hardware and software systems must be assembled and calibrated before operation. The hardware includes a KUKA Robot MED7 seven-DOF lightweight robot, whose end effector connects to a Nano25 six-dimensional force / torque sensor via a quick-change plate. A custom aluminum alloy L-shaped clamp is installed below the sensor to securely fix the linear array ultrasound probe. Experimental verification shows that a constant contact force between the probe and the skin surface is crucial for obtaining high-quality ultrasound imaging. Therefore, a five-DOF impedance control strategy is adopted, with rotation in the xt direction to maintain the position mode, and force control in the other directions. During scanning, the force control parameters are set to 5 N or 8 N for different applications, which can control the contact force fluctuation within ±7%, and no probe slippage or acoustic coupling failure occurs.

[0098] The software platform is based on Ubuntu 20.04 and ROS 2. Kinematics solving is performed using the eTaSL task graph language framework, which can map the target pose to joint motion commands in real time. Ultrasound data is input via a frame acquisition card at a sampling frequency of 15 Hz. A modified BCDU-Net network running on the GPU is used to perform bone contour segmentation. After 20 rounds of training, an F1-score of 0.67 and a balance accuracy of 0.91 were achieved on volunteer data, ensuring the accuracy of subsequent 3D point cloud reconstruction.

[0099] In "gesture-guided" mode, the operator guides the probe sequentially to key anatomical locations within the scanning area. Three sets of poses are automatically recorded and connected to form an "S"-shaped path traversing multiple anatomical units. The robot performs scanning along this path at a translational speed of 3 mm / s or 4 mm / s. During the scanning process, the probe's normal direction remains essentially perpendicular to the skin surface, ensuring the sound beam incident angle is within the optimal range. The bone structure edge pixels segmented from each frame are transformed using a spatial calibration matrix and accumulated into a sparse 3D point cloud, forming a preliminary reconstruction model.

[0100] To support subsequent surface normal calculations, the initial point cloud is projected onto the robot's working plane along its major axis. The upper envelope point sequence along the z-axis is extracted from the projection results, smoothed using a moving average, and then fitted with cubic splines to form a continuous curve. This curve contains an average of 120 to 150 sampling nodes, fully covering the length of a single anatomical unit, preserving surface geometry while effectively suppressing isolated noise, providing a stable reference for subsequent normal direction calculations.

[0101] By selecting any point on the curve and calculating the tangent direction, the normal direction can be derived. Extrapolate the ray along the normal direction to search for the intersection point on the skin surface, recording the coordinates of the intersection point and the normal direction. If the ray does not intersect the skin surface, automatically adjust the normal direction in 5° increments until an intersection point is found; if the calculated normal angle exceeds the ±20° safety threshold, truncate it to the limit angle and search again. In this way, a new set of corresponding sensor positions and ideal orientations can be generated on the skin side.

[0102] To reduce data redundancy and improve control efficiency, the entire skin trajectory was divided into several segments at 10 mm intervals. Each segment retained only two key points: the maximum and minimum pose. The robot performed linear interpolation within each segment to achieve a smooth pose transition. This method significantly reduced the number of control commands and eliminated discontinuities during motion. Verification results showed that pose changes were smooth throughout the entire operation without any noticeable stuttering.

[0103] The robot performs a secondary scan along the replanned path, maintaining force control to preserve constant contact force. Due to large-angle deflections in some areas, a 2 cm thick soft coupling pad was added to cover the probe surface in volunteer experiments to ensure acoustic coupling integrity. This design was verified to significantly improve the clarity of images at large tilt angles without affecting resolution. The secondary scan takes an average of approximately 3 minutes per dissection unit, and point cloud data is overlaid and displayed in real time, providing visual feedback.

[0104] After the second scan, the system automatically merged the new point cloud with the initial point cloud data, optimized the data density using a statistical outlier removal algorithm, and generated a high-density reconstruction model. Subsequently, the ICP algorithm was used to register the model with the preoperative reference CT model, and coverage, geometric accuracy, and overall F-score were calculated based on a 2 mm distance threshold. Validation results showed that the overall coverage increased from 32% to 56%, and the coverage of key anatomical regions increased by 153%.

[0105] Example Description: In this application, the robot's operating system is first adapted to the medical scenario, the robotic surgical platform is started, and a linear array ultrasound probe is installed at the end effector. The operator uses a gesture guidance module to mark the start and end points of the scan on the skin surface of the accessible area on the patient's abdomen, forming preliminary path data defining the range of motion. The path planning module connects the start and end points to generate a path curve for the robot's movement.

[0106] At the starting point of the initial path, the robot control system calculates the local normal direction at that location based on the 3D reconstructed model of the accessible surface, adjusts the angles of each joint of the robot to align the sensor's emission direction with the local normal direction, and completes the initialization of the robot's posture. Subsequently, the robot drive module translates along the initial path at a constant speed, while the pose acquisition module synchronously records the sensor's position and three-axis attitude angle data in real time, forming a motion trajectory record corresponding to each moment.

[0107] During robot movement, the imaging module emits a sound beam towards the internal skeletal structure of the target object and receives the echo signal. The imaging signal processing module converts the echo signal into multiple preliminary two-dimensional images in real time, forming a continuous two-dimensional image sequence. The data fusion module binds each preliminary two-dimensional image with the corresponding sensor spatial position and three-axis attitude angle at that moment, and writes it into the storage medium to form a collection of acquired data with spatial and attitude labels.

[0108] The image quality analysis module calculates the signal-to-noise ratio (SNR) of the bone structure edge pixels in each preliminary 2D image. If the SNR is below a threshold, the image is marked as invalid data. The path replanning module generates a local supplementary path based on the locations corresponding to these invalid data and triggers the robot to re-acquire data at those locations, ensuring the quality and integrity of the original data.

[0109] After data acquisition, the processing module reads the recorded preliminary 2D images and the corresponding sensor position and attitude data for each image. It then uses a deep learning segmentation model to extract the edge pixels of the bone structure frame by frame, converting them into a set of 2D coordinates in the image coordinate system. Combining the sensor's spatial position and three-axis attitude angles during acquisition, a spatial calibration matrix is ​​used to transform these 2D coordinate sets into the robot's 3D coordinate system, generating a set of 3D coordinate points for each image. All 3D coordinate points are aligned according to the acquisition timestamp, fused into a sparse 3D point cloud, and isolated points are removed using a statistical outlier removal algorithm to obtain a preliminary 3D point cloud representing the surface of the target object.

[0110] Based on the initial 3D point cloud, the operating system projects the point cloud data onto the robot's working plane along the long axis of the target object, obtaining a 2D distribution. The upper envelope point sequence of the 2D distribution is extracted, smoothed using moving average, and fitted with cubic splines to form a continuous skeletal contour curve. The tangent direction is calculated for each sampling point on the curve. At each point, the perpendicular direction of this tangent direction within the curve plane is determined as the local normal direction. This normal direction is used to define the ideal sensor pose at the sampling point location. Each pose includes the corresponding spatial position and three-axis pose angles. The system records and stores the ideal sensor poses of all sampling points as a geometric reference for replanning the path.

[0111] The system starts from the sampling point position corresponding to each ideal sensor posture, constructs a ray along the normal direction corresponding to that posture, and solves for the first intersection point between the ray and the accessible surface. This intersection point is taken as the corresponding new sensor position. When the posture tilt angle of the new sensor position exceeds a preset safety threshold, the system automatically adjusts the normal direction and recalculates to ensure safety. Each new sensor position is bound to its corresponding ideal sensor posture to form a path node. The nodes are connected sequentially to generate a preliminary path point sequence. The preliminary path point sequence is divided into equidistant segments. Within each segment, the two nodes with the smallest and largest tilt angles are selected as key points. Connecting all key points forms a smooth and continuous replanning scan path.

[0112] The robot re-executes the scan according to the replanned scanning path. It first moves to the new sensor position corresponding to the starting node of the path and adjusts to the ideal sensor posture at that node. The robot moves along the replanned path at a constant speed, continuously interpolating the three-axis posture angles in real time between adjacent path nodes to ensure a smooth transition in posture during movement. During the scan, the force feedback control module monitors the contact force between the probe and the skin in real time, dynamically adjusting the probe position based on real-time data to maintain the preset contact force value. The robot triggers the imaging unit to acquire the final two-dimensional image at preset anatomical key locations. Each final two-dimensional image, along with the sensor position and three-axis posture angle at the time of acquisition, is bound to the storage medium and written to form a complete, high-quality dataset with spatial and posture information.

[0113] Finally, the operating system merges the sparse preliminary 3D point cloud with the 3D point cloud data generated based on the final 2D image, performs statistical outlier removal to eliminate isolated points, and obtains a high-density 3D point cloud model. This high-density model is registered with a reference model using an iterative nearest-neighbor algorithm, and the surface coverage, geometric reconstruction accuracy, and overall F-value of the 3D point cloud model are calculated based on the registration results. Ultimately, a complete 3D reconstruction quality assessment report is generated, providing a reliable basis for subsequent analysis and applications.

[0114] This embodiment, through merging the preliminary and final scan point clouds, removing outliers, generating high-density models, registering and quantitatively analyzing them, can automatically, quantitatively and with high precision assess the quality of the 3D reconstruction results after the robot's autonomous scanning is completed. This ensures that the output 3D model meets the requirements for clinical or industrial applications in key indicators such as surface coverage, geometric consistency and spatial accuracy, thereby improving the overall system's usability, reliability and practical value.

[0115] In one embodiment, a robot scanning path adjustment device is provided, which corresponds one-to-one with the robot scanning path adjustment method in the above embodiments. (Refer to...) Figure 3 , Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the robot scanning path adjustment device of the present invention. The modules include a path execution module 10, a 3D point cloud generation module 20, an ideal pose calculation module 30, a replanning path generation module 40, and a replanning path execution module 50. Detailed descriptions of each functional module are as follows: The path execution module 10 is used to control the robot with sensors at its end, so that the sensors move along a preliminary path and acquire multiple preliminary two-dimensional images of the target object. The 3D point cloud generation module 20 is used to process the multiple preliminary 2D images to extract feature points of the target object, and combine the spatial information of the sensor when acquiring each preliminary 2D image to generate a preliminary 3D point cloud representing the surface of the target object. Ideal attitude calculation module 30 is used to determine multiple ideal sensor attitudes based on the preliminary three-dimensional point cloud, wherein each ideal sensor attitude corresponds to the normal direction of a local surface on the preliminary three-dimensional point cloud; The replanning path generation module 40 is used to generate a replanned scan path by determining the corresponding new sensor position on the accessible surface for each ideal sensor posture. The replanning path execution module 50 is used to control the robot to move the sensor along the replanned scanning path, adjust the posture of the sensor according to the replanned scanning path, and acquire multiple final two-dimensional images of the target object.

[0116] In one embodiment, the path execution module 10 is specifically used for: The gesture guidance module marks the start and end points on the accessible surface of the target object. The path planning module connects the start and end points to generate preliminary path data, thereby defining the range of motion of the robot with sensors at the end effector. At the starting position of the initial path data, the robot control system calculates the local normal direction based on the three-dimensional reconstruction model of the accessible surface, and adjusts the robot joint angle to make the emission direction of the sensor consistent with the local normal direction, so as to initialize the posture of the robot with the sensor at the end. The robot drive module controls the robot with sensors at the end effector to move along the initial path data at a constant translational speed, while the pose acquisition module synchronously records the spatial position and three-axis attitude angle data of the sensors in real time. As the sensor moves along the initial path data, the sensor's imaging module emits a sound beam toward the bone structure of the target object and receives echo signals. The imaging signal processing module then converts the received echo signals into multiple initial two-dimensional images of the target object, generating a two-dimensional image sequence. The data fusion module binds each preliminary two-dimensional image with the spatial position and three-axis attitude angle data of the sensor at the corresponding acquisition time, and writes it into the non-volatile memory array through the storage controller to form the acquisition result with spatial position and three-axis attitude angle labels. The image quality analysis module calculates the signal-to-noise ratio (SNR) of the bone structure edge pixels in each preliminary 2D image. If the SNR is lower than a preset threshold, the preliminary 2D image is marked as invalid data. The path replanning module generates a local supplementary path based on the location of the invalid data and triggers the robot to re-acquire the preliminary 2D image at the corresponding location.

[0117] In one embodiment, the 3D point cloud generation module 20 is specifically used for: Multiple preliminary two-dimensional images and the spatial position and three-axis attitude angle data of the sensor corresponding to each preliminary two-dimensional image are read as input data; The deep learning segmentation model is applied to process the multiple preliminary two-dimensional images respectively, and the bone structure edge pixels of the target object in each preliminary two-dimensional image are extracted as feature points; The bone structure edge pixels extracted from each preliminary two-dimensional image are converted into a two-dimensional coordinate set in the image coordinate system to form a feature point coordinate set. Based on the spatial position and three-axis attitude angle data of the sensor corresponding to each preliminary two-dimensional image, the set of feature point coordinates is transformed into a unified robot three-dimensional coordinate system through a spatial calibration matrix to generate the transformed three-dimensional coordinate data. All transformed 3D coordinate data are aligned according to the corresponding acquisition timestamps and fused to form a spatial point set; A statistical outlier removal algorithm is performed on the set of spatial points to remove isolated points, and a sparse preliminary 3D point cloud representing the surface of the target object is output.

[0118] In one embodiment, the ideal attitude calculation module 30 is specifically used for: The preliminary three-dimensional point cloud is projected onto the robot's working plane along the long axis of the target object to obtain the two-dimensional distribution of the preliminary three-dimensional point cloud; Extract the upper envelope point sequence from the two-dimensional distribution of the preliminary three-dimensional point cloud; The upper envelope point sequence is processed using a moving average algorithm, and a continuous bone surface contour curve is generated by fitting a cubic spline function. Calculate the tangent direction at each sampling point of the bone surface contour curve; At each sampling point, the perpendicular direction of the tangent direction in the plane of the bone surface contour curve is calculated based on the tangent direction as the normal direction of the local surface, and the ideal sensor posture is defined at the sampling point position using the normal direction, so that the ideal sensor posture includes the sampling point position and the ideal three-axis posture angle. Record and store the ideal sensor poses corresponding to all sampling points on the preliminary 3D point cloud.

[0119] In one embodiment, the replanning path generation module 40 is specifically used for: Starting from the sampling point position corresponding to each ideal sensor pose, construct a ray along the normal direction corresponding to the ideal sensor pose; Calculate the first intersection point between the ray and the accessible surface, and determine this intersection point as the new sensor position corresponding to the ideal sensor orientation; For each new sensor position, calculate the tilt angle of the ideal sensor orientation. When the tilt angle exceeds the preset safety threshold, adjust the normal direction and recalculate the new sensor position. Each new sensor position is bound to the ideal sensor attitude corresponding to that position to form a path node; Connect all path nodes in the order of the ideal sensor orientation to generate a preliminary path point sequence; The initial path point sequence is divided into equidistant segments, and the path nodes with the smallest and largest tilt angles within each segment are selected as key points. Connect all key points to generate a smooth and continuous replanning scan path.

[0120] In one embodiment, the replanning path execution module 50 is specifically used for: The robot is controlled to move the sensor to a new sensor position corresponding to the starting path node of the replanned scanning path, and the sensor's orientation is adjusted to the ideal sensor orientation corresponding to the starting path node. The robot is controlled to move the sensor along the replanned scanning path at a constant translational speed. Between adjacent path nodes, the three-axis attitude angles of the sensor are continuously interpolated in real time to ensure that the attitude of the sensor transitions smoothly during the movement. During the movement of the sensor along the replanned scanning path, the force feedback control module monitors the contact force between the sensor and the accessible surface in real time, and dynamically adjusts the position of the sensor based on the real-time contact force to maintain a preset contact force value. During the movement, the imaging unit is triggered at a preset anatomical position to acquire the final two-dimensional image of the target object; Each final two-dimensional image is bound to the sensor's position and three-axis attitude angle at the time of acquisition and written to a storage medium.

[0121] In one embodiment, the replanning path execution module 50 is specifically used for: Merge the sparse preliminary 3D point cloud representing the surface of the target object with the 3D point cloud data generated based on the multiple final 2D images; A statistical outlier removal algorithm is performed on the merged 3D point cloud data to remove isolated points and generate a high-density 3D point cloud model representing the surface of the target object. The iterative nearest point algorithm is applied to register the high-density 3D point cloud model with the reference model. Based on the registration results, the surface coverage, reconstruction accuracy, and comprehensive F-value of the high-density 3D point cloud model are calculated to generate a 3D reconstruction quality assessment report to characterize the reconstruction effect.

[0122] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external user terminals via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a robot scanning path adjustment method on the server side.

[0123] In one embodiment, a computer device is provided, which may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a robot scanning path adjustment method on the user side. In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: The robot, equipped with sensors at its end effector, is controlled to move along a preliminary path and acquire multiple preliminary two-dimensional images of the target object. The multiple preliminary two-dimensional images are processed to extract feature points of the target object, and combined with the spatial information of the sensor when each preliminary two-dimensional image is acquired, a preliminary three-dimensional point cloud representing the surface of the target object is generated. Based on the preliminary three-dimensional point cloud, multiple ideal sensor poses are determined, wherein each ideal sensor pose corresponds to the normal direction of a local surface on the preliminary three-dimensional point cloud; A replanned scan path is generated by determining the corresponding new sensor position on the accessible surface for each of the ideal sensor orientations. The robot is controlled to move the sensor along the replanned scanning path, and the orientation of the sensor is adjusted according to the replanned scanning path to acquire multiple final two-dimensional images of the target object.

[0124] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: The robot, equipped with sensors at its end effector, is controlled to move along a preliminary path and acquire multiple preliminary two-dimensional images of the target object. The multiple preliminary two-dimensional images are processed to extract feature points of the target object, and combined with the spatial information of the sensor when each preliminary two-dimensional image is acquired, a preliminary three-dimensional point cloud representing the surface of the target object is generated. Based on the preliminary three-dimensional point cloud, multiple ideal sensor poses are determined, wherein each ideal sensor pose corresponds to the normal direction of a local surface on the preliminary three-dimensional point cloud; A replanned scan path is generated by determining the corresponding new sensor position on the accessible surface for each of the ideal sensor orientations. The robot is controlled to move the sensor along the replanned scanning path, and the orientation of the sensor is adjusted according to the replanned scanning path to acquire multiple final two-dimensional images of the target object.

[0125] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and user side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0128] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for adjusting a robot scanning path, characterized in that, Includes the following steps: The robot, equipped with sensors at its end effector, is controlled to move along a preliminary path and acquire multiple preliminary two-dimensional images of the target object. The multiple preliminary two-dimensional images are processed to extract feature points of the target object, and combined with the spatial information of the sensor when each preliminary two-dimensional image is acquired, a preliminary three-dimensional point cloud representing the surface of the target object is generated. Based on the preliminary three-dimensional point cloud, multiple ideal sensor poses are determined, wherein each ideal sensor pose corresponds to the normal direction of a local surface on the preliminary three-dimensional point cloud; A replanned scan path is generated by determining the corresponding new sensor position on the accessible surface for each of the ideal sensor orientations. The robot is controlled to move the sensor along the replanned scanning path, and the orientation of the sensor is adjusted according to the replanned scanning path to acquire multiple final two-dimensional images of the target object.

2. The robot scanning path adjustment method as described in claim 1, characterized in that, Controlling a robot with sensors at its end effector to move the sensors along a preliminary path and acquire multiple preliminary two-dimensional images of the target object, including: The gesture guidance module marks the start and end points on the accessible surface of the target object. The path planning module connects the start and end points to generate preliminary path data, thereby defining the range of motion of the robot with sensors at the end effector. At the starting position of the initial path data, the robot control system calculates the local normal direction based on the three-dimensional reconstruction model of the accessible surface, and adjusts the robot joint angle to make the emission direction of the sensor consistent with the local normal direction, so as to initialize the posture of the robot with the sensor at the end. The robot drive module controls the robot with sensors at the end effector to move along the initial path data at a constant translational speed, while the pose acquisition module synchronously records the spatial position and three-axis attitude angle data of the sensors in real time. As the sensor moves along the initial path data, the sensor's imaging module emits a sound beam toward the bone structure of the target object and receives echo signals. The imaging signal processing module then converts the received echo signals into multiple initial two-dimensional images of the target object, generating a two-dimensional image sequence. The data fusion module binds each preliminary two-dimensional image with the spatial position and three-axis attitude angle data of the sensor at the corresponding acquisition time, and writes it into the non-volatile memory array through the storage controller to form the acquisition result with spatial position and three-axis attitude angle labels. The image quality analysis module calculates the signal-to-noise ratio (SNR) of the bone structure edge pixels in each preliminary 2D image. If the SNR is lower than a preset threshold, the preliminary 2D image is marked as invalid data. The path replanning module generates a local supplementary path based on the location of the invalid data and triggers the robot to re-acquire the preliminary 2D image at the corresponding location.

3. The robot scanning path adjustment method as described in claim 1, characterized in that, The process involves processing the multiple preliminary two-dimensional images to extract feature points of the target object, and combining this with the spatial information from the sensor acquired during the acquisition of each preliminary two-dimensional image to generate a preliminary three-dimensional point cloud representing the surface of the target object, including: Multiple preliminary two-dimensional images and the spatial position and three-axis attitude angle data of the sensor corresponding to each preliminary two-dimensional image are read as input data; The deep learning segmentation model is applied to process the multiple preliminary two-dimensional images respectively, and the bone structure edge pixels of the target object in each preliminary two-dimensional image are extracted as feature points; The bone structure edge pixels extracted from each preliminary two-dimensional image are converted into a two-dimensional coordinate set in the image coordinate system to form a feature point coordinate set. Based on the spatial position and three-axis attitude angle data of the sensor corresponding to each preliminary two-dimensional image, the set of feature point coordinates is transformed into a unified robot three-dimensional coordinate system through a spatial calibration matrix to generate the transformed three-dimensional coordinate data. All transformed 3D coordinate data are aligned according to the corresponding acquisition timestamps and fused to form a spatial point set; A statistical outlier removal algorithm is performed on the set of spatial points to remove isolated points, and a sparse preliminary 3D point cloud representing the surface of the target object is output.

4. The robot scanning path adjustment method as described in claim 1, characterized in that, Based on the preliminary 3D point cloud, multiple ideal sensor poses are determined, wherein each ideal sensor pose corresponds to the normal direction of a local surface on the preliminary 3D point cloud, including: The preliminary three-dimensional point cloud is projected onto the robot's working plane along the long axis of the target object to obtain the two-dimensional distribution of the preliminary three-dimensional point cloud; Extract the upper envelope point sequence from the two-dimensional distribution of the preliminary three-dimensional point cloud; The upper envelope point sequence is processed using a moving average algorithm, and a continuous bone surface contour curve is generated by fitting a cubic spline function. Calculate the tangent direction at each sampling point of the bone surface contour curve; At each sampling point, the perpendicular direction of the tangent direction in the plane of the bone surface contour curve is calculated based on the tangent direction as the normal direction of the local surface, and the ideal sensor posture is defined at the sampling point position using the normal direction, so that the ideal sensor posture includes the sampling point position and the ideal three-axis posture angle. Record and store the ideal sensor poses corresponding to all sampling points on the preliminary 3D point cloud.

5. The robot scanning path adjustment method as described in claim 1, characterized in that, By determining the corresponding new sensor position on the accessible surface for each of the ideal sensor orientations, a replanned scan path is generated, including: Starting from the sampling point position corresponding to each ideal sensor pose, construct a ray along the normal direction corresponding to the ideal sensor pose; Calculate the first intersection point between the ray and the accessible surface, and determine this intersection point as the new sensor position corresponding to the ideal sensor orientation; For each new sensor position, calculate the tilt angle of the ideal sensor orientation. When the tilt angle exceeds the preset safety threshold, adjust the normal direction and recalculate the new sensor position. Each new sensor position is bound to the ideal sensor attitude corresponding to that position to form a path node; Connect all path nodes in the order of the ideal sensor orientation to generate a preliminary path point sequence; The initial path point sequence is divided into equidistant segments, and the path nodes with the smallest and largest tilt angles within each segment are selected as key points. Connect all key points to generate a smooth and continuous replanning scan path.

6. The robot scanning path adjustment method as described in claim 1, characterized in that, The robot is controlled to move the sensor along the replanned scanning path, and the sensor's orientation is adjusted according to the replanned scanning path. Multiple final two-dimensional images of the target object are acquired, including: The robot is controlled to move the sensor to a new sensor position corresponding to the starting path node of the replanned scanning path, and the sensor's orientation is adjusted to the ideal sensor orientation corresponding to the starting path node. The robot is controlled to move the sensor along the replanned scanning path at a constant translational speed. Between adjacent path nodes, the three-axis attitude angles of the sensor are continuously interpolated in real time to ensure that the attitude of the sensor transitions smoothly during the movement. During the movement of the sensor along the replanned scanning path, the force feedback control module monitors the contact force between the sensor and the accessible surface in real time, and dynamically adjusts the position of the sensor based on the real-time contact force to maintain a preset contact force value. During the movement, the imaging unit is triggered at a preset anatomical position to acquire the final two-dimensional image of the target object; Each final two-dimensional image is bound to the sensor's position and three-axis attitude angle at the time of acquisition and written to a storage medium.

7. The robot scanning path adjustment method as described in claim 1, characterized in that, After controlling the robot to move the sensor along the replanned scanning path, adjusting the sensor's orientation according to the replanned scanning path, and acquiring multiple final two-dimensional images of the target object, the method further includes: Merge the sparse preliminary 3D point cloud representing the surface of the target object with the 3D point cloud data generated based on the multiple final 2D images; A statistical outlier removal algorithm is performed on the merged 3D point cloud data to remove isolated points and generate a high-density 3D point cloud model representing the surface of the target object. The iterative nearest point algorithm is applied to register the high-density 3D point cloud model with the reference model. Based on the registration results, the surface coverage, reconstruction accuracy, and comprehensive F-value of the high-density 3D point cloud model are calculated to generate a 3D reconstruction quality assessment report to characterize the reconstruction effect.

8. A robot scanning path adjustment device, characterized in that, The robot scanning path adjustment device includes: The path execution module is used to control the robot with sensors at its end effector, so that the sensors move along a preliminary path and acquire multiple preliminary two-dimensional images of the target object. The three-dimensional point cloud generation module is used to process the multiple preliminary two-dimensional images to extract feature points of the target object, and combine the spatial information of the sensor when acquiring each preliminary two-dimensional image to generate a preliminary three-dimensional point cloud representing the surface of the target object. An ideal attitude calculation module is used to determine multiple ideal sensor attitudes based on the preliminary three-dimensional point cloud, wherein each ideal sensor attitude corresponds to the normal direction of a local surface on the preliminary three-dimensional point cloud; The replanning path generation module is used to generate a replanned scan path by determining the corresponding new sensor position on the accessible surface for each ideal sensor orientation. The replanning path execution module is used to control the robot to move the sensor along the replanned scanning path, adjust the posture of the sensor according to the replanned scanning path, and acquire multiple final two-dimensional images of the target object.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a robot scanning path adjustment program stored in the memory and executable on the processor, wherein when executed by the processor, the robot scanning path adjustment program implements the steps of the robot scanning path adjustment method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a robot scanning path adjustment program, which, when executed by a processor, implements the steps of the robot scanning path adjustment method as described in any one of claims 1-7.

Images