Robot real-time normal perpendicular correction method and system based on point cloud technology

Abstract

The present application relates to the technical field of medical precise scanning technology, and discloses a robot real-time normal perpendicular correction method and system based on point cloud technology.The correction method is applied to a scanning robot and specifically comprises the following steps:S101: receiving a mimic learning trajectory planning request sent by a user terminal, wherein the mimic learning trajectory planning request carries expert demonstration learning mapping data, collected and optimized training data set and model training application data;S102: learning a mapping relationship from image observation to action based on the expert demonstration learning mapping data, and training and optimizing a neural network by using the collected and optimized training data set.The present application solves the error existing in the end fitting of the current intelligent trajectory planning algorithm, dynamically adjusts the attitude information by updating the target point cloud in real time, thereby effectively correcting the robot end attitude, and greatly improves the accuracy and stability of the scanning robot in the scanning task.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of medical precise scanning technology, in particular to a robot real-time normal vertical correction method and system based on point cloud technology. BACKGROUND

[0002] Modern scanning robots have been widely applied to medical ultrasound, industrial polishing and other fields. Although the scanning trajectory planning method can generate relatively smooth trajectories to ensure that the end position accurately reaches the target point, in a complex dynamic environment, the attitude of the end is difficult to realize accurate verticality with the surface of the object. This limitation makes it difficult for the originally highly intelligent algorithm to meet the needs of actual industrial applications.

[0003] In addition, the method based purely on three-dimensional point cloud reconstruction often needs to be re-modeled and trajectory planned when the physical position of the object changes, thereby seriously affecting the efficiency. SUMMARY

[0004] The application aims to provide a robot real-time normal vertical correction method and system based on point cloud technology, which dynamically adjusts the attitude information by updating the target point cloud in real time, thereby effectively correcting the attitude of the robot end, greatly improving the accuracy and stability of the scanning robot in the scanning task, and solving the problems in the prior art.

[0005] The application is implemented in the following manner. The robot real-time normal vertical correction method based on point cloud technology is applied to a scanning robot and comprises the following steps:

[0006] S101: receiving a mimic learning trajectory planning request sent by a user terminal, wherein the mimic learning trajectory planning request carries expert demonstration learning mapping data, training data set collection and optimization, and model training application data;

[0007] S102: learning the mapping relationship from image observation to action based on the expert demonstration learning mapping data, training and optimizing the neural network by using the training data set collection and optimization, and generating a high-precision trajectory according to real-time image observation;

[0008] S103: establishing a unified coordinate system, obtaining the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate marker of the point cloud data, mapping the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through a coordinate transformation matrix to obtain the point cloud data, and then completing preprocessing and feature extraction of the point cloud data to obtain three-dimensional space information and local normal characteristics of the point cloud data;

[0009] S104: Put the generated high-precision trajectory in the point cloud data, determine the contact point of the scanning point cloud coordinates according to the trajectory and the light center corresponding point cloud normal direction, and determine the normal perpendicular intersection point in the three-dimensional space information and the local normal characteristics in the point cloud coordinates;

[0010] S105: According to the point cloud normal vector of the perpendicular intersection point, the consistency adjustment is carried out, the target normal after the consistency adjustment is the final attitude direction required by the end execution adjustment, and then the final attitude direction is expanded Kalman filter attitude correction to complete the state prediction and error correction. Through continuous acquisition of sensor data, the end attitude of the scanning robot and the trajectory planning are adjusted in real time to ensure the task execution performance in complex dynamic environment.

[0011] Further, in S101, the imitation learning trajectory planning request sent by the user terminal is received, including:

[0012] The connection request sent by the user terminal is received through the preset transmission network, and the connection request is used to request to establish connection with the scanning robot;

[0013] Detect whether the current account of the user terminal is the target account;

[0014] If the current account of the user terminal is the target account, the connection with the user terminal is carried out according to the connection request, and after the connection is completed, the imitation learning trajectory planning request sent by the user terminal is received.

[0015] Further, collecting and optimizing the training data set and model training application data includes:

[0016] The collection and optimization of the training data set is to improve the accuracy and robustness of the model, collect the exclusive training data set, and train and optimize the neural network using the exclusive training data set. In the exclusive training data set, the scanning task of the scanning robot freedom degree is defined to ensure that the model can handle complex motion patterns and has strong generalization ability;

[0017] The model training application data is to directly call the trained trajectory planning model to complete the scanning task. The trajectory planning model can generate high-precision trajectory according to real-time image observation, which significantly improves the task efficiency and execution precision.

[0018] Further, in S102, based on constructing expert demonstration learning mapping data, the acquisition method of the expert demonstration learning mapping data includes:

[0019] Behavior cloning is used to construct a mapping from state to action to realize imitation;

[0020] Generative adversarial imitation learning, for combining generative adversarial network and imitation learning, using the adversarial training mechanism of GAN to optimize the policy network in imitation learning, which can learn similar behavior to the expert policy;

[0021] Inverse reinforcement learning, for generating a reward function inferred from expert demonstrations, and training reinforcement learning using the reward function;

[0022] Visual attention imitation learning, for improving individual visual attention through imitation.

[0023] Further, in S103, a unified coordinate system is established, the coordinate system of the scanning robot, the coordinate system of the RGB-D camera of the scanning robot, and the coordinate marker of the point cloud data are obtained, including:

[0024] The coordinate system of the scanning robot itself, the coordinate system of the RGB-D camera, and the coordinate system of the point cloud data are unified in one coordinate system, ensuring effective fusion with multi-source sensor data;

[0025] The scanning robot coordinate system is used as the base coordinate system to describe the trajectory point, the optical center position and the end effector pose, and the current time trajectory point coordinate is defined as .

[0026] Through the coordinate transformation matrix, the point cloud data collected by the RGB-D camera is mapped into the scanning robot coordinate system, realizing the unified expression of different information. In the scanning robot coordinate system, the optical center position coordinate is , and the point cloud coordinate set is .

[0027] Further, after the point cloud data is preprocessed and feature extracted, the three-dimensional spatial information and local normal characteristics of the point cloud data are obtained, including:

[0028] After obtaining the point cloud data from the RGB-D camera, the following processing is performed:

[0029] The point cloud data is down-sampled using voxel filtering to reduce the data volume, and statistical filtering is used to remove isolated points and noise points of the point cloud data;

[0030] For each point in the point cloud, its local domain is defined, and the covariance matrix formula of the points in the local domain is calculated as:

[0031] ;

[0032] Wherein, is the domain set of point , and is the centroid of the field, the eigenvectors of the covariance matrix corresponding to the smallest eigenvalues are taken as the normal vectors of the points The point cloud data contains three-dimensional space information and supplements the local normal characteristics.

[0033] Further, in S104, the point cloud coordinates of the contact point scanned by the trajectory and the optical center coordinates are determined, and the normal vertical intersection point in the three-dimensional space information and the local normal characteristics is determined, including:

[0034] According to the line equation: , the normal direction of the point cloud corresponding to the trajectory and the optical center is determined;

[0035] According to the square of the Euclidean distance function , by taking the derivative of D(t) with respect to t and setting the derivative equal to 0, the minimum value corresponding to the parameter is obtained;

[0036] Substituting the above equation into the equation , the point cloud coordinates of the contact point to be scanned are obtained by the current trajectory and the optical center coordinates.

[0037] Further, in S105, the point cloud normal vector of the vertical intersection point is adjusted for consistency, and the target normal after consistency adjustment is the final pose direction adjusted by the end execution, including:

[0038] Adjust the point cloud normal direction to ensure that the point cloud normal points to the correct direction and is consistent with the current end pose normal , and the point cloud normal adjustment formula is:

[0039] ;

[0040] The target normal after consistency adjustment is the final pose direction adjusted by the end execution.

[0041] Further, the final pose direction is further extended Kalman filter pose correction to complete state prediction and error correction, by continuously collecting sensor data, real-time adjustment of the scanning robot end pose and trajectory planning, including:

[0042] Use the extended Kalman filter to combine the data collected by the sensor with the predefined trajectory to perform state prediction and error correction, ensuring the stability and adaptability of task execution, even in a dynamic and complex environment, the specified task can be accurately completed;

[0043] Through the RGB-D camera or other sensors, the target surface point cloud data is continuously acquired, the current pose normal and the target normal are compared, the error vector is calculated in real time, and the end pose is dynamically adjusted by the extended Kalman filter according to the error size and direction.

[0044] The newly collected point cloud data is aligned with the scanning robot coordinate system after denoising and downsampling processing, used to generate the latest target point cloud normal, and ensure that the direction of the current normal adjustment is consistent with the task requirements, the updated target normal is fused with the current trajectory, the intersection coordinates are recalculated through the trajectory equation, and the dynamic correction of the posture is realized;

[0045] The point cloud data preprocessing, error calculation and trajectory updating are processed in parallel, and then the data acquisition frequency and the smoothing factor of the sensor are dynamically adjusted according to the frequency of environmental changes, and the feedback control is optimized.

[0046] Compared with the prior art, the robot real-time normal vertical correction method and system based on point cloud technology provided by the application have the following beneficial effects:

[0047] 1. The error existing in the current intelligent trajectory planning algorithm in the end fitting aspect is solved, the posture information is dynamically adjusted by updating the target point cloud in real time, so that the robot end posture is effectively corrected, the accuracy and stability of the scanning robot in the scanning task are greatly improved, and the adaptability of the scanning robot in the dynamic complex environment is significantly enhanced;

[0048] 2. The relative position and direction change of the robot and the target environment are perceived and calculated in real time, the spatial posture of the scanning robot end effector is adjusted, so that it always maintains perpendicular to the target surface normal, and the specific design and implementation of each link mainly includes: imitation learning trajectory planning, unified coordinate system, point cloud preprocessing, perpendicular point confirmation, calculation of target normal vector, end posture correction, real-time update feedback mechanism, realization of whole process moving along the trajectory, comprehensive utilization of trajectory planning and point cloud characteristics, this method provides high precision and real-time posture adjustment capability for the scanning robot, not only solves the perpendicularity problem of the end posture, but also optimizes the task completion efficiency, greatly expands the application prospect of the robot in the medical and industrial fields.

[0049] The robot real-time normal vertical correction system based on point cloud technology is executed in the robot real-time normal vertical correction method described above, and the correction system comprises:

[0050] The acquisition module is used for receiving the imitation learning trajectory planning request sent by the user terminal;

[0051] The trajectory generation module is used for training and optimizing the neural network by using the collected and optimized training data set, and generating a high-precision trajectory according to real-time image observation;

[0052] The coordinate execution module is used for establishing a unified coordinate system, acquiring the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot and the coordinate marker of the point cloud data;

[0053] The point cloud data acquisition module is configured to acquire point cloud data by mapping the point cloud data collected by the RGB-D camera into the scanning robot coordinate system through a coordinate transformation matrix, and then pre-process and extract features of the point cloud data to acquire three-dimensional spatial information and local normal characteristics of the point cloud data.

[0054] The adjusting module is configured to determine the point cloud coordinates of the contact point scanned by the trajectory and the optical center coordinates according to the normal direction of the point cloud corresponding to the trajectory and the optical center, and determine the normal perpendicular intersection point in the three-dimensional spatial information and the local normal characteristics in the point cloud coordinates.

[0055] The end posture planning module is configured to perform consistency adjustment according to the point cloud normal vector of the perpendicular intersection point, and the target normal after the consistency adjustment is the final posture direction required by the end execution, and the scanning robot end posture and trajectory planning are adjusted in real time by continuously collecting sensor data to ensure the task execution performance in a complex dynamic environment. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flowchart of the robot real-time normal perpendicular correction method based on the point cloud technology is provided.

[0057] Figure 2 A flowchart of the robot real-time normal perpendicular correction method based on the point cloud technology is provided.

[0058] Figure 3 A flowchart of the robot real-time normal perpendicular correction method based on the point cloud technology is provided.

[0059] Figure 4 A structure diagram of the robot real-time normal perpendicular correction system based on the point cloud technology is provided. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0061] The implementation of the present application will be described in detail below in combination with specific examples.

[0062] The same or similar reference numerals in the drawings of the embodiments correspond to the same or similar components; in the description of the present application, it should be understood that if the orientations or positional relationships indicated by the terms "upper", "lower", "left", "right" and the like are based on the orientations or positional relationships shown in the drawings, they are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore the terms describing the positional relationship in the drawings are only used for exemplary illustration, and cannot be understood as a limitation on the present application. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to the specific circumstances.

[0063] Referring to Figures 1-3 The robot real-time normal perpendicular correction method based on point cloud technology is applied to a scanning robot, and specifically includes the following steps:

[0064] S101: receiving a mimic learning trajectory planning request sent by a user terminal, the mimic learning trajectory planning request carrying expert demonstration learning mapping data, collected and optimized training data set and model training application data;

[0065] The receiving of the mimic learning trajectory planning request sent by the user terminal includes:

[0066] The connection request is used to request to establish a connection with the scanning robot;

[0067] Detecting whether the current account of the user terminal is a target account;

[0068] If the current account of the user terminal is the target account, connect with the user terminal according to the connection request, and after the connection is completed, receive the mimic learning trajectory planning request sent by the user terminal;

[0069] S102: learning the mapping relationship from image observation to action based on the constructed expert demonstration learning mapping data, training and optimizing the neural network using the collected and optimized training data set, and generating a high-precision trajectory according to real-time image observation;

[0070] The expert demonstration learning mapping data is acquired in the following manner:

[0071] Behavior cloning is used to construct a mapping from state to action to realize imitation;

[0072] Generative adversarial imitation learning is used to combine generative adversarial networks and imitation learning, and use the adversarial training mechanism of GAN to optimize the policy network in imitation learning, which can learn behaviors similar to expert policies;

[0073] Inverse reinforcement learning for generating a reward function inferred from expert demonstrations and training reinforcement learning with the reward function;

[0074] Visual attention imitation learning for improving individual visual attention through imitation;

[0075] S103: Establish a unified coordinate system, obtain the coordinate system of the scanning robot, the coordinate system of the RGB-D camera of the scanning robot, and the coordinate marker of the point cloud data, map the point cloud data collected by the RGB-D camera to the coordinate system of the scanning robot through the coordinate transformation matrix to obtain the point cloud data, and then complete the preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data;

[0076] The establishment of the unified coordinate system, the obtaining of the coordinate system of the scanning robot, the coordinate system of the RGB-D camera of the scanning robot, and the coordinate marker of the point cloud data include:

[0077] The coordinate system of the scanning robot itself, the coordinate system of the RGB-D camera, and the coordinate system of the point cloud data are unified in one coordinate system, ensuring effective fusion with multi-source sensor data;

[0078] The coordinate system of the scanning robot is used as the base coordinate system to describe the trajectory point, the optical center position, and the end effector attitude, and the coordinate of the trajectory point at the current time is defined as .

[0079] The point cloud data collected by the RGB-D camera is mapped to the coordinate system of the scanning robot through the coordinate transformation matrix, realizing the unified expression of different information. In the coordinate system of the scanning robot, the coordinate of the optical center position is , and the point cloud coordinate set is .

[0080] S104: Place the generated high-precision trajectory in the point cloud data, determine the point cloud coordinates of the contact points scanned by the trajectory and the optical center according to the normal direction of the corresponding point cloud of the trajectory and the optical center, and determine the normal perpendicular intersection point in the three-dimensional spatial information and the local normal characteristics in the point cloud coordinates;

[0081] The determination of the point cloud coordinates of the contact points scanned by the trajectory and the optical center, and the determination of the normal perpendicular intersection point in the three-dimensional spatial information and the local normal characteristics in the point cloud coordinates include:

[0082] According to the line equation: , the normal direction of the corresponding point cloud of the trajectory and the optical center is determined;

[0083] According to the square of the Euclidean distance function , the derivative of D(t) with respect to t is taken, and the derivative is set to 0 to obtain the minimum value corresponding to the parameter ;

[0084] Bring in to get the normal vertical intersection point coordinates , the point cloud coordinates of the contact point to be scanned are obtained from the current trajectory and the optical center coordinates;

[0085] S105: consistency adjustment according to the point cloud normal vector of the vertical intersection point, the target normal after consistency adjustment is the final attitude direction that the end execution needs to adjust to, and then the final attitude direction is subjected to extended Kalman filter attitude correction to complete state prediction and error correction, through continuous acquisition of sensor data, real-time adjustment of scanning robot end attitude and trajectory planning is realized, and task execution performance is ensured in complex dynamic environment;

[0086] Among them, according to the point cloud normal vector of the vertical intersection point, the target normal after consistency adjustment is the final attitude direction that the end execution needs to adjust to, including:

[0087] Adjust the point cloud normal direction to ensure that the point cloud normal points to the correct direction, and is consistent with the current end attitude normal The point cloud normal adjustment formula is:

[0088] ;

[0089] The target normal after consistency adjustment is the final attitude direction that the end execution adjusts to.

[0090] In this embodiment, collecting and optimizing the training data set and model training application data includes:

[0091] Collecting and optimizing the training data set is to improve the accuracy and robustness of the model, collecting the exclusive training data set, and training and optimizing the neural network using the exclusive training data set, defining the scanning task of the scanning robot freedom degree in the exclusive training data set, ensuring that the model can handle complex motion patterns and has strong generalization ability;

[0092] The model training application data is to directly call the trained trajectory planning model to complete the scanning task. The trajectory planning model can generate high-precision trajectory according to real-time image observation, significantly improving task efficiency and execution precision. The current intelligent trajectory planning algorithm solves the error in end fitting, dynamically adjusts the attitude information by updating the target point cloud in real time, effectively corrects the robot end attitude, greatly improves the accuracy and stability of the scanning robot in the scanning task, and significantly enhances its adaptability in dynamic and complex environment.

[0093] In this embodiment, after the point cloud data is preprocessed and feature extracted, the three-dimensional space information and local normal characteristics of the point cloud data are obtained, including:

[0094] After obtaining the point cloud data from the RGB-D camera, the following processing is performed:

[0095] Downsampling of point cloud data is performed using voxel filtering to reduce data volume, and statistical filtering is used to remove isolated points and noise points in the point cloud data.

[0096] For each point in the point cloud Define its local domain The formula for calculating the covariance matrix of points in the local domain is:

[0097] ;

[0098] Where, is the domain set of point , is the centroid of the domain, and the eigenvectors corresponding to the smallest eigenvalues of the covariance matrix are taken as the normal vectors of the point Point cloud data contains three-dimensional spatial information and supplements local normal characteristics.

[0099] In this embodiment, the final pose direction is further extended Kalman filter pose correction to complete state prediction and error correction. By continuously collecting sensor data, the end pose of the scanning robot and the trajectory planning are adjusted in real time, including:

[0100] The data collected by the sensor is combined with the predefined trajectory using extended Kalman filter to perform state prediction and error correction, ensuring the stability and adaptability of task execution, even in dynamic and complex environments.

[0101] Through the RGB-D camera or other sensors, the target surface point cloud data is continuously acquired, and the error vector is calculated in real time by comparing the current pose normal with the target normal. According to the error size and direction, the end pose is dynamically adjusted through extended Kalman filter.

[0102] The newly acquired point cloud data is processed by denoising and downsampling, and is aligned with the scanning robot coordinate system to generate the latest target point cloud normal, and ensure that the direction after adjusting the current normal is consistent with the task requirements. The updated target normal is combined with the current trajectory, and the intersection coordinates are recalculated through the trajectory equation to realize dynamic correction of the pose.

[0103] Parallel processing of point cloud data preprocessing, error calculation and trajectory update, and then dynamically adjusting the data acquisition frequency and smoothing factor of the sensor according to the frequency of environmental changes, optimizing feedback control, referring to Figure 3 ​It can be known that for the end posture realization and the precise perpendicularity to the object surface in the complex dynamic environment, the technical scheme of realizing the image-to-action based imitative learning trajectory planning and then the point cloud real-time normal correction is realized without re-modeling and trajectory planning when the physical position of the object changes.

[0104] The technical scheme adjusts the spatial posture of the scanning robot end effector to keep it always perpendicular to the target surface normal by real-time sensing and computing the relative position and direction change of the robot and the target environment, and the specific design and implementation of each link mainly include: imitative learning trajectory planning, unified coordinate system, point cloud preprocessing, perpendicular point confirmation, target normal vector calculation, end posture correction, real-time update feedback mechanism, and realization of the whole trajectory movement. The method comprehensively utilizes the trajectory planning and point cloud characteristics to provide the scanning robot with high-precision and real-time posture adjustment capability, not only solves the perpendicularity problem of the end posture, but also optimizes the task completion efficiency, greatly expands the application prospect of the robot in the medical and industrial fields.

[0105] Reference Figure 4As shown, the robot real-time normal vertical correction system based on point cloud technology executes the robot real-time normal vertical correction method described above, and the correction system includes: an acquisition module for receiving a mimic learning trajectory planning request sent by a user terminal; a trajectory generation module for training and optimizing a neural network using collected and optimized training data sets to generate high-precision trajectories according to real-time image observations; a coordinate execution module for establishing a unified coordinate system, obtaining the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate marker of the point cloud data; a point cloud data acquisition module for mapping the point cloud data collected by the RGB-D camera into the scanning robot coordinate system through a coordinate transformation matrix to obtain the point cloud data, and then performing preprocessing and feature extraction on the point cloud data to obtain three-dimensional spatial information and local normal characteristics of the point cloud data; an adjustment module for determining the point cloud coordinates of the contact point scanned by the trajectory and the optical center coordinates according to the corresponding point cloud normal direction of the trajectory and the optical center, and determining the normal vertical intersection point in the three-dimensional spatial information and the local normal characteristics in the point cloud coordinates; an end pose planning module for adjusting the consistency according to the point cloud normal vector of the vertical intersection point, and the target normal after consistency adjustment is the final pose direction of the end execution that needs to be adjusted, and the end pose and trajectory planning of the scanning robot are adjusted in real time by continuously collecting sensor data, to ensure the task execution performance in a complex dynamic environment, the relative position and direction change of the robot and the target environment are perceived and calculated in real time, the spatial pose of the scanning robot end effector is adjusted to keep it always normal to the target surface, and the specific design and implementation of each link mainly include: mimic learning trajectory planning, unified coordinate system, point cloud preprocessing, vertical point confirmation, target normal vector calculation, end pose correction, real-time update feedback mechanism, and implementation of full-range trajectory movement, which comprehensively utilizes trajectory planning and point cloud characteristics, and the method provides high-precision and real-time pose adjustment capability for the scanning robot, not only solves the problem of end pose normality, but also optimizes the task completion efficiency, greatly expanding the application prospect of the robot in the medical and industrial fields.

[0106] The technical solution dynamically adjusts the pose information by updating the target point cloud in real time, thereby effectively correcting the end pose of the robot, greatly improving the accuracy and stability of the scanning robot in the scanning task, and significantly enhancing its adaptability in dynamic and complex environments.

[0107] In the embodiment, the entire operation process can be controlled by a computer, and in each operation link, signal feedback can be performed through the setting of sensors to realize the sequential performance of the steps. These are all conventional knowledge of automatic control, which will not be described one by one in the embodiment.

[0108] The above merely describes preferred embodiments of the present application, and is not used to limit the present application, any modification, equivalent replacement and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for real-time vertical normal correction of a robot based on point cloud technology, characterized in that, When applied to scanning robots, the specific steps include: S101: Receive a simulation learning trajectory planning request sent by a user terminal, wherein the simulation learning trajectory planning request carries the construction of expert demonstration learning mapping data, collection and optimization of training dataset and model training application data; S102: Based on the construction of expert demonstration learning mapping data, learn the mapping relationship from image observation to action, use the collected and optimized training dataset to train and optimize the neural network, and generate high-precision trajectories based on real-time image observation; S103: Establish a unified coordinate system, obtain the coordinate system of the scanning robot, the coordinate system of the scanning robot's RGB-D camera, and the coordinate markers of the point cloud data. Map the point cloud data collected by the RGB-D camera to the coordinate system of the scanning robot through the coordinate transformation matrix to obtain the point cloud data. Then, perform preprocessing and feature extraction on the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data. S104: The generated high-precision trajectory is placed in the point cloud data. Based on the point cloud normal direction corresponding to the trajectory and the optical center, the coordinates of the trajectory and the optical center are determined to obtain the point cloud coordinates of the scanned contact point. The perpendicular intersection point of the three-dimensional spatial information and the local normal characteristics is determined in the point cloud coordinates. Specifically, determining the trajectory and optical center coordinates yields the point cloud coordinates of the scanned contact point. Within these point cloud coordinates, the perpendicular intersection point between the 3D spatial information and the local normal characteristics is determined, including: According to the equation of the connecting line: Determine the cloud normal direction at the point corresponding to the optical center of the trajectory; According to the square of the Euclidean distance function By taking the derivative of D(t) with respect to t and setting the derivative to zero, we can obtain the parameter corresponding to the minimum value. ; Substituting the values, we obtain the coordinates of the perpendicular intersection point of the required normal vectors. The point cloud coordinates of the contact point to be scanned are obtained from the current trajectory and the coordinates of the optical center. S105: Based on the point cloud normal vector of the vertical intersection point, the consistency adjustment is performed. The target normal after the consistency adjustment is the final attitude direction that the end effector needs to adjust to. Then, the attitude correction is performed by extended Kalman filter to complete the state prediction and error correction. By continuously collecting sensor data, the end effector attitude and trajectory planning of the scanning robot are adjusted in real time to ensure that the task execution performance is maintained in complex dynamic environments. Among them, consistency adjustment is performed based on the point cloud normal vectors of the perpendicular intersection points. The target normal vector after consistency adjustment is the final attitude direction that the end effector needs to adjust to, including: Adjust the point cloud normal direction to ensure that the point cloud normal points in the correct direction and is consistent with the current end-effector attitude normal. Consistent, the formula for adjusting the point cloud normal is: ； The target normal after consistency adjustment is the final attitude direction adjusted by the end effector. Where t is the base coordinate system used to describe the parameters corresponding to the trajectory points. The coordinates of the trajectory point at the current moment. The coordinates of the optical center position; For point cloud coordinates, The base coordinate system is used to describe the parameters corresponding to the minimum value of the trajectory points. Let be the minimum cloud normal of the point corresponding to the trajectory and the optical center. The current end-effector attitude normal. For the target normal direction, This refers to the direction of point cloud.

2. The robot real-time normal vertical correction method based on point cloud technology as described in claim 1, characterized in that, In S101, a request for imitation learning trajectory planning sent by the user terminal is received, including: The system receives a connection request sent by a user terminal through a preset transmission network. The connection request is used to request the establishment of a connection with the scanning robot. Detect whether the current account on the user terminal is the target account; If the current account of the user terminal is the target account, a connection is established with the user terminal according to the connection request. After the connection is established, a simulation learning trajectory planning request is received from the user terminal.

3. The robot real-time normal vertical correction method based on point cloud technology as described in claim 2, characterized in that, Collecting and optimizing training datasets and model training application data includes: The purpose of collecting and optimizing the training dataset is to improve the accuracy and robustness of the model. A dedicated training dataset is collected and used to train and optimize the neural network. The scanning task of the scanning robot's degrees of freedom is defined in the dedicated training dataset to ensure that the model can handle complex motion patterns and has strong generalization ability. The model training application data directly calls the trained trajectory planning model to complete the scanning task. The trajectory planning model can generate high-precision trajectories based on real-time image observations, significantly improving task efficiency and execution accuracy.

4. The robot real-time normal vertical correction method based on point cloud technology as described in claim 3, characterized in that, In S102, based on the construction of expert demonstration learning mapping data, the methods for obtaining the expert demonstration learning mapping data include: Behavioral cloning is used to construct a mapping from state to action to achieve imitation; Generative adversarial imitation learning combines generative adversarial networks (GANs) and imitation learning. It utilizes the adversarial training mechanism of GANs to optimize the policy network in imitation learning, enabling it to learn behaviors similar to expert policies. Inverse reinforcement learning is used to generate reward functions inferred from expert demonstrations and to train reinforcement learning using those reward functions. Visual attention imitation learning is used to improve an individual's visual attention through imitation.

5. The robot real-time normal vertical correction method based on point cloud technology as described in claim 4, characterized in that, In S103, a unified coordinate system is established, and the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate markers of the point cloud data are obtained, including: By unifying the coordinate system of the scanning robot itself, the coordinate system of the RGB-D camera, and the coordinate system of the point cloud data into one coordinate system, effective fusion with multi-source sensor data is ensured. The scanning robot coordinate system is used as the base coordinate system to describe the trajectory points, optical center positions, and end effector attitudes. The coordinates of the trajectory points at the current moment are defined as follows: ; By using a coordinate transformation matrix, the point cloud data acquired by the RGB-D camera is mapped onto the scanning robot's coordinate system, achieving a unified representation of different information. In the scanning robot's coordinate system, the coordinates of the optical center are... The point cloud coordinate set is .

6. The robot real-time normal vertical correction method based on point cloud technology as described in claim 5, characterized in that, Next, preprocessing and feature extraction are performed on the point cloud data to obtain its 3D spatial information and local normal characteristics, including: After acquiring point cloud data from the RGB-D camera, the following processing is performed: Voxel filtering is used to downsample point cloud data to reduce the amount of data, and statistical filtering is used to remove isolated and noisy points from the point cloud data. For each point cloud Define its local domain The formula for calculating the covariance matrix of points within a local neighborhood is: ； in, For point The collection of fields, The centroid of the region is used to perform eigenvalue decomposition on the covariance matrix, and the eigenvector corresponding to the smallest eigenvalue is taken as the normal vector of the point. Point cloud data contains three-dimensional spatial information and supplements local normal properties.

7. The real-time normal vertical correction method for robots based on point cloud technology as described in claim 6, characterized in that, Then, extended Kalman filtering is applied to the final attitude direction to complete state prediction and error correction. By continuously collecting sensor data, the end-effector attitude and trajectory planning of the scanning robot are adjusted in real time, including: Extended Kalman filtering is used to combine sensor-acquired data with predefined trajectories for state prediction and error correction, ensuring the stability and adaptability of task execution and enabling accurate completion of specified tasks even in dynamic and complex environments. By continuously acquiring point cloud data of the target surface through an RGB-D camera or other sensors, comparing the current attitude normal with the target normal, calculating the error vector in real time, and dynamically adjusting the end attitude through an extended Kalman filter based on the magnitude and direction of the error; The newly acquired point cloud data is processed by denoising and downsampling, and then aligned with the coordinate system of the scanning robot to generate the latest target point cloud normal. It also ensures that the direction of the current normal after adjustment is consistent with the task requirements. The updated target normal is fused with the current trajectory, and the intersection coordinates are recalculated through the trajectory equation to achieve dynamic attitude correction. Parallel processing of point cloud data preprocessing, error calculation, and trajectory update is performed. Then, based on the frequency of environmental changes, the data acquisition frequency and smoothing factor of the sensors are dynamically adjusted to optimize feedback control.

8. A robot real-time normal vertical correction system based on point cloud technology, characterized in that, The robot real-time normal vertical correction method according to any one of claims 1-7, wherein the correction system comprises: The acquisition module is used to receive imitation learning trajectory planning requests sent by user terminals; The trajectory generation module is used to train and optimize the neural network using the collected and optimized training dataset, and generate high-precision trajectories based on real-time image observations. The coordinate execution module is used to establish a unified coordinate system and obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate markers of the point cloud data. The point cloud data acquisition module is used to map the point cloud data collected by the RGB-D camera to the coordinate system of the scanning robot through a coordinate transformation matrix to obtain the point cloud data. Then, it performs preprocessing and feature extraction on the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data. The adjustment module is used to determine the point cloud coordinates of the contact point obtained by scanning the trajectory and the optical center based on the point cloud normal direction corresponding to the trajectory and the optical center coordinates, and to determine the perpendicular intersection point of the three-dimensional spatial information and the normal in the local normal characteristics in the point cloud coordinates. The end-effector attitude planning module is used to make consistent adjustments based on the point cloud normal vectors of the vertical intersection points. The target normal after consistency adjustment is the final attitude direction that the end-effector needs to adjust to. By continuously collecting sensor data, the end-effector attitude and trajectory planning of the scanning robot are adjusted in real time to ensure that the task execution performance is maintained in complex dynamic environments.

Images