Robot real-time normal vertical correction method and system based on point cloud technology

Abstract

The invention relates to the technical field of medical precise scanning technologies, and discloses a robot real-time normal vertical correction method and system based on a point cloud technology. The correction method is applied to the scanning robot and specifically comprises the following steps that S101, an imitation learning track planning request sent by a user terminal is received, and the imitation learning track planning request carries construction expert demonstration learning mapping data, a collection and optimization training data set and model training application data; s102, learning a mapping relation from image observation to action based on construction of expert demonstration learning mapping data, and training and optimizing a neural network by utilizing a collected and optimized training data set. According to the invention, the error of the current intelligent trajectory planning algorithm in the aspect of end fitting is solved, and the attitude information is dynamically adjusted by updating the target point cloud in real time, so that the end attitude of the robot is effectively corrected, and the accuracy and stability of the scanning robot in a scanning task are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical precision scanning technology, and in particular to a robot real-time normal vertical correction method and system based on point cloud technology. Background Art

[0002] Modern scanning robots have been widely used in medical ultrasound, industrial polishing and other fields. Although the scanning trajectory planning method can generate a relatively smooth trajectory to ensure that the end position accurately reaches the target point, in a complex dynamic environment, the posture of the end is difficult to achieve precise perpendicularity to the surface of the object. This limitation makes it difficult for the originally highly intelligent algorithm to meet the needs of actual industrial applications.

[0003] In addition, the method based purely on 3D point cloud reconstruction often requires re-modeling and trajectory planning when the physical position of the object changes, which seriously affects efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a real-time normal vertical correction method and system for a robot based on point cloud technology, which dynamically adjusts the posture information by updating the target point cloud in real time, thereby effectively correcting the robot's terminal posture, greatly improving the accuracy and stability of the scanning robot in scanning tasks, and aims to solve the problems in the prior art.

[0005] The present invention is implemented in this way: a robot real-time normal vertical correction method based on point cloud technology is applied to a scanning robot, specifically comprising the following steps:

[0006] S101: receiving an imitation learning trajectory planning request sent by a user terminal, wherein the imitation learning trajectory planning request carries constructing expert demonstration learning mapping data, collecting and optimizing training data sets, and model training application data;

[0007] S102: Based on constructing expert demonstration learning mapping data, learn the mapping relationship from image observation to action, train and optimize the neural network by collecting and optimizing the training data set, and generate high-precision trajectories based on real-time image observations;

[0008] S103: Establishing a unified coordinate system, obtaining the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data, mapping the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through a coordinate transformation matrix to obtain the point cloud data, and then completing preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data;

[0009] S104: placing the generated high-precision trajectory in the point cloud data, determining the point cloud coordinates of the contact point of the scan according to the normal direction of the trajectory and the optical center corresponding to the point cloud, and determining the normal vertical intersection point in the three-dimensional space information and the local normal characteristics in the point cloud coordinates;

[0010] S105: Consistency adjustment is performed based on the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that needs to be adjusted to the end execution. The final posture direction is then subjected to extended Kalman filter posture correction to complete state prediction and error correction. By continuously collecting sensor data, the scanning robot's end posture and trajectory planning are adjusted in real time to ensure that the task execution performance is maintained in a complex dynamic environment.

[0011] Further, in S101, receiving an imitation learning trajectory planning request sent by a user terminal includes:

[0012] Receiving a connection request sent by a user terminal through a preset transmission network, wherein the connection request is used to request to establish a connection with the scanning robot;

[0013] Detecting whether the current account of the user terminal is the target account;

[0014] If the current account of the user terminal is the target account, a connection is made with the user terminal according to the connection request, and after the connection is completed, an imitation learning trajectory planning request sent by the user terminal is received.

[0015] Furthermore, collecting and optimizing training data sets and model training application data includes:

[0016] The purpose of collecting and optimizing the training data set is to improve the accuracy and robustness of the model. The exclusive training data set is collected and used to train and optimize the neural network. The scanning task of scanning the robot's degrees of freedom is defined in the exclusive training data set to ensure that the model can handle complex motion patterns and has strong generalization capabilities.

[0017] The model training application data is to directly call the trained trajectory planning model to complete the scanning task. The trajectory planning model can generate high-precision trajectories based on real-time image observations, significantly improving task efficiency and execution accuracy.

[0018] Further, in S102, based on constructing expert demonstration learning mapping data, the method for acquiring the expert demonstration learning mapping data includes:

[0019] Behavioral cloning, which is used to build a mapping from state to action to achieve imitation;

[0020] Generative adversarial imitation learning, which combines generative adversarial networks and imitation learning, uses the adversarial training mechanism of GAN to optimize the policy network in imitation learning, and can learn behaviors similar to expert strategies;

[0021] Inverse reinforcement learning, for generating reward functions inferred from expert demonstrations and using the reward functions to train reinforcement learning;

[0022] Visual attention imitation learning is used to improve individual visual attention through imitation.

[0023] Furthermore, in S103, a unified coordinate system is established to obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data, including:

[0024] The coordinate system of the scanning robot itself, the coordinate system of the RGB-D camera, and the coordinate system of the point cloud data are combined in one coordinate system to ensure effective fusion with multi-source sensor data;

[0025] The scanning robot coordinate system is used as the base coordinate system to describe the trajectory point, optical center position and end effector posture, and the coordinate of the trajectory point at the current moment is defined as p t .

[0026] Through the coordinate transformation matrix, the point cloud data collected by the RGB-D camera is mapped to the scanning robot coordinate system to achieve a unified expression of different information. In the scanning robot coordinate system, the optical center position coordinate is p optical , the point cloud coordinate set is P cloud .

[0027] Furthermore, the point cloud data is preprocessed and feature extracted to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data, including:

[0028] After acquiring the point cloud data from the RGB-D camera, the following processing is performed:

[0029] Use voxel filtering to downsample point cloud data to reduce the amount of data, and use statistical filtering to remove isolated points and noise points in point cloud data;

[0030] For each P in the point cloud i Define its local area V i ,The formula for calculating the covariance matrix of points in the local area is:

[0031]

[0032] Among them, V i For point P i The domain set, μ iis the centroid of the field, perform eigenvalue decomposition on the covariance matrix, and take the eigenvector of the minimum eigenvalue as the normal vector of the point n i ,The point cloud data contains three-dimensional spatial information, and supplements the local normal characteristics.

[0033] Further, in S104, the trajectory and the optical center coordinates are determined to obtain the point cloud coordinates of the scanned contact point, and the normal vertical intersection point in the three-dimensional space information and the local normal characteristics is determined in the point cloud coordinates, including:

[0034] According to the line equation: L(t)=P t +t·(P t -P optical ), determine the cloud normal direction of the point corresponding to the trajectory and the optical center;

[0035] According to the square of the Euclidean distance function D(t) = || L(t) - P cloud || 2 , by taking the derivative of D(t) with respect to t and making the derivative equal to 0, we can get the parameter corresponding to the minimum value

[0036] Substitute the required normal perpendicular intersection coordinates P target =L(t * )=P t +t * ·(P t -P optical ), the point cloud coordinates of the contact point to be scanned are obtained from the current trajectory and the optical center coordinates.

[0037] Further, in S105, consistency adjustment is performed according to the point cloud normal vector of the vertical intersection, and the target normal after consistency adjustment is the final posture direction to which the end execution needs to be adjusted, including:

[0038] Adjust the point cloud normal direction to ensure that the point cloud normal points in the correct direction and is consistent with the current end posture normal n current The point cloud normal adjustment formula is:

[0039]

[0040] The target normal after consistency adjustment is the final posture direction adjusted by the end execution.

[0041] Furthermore, the final attitude direction is corrected by extended Kalman filtering to complete state prediction and error correction. By continuously collecting sensor data, the end attitude and trajectory planning of the scanning robot are adjusted in real time, including:

[0042] The extended Kalman filter is used to combine the data collected by the sensor with the predefined trajectory to perform state prediction and error correction, ensuring the stability and adaptability of task execution, and accurately completing the specified task even in a dynamic and complex environment;

[0043] Through RGB-D cameras or other sensors, the target surface point cloud data is continuously acquired, the current posture normal is compared with the target normal, the error vector is calculated in real time, and the terminal posture is dynamically adjusted through the extended Kalman filter according to the error size and direction.

[0044] After denoising and downsampling, the newly collected point cloud data is aligned with the scanning robot coordinate system to generate the latest target point cloud normal and ensure that the direction of the current normal after adjustment is consistent with the task requirements. The updated target normal is merged with the current trajectory, and the intersection coordinates are recalculated through the trajectory equation to achieve dynamic posture correction.

[0045] Point cloud data preprocessing, error calculation and trajectory update are processed in parallel, and then the data acquisition frequency and smoothing factor of the sensor are dynamically adjusted according to the frequency of environmental changes to optimize feedback control.

[0046] Compared with the prior art, the robot real-time normal vertical correction method and system based on point cloud technology provided by the present invention has the following beneficial effects:

[0047] 1. It solves the errors existing in the terminal fitting of the current intelligent trajectory planning algorithm, dynamically adjusts the posture information by updating the target point cloud in real time, and effectively corrects the terminal posture of the robot, greatly improving the accuracy and stability of the scanning robot in the scanning task, and significantly enhancing its adaptability in dynamic and complex environments;

[0048] 2. By real-time perception and calculation of the relative position and direction changes between the robot and the target environment, the spatial posture of the end effector of the scanning robot is adjusted to keep it always perpendicular to the normal of the target surface. The specific design and implementation of each link mainly include: imitation learning trajectory planning, unified coordinate system, point cloud preprocessing, vertical point confirmation, calculation of target normal vector, end posture correction, real-time update feedback mechanism, and realization of full trajectory movement. By comprehensively utilizing trajectory planning and point cloud characteristics, this method provides the scanning robot with high-precision and real-time posture adjustment capabilities, which not only solves the problem of verticality of the end posture, but also optimizes the task completion efficiency, greatly expanding the application prospects of robots in the medical and industrial fields.

[0049] A robot real-time normal vertical correction system based on point cloud technology is implemented in the above-mentioned robot real-time normal vertical correction method, and the correction system includes:

[0050] An acquisition module, used to receive an imitation learning trajectory planning request sent by a user terminal;

[0051] The trajectory generation module is used to train and optimize the neural network by collecting and optimizing the training data set, and generate high-precision trajectories based on real-time image observations;

[0052] The coordinate execution module is used to establish a unified coordinate system, obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data;

[0053] The point cloud data acquisition module is used to map the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through the coordinate transformation matrix to obtain the point cloud data, and then complete the preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data;

[0054] An adjustment module is used to determine the point cloud coordinates of the contact point scanned by the trajectory and the optical center coordinates according to the normal direction of the point cloud corresponding to the trajectory and the optical center, and determine the normal vertical intersection point in the three-dimensional space information and the local normal characteristics in the point cloud coordinates;

[0055] The terminal posture planning module is used to make consistency adjustments based on the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that the terminal needs to be adjusted to for execution. By continuously collecting sensor data, the terminal posture and trajectory planning of the scanning robot are adjusted in real time to ensure that the task execution performance is maintained in a complex dynamic environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic flow chart of the method for real-time normal vertical correction of a robot based on point cloud technology proposed by the present invention;

[0057] Figure 2 A schematic flow chart of the process of performing extended Kalman filter posture correction on the final posture direction in the robot real-time normal vertical correction method based on point cloud technology proposed by the present invention to complete state prediction and error correction, and adjusting the terminal posture and trajectory planning of the scanning robot in real time;

[0058] Figure 3 A schematic diagram of the robot's real-time normal vertical correction method based on point cloud technology proposed in the present invention, in which the final posture direction is corrected by extended Kalman filtering to complete state prediction and error correction, and the terminal posture and trajectory planning of the scanning robot are adjusted by continuously collecting sensor data;

[0059] Figure 4 This is a schematic structural diagram of the robot real-time normal vertical correction system based on point cloud technology proposed in the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0061] The implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0062] The same or similar numbers in the drawings of this embodiment correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0063] Reference Figure 1-3 As shown, the robot real-time normal vertical correction method based on point cloud technology is applied to the scanning robot, which specifically includes the following steps:

[0064] S101: receiving an imitation learning trajectory planning request sent by a user terminal, where the imitation learning trajectory planning request carries building expert demonstration learning mapping data, collecting and optimizing training data sets, and model training application data;

[0065] The step of receiving the imitation learning trajectory planning request sent by the user terminal includes:

[0066] Receive a connection request sent by a user terminal through a preset transmission network, where the connection request is used to request to establish a connection with the scanning robot;

[0067] Check whether the current account of the user terminal is the target account;

[0068] If the current account of the user terminal is the target account, connect with the user terminal according to the connection request, and after the connection is completed, receive the imitation learning trajectory planning request sent by the user terminal;

[0069] S102: Based on constructing expert demonstration learning mapping data, learn the mapping relationship from image observation to action, train and optimize the neural network by collecting and optimizing the training data set, and generate high-precision trajectories based on real-time image observations;

[0070] Wherein, based on constructing expert demonstration learning mapping data, the method for obtaining the expert demonstration learning mapping data includes:

[0071] Behavioral cloning, which is used to build a mapping from state to action to achieve imitation;

[0072] Generative adversarial imitation learning, which combines generative adversarial networks and imitation learning, uses the adversarial training mechanism of GAN to optimize the policy network in imitation learning, and can learn behaviors similar to expert strategies;

[0073] Inverse reinforcement learning, for generating reward functions inferred from expert demonstrations and using the reward functions to train reinforcement learning;

[0074] Visual attention imitation learning, used to improve individual visual attention through imitation;

[0075] S103: Establishing a unified coordinate system, obtaining the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data, mapping the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through a coordinate transformation matrix to obtain the point cloud data, and then completing preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data;

[0076] Among them, a unified coordinate system is established to obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data, including:

[0077] The coordinate system of the scanning robot itself, the coordinate system of the RGB-D camera, and the coordinate system of the point cloud data are combined in one coordinate system to ensure effective fusion with multi-source sensor data;

[0078] The scanning robot coordinate system is used as the base coordinate system to describe the trajectory point, optical center position and end effector posture, and the coordinate of the trajectory point at the current moment is defined as p t .

[0079] Through the coordinate transformation matrix, the point cloud data collected by the RGB-D camera is mapped to the scanning robot coordinate system to achieve a unified expression of different information. In the scanning robot coordinate system, the optical center position coordinate is p optical , the point cloud coordinate set is P cloud .

[0080] S104: placing the generated high-precision trajectory in the point cloud data, determining the point cloud coordinates of the contact point of the scan according to the normal direction of the trajectory and the optical center corresponding to the point cloud, and determining the normal vertical intersection point in the three-dimensional space information and the local normal characteristics in the point cloud coordinates;

[0081] The point cloud coordinates of the contact point scanned are obtained by determining the trajectory and the optical center coordinates, and the normal vertical intersection point in the three-dimensional space information and the local normal characteristics is determined in the point cloud coordinates, including:

[0082] According to the connection equation: L(t)=P t +t·(P t -P optical ), determine the cloud normal direction of the point corresponding to the trajectory and the optical center;

[0083] According to the square of the Euclidean distance function D(t) = || L(t) - P cloud || 2 , by taking the derivative of D(t) with respect to t and making the derivative equal to 0, we can get the parameter corresponding to the minimum value

[0084] Substitute the required normal perpendicular intersection coordinates P target =L(t * )=P t +t * ·(P t -P optical ), the point cloud coordinates of the contact point to be scanned are obtained from the current trajectory and the optical center coordinates;

[0085] S105: Consistency adjustment is performed based on the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that the end execution needs to adjust to. The final posture direction is then subjected to extended Kalman filter posture correction to complete state prediction and error correction. By continuously collecting sensor data, the end posture and trajectory planning of the scanning robot are adjusted in real time to ensure that the task execution performance is maintained in a complex dynamic environment.

[0086] Among them, consistency adjustment is performed according to the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that the end execution needs to be adjusted to, including:

[0087] Adjust the point cloud normal direction to ensure that the point cloud normal points in the correct direction and is consistent with the current end posture normal n current The point cloud normal adjustment formula is:

[0088]

[0089] The target normal after consistency adjustment is the final posture direction adjusted by the end execution.

[0090] In this embodiment, collecting and optimizing the training data set and model training application data includes:

[0091] The purpose of collecting and optimizing training data sets is to improve the accuracy and robustness of the model. We collect exclusive training data sets and use them to train and optimize the neural network. We define the scanning task of scanning the robot's degrees of freedom in the exclusive training data sets to ensure that the model can handle complex motion patterns and has strong generalization capabilities.

[0092] The model training application data is to directly call the trained trajectory planning model to complete the scanning task. The trajectory planning model can generate high-precision trajectories based on real-time image observations, significantly improving task efficiency and execution accuracy. This technology solves the errors in the terminal fitting of the current intelligent trajectory planning algorithm. It dynamically adjusts the posture information by updating the target point cloud in real time, thereby effectively correcting the terminal posture of the robot, greatly improving the accuracy and stability of the scanning robot in scanning tasks, and significantly enhancing its adaptability in dynamic and complex environments.

[0093] In this embodiment, the point cloud data is preprocessed and feature extracted to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data, including:

[0094] After acquiring the point cloud data from the RGB-D camera, the following processing is performed:

[0095] Use voxel filtering to downsample point cloud data to reduce the amount of data, and use statistical filtering to remove isolated points and noise points in point cloud data;

[0096] For each P in the point cloud i Define its local area V i ,The formula for calculating the covariance matrix of points in the local area is:

[0097]

[0098] Among them, V i For point P i The domain set, μ i is the centroid of the field, perform eigenvalue decomposition on the covariance matrix, and take the eigenvector of the minimum eigenvalue as the normal vector of the point n i ,The point cloud data contains three-dimensional spatial information, and supplements the local normal characteristics.

[0099] In this embodiment, the final attitude direction is further subjected to extended Kalman filter attitude correction to complete state prediction and error correction, and the terminal attitude and trajectory planning of the scanning robot are adjusted in real time by continuously collecting sensor data, including:

[0100] The extended Kalman filter is used to combine the data collected by the sensor with the predefined trajectory to perform state prediction and error correction, ensuring the stability and adaptability of task execution, and accurately completing the specified task even in a dynamic and complex environment;

[0101] Through RGB-D cameras or other sensors, the target surface point cloud data is continuously acquired, the current posture normal is compared with the target normal, the error vector is calculated in real time, and the terminal posture is dynamically adjusted through the extended Kalman filter according to the error size and direction.

[0102] After denoising and downsampling, the newly collected point cloud data is aligned with the scanning robot coordinate system to generate the latest target point cloud normal and ensure that the direction of the current normal after adjustment is consistent with the task requirements. The updated target normal is merged with the current trajectory, and the intersection coordinates are recalculated through the trajectory equation to achieve dynamic posture correction.

[0103] Parallel processing of point cloud data preprocessing, error calculation and trajectory update, and then dynamically adjusting the sensor data acquisition frequency and smoothing factor according to the frequency of environmental changes, optimizing feedback control, refer to Figure 3 It can be seen that in a complex dynamic environment, the posture of the end can be accurately perpendicular to the surface of the object. When the physical position of the object changes, there is no need to re-model and plan the trajectory. The technical solution of "first image-to-action based imitation learning trajectory planning, and then real-time point cloud normal vertical correction" is implemented.

[0104] This technical solution adjusts the spatial posture of the end effector of the scanning robot by real-time sensing and calculating the relative position and direction changes between the robot and the target environment, so that it always remains perpendicular to the normal of the target surface. The specific design and implementation of each link mainly include: imitation learning trajectory planning, unified coordinate system, point cloud preprocessing, vertical point confirmation, calculation of target normal vector, end posture correction, real-time update feedback mechanism, and realization of full trajectory movement. By comprehensively utilizing trajectory planning and point cloud characteristics, this method provides the scanning robot with high-precision and real-time posture adjustment capabilities, which not only solves the problem of verticality of the end posture, but also optimizes the task completion efficiency, greatly expanding the application prospects of robots in the medical and industrial fields.

[0105] Reference Figure 4As shown, a robot real-time normal vertical correction system based on point cloud technology is executed in the above-mentioned robot real-time normal vertical correction method, and the correction system includes: an acquisition module, which is used to receive an imitation learning trajectory planning request sent by a user terminal; a trajectory generation module, which is used to train and optimize the neural network by collecting and optimizing the training data set, and generate a high-precision trajectory according to real-time image observation; a coordinate execution module, which is used to establish a unified coordinate system, obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data; a point cloud data acquisition module, which is used to map the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through a coordinate transformation matrix to obtain point cloud data, and then complete preprocessing and feature extraction of the point cloud data to obtain three-dimensional spatial information and local normal characteristics of the point cloud data; an adjustment module, which is used to determine the point cloud coordinates of the contact point of the scan between the trajectory and the optical center coordinates according to the point cloud normal direction corresponding to the trajectory and the optical center, and determine the three-dimensional spatial information and local normal characteristics in the point cloud coordinates. Normal vertical intersection; terminal posture planning module, used to make consistency adjustment according to the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that the terminal needs to adjust to. By continuously collecting sensor data, the terminal posture and trajectory planning of the scanning robot are adjusted in real time to ensure the task execution performance in a complex dynamic environment. By real-time perception and calculation of the relative position and direction changes between the robot and the target environment, the spatial posture of the end effector of the scanning robot is adjusted to keep it perpendicular to the target surface normal. The specific design and implementation of each link mainly include: imitation learning trajectory planning, unified coordinate system, point cloud preprocessing, vertical point confirmation, calculation of target normal vector, terminal posture correction, real-time update feedback mechanism, and realization of full trajectory movement. By comprehensively utilizing trajectory planning and point cloud characteristics, this method provides the scanning robot with high-precision and real-time posture adjustment capabilities, which not only solves the verticality problem of the terminal posture, but also optimizes the task completion efficiency, greatly expanding the application prospects of robots in the medical and industrial fields.

[0106] This technical solution dynamically adjusts the posture information by updating the target point cloud in real time, thereby effectively correcting the robot's terminal posture, greatly improving the accuracy and stability of the scanning robot in scanning tasks, and significantly enhancing its adaptability in dynamic and complex environments.

[0107] In this embodiment, the entire operation process can be controlled by a computer, and in each operation link, sensors can be set to provide signal feedback to achieve sequential execution of steps. These are all common knowledge of current automated control and will not be described in detail in this embodiment.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A robot real-time normal vertical correction method based on point cloud technology, characterized in that: Applied to scanning robots, specifically including the following steps: S101: receiving an imitation learning trajectory planning request sent by a user terminal, wherein the imitation learning trajectory planning request carries constructing expert demonstration learning mapping data, collecting and optimizing training data sets, and model training application data; S102: Based on constructing expert demonstration learning mapping data, learn the mapping relationship from image observation to action, train and optimize the neural network by collecting and optimizing the training data set, and generate high-precision trajectories based on real-time image observations; S103: Establishing a unified coordinate system, obtaining the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data, mapping the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through a coordinate transformation matrix to obtain the point cloud data, and then completing preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data; S104: placing the generated high-precision trajectory in the point cloud data, determining the point cloud coordinates of the contact point of the scan according to the normal direction of the trajectory and the optical center corresponding to the point cloud, and determining the normal vertical intersection point in the three-dimensional space information and the local normal characteristics in the point cloud coordinates; S105: Consistency adjustment is performed based on the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that needs to be adjusted to the end execution. The final posture direction is then subjected to extended Kalman filter posture correction to complete state prediction and error correction. By continuously collecting sensor data, the scanning robot's end posture and trajectory planning are adjusted in real time to ensure that the task execution performance is maintained in a complex dynamic environment.

2. The robot real-time normal vertical correction method based on point cloud technology according to claim 1, characterized in that: In S101, receiving an imitation learning trajectory planning request sent by a user terminal includes: Receiving a connection request sent by a user terminal through a preset transmission network, wherein the connection request is used to request to establish a connection with the scanning robot; Detecting whether the current account of the user terminal is the target account; If the current account of the user terminal is the target account, a connection is made with the user terminal according to the connection request, and after the connection is completed, an imitation learning trajectory planning request sent by the user terminal is received.

3. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 2, characterized in that: Collecting and optimizing training data sets and model training application data includes: The purpose of collecting and optimizing the training data set is to improve the accuracy and robustness of the model. The exclusive training data set is collected and used to train and optimize the neural network. The scanning task of scanning the robot's degrees of freedom is defined in the exclusive training data set to ensure that the model can handle complex motion patterns and has strong generalization capabilities. The model training application data is to directly call the trained trajectory planning model to complete the scanning task. The trajectory planning model can generate high-precision trajectories based on real-time image observations, significantly improving task efficiency and execution accuracy.

4. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 3, characterized in that: In S102, based on constructing expert demonstration learning mapping data, the method for acquiring the expert demonstration learning mapping data includes: Behavioral cloning, which is used to build a mapping from state to action to achieve imitation; Generative adversarial imitation learning, which combines generative adversarial networks and imitation learning, uses the adversarial training mechanism of GAN to optimize the policy network in imitation learning, and can learn behaviors similar to expert strategies; Inverse reinforcement learning, for generating reward functions inferred from expert demonstrations and using the reward functions to train reinforcement learning; Visual attention imitation learning is used to improve individual visual attention through imitation.

5. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 4, characterized in that: In S103, a unified coordinate system is established to obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data, including: The coordinate system of the scanning robot itself, the coordinate system of the RGB-D camera, and the coordinate system of the point cloud data are combined in one coordinate system to ensure effective fusion with multi-source sensor data; The scanning robot coordinate system is used as the base coordinate system to describe the trajectory point, optical center position and end effector posture, and the coordinate of the trajectory point at the current moment is defined as p t . Through the coordinate transformation matrix, the point cloud data collected by the RGB-D camera is mapped to the scanning robot coordinate system to achieve a unified expression of different information. In the scanning robot coordinate system, the optical center position coordinate is p optical , the point cloud coordinate set is P cloud .

6. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 5, characterized in that: Then complete the preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data, including: After acquiring the point cloud data from the RGB-D camera, the following processing is performed: Use voxel filtering to downsample point cloud data to reduce the amount of data, and use statistical filtering to remove isolated points and noise points in point cloud data; For each P in the point cloud i Define its local area V i ,The formula for calculating the covariance matrix of points in the local area is: Among them, V i For point P i The domain set, μ i is the centroid of the field, perform eigenvalue decomposition on the covariance matrix, and take the eigenvector of the minimum eigenvalue as the normal vector of the point n i ,The point cloud data contains three-dimensional spatial information, and supplements the local normal characteristics.

7. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 6, characterized in that: In S104, the trajectory and the optical center coordinates are determined to obtain the point cloud coordinates of the scanned contact point, and the normal vertical intersection point in the three-dimensional space information and the local normal characteristics is determined in the point cloud coordinates, including: According to the line equation: L(t)=P t +t·(P t -P optical ), determine the cloud normal direction of the point corresponding to the trajectory and the optical center; According to the square of the Euclidean distance function D(t) = || L(t) - P cloud || 2 , by taking the derivative of D(t) with respect to t and making the derivative equal to 0, we can get the parameter corresponding to the minimum value Substitute the required normal perpendicular intersection coordinates P tareet =L(t * )=P t +t * ·(P t -P optical ), the point cloud coordinates of the contact point to be scanned are obtained from the current trajectory and the optical center coordinates.

8. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 7, characterized in that: In S105, consistency adjustment is performed according to the point cloud normal vector of the vertical intersection point. The target normal after consistency adjustment is the final posture direction to which the end execution needs to be adjusted, including: Adjust the point cloud normal direction to ensure that the point cloud normal points in the correct direction and is consistent with the current end posture normal n current The point cloud normal adjustment formula is: The target normal after consistency adjustment is the final posture direction adjusted by the end execution.

9. The robot real-time normal vertical correction method based on point cloud technology as claimed in claim 8, characterized in that: Then, the final attitude direction is corrected by extended Kalman filtering to complete state prediction and error correction. By continuously collecting sensor data, the terminal attitude and trajectory planning of the scanning robot are adjusted in real time, including: The extended Kalman filter is used to combine the data collected by the sensor with the predefined trajectory to perform state prediction and error correction, ensuring the stability and adaptability of task execution, and accurately completing the specified task even in a dynamic and complex environment; Through RGB-D cameras or other sensors, the target surface point cloud data is continuously acquired, the current posture normal is compared with the target normal, the error vector is calculated in real time, and the terminal posture is dynamically adjusted through the extended Kalman filter according to the error size and direction. After denoising and downsampling, the newly collected point cloud data is aligned with the scanning robot coordinate system to generate the latest target point cloud normal and ensure that the direction of the current normal after adjustment is consistent with the task requirements. The updated target normal is merged with the current trajectory, and the intersection coordinates are recalculated through the trajectory equation to achieve dynamic posture correction. Point cloud data preprocessing, error calculation and trajectory update are processed in parallel, and then the data acquisition frequency and smoothing factor of the sensor are dynamically adjusted according to the frequency of environmental changes to optimize feedback control.

10. The robot real-time normal vertical correction system based on point cloud technology is characterized by: The robot real-time normal vertical correction method according to any one of claims 1 to 9, wherein the correction system comprises: An acquisition module, used to receive an imitation learning trajectory planning request sent by a user terminal; The trajectory generation module is used to train and optimize the neural network by collecting and optimizing the training data set, and generate high-precision trajectories based on real-time image observations; The coordinate execution module is used to establish a unified coordinate system, obtain the coordinate system of the scanning robot, the RGB-D camera coordinate system of the scanning robot, and the coordinate mark of the point cloud data; The point cloud data acquisition module is used to map the point cloud data collected by the RGB-D camera to the scanning robot coordinate system through the coordinate transformation matrix to obtain the point cloud data, and then complete the preprocessing and feature extraction of the point cloud data to obtain the three-dimensional spatial information and local normal characteristics of the point cloud data; An adjustment module is used to determine the point cloud coordinates of the contact point scanned by the trajectory and the optical center coordinates according to the normal direction of the point cloud corresponding to the trajectory and the optical center, and determine the normal vertical intersection point in the three-dimensional space information and the local normal characteristics in the point cloud coordinates; The terminal posture planning module is used to make consistency adjustments based on the point cloud normal vector of the vertical intersection. The target normal after consistency adjustment is the final posture direction that the terminal needs to be adjusted to for execution. By continuously collecting sensor data, the terminal posture and trajectory planning of the scanning robot are adjusted in real time to ensure that the task execution performance is maintained in a complex dynamic environment.

Images