Robot control method and control system

Abstract

The invention provides a robot control method and system. Relates to the technical field of robots. The method comprises the following steps: extracting a first visual feature representing a geometric structure of a target workpiece from visual data, and extracting a tactile feature representing a contact state from tactile data; splicing the first visual feature, the touch feature and the action data of the target robot in the previous time step into a current state vector, inputting the current state vector into a strategy network in a robot control model, outputting an action instruction of the next time step, and controlling the target robot to execute a polishing task on the target workpiece based on the action instruction; wherein the robot control model is obtained through reinforcement learning algorithm training, and in the training process, the difference between the burr residual state fed back after the polishing task is executed in the training environment based on the action instruction and the second visual feature representing the workpiece edge burr feature serves as a reward signal to be input into the evaluation network; and updating the parameters of the policy network.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a robot control method and control system. Background Technology

[0002] In the current electronics manufacturing industry, deburring is a key step in product post-processing. Currently, the industry generally uses fixed deburring workstations or single-arm robots to deburr simple, regular workpieces.

[0003] Existing technologies typically operate using pre-programmed trajectories. This rigid and fixed programming mode lacks flexibility when dealing with products of different specifications, lacks the ability to adapt to complex curved workpieces, cannot dynamically adjust grinding parameters according to the actual condition of the workpiece surface, and is difficult to accurately locate tiny burrs or adapt to the positional deviation of the workpiece. This can easily lead to incomplete grinding or damage to the chip substrate, resulting in unstable processing quality and limiting its application in precision machining scenarios. Summary of the Invention

[0004] Based on this, it is necessary to provide a robot control method and control system to address the above-mentioned technical problems, so as to solve the defects of the existing technology, such as lack of adaptability of pre-programmed trajectories, inability to accurately identify and process small burrs, difficulty in dealing with workpiece position deviations and poor adaptability to complex curved surfaces, and to realize real-time adaptive grinding control based on multimodal perception fusion.

[0005] The following technical solution is adopted in this specification: This specification provides a robot control method, including: Acquire visual and tactile data of the target robot at the current time step; Extract a first visual feature characterizing the geometric structure of the target workpiece from the visual data, and extract tactile features characterizing the contact state from the tactile data; The first visual feature, the tactile feature, and the target robot's motion data from the previous time step are concatenated to form the current state vector. This current state vector is then input into the policy network of the robot control model to output the motion command for the next time step. The robot control model is trained using a reinforcement learning algorithm. During the training process, the difference between the burr residue state fed back after performing the grinding task in the training environment based on action commands and the second visual feature representing the burr characteristics of the workpiece edge is used as a reward signal and input into the evaluation network to update the parameters of the policy network. Based on the action instructions, the target robot is controlled to perform a grinding task on the target workpiece.

[0006] Furthermore, the step of inputting the difference between the residual burr state feedback after performing the grinding task in the training environment based on action instructions and the second visual feature characterizing the burr features of the workpiece edge as a reward signal into the evaluation network specifically includes: A first quantitative index is determined based on the burr residue state. The first quantitative index is used to characterize the current degree of burr removal or the amount of remaining burrs reported by the simulation platform. A second quantitative index is determined based on the second visual feature. The second quantitative index is used to characterize the current degree of burr removal or the amount of remaining burrs predicted based on visual data. Calculate the absolute value or mean square error of the difference between the first quantitative indicator and the second quantitative indicator, and determine the absolute value or mean square error of the difference as the gap; A reward value is calculated based on the gap, and the reward value is input into the evaluation network; wherein the gap and the reward value are negatively correlated.

[0007] Furthermore, updating the parameters of the policy network specifically includes: The evaluation network is used to calculate the time-series difference error based on the current state vector and the reward signal; The dominance function value is calculated based on the aforementioned time-series difference error; Calculate the policy gradient based on the advantage function value, and construct the loss function using the policy gradient; The parameters of the policy network are updated by backpropagation by minimizing the loss function.

[0008] Furthermore, the policy network includes a fully connected feature extraction layer, a feature fusion layer, and an action output layer connected in sequence; the step of inputting the current state vector into the policy network in the robot control model and outputting the action command for the next time step specifically includes: The current state vector is input into the fully connected feature extraction layer and mapped to a high-dimensional feature space. The features in the high-dimensional feature space are input into the feature fusion layer, and the features of different modalities are weighted and fused using an attention mechanism to obtain the fused features. The fused features are input into the action output layer, which outputs the probability distribution parameters of the action commands. Based on the probability distribution parameters, the action command for the next time step is obtained by sampling.

[0009] Furthermore, the tactile data includes the pressure distribution, contact area, contact torque, and tactile feedback signal in the contact area between the robot end effector and the target workpiece; the extraction of tactile features characterizing the contact state from the tactile data specifically includes: The frequency domain decomposition of the tactile feedback signal is performed using the Fourier transform method. Combined with the analysis of pressure and deformation data at each contact point on the surface of the target workpiece, the frequency characteristic distribution of the tactile feedback signal is calculated. Based on the frequency characteristic distribution of the tactile feedback signal, the local and global frequency characteristics of the tactile feedback signal are captured by fusing convolutional kernels of different receptive fields. The stiffness gradient of the contact area of ​​the target workpiece is then classified in layers to determine the preliminary estimate of the contact stiffness. The stiffness gradient distribution of the object is calculated by combining the time-domain processing results of the tactile feedback signal and the geometric contour of the target workpiece surface; Based on the object's stiffness gradient distribution, pressure distribution, and contact torque, a tactile feature vector containing information on contact stiffness changes and surface morphology is constructed.

[0010] Furthermore, the visual data includes three-dimensional point cloud data of the target workpiece, and the extraction of a first visual feature characterizing the geometric structure of the target workpiece from the visual data specifically includes: The 3D point cloud data is input into a pre-trained lightweight point cloud segmentation model to obtain a workpiece surface segmentation map; the lightweight point cloud segmentation model includes at least one of PointNet, PointNet++, and an improved network based on PointNet. The workpiece surface segmentation map is sampled, and a preset number of key points are selected; for each key point, a set of point clouds within its preset radius neighborhood is determined. Based on the point cloud set, the surface normal vector and curvature value of the key points are calculated by fitting the point cloud using principal component analysis or least squares method; the spatial coordinates, surface normal vector and curvature value of each key point are then concatenated to form a local geometric feature descriptor. A feature aggregation operation is performed on all local geometric feature descriptors to obtain the first visual feature representing the overall geometric structure of the target workpiece.

[0011] Furthermore, the visual data includes image data of the target workpiece, and the step of obtaining the second visual feature characterizing the edge burr features of the workpiece includes: The image data is preprocessed to extract the region of interest containing the edge of the target workpiece; The region of interest is input into a pre-trained lightweight image segmentation model, which outputs a spur candidate region confidence map. The spur candidate region confidence map is used to characterize the probability distribution of each pixel in the image belonging to a spur. The lightweight image segmentation model includes at least one of the MobileNet series, ShuffleNet series, or GhostNet. Statistical analysis is performed on the confidence map of the burr candidate region to extract the burr probability distribution features in the edge normal direction, the connected domain area features of the high probability region, and the gradient features of the burr texture. The burr probability distribution features in the edge normal direction, the connected domain area features of the high probability region, and the gradient features of the burr texture are combined to obtain the second visual feature that characterizes the burr features of the workpiece edge.

[0012] This specification provides a robot control system, including: The multimodal data acquisition module is used to acquire visual and tactile data of the target robot at the current time step; The feature extraction module is used to extract first visual features representing the geometric structure of the target workpiece from the visual data, and to extract tactile features representing the contact state from the tactile data. The instruction generation module is used to concatenate the first visual feature, the tactile feature, and the target robot's action data from the previous time step into a current state vector, input the current state vector into the policy network in the robot control model, and output the action instruction for the next time step; wherein... The robot control model is trained using a reinforcement learning algorithm. During the training process, the difference between the burr residue state fed back after performing the grinding task in the training environment based on action commands and the second visual feature representing the burr characteristics of the workpiece edge is used as a reward signal and input into the evaluation network to update the parameters of the policy network. The execution control module is used to control the target robot to perform a grinding task on the target workpiece based on the action instructions.

[0013] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects: This invention acquires visual and tactile data in real time and extracts corresponding features. Combined with action data from the previous time step, a state vector is constructed. A policy network trained through reinforcement learning dynamically outputs action commands—geometric structural features extracted from visual data and contact state features from tactile data—replacing the fixed trajectory parameters in pre-programming. This allows the robot to perceive the actual geometric specifications and surface condition of the workpiece, thus adapting to different product specifications and complex curved surfaces. Simultaneously, during training, the difference between the residual burr state and the burr features at the workpiece edge serves as a reward signal, driving the policy network to continuously optimize actions. This achieves precise positioning of minute burrs and dynamically adjusts grinding parameters based on workpiece position deviations, avoiding incomplete grinding or damage to the chip substrate. This closed-loop perception-decision-optimization model not only overcomes the limitations of pre-programming for fine processing scenarios but also enables the robot to develop "adaptive learning capabilities" for individual workpiece differences during grinding. Even when faced with minor dimensional deviations within a batch of workpieces, it can autonomously adjust through real-time feature feedback, significantly improving the stability of processing quality. Simultaneously, it reduces the time cost of reprogramming for different product specifications, expanding the applicable scenarios of the equipment while ensuring processing accuracy. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating a robot control method provided in this specification; Figure 2 This document provides a schematic diagram of a reinforcement learning training process. Figure 3 This is a flowchart illustrating a process for obtaining the current state, as provided in this specification. Figure 4 This is a flowchart illustrating a process for obtaining evaluation network inputs, as provided in this specification. Figure 5 This is a flowchart illustrating a training and evaluation network provided in this manual; Figure 6 This is a flowchart illustrating a network parameter update strategy provided in this manual. Figure 7 This is a flowchart illustrating the operation of a strategy network as provided in this manual; Figure 8 This is a schematic diagram of the structure of a robot control system provided by the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in this specification without creative effort are within the scope of protection of this application.

[0016] The technical solution provided by this invention can be applied to electronic manufacturing scenarios to intelligently control robots performing grinding tasks. Existing robotic grinding systems have significant shortcomings in core perception capabilities. Their vision systems typically only provide a rough two-dimensional outline or macroscopic geometric information of the workpiece, lacking the ability to accurately capture microscopic surface features such as burr morphology and texture changes. Simultaneously, the tactile feedback mechanism is still imperfect, unable to perceive minute fluctuations in contact force and real-time changes in material removal rate during high-speed dynamic processes, making it difficult for the system to establish an accurate perception model. Therefore, a new generation of grinding solutions integrating multimodal perception, intelligent decision-making, and collaborative control is urgently needed.

[0017] The following is combined Figures 1-7 The robot control method of the present invention is described.

[0018] Figure 1 This is a flowchart illustrating the robot control method provided by the present invention, as shown below. Figure 1 As shown, the method includes: S1. Obtain the visual and tactile data of the target robot at the current time step.

[0019] Among them, the target robot is an industrial robot specifically designed to perform grinding tasks. It has a multimodal sensing interface and a high-degree-of-freedom robotic arm. The robot's end effector is equipped with a six-dimensional force sensor, a vision acquisition unit, and a flexible grinding tool. It can maintain a stable working posture under complex working conditions and respond to changes in contact force during the grinding process.

[0020] For example, visual data includes 3D point cloud data, image data, and attitude information reflecting the spatial position of the workpiece. It can be acquired directly via a structured light camera mounted at the end of a robotic arm or a fixed workstation, or via a visual sensor array fixed to the work environment; no limitation is made here.

[0021] For example, tactile data includes pressure distribution, contact area and contact torque in the contact area between the robot's end effector and the target workpiece, high-frequency vibration signals and tactile feedback signals, as well as characteristic information for characterizing the hardness of the contact material. This data can be acquired using flexible electronic skin or an array of pressure sensors mounted on the end effector, or calculated using a highly sensitive six-dimensional force sensor combined with a dynamic model; no limitation is imposed here.

[0022] S2. Extract the first visual features representing the geometric structure of the target workpiece and the second visual features representing the edge burr features of the workpiece from the visual data, and extract the tactile features representing the contact state from the tactile data.

[0023] For example, the first visual feature is used to characterize the overall geometric structure and shape of the target workpiece. The specific acquisition steps are as follows: To effectively extract this feature, the collected 3D point cloud data is first input into a pre-trained lightweight point cloud segmentation model to remove background noise and fixture interference, accurately outputting a point cloud segmentation map containing only the workpiece surface. This lightweight point cloud segmentation model aims to ensure real-time performance while maintaining feature extraction accuracy; the specific architecture can be any of PointNet, PointNet++, or a variant network based on the PointNet architecture. Subsequently, the obtained workpiece surface segmentation map is downsampled, selecting a preset number of key points to retain regions with significant geometric features while reducing subsequent computational complexity. For each selected key point, the system determines a spherical neighborhood within a preset radius centered on it and collects all point cloud data within this neighborhood to form a local point cloud set. Then, based on this local point cloud set, geometric calculations are performed using Principal Component Analysis (PCA) or least squares plane fitting techniques. By analyzing the distribution covariance matrix of the point cloud data, the normal vector direction of the fitting plane is calculated. Based on the distance distribution or eigenvalue ratio from the point cloud to the fitting plane, the curvature values ​​at key points are calculated to quantify the local surface curvature. Finally, after obtaining the basic geometric parameters, the three-dimensional spatial coordinates of each key point, the calculated surface normal vector, and the curvature value are sequentially concatenated to form a high-dimensional vector, i.e., the local geometric feature descriptor for that point. To obtain the first visual feature that can characterize the overall geometric structure of the target workpiece, feature aggregation operations (such as max pooling or global average pooling) are performed on all local geometric feature descriptors, thereby generating a global feature vector that comprehensively reflects the geometric shape of the workpiece.

[0024] For example, the second visual feature is specifically used to characterize the subtle morphology and distribution of burrs on the edge of a workpiece. The specific acquisition steps are as follows: First, the system preprocesses the acquired raw image data, including denoising, histogram equalization, and distortion correction, aiming to improve image quality and eliminate ambient lighting interference. Based on the preprocessed image, an edge detection algorithm is used to extract the region of interest (ROI) containing the edge of the target workpiece to eliminate interference from irrelevant devices in the background and focus on key edge areas. Next, the extracted ROI is input into a pre-trained lightweight image segmentation model, which performs point-by-point classification and inference on the pixels within the region, thereby outputting a burr candidate region confidence map. This confidence map is essentially a two-dimensional probability matrix, intuitively representing the probability distribution of each pixel in the image belonging to the burr category. The higher the pixel value in the map, the greater the probability that a burr exists at that location. To meet the stringent real-time requirements of robot control systems, this lightweight image segmentation model can utilize any architecture from the MobileNet series (such as MobileNetV3), ShuffleNet series, or GhostNet. These network structures can significantly reduce computational complexity and model parameter size while maintaining high segmentation accuracy. Finally, in-depth statistical analysis is performed on the generated burr candidate region confidence map. The system first determines the edge normal direction and statistically analyzes the distribution of probability values ​​along this direction, extracting the burr probability distribution features along the edge normal direction to reflect the thickness trend of the burr in the vertical edge direction. Simultaneously, by setting a high confidence threshold, high-probability regions are selected, and their connected region area features are calculated to quantify the continuous coverage of the burr. Furthermore, gradient operators are used to calculate the texture gradient features of the burr region to describe the sharpness and surface roughness of the burr edge. Finally, the extracted edge normal direction burr probability distribution features, high-probability region connected region area features, and burr texture gradient features are standardized, concatenated, and combined to form a high-dimensional feature vector, thus obtaining the second visual feature representing the burr characteristics of the workpiece edge.

[0025] For example, tactile features are used to finely characterize the physical contact state when a robot end effector contacts a workpiece. The extraction process specifically includes the following steps: First, the acquired raw tactile feedback signal is decomposed in the frequency domain using Fourier transform. During this process, the system combines real-time pressure data and minute deformation data at various contact points on the target workpiece surface for joint analysis. By calculating the energy spectral density of the signal at different frequencies, the frequency characteristic distribution of the tactile feedback signal is accurately calculated to reveal the vibration characteristics caused by uneven material hardness or surface roughness during contact. Subsequently, based on the calculated frequency characteristic distribution, a multi-scale convolutional neural network is used for deep feature extraction. This network integrates convolutional kernels with different receptive fields. Smaller kernels are responsible for capturing subtle high-frequency local frequency features in the tactile signal, while larger kernels are used to capture low-frequency global frequency features. Through this hierarchical feature extraction method, the system can perform fine-grained hierarchical classification of the stiffness gradient of the target workpiece contact area and determine a preliminary estimate of the contact stiffness accordingly. To improve the completeness of the features, the system also combines the time-domain processing results of the tactile feedback signal with the geometric contour of the target workpiece surface for comprehensive calculation. Temporal processing involves the effective value (RMS), peak factor, and waveform indices of the statistical signal, which reflect the dynamic range of contact force changes. By aligning and fusing the temporal features with the spatial geometric contour, the system can calculate the object stiffness gradient distribution along the contact path, thereby accurately identifying abrupt hardness changes between the burr root and the substrate, or at the interface between different materials. Finally, based on the object stiffness gradient distribution, pressure distribution on the contact interface, and contact torque information obtained from the above analysis, the final tactile feature vector is constructed. This vector is a multi-dimensional data combination that not only contains dynamic information about the changes in contact stiffness but also implicitly contains rich surface morphology information (such as roughness texture), providing a high-precision decision-making basis for the robot to adjust the polishing force and path based on tactile perception during the polishing process.

[0026] S3. The first visual feature, tactile feature, and the action data of the target robot in the previous time step are concatenated into the current state vector. The current state vector is input into the policy network in the robot control model, and the action command for the next time step is output. The robot control model is trained by a reinforcement learning algorithm. During the training process, the difference between the burr residue state and the second visual feature after performing the polishing task in the training environment based on the action command is used as a reward signal and input into the evaluation network to update the parameters of the policy network.

[0027] For example, Figure 2 This manual provides a schematic diagram of a reinforcement learning training process, such as... Figure 2As shown, this reinforcement learning framework adopts an actor-critic architecture, mainly consisting of a policy network (Actor) and an evaluation network (Critic). In the critic value network, S... t The state vector at the current time step t is composed of the first visual feature, tactile feature, and action data extracted from the previous time step, and serves as the input to the network. This indicates that the critic network is in the current parameters Next to state S t Value estimation; S t+1 This indicates the state of the environment at the next time step after the action is performed; This indicates the commentator network's opinion on the next state S. t+1 Value estimation; The loss function of the critic network is represented by S. t The network parameters are updated using the timing difference (TD) error of the direction propagation. To improve the accuracy of value estimation. In the Actor Strategy Network (Actor) section, This indicates that the policy network outputs an action given the current parameters w and the state s. The probability distribution; the weighted gradient represents the policy gradient multiplied by the advantage estimate provided by the critic, used to guide the direction of policy updates. The gradient of the policy objective function J(W) with respect to parameter w is represented by the gradient. The policy network parameters are updated by maximizing the expected cumulative reward, enabling the agent to learn a better action selection policy. In this embodiment, the agent refers to the subject that makes decisions and executes actions, specifically manifested as a robot entity loaded with the policy network or its agent model in a simulation environment. The current state of reinforcement learning is composed of the first visual feature, second visual feature, tactile feature extracted from the previous time step, and the executed action data, serving as the input to the policy network. The policy network infers based on this state vector and outputs the action command for the next time step. The agent executes the polishing task in a preset training environment according to the action command, and the environment provides feedback on the residual burr state after execution (i.e., the actual processing result). Subsequently, the difference between the residual burr state and the second visual feature extracted from the visual data is calculated, and this difference is converted into a reward signal and input into the evaluation network. The core objective of training is to minimize this difference (i.e., remove burrs as much as possible or improve perceptual accuracy) by adjusting the policy network parameters. The evaluation network evaluates the value of the current policy based on the reward signal and guides the parameter update of the policy network, thereby achieving continuous optimization of the polishing control policy.

[0028] For example, Figure 3 This document provides a flowchart illustrating a process for obtaining the current state, as shown below. Figure 3As shown, the process begins with a multimodal sensor array. The vision sensor uses a structured light camera to scan the workpiece to be grasped, while a six-dimensional force sensor of the tactile sensor is mounted at the end of the robotic arm, and flexible electronic skin covers the surface of the robotic gripper. After denoising and distortion correction preprocessing, the first visual feature extraction path segments the 3D point cloud of the vase to separate the target object. Then, keypoint sampling is performed, and the normal vector and curvature of each point are calculated, ultimately aggregating to form geometric surface features. Simultaneously, the second visual feature extraction path focuses on the bottleneck area of ​​the vase through ROI extraction, using a lightweight segmentation algorithm to obtain the local contour. Probabilistic graphical analysis is combined to evaluate grasping stability, and multi-dimensional features are combined. In the tactile data stream… In this process, when the robotic arm approaches the vase, a six-dimensional force sensor collects contact force / torque signals in real time, and a flexible electronic skin senses the contact pressure distribution. After filtering and normalization preprocessing, the tactile feature extraction module performs frequency domain decomposition on the signals to identify material vibration characteristics, calculates local stiffness gradients to determine surface hardness, and integrates dynamic contact process information through temporal fusion to finally construct a tactile feature vector. The geometric surface features extracted by the first visual path, the local grasping features obtained by the second path, and the tactile feature vector are input together to the current state vector node, fusing to form a complete representation that includes the three-dimensional shape of the target object, local key areas, material physical properties, and real-time contact state, providing multimodal perception support for the robotic arm to plan the optimal grasping posture and force control strategy.

[0029] S4. Control the target robot to perform grinding tasks on the target workpiece based on motion commands.

[0030] Based on the above Figure 1 The illustrated embodiment, for example, Figure 4 This is a flowchart illustrating a process for obtaining evaluation network inputs, as provided in this specification. Figure 4 As shown, in S3 above, the difference between the residual burr state feedback after performing a grinding task in the training environment based on action commands and the second visual feature representing the burr characteristics of the workpiece edge is used as a reward signal input to the evaluation network. By constructing a reward mechanism based on the deviation between "simulation truth" and "visual prediction," the problem of difficulty in training and inconsistency between visual perception and physical state in grinding tasks by relying solely on sparse end feedback is effectively solved. This method can guide the agent to actively focus on the correspondence between visual features and actual physical removal effect during simulation training, improving the robot's perception accuracy of burr state and the robustness of grinding strategy, thereby achieving efficient transfer from simulation environment to real workspace. Specifically, it includes: S311. Determine the first quantitative index based on the burr residue state. The first quantitative index is used to characterize the current burr removal degree or the amount of remaining burrs reported by the simulation platform.

[0031] For example, the burr residue state is generated in real time by a high-precision geometric comparison module built into the simulation platform. Its input is the 3D point cloud data of the workpiece before and after the grinding task—the point cloud before grinding is acquired by a laser scanner and its coordinate system is calibrated, while the point cloud after grinding is acquired synchronously by the same device at the moment the task ends. This module first performs rigid registration on the two sets of point clouds, and then extracts the normal cross-sectional area along the theoretical edge curve of the workpiece design model at a preset sampling interval (e.g., every 0.5 mm). Within each cross-section, the set of offset distances of the actual point cloud data relative to the theoretical edge curve in the normal direction is calculated. For each cross-section, the maximum single-sided offset distance of the actual point cloud relative to the theoretical edge (i.e., the maximum residual height in the normal direction) is selected. The maximum offset distance in all cross-sections is extracted as the first quantitative index. This index directly reflects the physical limit removal effect of the current grinding action on the real burrs in the simulation environment; the smaller the value, the less burr residue and the higher the grinding quality. Finally, the index is normalized and mapped to the [0,1] interval for network calculation.

[0032] S312. Determine a second quantitative index based on the second visual feature. The second quantitative index is used to characterize the current degree of burr removal or the amount of remaining burrs predicted based on visual data.

[0033] For example, the second visual feature is obtained by standardizing the spur probability distribution features along the edge normal direction, the connected region area features of high-probability regions, and the gradient features of the spur texture, and then concatenating these features into a high-dimensional feature vector. Since the feature vector itself is not a direct physical quantity, this step uses a pre-trained regression mapping network or fully connected layer to map this high-dimensional feature vector to a specific physical size value. This mapping network is trained on a labeled sample dataset and can establish a non-linear relationship between visual features and spur size. The output value calculated by the mapping network is determined as the second quantification index, which represents the current remaining amount of spurs perceived by the robot's vision, and is used for comparison with the physical ground truth of the simulation platform.

[0034] S313. Calculate the absolute value or mean square error of the difference between the first quantitative indicator and the second quantitative indicator, and determine the absolute value or mean square error of the difference as the gap.

[0035] For example, the absolute value or mean square error of the difference between the first quantification index and the second quantification index can be selected as the final gap according to the actual training needs. This gap represents the degree of consistency between the visual observation results and physical reality under the current state.

[0036] S314. Calculate the reward value based on the gap and input the reward value into the evaluation network; wherein the gap and the reward value are negatively correlated.

[0037] For example, the reward function is designed to guide the agent to pursue the state of "what is seen is what is physically realized", and a reward function of the following form is constructed:

[0038] in, It is a preset penalty coefficient used to adjust the weight of the gap in the total reward; This is the base reward value, which can be 0 or an inherent reward based on the amount of burr removal. When the gap increases, meaning the difference between the visually predicted burr amount and the simulated actual burr amount becomes larger (e.g., visual interference leading to misjudgment), the reward value R decreases significantly; conversely, when the gap decreases or even approaches 0, it indicates that the visual perception accurately reflects the workpiece state, and the reward value R increases.

[0039] For example, Figure 5 This is a flowchart illustrating a training and evaluation network provided in this manual, such as... Figure 5 As shown, the robotic polishing system is first deployed in a polishing platform training environment. The platform is equipped with an industrial robotic arm, a force-controlled polishing head, and a binocular vision system. Initially, the residual burr state of the blades is unknown. The policy network takes the current environmental observation as input, and outputs action commands after deep neural network inference, specifically including polishing force, polishing path, and robotic arm end-effector posture. After executing the polishing action, the system obtains feedback information through two channels: on the one hand, it uses a second visual feature extraction module to perform image analysis on the polished area and generate a second quantitative index; on the other hand, it obtains a first quantitative index through contact measurement or physical ground truth calibration. Subsequently, the calculation module compares the first quantitative index with the second quantitative index. The system calculates the gap between visual prediction and actual physical state using a quantitative indicator. This gap value is input into the reward signal generation module to generate a negatively correlated reward value. The larger the gap, the more severe the reward penalty. Simultaneously, the evaluation network calculates the TD error based on the current state-action pair to estimate the advantage function and uses a gradient strategy method to evaluate the value of the current strategy. Finally, based on the reward signal and TD error, the strategy network parameters are updated through backpropagation, and the neural network weights are adjusted to optimize the mapping relationship between polishing intensity, path, and posture. This forms a closed-loop training process of perception-decision-execution-evaluation-optimization, enabling the robot to gradually learn the optimal polishing strategy that can simultaneously minimize visual prediction error and physical burr residue.

[0040] Based on any of the above embodiments, for example, Figure 6 This is a flowchart illustrating a network parameter update strategy provided in this manual, such as... Figure 6 As shown, in S3 above, updating the parameters of the policy network specifically includes: S321. Calculate the timing difference error using the evaluation network based on the current state vector and the reward signal.

[0041] For example, the evaluation network receives the current state vector and the immediate reward signal obtained after performing an action. The current state vector includes a first visual feature, a second visual feature, a tactile feature, and data on the performed action. The reward signal is R, calculated in S314 above based on the difference between the first and second quantification indicators. The evaluation network predicts the value of the current state based on the current state vector, and combines the predicted value of the next state vector with the reward signal to calculate the target value using the Bellman equation. Finally, the deviation between the target value and the current predicted value is calculated and determined as the temporal difference error. This error reflects whether the reward signal based on the visual-physical gap exceeds the evaluation network's expectations.

[0042] S322. Calculate the dominance function value based on time-series difference error.

[0043] For example, to evaluate how good or bad the current action is relative to the average level, the system uses the Generalized Dominance Estimation (GAE) algorithm or a simple temporal difference residual method to calculate the dominance function value based on the temporal difference error calculated in step S321. The dominance function value measures the extent to which the cumulative reward (i.e., the consistency between visual perception and physical truth) obtained by performing a specific polishing action is better than (or worse than) the average level under the current visual feature state. If the difference between the visual features and the physical state is small (i.e., the reward value is high), the dominance function value is positive, indicating that the current action strategy is effective; otherwise, it is negative.

[0044] S323. Calculate the policy gradient based on the advantage function value, and construct the loss function using the policy gradient.

[0045] For example, the policy network outputs an action probability distribution based on the current state vector. To optimize the policy, the output of the policy network is weighted using the advantage function value obtained above, and the policy gradient is calculated. Specifically, when constructing the loss function, the negative log-likelihood form of the policy gradient is used:

[0046] in, The action is selected when the policy network parameter is w. The probability, S t This is the current state, A t The value of the dominance function is used to guide the policy network to learn action strategies that simultaneously ensure both polishing effectiveness and visual perception accuracy. If the dominance function is positive (meaning the current action effectively reduces the gap between visual and physical states), the probability of that action is increased; if it is negative (meaning the current action leads to visual misjudgment or increases the deviation from the physical state), the probability of that action is decreased.

[0047] S324. Update the parameters of the policy network by backpropagation by minimizing the loss function.

[0048] For example, gradient descent (such as the Adam optimizer) is used to backpropagate and update the parameters of the policy network. Through continuous iterative updates, the policy network can gradually learn how to adjust the end effector action of the robotic arm under complex ambient lighting and workpiece posture, so as to minimize the gap between the burr residue state (physical truth) fed back after grinding and the second visual feature (perceptual prediction) observed by the vision system, thereby achieving high-precision and robust automated grinding operations.

[0049] Based on any of the above embodiments, for example, for example... Figure 7 This is a flowchart illustrating the operation of a policy network as provided in this manual, such as... Figure 7 As shown, in S3 above, the policy network includes a fully connected feature extraction layer, a feature fusion layer, and an action output layer connected in sequence; the current state vector is input into the policy network in the robot control model, and the action command for the next time step is output, specifically including: S331. Input the current state vector into the fully connected feature extraction layer and map it to the high-dimensional feature space.

[0050] For example, the current state vector S t Composed of multi-source heterogeneous data, the fully connected feature extraction layer is composed of stacked multilayer perceptrons (MLPs), each layer containing linear transformation units and non-linear activation functions (such as ReLU or Tanh). This layer is responsible for receiving the original state vector S. t By mapping the data to a high-dimensional feature space through linear transformation, the deep semantic information implicit in the state is extracted, resulting in a primary high-dimensional feature vector. This process realizes the transformation from low-dimensional observation data to information-rich representations, laying the foundation for subsequent feature fusion.

[0051] S332. Input the features in the high-dimensional feature space into the feature fusion layer, and combine the attention mechanism to perform weighted fusion of features of different modalities to obtain fused features.

[0052] For example, considering that the importance of visual information (reflecting the state of the burr) and proprioceptive information (reflecting the ability to move) in the polishing task will change dynamically as the task progresses (e.g., visual features have a greater weight when approaching the burr, while proprioceptive force / position information may have a greater weight when in contact), the feature fusion layer introduces an attention mechanism for processing.

[0053] In practical implementation, the aforementioned primary high-dimensional feature vectors are divided into corresponding visual modality sub-vectors and ontology modality sub-vectors. The attention mechanism module calculates the correlation weights between features of each modality. The attention mechanism can be self-attention or cross-attention. Self-attention calculates the correlation between dimensions within visual features to strengthen salient feature regions (such as texture features of burr edges). Cross-attention calculates the dependency between visual and ontology modalities, for example, dynamically adjusting the attention to visually blurred regions based on the current posture of the robotic arm. Through weighted summation or other aggregation methods, features from different modalities are dynamically fused according to the calculated weights to generate a fused feature vector. This fused feature vector can adaptively focus on the most critical state information at the current moment, improving the policy network's ability to perceive complex working conditions.

[0054] S333. Input the fused features into the action output layer and output the probability distribution parameters of the action command.

[0055] For example, the action output layer receives a fused feature vector and outputs a probability distribution parameter used to generate action commands. For grinding tasks in a continuous action space (such as controlling the 6-DOF pose or contact force of the grinding end), the policy network typically uses a Gaussian distribution as the output distribution. The action output layer contains two parallel fully connected branches: a mean branch, which outputs the action mean vector. , represents the optimal action suggested in the current state, standard deviation branch, output action standard deviation vector. This indicates the uncertainty of the action or the scope of exploration. These two parameters... Together they define the probability distribution of the action command at the current moment. .

[0056] S334. Sample based on probability distribution parameters to obtain the action command for the next time step.

[0057] For example, this can be achieved by using reparameterization techniques or by directly sampling noise from a Gaussian distribution. Calculate the action command for the next time step. ,in, This represents element-wise multiplication. The sampled continuous numerical vector usually needs to be amplitude-limited to map it to the robot's actual physical execution range. The final determined specific action command is sent to the robot controller to drive the robot to perform the corresponding polishing operation. Through the sampling mechanism, the policy network can maintain a certain degree of randomness to cope with unknown burr distribution changes while ensuring the smoothness of the action, thereby gradually converging to the optimal polishing strategy.

[0058] The robot control system provided by the present invention is described below. The robot control system described below can be referred to in correspondence with the robot control method described above.

[0059] Figure 8 For an example, please refer to the structural schematic diagram of a robot control system provided by the present invention. Figure 8 As shown, the robot control system may include: The multimodal data acquisition module is used to acquire visual and tactile data of the target robot at the current time step.

[0060] The feature extraction module is used to extract first visual features representing the geometric structure of the target workpiece from visual data and tactile features representing the contact state from tactile data.

[0061] The instruction generation module is used to concatenate the first visual features, tactile features, and the target robot's action data from the previous time step into a current state vector. This current state vector is then input into the policy network of the robot control model to output the action instruction for the next time step. The robot control model is trained using a reinforcement learning algorithm. During the training process, the difference between the burr residue state fed back after performing the grinding task in the training environment based on action commands and the second visual feature representing the burr characteristics of the workpiece edge is used as a reward signal input into the evaluation network to update the parameters of the policy network.

[0062] The execution control module is used to control the target robot to perform grinding tasks on the target workpiece based on motion commands.

[0063] For specific limitations regarding the robot control system, please refer to the limitations on the robot control method above, which will not be repeated here. Each module in the aforementioned robot control system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0064] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A robot control method, characterized in that, include: Acquire visual and tactile data of the target robot at the current time step; Extract a first visual feature characterizing the geometric structure of the target workpiece and a second visual feature characterizing the burr features on the edge of the workpiece from the visual data; extract tactile features characterizing the contact state from the tactile data. The first visual feature, the tactile feature, and the target robot's motion data from the previous time step are concatenated to form the current state vector. This current state vector is then input into the policy network of the robot control model to output the motion command for the next time step. The robot control model is trained using a reinforcement learning algorithm. During the training process, the gap between the residual burr state and the second visual feature fed back after performing the polishing task in the training environment based on the action command is input into the evaluation network to update the parameters of the policy network. Based on the action instructions, the target robot is controlled to perform a grinding task on the target workpiece.

2. The robot control method according to claim 1, characterized in that, The method of inputting the difference between the residual burr state feedback after performing a grinding task in the training environment based on action instructions and the second visual feature characterizing the burr features of the workpiece edge as a reward signal into the evaluation network specifically includes: A first quantitative index is determined based on the burr residue state. The first quantitative index is used to characterize the current degree of burr removal or the amount of remaining burrs reported by the simulation platform. A second quantitative index is determined based on the second visual feature. The second quantitative index is used to characterize the current degree of burr removal or the amount of remaining burrs predicted based on visual data. Calculate the absolute value or mean square error of the difference between the first quantitative indicator and the second quantitative indicator, and determine the absolute value or mean square error of the difference as the gap; A reward value is calculated based on the gap, and the reward value is input into the evaluation network; wherein the gap and the reward value are negatively correlated.

3. The robot control method according to claim 2, characterized in that, The updating of the parameters of the policy network specifically includes: The evaluation network is used to calculate the time-series difference error based on the current state vector and the reward signal; The dominance function value is calculated based on the aforementioned time-series difference error; Calculate the policy gradient based on the advantage function value, and construct the loss function using the policy gradient; The parameters of the policy network are updated by backpropagation by minimizing the loss function.

4. The robot control method according to claim 1, characterized in that, The policy network includes a fully connected feature extraction layer, a feature fusion layer, and an action output layer connected in sequence; the step of inputting the current state vector into the policy network of the robot control model and outputting the action command for the next time step specifically includes: The current state vector is input into the fully connected feature extraction layer and mapped to a high-dimensional feature space. The features in the high-dimensional feature space are input into the feature fusion layer, and the features of different modalities are weighted and fused using an attention mechanism to obtain the fused features. The fused features are input into the action output layer, which outputs the probability distribution parameters of the action commands. Based on the probability distribution parameters, the action command for the next time step is obtained by sampling.

5. The robot control method according to claim 1, characterized in that, The tactile data includes pressure distribution, contact area and contact torque, and tactile feedback signals in the contact area between the robot end effector and the target workpiece; the extraction of tactile features characterizing the contact state from the tactile data specifically includes: The frequency domain decomposition of the tactile feedback signal is performed using the Fourier transform method. Combined with the analysis of pressure and deformation data at each contact point on the surface of the target workpiece, the frequency characteristic distribution of the tactile feedback signal is calculated. Based on the frequency characteristic distribution of the tactile feedback signal, the local and global frequency characteristics of the tactile feedback signal are captured by fusing convolutional kernels of different receptive fields. The stiffness gradient of the contact area of ​​the target workpiece is then classified in layers to determine the preliminary estimate of the contact stiffness. The stiffness gradient distribution of the object is calculated by combining the time-domain processing results of the tactile feedback signal and the geometric contour of the target workpiece surface; Based on the object's stiffness gradient distribution, pressure distribution, and contact torque, a tactile feature vector containing information on contact stiffness changes and surface morphology is constructed.

6. The robot control method according to claim 1, characterized in that, The visual data includes three-dimensional point cloud data of the target workpiece, and the extraction of the first visual feature characterizing the geometric structure of the target workpiece from the visual data specifically includes: Three-dimensional point cloud data is input into a pre-trained lightweight point cloud segmentation model to obtain a workpiece surface segmentation map; the lightweight point cloud segmentation model includes at least one of PointNet, PointNet++, and an improved network based on PointNet. The workpiece surface segmentation map is sampled, and a preset number of key points are selected; for each key point, a set of point clouds within its preset radius neighborhood is determined. Based on the point cloud set, the surface normal vector and curvature value of the key points are calculated by fitting the point cloud using principal component analysis or least squares method; the spatial coordinates, surface normal vector and curvature value of each key point are then concatenated to form a local geometric feature descriptor. A feature aggregation operation is performed on all local geometric feature descriptors to obtain the first visual feature representing the overall geometric structure of the target workpiece.

7. The robot control method according to claim 1, characterized in that, The visual data includes image data of the target workpiece, and the step of obtaining the second visual feature characterizing the edge burr features of the workpiece includes: The image data is preprocessed to extract the region of interest containing the edge of the target workpiece; The region of interest is input into a pre-trained lightweight image segmentation model, which outputs a spur candidate region confidence map. The spur candidate region confidence map is used to characterize the probability distribution of each pixel in the image belonging to a spur. The lightweight image segmentation model includes at least one of the MobileNet series, ShuffleNet series, or GhostNet. Statistical analysis is performed on the confidence map of the burr candidate region to extract the burr probability distribution features in the edge normal direction, the connected domain area features of the high probability region, and the gradient features of the burr texture. The burr probability distribution features in the edge normal direction, the connected domain area features of the high probability region, and the gradient features of the burr texture are combined to obtain the second visual feature that characterizes the burr features of the workpiece edge.

8. A robot control system, characterized in that, The system includes: The multimodal data acquisition module is used to acquire visual and tactile data of the target robot at the current time step; The feature extraction module is used to extract first visual features representing the geometric structure of the target workpiece from the visual data, and to extract tactile features representing the contact state from the tactile data. The instruction generation module is used to concatenate the first visual feature, the tactile feature, and the target robot's action data from the previous time step into a current state vector, input the current state vector into the policy network in the robot control model, and output the action instruction for the next time step; wherein... The robot control model is trained using a reinforcement learning algorithm. During the training process, the difference between the burr residue state fed back after performing the grinding task in the training environment based on action commands and the second visual feature representing the burr characteristics of the workpiece edge is used as a reward signal and input into the evaluation network to update the parameters of the policy network. The execution control module is used to control the target robot to perform a grinding task on the target workpiece based on the action instructions.

9. The robot control system according to claim 8, characterized in that, The feature extraction module includes a tactile feature extraction unit and a visual feature extraction unit; The tactile feature extraction unit is used for: The frequency domain decomposition of the tactile feedback signal is performed using the Fourier transform method. Combined with the analysis of pressure and deformation data at each contact point on the surface of the target workpiece, the frequency characteristic distribution of the tactile feedback signal is calculated. Based on the frequency characteristic distribution of the tactile feedback signal, the local and global frequency characteristics of the tactile feedback signal are captured by fusing convolutional kernels of different receptive fields. The stiffness gradient of the contact area of ​​the target workpiece is then classified in layers to determine the preliminary estimate of the contact stiffness. The stiffness gradient distribution of the object is calculated by combining the time-domain processing results of the tactile feedback signal and the geometric contour of the target workpiece surface; Based on the object's stiffness gradient distribution, pressure distribution, and contact torque, a tactile feature vector containing information on contact stiffness changes and surface morphology is constructed. The visual feature extraction unit is used for: Three-dimensional point cloud data is input into a pre-trained lightweight point cloud segmentation model to obtain a workpiece surface segmentation map; the lightweight point cloud segmentation model includes at least one of PointNet, PointNet++, and an improved network based on PointNet. The workpiece surface segmentation map is sampled, and a preset number of key points are selected; For each key point, determine the set of point clouds within its preset radius neighborhood; Based on the point cloud set, the surface normal vector and curvature value of the key points are calculated by fitting the point cloud using principal component analysis or least squares method; the spatial coordinates, surface normal vector and curvature value of each key point are then concatenated to form a local geometric feature descriptor. Perform feature aggregation on all local geometric feature descriptors to obtain the first visual feature representing the overall geometric structure of the target workpiece. The visual feature extraction unit is also used for: The image data is preprocessed to extract the region of interest containing the edge of the target workpiece; The region of interest is input into a pre-trained lightweight image segmentation model, which outputs a spur candidate region confidence map. The spur candidate region confidence map is used to characterize the probability distribution of each pixel in the image belonging to a spur. The lightweight image segmentation model includes at least one of the MobileNet series, ShuffleNet series, or GhostNet. Statistical analysis is performed on the confidence map of the burr candidate region to extract the burr probability distribution features in the edge normal direction, the connected domain area features of the high probability region, and the gradient features of the burr texture. The burr probability distribution features in the edge normal direction, the connected domain area features of the high probability region, and the gradient features of the burr texture are combined to obtain the second visual feature that characterizes the burr features of the workpiece edge.

Images