Reinforcement learning polishing control method and system for intelligent mechanical arm

Abstract

The invention discloses an intelligent mechanical arm reinforcement learning grinding control method and system. The intelligent mechanical arm reinforcement learning grinding control method comprises the following steps that S1, grinding state information generated when a mechanical arm grinds a workpiece is collected; s2, processing the data by using a GRU network and a CNN network according to the polishing state information to construct modal fusion features; s3, joint expectation data are generated through MLP with the purpose of maximizing grinding quality, grinding efficiency and workpiece safety according to the modal fusion features, wherein the joint expectation data comprise an expectation joint angle and an expectation joint torque; and S4, feedforward compensation is conducted on the basic dynamic model based on the joint expectation data to obtain a joint torque control model, and the mechanical arm body is controlled according to the joint torque control model. According to the invention, the safe operation capability of the mechanical arm in a complex grinding environment can be effectively improved, and the self-adaptive precise control of the grinding task of the mechanical arm is realized.

Description

Technical Field

[0001] This invention belongs to the field of robot perception and control technology, specifically relating to a reinforcement learning-based polishing control method and system for a robotic arm with embodied intelligence. Background Technology

[0002] Current robotic arm grinding control technology is mainly based on multi-sensor information fusion and control algorithm optimization. However, its adaptability and intelligent decision-making level under complex working conditions still face bottlenecks: 1. Shallow application of multi-sensor fusion: Existing solutions generally integrate force / torque sensor and encoder data to achieve force / position coupling control, adjusting the contact force at the end of the robotic arm through force control strategies. For example, some technologies dynamically adjust grinding pressure through force sensor feedback and combine encoder information to achieve trajectory tracking. However, such solutions only rely on the superposition of basic sensor data and do not fully explore the complementary value of multi-source data such as vision and joint current. Their ability to perceive complex states such as grinding wheel wear and workpiece surface defects is limited. 2. Static limitations of control strategies: Traditional control methods (such as PID and adaptive algorithms) rely on preset rules or offline parameter tuning, making it difficult to cope with dynamic changes such as grinding wheel wear and uneven workpiece material during grinding. For example, fixed force threshold strategies require manual reconfiguration when switching between rough grinding and fine grinding conditions, and their response to sudden load changes (such as the grinding head contacting the workpiece edge) is insufficient, easily leading to over-grinding or fluctuations in processing accuracy. 3. Fragmented exploration of reinforcement learning: such as Figure 1 As shown, a few studies have attempted to introduce reinforcement learning into polishing parameter optimization, but their applications are mostly limited to single sensor inputs (such as visual point clouds or force sensor signals alone). They have not constructed comprehensive decision-making models that incorporate multi-dimensional information such as environmental perception, motion state, and actuator load, resulting in weak algorithm generalization ability and difficulty in stable operation under real-world complex conditions. The following examples illustrate this further:

[0003] Dynamic weight adjustment scheme based on force / position coupling control: This type of scheme is typically represented by a force-position coupling compliant grinding control method and compliant grinding control system (Publication No.: CN112658808B, Publication Date: 2021 / 4 / 16). Its core achieves force / position hybrid control through information fusion from a six-dimensional force sensor and an encoder. At the hardware level, a six-dimensional force sensor (such as the ATI Mini45 series) monitors the contact force in real time, and the encoder acquires joint position information. At the algorithm level, a weighted matrix is ​​used to dynamically adjust the weights of the force / position controller, combined with an impedance model to compensate for changes in environmental stiffness. For example, the force controller and position controller switch modes through weight parameters, and a smooth transition of control mode is triggered when the force sensor feedback value exceeds a threshold (such as 100N). Its technical advantages are: achieving dynamic weight adjustment of force / position hybrid control in grinding control, improving force control accuracy compared to traditional PID control; and the impedance model can partially compensate for environmental changes such as differences in workpiece materials. Its technical limitations are: the sensor has a single dimension, does not integrate visual information, and cannot identify workpiece surface defects (such as weld slag and cracks); the impedance model parameters need to be tuned offline, making it difficult to adapt to dynamic changes such as grinding wheel wear.

[0004] A Force Control Scheme Integrating Active and Passive Compliantness: Chinese Patent No. CN110561237B (Publication Date: 2019 / 12 / 13) proposes an active-passive compliantness integrated control architecture. The hardware layer integrates a six-dimensional force sensor (ATI Mini45) and a one-dimensional force sensor from the belt abrasive machine. The algorithm layer fuses the two types of force signals through Kalman filtering and combines a mass-spring-damping model to achieve precise contact force control. For example, the active-passive compliantness model dynamically adjusts the robot's end effector position and the belt abrasive machine's axial float based on force deviation (the difference between the desired force and the actual force). Its technical advantages are: multi-sensor fusion improves the accuracy of contact force measurement, and the combination of active and passive compliantness can compensate for stiffness changes during complex curved surface machining. Its technical limitations are: the mass and stiffness parameters of the impedance model still require manual adjustment, making it difficult to adapt to sudden changes in cutting force caused by grinding wheel wear; and the lack of visual or joint current data means it lacks the ability to perceive potential risks such as grinding head posture deviation.

[0005] A Single-Modal Parameter Optimization Scheme Based on Reinforcement Learning: The Jiangsu University of Science and Technology's publicly disclosed method, system, and storage medium for robot weld grinding based on the DDPG reinforcement learning algorithm (Publication No.: CN120480905A, Publication Date: 2025 / 8 / 15) uses the DDPG reinforcement learning algorithm to generate weld grinding parameters. Its technical approach involves extracting weld features from visual point cloud data and inputting them into a pre-trained reinforcement learning model to generate parameters such as grinding pressure and speed. For example, the model outputs the grinding wheel feed speed and contact force based on visually recognized weld width and depth features. Its technical advantages include improved efficiency compared to traditional manual parameter tuning and reduced trajectory deviation rate in standard weld grinding. Its limitations include: relying solely on visual data and not constructing a multi-modal state space including force perception and current fluctuations, resulting in insufficient generalization ability in complex curved surface grinding; lacking a real-time feedback closed loop, it cannot cope with unexpected situations such as accidental collisions between the grinding head and the workpiece. Summary of the Invention

[0006] To address the above problems, this invention proposes a reinforcement learning-based grinding control method and system for robotic arms with embodied intelligence. The problem it solves is how to integrate multi-source data such as encoders, force / torque, vision, and joint current during robotic arm grinding tasks to construct a comprehensive grinding state perception system. Through reinforcement learning, it achieves dynamic adaptive control under complex working conditions, forming a closed-loop optimization mechanism of "perception-decision-execution." This addresses the issues of insufficient multi-source data fusion, control strategies relying on fixed rules or offline parameter tuning, and lack of real-time autonomous decision-making capabilities in existing technologies, thereby improving the safe operation and processing accuracy of robotic arms in complex environments such as grinding wheel wear and workpiece surface variations. The technical solution adopted by this invention to solve the above technical problems is as follows:

[0007] A reinforcement learning-based polishing control method for a robotic arm with embodied intelligence includes the following steps:

[0008] S1, Collect grinding status information when the robotic arm grinds the workpiece. The grinding status information includes grinding scene image data, contact interaction data between the robotic arm and the workpiece, and robotic arm operation status data.

[0009] S2, Based on the polishing status information obtained in step S1, use GRU network and CNN network to process the data and construct modality fusion features;

[0010] S3. Based on the modal fusion features obtained in step S2, MLP is used to generate joint expectation data with the goal of maximizing grinding quality, grinding efficiency and workpiece safety. The joint expectation data includes expected joint angle and expected joint torque.

[0011] S4. Based on the joint expectation data obtained in step S3, feedforward compensation is performed on the basic dynamic model to obtain the joint torque control model, and the robotic arm is controlled according to the joint torque control model.

[0012] Step S2 includes the following steps:

[0013] S2.1, the various feature data in the robotic arm's operating status data are concatenated to obtain the first time-series vector, and the first time-series vector is input into the GRU network to obtain the time-series feature vector;

[0014] S2.2, using a CNN network to convert the polishing scene image data into visual feature vectors;

[0015] S2.3, preprocess the contact interaction data between the robotic arm and the workpiece to construct the interaction feature vector;

[0016] S2.4, based on the temporal feature vector obtained in step S2.1, the visual feature vector obtained in step S2.2, and the interaction feature vector obtained in step S2.3, the importance score of each modality data is calculated using MLP;

[0017] S2.5, Based on the importance score obtained in step S2.4, the normalized weight of each vector is calculated using the softmax function, and the vectors are fused based on the obtained normalized weights to obtain modality fusion features.

[0018] In step S3, the formula for the reward function aimed at maximizing grinding quality, grinding efficiency, and workpiece safety is as follows:

[0019] ;

[0020] In the formula, , and All are weights. Indicates safety constraint rewards. This indicates a reward for the quality of the polishing process. This indicates a reward for improving polishing efficiency.

[0021] The security constraint reward The corresponding formula is:

[0022] ;

[0023] In the formula, Indicates the maximum joint torque. This represents the maximum contact force when the robotic arm comes into contact with the workpiece. This represents the contact force when the robotic arm comes into contact with the workpiece. This indicates the joint torque.

[0024] The polishing quality reward The corresponding formula is:

[0025] ;

[0026] In the formula, Indicates the maximum surface flatness. Indicates the maximum trajectory deviation. This indicates a reward for the quality of the polishing process. Indicates surface smoothness. This indicates a deviation in the trajectory.

[0027] The grinding efficiency reward The corresponding formula is:

[0028] ;

[0029] In the formula, Indicates the desired polishing speed. This indicates the actual grinding speed of the robotic arm.

[0030] Step S4 includes the following steps:

[0031] S4.1, Based on the desired joint angle obtained in step S3 and actual joint angle Calculate the position error;

[0032] S4.2, Based on the position error obtained in step S4.1, the PD controller is used to generate the basic control quantity;

[0033] S4.3, Based on the basic control quantity obtained in step S4.2 and the joint expectation data obtained in step S3, feedforward compensation is performed on the basic dynamic model to obtain the joint torque control model, and the robotic arm is controlled based on the joint torque control model.

[0034] A reinforcement learning-based polishing control system for a robotic arm with embodied intelligence, comprising:

[0035] Data acquisition module: used to collect grinding status information when the robotic arm grinds the workpiece. The grinding status information includes grinding scene image data, contact interaction data between the robotic arm and the workpiece, and robotic arm operation status data.

[0036] Feature fusion module: Used to process the grinding status information obtained from the data acquisition module using GRU and CNN networks to construct modality fusion features;

[0037] Joint data optimization module: Based on the modal fusion features obtained by the feature fusion module, it generates joint expectation data using MLP with the goal of maximizing grinding quality, grinding efficiency and workpiece safety. The joint expectation data includes expected joint angles and expected joint torques.

[0038] Robotic arm optimization module: It is used to perform feedforward compensation on the basic dynamic model based on the joint expectation data obtained by the joint data optimization module to obtain the joint torque control model, and then control the robotic arm according to the joint torque control model.

[0039] The beneficial effects of this invention are:

[0040] By deeply integrating multi-source sensory information fusion (vision, contact force, joint data) with reinforcement learning strategies (GRU network + MLP) and robot dynamics models (including PD + feedforward compensation control law), a fusion architecture for reinforcement learning strategy training and robot dynamics model control was constructed. This not only significantly improved the trajectory and force control accuracy of the robotic arm grinding, enabling the robotic arm to adapt to dynamic working conditions such as workpiece surface changes and grinding wheel wear, but also avoided collision and overload risks in advance through all-dimensional state monitoring. This achieved unmanned intelligent decision-making and closed-loop optimization of "safety detection-adaptive control-quality assurance", comprehensively improving the safety, adaptability and intelligence level of robotic arm grinding tasks. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A schematic diagram of the reinforcement learning algorithm framework for existing polishing robots;

[0043] Figure 2 This is a schematic diagram of the modal fusion feature fusion process;

[0044] Figure 3 This is a schematic diagram of the control process for the robotic arm. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] A reinforcement learning-based polishing control method for a robotic arm with embodied intelligence includes the following steps:

[0047] S1, collect grinding status information when the robot is grinding the workpiece;

[0048] The grinding status information includes grinding scene image data, contact interaction data between the robotic arm and the workpiece, and robotic arm operation status data. The grinding scene image data can be obtained by acquiring the data with a depth camera, processing it through linearization and normalization, and then extracting the three-dimensional structural features of the scene. This is existing technology and not the focus of this application, so it will not be elaborated further. The robotic arm operation status data can be acquired through various sensors, including the joint angles of the robotic arm. angular velocity Joint torque Data indicating the robot's own motion / load status. The contact interaction data between the robot arm and the workpiece includes the contact force when the robot arm contacts the workpiece. Trajectory deviation The trajectory deviation refers to the deviation between the actual running position trajectory of the robotic arm and the expected running position trajectory.

[0049] S2, as Figure 2 As shown, the grinding state information obtained in step S1 is processed using GRU network, CNN network, and MLP model to construct modality fusion features, including the following steps:

[0050] S2.1, the various feature data in the robotic arm's operating status data are concatenated to obtain the first time-series vector, and the first time-series vector is input into the GRU network to obtain the time-series feature vector;

[0051] In this application, the joint angle angular velocity Joint torque Each dimension is 6, therefore we can obtain an 18-dimensional first time-series vector. , This indicates the time step, which can be set to values ​​such as 20 or 30. It is a real number.

[0052] The hidden layer dimension of the GRU network described in this application is 64. After inputting the first temporal vector into the GRU network, a temporal feature vector is obtained. In other words, the hidden state of the last time step is used to represent the overall dynamic features. Using a GRU network to process temporal ontology perception data is beneficial for capturing the dynamic changes in the robotic arm's motion state.

[0053] S2.2, using a CNN network to convert the polishing scene image data into visual feature vectors;

[0054] The CNN network can be a ResNet-18 truncated up to the fully connected layers. This application uses depth maps acquired by a 640*480 resolution depth camera for feature extraction. The visual feature vectors are... express, This reflects the contour and defect information of the workpiece being ground by the robotic arm.

[0055] S2.3, preprocess the contact interaction data between the robotic arm and the workpiece to construct the interaction feature vector;

[0056] Specifically, each feature data in the contact interaction data is first normalized, and then the normalized data is concatenated. The contact force when the robotic arm contacts the workpiece in this application... The dimension is 6-dimensional, and the trajectory deviation is... The dimension is 3. Two data points are mapped to the [0,1] interval, and then the mapped data are concatenated to obtain a 9-dimensional interaction feature vector. The interaction feature vector uses... express, This reflects the interactive state when the robotic arm comes into contact with the workpiece.

[0057] The above process first uses a GRU network to process "body perception data", which is the robotic arm's operating status data, and then integrates it with "visual features" and "contact features". Body data is time-series data, so dynamic features need to be extracted by GRU first. Visual data is static features, and contact data is quasi-static features. The fusion of the two can avoid interference with time-series information.

[0058] S2.4, based on the temporal feature vector obtained in step S2.1, the visual feature vector obtained in step S2.2, and the interaction feature vector obtained in step S2.3, the importance score of each modality data is calculated using MLP;

[0059] Specifically, first, the 64-dimensional temporal feature vector 128-dimensional visual feature vector and 9-dimensional interaction feature vector Mapping to the same dimension, such as 64 dimensions, yields the time-series feature vector mapping value. Visual feature vector mapping value and interaction feature vector mapping value The mapping method includes linear layers or autoencoders. A single-hidden-layer MLP is then used to calculate the importance score of each modality's data. The MLP can be a single-hidden-layer or double-hidden-layer MLP, but the input dimension of the MLP must be consistent with the dimension of the mapped vector. This application uses a single-hidden-layer MLP with a 64-dimensional input and a 1-dimensional output.

[0060] The formulas for calculating the importance scores of temporal feature vectors, visual feature vectors, and interaction feature vectors are as follows:

[0061] (1)

[0062] (2)

[0063] (3)

[0064] S2.5, Based on the importance score obtained in step S2.4, the normalized weight of each vector is calculated using the softmax function, and the vectors are fused based on the obtained normalized weights to obtain modality fusion features;

[0065] The formula for calculating the normalized weights of the time series feature vector is:

[0066] (4)

[0067] In the formula, This represents an exponential function.

[0068] The formula for calculating the normalized weights of visual feature vectors is:

[0069] (5)

[0070] The formula for calculating the normalized weights of the interaction feature vector is:

[0071] (6)

[0072] The importance of different modes during the grinding process varies with the working conditions (e.g., contact force is more important during curved surface transitions, while vision is more important for workpiece defect identification). An attention mechanism can be used to dynamically allocate these weights. Taking curved surface grinding as an example... , , When sanding a flat surface, , , .

[0073] After obtaining the normalized weights of each feature vector, the mapped feature vectors are then weighted and fused based on the normalized weights to obtain the modality fusion feature. The corresponding calculation formula for the modality fusion feature is as follows:

[0074] (7)

[0075] S3, based on the modal fusion features obtained in step S2, and with the goal of maximizing grinding quality, grinding efficiency, and workpiece safety, an MLP is used to obtain a joint data optimization model for generating joint expectation data. The joint expectation data includes the expected joint angles. and desired joint torque ;

[0076] The formula for the reward function aimed at maximizing grinding quality, grinding efficiency, and workpiece safety is as follows:

[0077] (8)

[0078] In the formula, , and All are weights, and , The reward value representing the security constraint. The bonus value represents the quality of the polishing process. The reward value represents the efficiency of polishing.

[0079] Based on joint torque, i.e., joint output torque Contact force when the robotic arm contacts the workpiece The safety threshold is set using a "penalty" function. When the data exceeds the threshold, the reward drops sharply; when the data does not exceed the threshold, the basic reward is maintained. The corresponding expression is:

[0080] (9)

[0081] In the formula, Indicates the maximum joint torque. This represents the maximum contact force when the robotic arm comes into contact with the workpiece. This indicates a safety constraint reward. By setting safety constraints, the robotic arm and the target workpiece can be effectively protected, thereby ensuring the continuous operation of the equipment.

[0082] Based on the surface roughness of the target workpiece Equal to trajectory deviation Set up a "positive incentive" function so that the higher the polishing quality, the higher the reward. The corresponding expression is:

[0083] (10)

[0084] In the formula, Indicates the maximum surface flatness. This indicates the preset maximum trajectory deviation. This indicates a reward for polishing quality. Controlling polishing quality can effectively improve the consistency of polished products, reduce rework costs, and effectively ensure that surface treatment quality meets standards. The surface flatness laser rangefinder (profilometer) is measured according to the process.

[0085] Based on the actual grinding speed of the robotic arm and the preset expected grinding speed The matching degree between them sets the polishing efficiency reward function to avoid inefficiency caused by excessively slow polishing speed. The corresponding expression is:

[0086] (11)

[0087] In the formula, This indicates a reward for improving polishing efficiency. The actual grinding speed of the robotic arm, also known as the actual feed speed at the end of the robotic arm, is calculated by the encoder and is expressed in mm / s. This indicates the desired grinding speed, which is also the desired feed speed at the end of the robotic arm, and is set according to the process.

[0088] The obtained modal fusion features are input into the MLP to construct a joint data optimization model to obtain the control quantities of the robot body, which include the desired joint angles. and desired joint torque In this application, the MLP is a 3-layer MLP, including a 64-dimensional input layer, a 32-dimensional hidden layer, and a 12-dimensional output layer, with the desired joint angle. and desired joint torque Each of them has 6 dimensions, and the corresponding units are rad and . .

[0089] S4, such as Figure 3 As shown, online grinding status information is collected, and the corresponding joint expectation data is obtained using the methods in steps S2 and S3. Based on the obtained joint expectation data, feedforward compensation is performed on the basic dynamic model to obtain the joint torque control model. The robotic arm body is then controlled according to the joint torque control model, including the following steps:

[0090] S4.1, collect grinding status information online, and obtain the corresponding joint expectation data using the methods in steps S2 and S3. Based on the obtained expected joint angles... and actual joint angle Calculate the position error;

[0091] The position error is adopted The corresponding calculation formula is:

[0092] (12)

[0093] S4.2, Based on the position error obtained in step S4.1, the PD controller is used to generate the basic control quantity;

[0094] The formula for calculating the basic control quantity is:

[0095] (13)

[0096] In the formula, Indicates the proportional gain of the PD controller. This represents the derivative gain of the PD controller. Indicates position error The differential.

[0097] S4.3, Based on the basic control quantity obtained in step S4.2 and the joint expectation data obtained in step S3, feedforward compensation is performed on the basic dynamic model to obtain the joint torque control model, and the robot arm body is controlled according to the joint torque control model.

[0098] The dynamics of a single robotic arm can be calculated using the dynamic equations obtained by the Newton-Euler recursive method to calculate the driving torque of each joint. This is existing technology and will not be elaborated here. The corresponding formulas for calculating the joint torques are as follows:

[0099] (14)

[0100] In the formula, Represents generalized acceleration. Represents generalized acceleration Proportional inertial force, Each component is a quadratic form of a generalized acceleration, express quadratic function, This represents the gravity term.

[0101] This application designs a PD robotic arm controller in joint space and Cartesian space, and introduces control signals from reinforcement learning output. That is, the desired joint torque obtained in step S3 Combined with the dynamic model Compensation is performed to obtain a joint torque control model. The above achieves high-precision trajectory and force control by receiving the control quantity output by reinforcement learning and combining it with the robot dynamics model.

[0102] like Figure 3 As shown, the formula for the control output corresponding to the joint torque control model is:

[0103] (15)

[0104] Figure 3 For the expression of the controller, For the Laplace operator, it is the error. In the frequency domain, Figure 3 The data introduced in the second summation includes data from the robot's sensors, such as joint torque, joint position, and velocity. Finally, the control output joint torque is calculated. The output to the robotic arm enables high-precision and highly compliant grinding motion control. The improvement of this application lies in constructing a collaborative control architecture of "reinforcement learning dynamic correction + dynamic model-based compensation," which overcomes the limitations of existing technologies' "fixed compensation, static response." In a PD + dynamic compensation control model similar to existing technologies (Equation 15), the application innovatively introduces the control signal output from reinforcement learning. That is, the desired joint torque, and through This is deeply integrated into the dynamic compensation process.

[0105] This application uses iterative learning based on multi-source sensing data and reward signals to output control signals adapted to the working conditions. It receives these control signals in real time and drives the robotic arm to perform grinding actions, while also recording the actual state (such as joint angles). angular velocity Feedback is then provided again, forming a closed-loop iteration that enables the robotic arm to adapt to dynamic working conditions such as changes in workpiece surface curvature and mold wear, ensuring processing quality and safety. A joint data optimization model based on reinforcement learning to generate joint expectation data and a robot dynamics model for adjusting and controlling the robotic arm work together to form a closed-loop system of "perception-decision-control-feedback".

[0106] Taking the transition of a workpiece from a flat surface to a curved surface (such as a cylindrical surface) as an example, the steps of a collaborative robotic arm for grinding with a 5kg load are as follows: Data collection and identification of working condition changes: Determine the rate of change of the Z-axis coordinate in the image based on the grinding scene image data. The Z-axis coordinate change rate is a specific operation for identifying the "plane to curved surface transition" condition. Its purpose is to accurately locate the starting boundary of the curved surface. Since steps S1 and S2 need to adapt to all polishing scenarios (such as pure planes and irregular surfaces), there is no need to fix a single slope threshold; therefore, this judgment is unnecessary. Indicates the change in Z-axis data. This indicates the change in X-axis data; a slope greater than 10% indicates a surface. The output is a "surface transition signal," representing the operational decision command generated during the "plane to surface switch." This signal is a key link between multi-source sensing and reinforcement learning decision-making, containing the operational condition type (as a transition phase identifier), transition boundary (surface start position parameter), and policy trigger signal (used to trigger reinforcement learning control policy switching). Then, the fluctuation amplitude of the contact force is calculated based on the collected contact interaction data. The fluctuation range is a secondary verification operation under the working condition. It confirms the "true contact on the curved surface" by the difference in contact force stability (stable on a plane, large fluctuation on a curved surface), avoiding visual misjudgment. Steps S1 and S2 do not require additional verification because the contact force state is stable under a single working condition (pure plane / pure curved surface), therefore no calculation is needed. During plane grinding... When polishing curved surfaces According to the fluctuation range Further confirmation of curved surface contact. The control quantity is adjusted based on reinforcement learning: specifically, based on the surface radius extracted visually. (Example:) ), calculate the Z-axis following trajectory at the end This trajectory is a customized operation for cylindrical surfaces, requiring adaptation to the surface radius and its geometric characteristics to avoid rigid collisions. Steps S1 and S2 need to adapt to multiple workpiece types (such as spherical surfaces and irregularly shaped surfaces), therefore they do not require binding to a single surface parameter, and thus this operation is unnecessary. This indicates the x-coordinate corresponding to the center of the surface, and then outputs the desired joint angle. Compared to planar grinding, the fixed angle increment of the 3rd / 4th joint of the robotic arm is reduced by 50% (as in planar grinding). rad, curved surface (rad) to avoid trajectory overshoot. Then, based on contact force fluctuations, adjust the torque correction coefficient. The value is reduced from 1.0 to 0.8 (see PD+ compensation control rate, i.e., step S4.3) to make the joint torque output smoother and avoid a sudden increase in contact force; a target value for contact force is set. The N is reduced from 80N to 60N, which reduces the risk of over-grinding when polishing curved surfaces. Control and adjustment: when position error mm (prone to occur during curved surface transitions), automatically adjust the proportional gain. Increasing by 20% (e.g., from 500 to 600), the PD controller in S4.2 is a general fixed-parameter controller, only outputting the basic control quantity according to the preset proportional gain. The increase in proportional gain here is an error compensation operation for "surface transition," used to address error deviations that are prone to occur in transition scenarios and accelerate convergence. Since step S4.2 has a small error in a single operating condition (pure plane), the fixed gain is sufficient and therefore does not require dynamic adjustment; Dynamic compensation: Real-time update of the inertia matrix. (Due to changes in joint angle,) (Needs to be recalculated) to ensure... The compensation terms are precise, avoiding torque output deviations.

[0107] The diameter of the grinding wheel decreases due to wear (e.g., from...) mm down to When grinding a collaborative robotic arm with a load of 5kg (mm), the steps include the following: Data acquisition and wear condition identification: The diameter of the grinding wheel is calculated by photographing its profile with a depth camera. The diameter calculation here is a specific operation for "grinding wheel wear identification," used to determine the degree of wear. (i.e., wear exceeds 5%), outputting a "grinding wheel wear signal". This grinding wheel wear signal is a key carrier connecting grinding wheel state perception and control strategy optimization. It includes a wear state identifier (used to confirm whether the grinding wheel is worn), wear degree parameters (quantitative information about wear), and a strategy trigger signal (triggering reinforcement learning to adjust feed rate and torque). Then, the joint torque is judged, and the actual joint torque is determined. Increase by more than 15% (wear leads to increased grinding load) to further confirm the wear; Adjusting the control quantity based on reinforcement learning: Specifically, first based on the wear quantity. (Example) mm), the end feed speed The speed decreased from 80 mm / s to 65 mm / s (the greater the wear, the greater the speed reduction, formula: ), This refers to the initial end feed speed preset according to the grinding process when the grinding wheel is not worn. It is the reference value for adjusting the feed speed after wear, used to ensure a balance between grinding efficiency and quality. As the grinding wheel diameter decreases, the required joint torque increases under the same contact force. Therefore, the desired joint torque is... Increase from 10 N·m to 12 N·m (Formula: ,in, This refers to the initial expected joint torque set based on process requirements when the grinding wheel is not worn. It serves as the benchmark value for calculating torque adjustments after wear. This refers to the actual diameter of the grinding wheel, calculated in real time after its profile is captured by a depth camera, and is used to determine the degree of wear. The original diameter of the grinding wheel before wear is used as a benchmark for determining whether wear exceeds a threshold and calculating the amount of wear; adjusting the reward function... torque threshold From 64 (Original Increased to 68 The threshold adjustment here is a "special optimization for grinding wheel wear scenarios". Because the joint torque demand increases after wear, a new safety threshold needs to be adapted to adapt to the change in torque demand. Control and Adjustment: Grinding wheel wear leads to increased end load mass Reduce the weight (e.g., from 0.5 kg to 0.45 kg) and recalculate the gravity term. ( The length of the end link. (For gravitational acceleration), ensuring accurate compensation; every 5 polishing cycles (example 1 cycle = 2 s), the actual joint angle is adjusted. Contact force Feedback is sent to the reinforcement learning module, if (i.e., insufficient contact force) This refers to a preset target contact force value based on the grinding process (such as flat / curved surface grinding requirements), used to determine whether the actual contact force meets the standard, and further... Increase by 5%~10%, until Returning to the target scope, for the periodic iteration of grinding wheel wear scenarios, the contact force caused by wear optimization is continuously insufficient. Therefore, it is necessary to periodically correct the torque based on real-time data feedback. However, general steps (such as S4.3) do not require periodic adjustments for wear.

[0108] This application also provides an embodiment of a robotic arm reinforcement learning-based polishing control system, including:

[0109] Data acquisition module: used to collect grinding status information when the robotic arm grinds the workpiece. The grinding status information includes grinding scene image data, contact interaction data between the robotic arm and the workpiece, and robotic arm operation status data.

[0110] Feature fusion module: Used to process the grinding status information obtained from the data acquisition module using GRU and CNN networks to construct modality fusion features;

[0111] Joint data optimization module: Based on the modal fusion features obtained by the feature fusion module, it generates joint expectation data using MLP with the goal of maximizing grinding quality, grinding efficiency and workpiece safety. The joint expectation data includes expected joint angles and expected joint torques.

[0112] Robotic arm optimization module: It is used to perform feedforward compensation on the basic dynamic model based on the joint expectation data obtained by the joint data optimization module to obtain the joint torque control model, and then control the robotic arm according to the joint torque control model.

[0113] This application also provides an electronic device, which includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it performs the steps of the above-described embodied intelligence robotic arm reinforcement learning polishing control method.

[0114] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the steps of the aforementioned embodied intelligence robotic arm reinforcement learning polishing control method. Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard disk. When the computer program on the storage medium is run, it can execute the aforementioned embodiment of the embodied intelligence robotic arm reinforcement learning polishing control method.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A reinforcement learning-based polishing control method for a robotic arm with embodied intelligence, characterized in that, Includes the following steps: S1, Collect grinding status information when the robotic arm grinds the workpiece. The grinding status information includes grinding scene image data, contact interaction data between the robotic arm and the workpiece, and robotic arm operation status data. S2, Based on the polishing status information obtained in step S1, use GRU network and CNN network to process the data and construct modality fusion features; S3. Based on the modal fusion features obtained in step S2, MLP is used to generate joint expectation data with the goal of maximizing grinding quality, grinding efficiency and workpiece safety. The joint expectation data includes expected joint angle and expected joint torque. S4. Based on the joint expectation data obtained in step S3, feedforward compensation is performed on the basic dynamic model to obtain the joint torque control model, and the robotic arm is controlled according to the joint torque control model.

2. The embodied intelligence robotic arm reinforcement learning polishing control method according to claim 1, characterized in that, Step S2 includes the following steps: S2.1, the various feature data in the robotic arm's operating status data are concatenated to obtain the first time-series vector, and the first time-series vector is input into the GRU network to obtain the time-series feature vector; S2.2, using a CNN network to convert the polishing scene image data into visual feature vectors; S2.3, preprocess the contact interaction data between the robotic arm and the workpiece to construct the interaction feature vector; S2.4, based on the temporal feature vector obtained in step S2.1, the visual feature vector obtained in step S2.2, and the interaction feature vector obtained in step S2.3, the importance score of each modality data is calculated using MLP; S2.5, Based on the importance score obtained in step S2.4, the normalized weight of each vector is calculated using the softmax function, and the vectors are fused based on the obtained normalized weights to obtain modality fusion features.

3. The embodied intelligence robotic arm reinforcement learning polishing control method according to claim 1, characterized in that, In step S3, the formula for the reward function aimed at maximizing grinding quality, grinding efficiency, and workpiece safety is as follows: ； In the formula, , and All are weights. This indicates a safety constraint reward. This indicates a reward for the quality of the polishing process. This indicates a reward for improving polishing efficiency.

4. The embodied intelligence robotic arm reinforcement learning polishing control method according to claim 3, characterized in that, The security constraint reward The corresponding formula is: ； In the formula, Indicates the maximum joint torque. This represents the maximum contact force when the robotic arm comes into contact with the workpiece. This represents the contact force when the robotic arm comes into contact with the workpiece. This indicates the joint torque.

5. The embodied intelligence robotic arm reinforcement learning polishing control method according to claim 3, characterized in that, The polishing quality reward The corresponding formula is: ； In the formula, Indicates the maximum surface flatness. Indicates the maximum trajectory deviation. This indicates a reward for the quality of the polishing process. Indicates surface smoothness. This indicates a deviation in the trajectory.

6. The embodied intelligence robotic arm reinforcement learning polishing control method according to claim 3, characterized in that, The grinding efficiency reward The corresponding formula is: ； In the formula, Indicates the desired polishing speed. This indicates the actual grinding speed of the robotic arm.

7. The reinforcement learning-based polishing control method for a robotic arm with embodied intelligence according to claim 1, characterized in that, Step S4 includes the following steps: S4.1, Based on the desired joint angle obtained in step S3 and actual joint angle Calculate the position error; S4.2, Based on the position error obtained in step S4.1, the PD controller is used to generate the basic control quantity; S4.3, Based on the basic control quantity obtained in step S4.2 and the joint expectation data obtained in step S3, feedforward compensation is performed on the basic dynamic model to obtain the joint torque control model, and the robotic arm is controlled based on the joint torque control model.

8. A reinforcement learning-based polishing control system for a robotic arm with embodied intelligence, characterized in that, include: Data acquisition module: used to collect grinding status information when the robotic arm grinds the workpiece. The grinding status information includes grinding scene image data, contact interaction data between the robotic arm and the workpiece, and robotic arm operation status data. Feature fusion module: Used to process the grinding status information obtained from the data acquisition module using GRU and CNN networks to construct modality fusion features; Joint data optimization module: Based on the modal fusion features obtained by the feature fusion module, it generates joint expectation data using MLP with the goal of maximizing grinding quality, grinding efficiency and workpiece safety. The joint expectation data includes expected joint angles and expected joint torques. Robotic arm optimization module: It is used to perform feedforward compensation on the basic dynamic model based on the joint expectation data obtained by the joint data optimization module to obtain the joint torque control model, and then control the robotic arm according to the joint torque control model.

Images