Robot grinding and polishing parameter decision-making system based on deep reinforcement learning
The robot grinding and polishing parameter decision system, which utilizes deep reinforcement learning, solves the problem of autonomous optimization of grinding and polishing parameters, realizes intelligent control of the grinding and polishing process, improves the stability and efficiency of processing quality, reduces manual intervention, and adapts to different workpieces and working conditions.
Patent Information
- Application Number
- CN202511667723.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-14
- Publication Date
- 2026-03-20
AI Technical Summary
Existing automated grinding and polishing systems struggle to achieve autonomous optimization and efficient control of grinding and polishing parameters. In particular, when faced with multivariable coupling, strong nonlinearity, and time-varying characteristics, traditional methods are inefficient and unstable, making it difficult to adapt to the processing needs of different materials and complex curved surfaces.
A robot grinding and polishing parameter decision system based on deep reinforcement learning is adopted. Through a closed-loop optimization process consisting of a state perception module, a feature extraction and modeling module, a reinforcement learning decision module, a parameter execution and feedback module, and an experience playback and strategy update module, intelligent control of the grinding and polishing process is achieved.
The system is self-adaptive, automatically adapting to different grinding and polishing operation requirements, improving processing quality stability, reducing manual parameter adjustment, and is highly scalable. It balances real-time performance and safety, achieving highly consistent and high-precision automated processing.
Smart Images
Figure CN121696936A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of industrial automation and artificial intelligence control, in particular to a robot polishing parameter decision system based on deep reinforcement learning. BACKGROUND
[0002] In the automatic polishing system, the setting of polishing parameters (such as pressure, speed, angle, feed rate and abrasive type, etc.) directly affects the processing quality and production efficiency. The traditional process usually relies on manual experience, and the parameters are adjusted through repeated tests, which is not only low in efficiency and poor in stability, but also difficult to adapt to the diversified processing needs of different materials and complex surfaces.
[0003] Although some existing adaptive control systems (such as pressure regulation based on force control feedback) can achieve limited parameter correction, their control logic relies on predefined rules or static models, which is difficult to cope with the multivariable coupling, strong nonlinearity and time-varying characteristics in the polishing process.
[0004] With the development of artificial intelligence and reinforcement learning, the control method based on deep reinforcement learning (DRL) has shown superior performance in dynamic optimization problems. However, there is currently a lack of an intelligent decision system that can combine real-time feedback from the polishing process and dynamically adjust multiple parameters, enabling the polishing equipment to have self-learning and self-optimization capabilities to achieve high consistency, high precision and high efficiency of automatic processing.
[0005] Therefore, it is urgent to propose a polishing parameter decision system based on deep reinforcement learning to realize the intelligentization, autonomous optimization and optimal control of the polishing process. SUMMARY
[0006] The purpose of the present application is to provide a robot polishing parameter decision system based on deep reinforcement learning to solve the problems in the background art.
[0007] To achieve the above purpose, the present application provides the following technical solution: a robot polishing parameter decision system based on deep reinforcement learning, comprising a state perception module, a feature extraction and modeling module, a reinforcement learning decision module, a parameter execution and feedback module, an experience replay and policy update module, and an actuator.
[0008] The state perception module is output connected to the feature extraction and modeling module;
[0009] The feature extraction and modeling module is output connected to the reinforcement learning decision module;
[0010] The reinforcement learning decision module is output connected to the parameter execution and feedback module;
[0011] The parameter execution and feedback module is connected with a state perception module and an actuator;
[0012] The parameter execution and feedback module is connected with an experience replay and policy update module;
[0013] The experience replay and policy update module is reversely connected with a reinforcement learning and decision module, forming a closed-loop learning and optimization process;
[0014] The state perception module is used for real-time monitoring of multi-dimensional physical quantities and surface characteristics in the polishing process, and constructing a comprehensive representation of the process state;
[0015] The feature extraction and modeling module converts the multi-modal raw signals collected by the perception module into a high-dimensional state vector through data preprocessing, feature engineering, feature dimension reduction, and multi-modal fusion , providing input for reinforcement learning decision-making and outputting an action vector ;
[0016] The reinforcement learning and decision module realizes autonomous decision-making through a deep reinforcement learning algorithm based on policy gradient;
[0017] The parameter execution and feedback module real-time issues the polishing parameters output by the reinforcement learning module to the robot controller, and collects the result feedback for the next round of optimization;
[0018] The experience replay and policy update module stores the state, action, and reward data of each polishing process into an experience pool for subsequent policy training and adaptive optimization.
[0019] Preferably, the state perception module collects mechanical, thermal, surface quality, stress, and energy consumption characteristics, but is not limited to them.
[0020] Preferably, the reinforcement learning and decision module is based on a reinforcement learning algorithm of Actor-Critic architecture, with the state vector as input and the action output as polishing parameters. The reward function is constructed based on surface quality target indicators, surface quality consistency indicators, grinding removal amount, and polishing processing efficiency indicators, realizing optimal policy learning driven by quality and efficiency, and having both online training and offline simulation modes.
[0021] Preferably, the reinforcement learning and decision module algorithm selection adopts a PPO algorithm based on policy gradient, with the state vector including the following characteristic quantities:
[0022] Surface roughness change: reflects the processing quality;
[0023] Polishing force average and fluctuation: reflects the contact stability;
[0024] End effector velocity: reflects dynamic response characteristics;
[0025] Abrasive wear degree: reflects the abrasive attenuation characteristics;
[0026] Energy consumption level: reflects energy efficiency performance;
[0027] Processing time characteristics: reflects the efficiency of the beat;
[0028] Removal amount: used to measure the efficiency of removing material.
[0029] Preferably, the action vector Including continuous control variables: polishing pressure, polishing speed, feed rate, tool angle and abrasive type, realizing abrasive autonomous selection and adaptive adjustment of processing parameters.
[0030] Preferably, the reward function considers factors such as processing quality, efficiency, energy consumption and consistency, and defines the reward function as follows:
[0031] R t =w1·(−∣R atarget −R areal ∣)+w2·(−F fluct )+w3·(−E energy )+w4·(−C var )
[0032] Where:
[0033] R areal : current surface roughness;
[0034] R atarget : target surface roughness, specified by workpiece type or user demand, so that the strategy can be dynamically optimized for different workpiece demands;
[0035] F fluct : polishing force fluctuation index;
[0036] E energy : energy consumption index;
[0037] C var : processing consistency index, calculated by the variance or normalized fluctuation degree of removal amount;
[0038] W1, W2, W3, W4: adjustable weight coefficients, used to balance the influence of each performance index.
[0039] Preferably, the parameter execution and feedback module receives the control parameters output by the reinforcement learning decision module, realizes real-time polishing control through the robot controller interface, the control instructions include speed adjustment, pressure correction, path offset correction, supports force control feedback loop, and realizes flexible polishing.
[0040] Preferably, the experience replay and policy update module is used to store the state-action-reward sequence in the polishing process, periodically resample the experience pool samples and update the policy, improve the learning efficiency and robustness, support the safety constraint training based on model predictive control, and prevent over-polishing.
[0041] Preferably, the experience replay and policy update module is responsible for optimizing the reinforcement learning model according to the interaction data stored in the experience replay pool, to continuously improve the adaptability and robustness of the polishing strategy. Its main features include Actor-Critic network optimization, experience replay and batch update, adaptive learning rate and policy stability, multi-objective and multi-workpiece demand adaptability, and policy iteration and update.
[0042] The present application has at least the following advantages:
[0043] (1) Strong adaptability: the system automatically adapts to different polishing operation requirements, different workpieces, and different working conditions of different wear states through deep reinforcement learning;
[0044] (2) High quality stability: closed-loop control is formed through surface quality feedback, significantly reducing uneven polishing, over-polishing or under-polishing phenomena;
[0045] (3) No need for manual parameter adjustment: the system can automatically learn the optimal parameters after initial setting, reducing human involvement;
[0046] (4) Strong scalability: the algorithm model and robot hardware are decoupled, and can be ported to different brands of industrial robots;
[0047] (5) Real-time and safety: the system realizes real-time optimization and stable policy update through parallel computing and experience replay mechanism. BRIEF DESCRIPTION OF DRAWINGS
[0048] Figure 1 is a schematic diagram of the overall module structure of the present application;
[0049] Figure 2 is a schematic diagram of the feature extraction and modeling module process of the present application. DETAILED DESCRIPTION
[0050] The technical solutions in the embodiments of the present application will be described below in a clear and complete manner. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.
[0051] EMBODIMENT
[0052] The embodiment provides a robot polishing parameter decision system based on deep reinforcement learning, which is used for realizing planning, adaptive adjustment and optimal control of polishing parameters under complex working conditions.
[0053] The system comprises the following modules:
[0054] ① State perception module: the state perception module is used for monitoring multi-dimensional physical quantities and surface characteristics in the polishing process in real time, and constructing comprehensive characterization of the process state. The module collects multi-source signals including mechanics, kinematics, thermology and vision through a multi-modal sensor system, and realizes dynamic perception of the polishing process. The main perception contents are:
[0055] Mechanical characteristics: used for characterizing polishing force, normal pressure and contact state; polishing force and normal pressure are obtained through force / torque sensors and robot joint force control units, and are used for reflecting the polishing contact state;
[0056] Energy consumption and impedance characteristics: the change of grinding load is reflected through current and vibration feedback; the real-time change of motor current and the change of polishing load are obtained through the IO signal of the end effector, and when the current or load instantaneously rises or fluctuates significantly, it usually indicates that the tool or grinding head encounters an abrupt slope, edge or local high point area;
[0057] Thermal characteristics: the temperature rise and temperature gradient of the polishing area are monitored to evaluate the surface damage risk and the effectiveness of abrasive grinding; the temperature rise and distribution gradient of the polishing area are monitored through temperature sensors, the friction strength between the abrasive and the workpiece and the surface thermal damage risk are reflected, and whether the workpiece surface is in the over-grinding or ablation risk interval is judged, so as to guide the dynamic adjustment of the polishing process;
[0058] Surface quality characteristics: described by roughness, glossiness and grinding mark consistency, the material removal and surface forming effect; the roughness reflects the material removal effect, and the glossiness evaluates the surface reflection characteristics and the final appearance quality;
[0059] Stress and wear characteristics: used for reflecting the residual stress of the workpiece surface and the abrasive wear state; a residual stress analyzer is used to obtain the stress distribution information of the workpiece surface, combined with the grinding head wear amount and the processing force fluctuation, the abrasive failure state and the mechanical properties of the workpiece can be evaluated, and for specific process requirements, the system can also maintain moderate residual stress through parameter optimization to improve the surface strengthening effect.
[0060] The module supports multi-modal data synchronous acquisition, realizes real-time transmission and fusion of multi-source signals through communication with the upper computer; the system applies a unified synchronous signal at the hardware level, calibrates the time base of the sampling clock of each sensor, thereby ensuring the timing consistency and data alignment accuracy of low-level signals such as force, temperature and current; through this mechanism, it can ensure that multi-modal data strictly corresponds to the processing state change in the time domain, and provide a reliable data basis for subsequent feature extraction and intelligent modeling.
[0061] ②Feature extraction and modeling module: the feature extraction and modeling module converts the multi-modal original signals collected by the perception module into high-dimensional state vectors by data preprocessing, feature engineering, feature dimension reduction and multi-modal fusion, providing input for reinforcement learning decision-making and output action vector ;
[0062] The multi-modal original signals collected during the polishing process are preprocessed and features are extracted to generate high-dimensional state vectors describing the current polishing state for the reinforcement learning module; the main steps include:
[0063] a. Data preprocessing: filter, denoise and normalize the collected multi-modal signals (mechanical, kinematic, thermal, visual, stress and end IO signals), and based on the unified time synchronization mechanism applied by the perception module, perform time domain alignment and synchronization on multi-source sensor data;
[0064] b. Feature engineering: extract statistical and frequency domain features (such as mean, variance, RMS, dominant frequency, spectral energy, etc.) from low-level signal features (force, current and temperature, etc.) to describe the dynamic stability and energy characteristics of the polishing process; use high-level process features (roughness, gloss, removal amount and residual stress, etc. feedback by multi-modal sensors) to represent the workpiece surface state and processing effect;
[0065] c. Representation learning and dimension reduction: reduce the dimension and feature representation of multi-modal features through principal component analysis (PCA) and one-dimensional convolutional neural network (1D-CNN), so that the system can automatically extract key feature components and generate compact state vectors to describe the polishing process state.
[0066] d. Multi-modal fusion: fuse the features of different sensor channels to generate the final state vector , which provides input for reinforcement learning decision-making, and the process is shown in Figure 2 .
[0067] ③ Reinforcement Learning Decision Module: autonomous decision-making is achieved through a deep reinforcement learning algorithm based on policy gradient (Proximal Policy Optimization, PPO); system state vector includes multi-modal sensor information and workpiece surface quality target indicators (such as roughness target, gloss requirement); reinforcement learning module with comprehensive reward signal as a basis, considering factors such as surface roughness error, force control stability, energy consumption, material removal consistency, and abrasive selection effect, etc.
[0068] Reinforcement learning algorithm based on Actor-Critic architecture, state vector as input, action output as polishing parameters (pressure, speed, angle, feed rate and abrasive type, etc.), reward function based on surface quality target indicators, surface quality consistency indicators, grinding removal amount and polishing processing efficiency, etc. to achieve optimal policy learning driven by quality and efficiency, with online training and offline simulation modes;
[0069] a. Algorithm selection: PPO algorithm based on policy gradient is adopted;
[0070] b. State space: state vector includes the following characteristic quantities:
[0071] Surface roughness variation: reflects processing quality;
[0072] Grinding and polishing force mean and fluctuation: reflects contact stability;
[0073] End effector speed: reflects dynamic response characteristics;
[0074] Abrasive wear degree: reflects the attenuation characteristics of abrasives;
[0075] Energy consumption level: reflects energy efficiency performance;
[0076] Processing time characteristics: reflects beat efficiency;
[0077] Removal amount: used to measure the efficiency of material removal;
[0078] Surface quality target indicators (optional state): such as target roughness or gloss, used to support directional optimization for different workpiece requirements;
[0079] c. Action space: action vector includes continuous control variables: polishing pressure, polishing speed, feed rate, tool angle and abrasive type, realizing autonomous selection of abrasives and adaptive adjustment of processing parameters
[0080] d. Reward function: considering factors such as processing quality, efficiency, energy consumption and consistency, the reward function is defined as follows:
[0081] R t = w1 · ( - |R atarget - R areal | ) + w2 · ( - F fluct ) + w3 · ( - E energy ) + w4 · ( - C var )
[0082] wherein:
[0083] R areal : current surface roughness;
[0084] R atarget : target surface roughness, specified by workpiece type or user requirement, enabling the strategy to dynamically optimize for different workpiece requirements;
[0085] F fluct : polishing force fluctuation index;
[0086] E energy : energy consumption index;
[0087] C var : machining consistency index, calculated from the variance or normalized fluctuation degree of removal amount;
[0088] W1, W2, W3, W4: adjustable weight coefficients, used to balance the influence of each performance index.
[0089] ④ Parameter execution and feedback module: real-time issue polishing parameters (such as pressure, speed, angle, abrasive replacement strategy, etc.) output by the reinforcement learning module to the robot controller, and collect result feedback for the next round of optimization;
[0090] Receive the control parameters output by the reinforcement learning decision module, realize real-time polishing control through the robot controller interface, the control instructions include speed adjustment, pressure correction, path offset correction, support force control feedback loop, realize flexible polishing.
[0091] ⑤ Experience replay and strategy update module: the system stores the state, action, reward data of each polishing process into the experience pool for subsequent strategy training and adaptive optimization;
[0092] Used to store the state-action-reward sequence in the polishing process, periodically resample and update the strategy of the experience pool samples, improve the learning efficiency and robustness, support safety constraint training based on model predictive control (Model Predictive Control, MPC), prevent over-polishing;
[0093] Responsible for optimizing the reinforcement learning model according to the interaction data stored in the experience replay pool to continuously improve the adaptability and robustness of the polishing strategy, its main features include:
[0094] a. Actor-Critic network optimization: based on gradient descent method, respectively optimize Actor policy network and Critic value network; Actor network outputs continuous control action, including polishing force, speed, tool angle, feed rate and abrasive type, realizes the autonomous selection of parameters and abrasive; Critic network evaluates the value of the current policy, provides gradient information for policy update;
[0095] b. Experience replay and batch update: sample state-action-reward-next state sequence from experience replay pool for training; support random resampling and prioritized sampling (Prioritized Experience Replay), enhance the learning ability of key states and abnormal working conditions; , , , ) for training; support random resampling and prioritized sampling (Prioritized Experience Replay), enhance the learning ability of key states and abnormal working conditions;
[0096] c. Adaptive learning rate and policy stability: adaptively adjust the learning rate according to the policy change, ensure the convergence stability of the update process, and avoid overfitting to a single workpiece type;
[0097] d. Multi-objective and multi-workpiece demand adaptability: the model can dynamically optimize abrasive selection and polishing parameters according to different surface quality targets of workpieces, realize adaptive control of different workpiece types and processing requirements; In the multi-objective training stage, the system continuously explores and accumulates experience, so that the strategy can quickly generate optimal control actions when facing different roughness, glossiness or removal amount requirements;
[0098] e. Policy iteration and update: in the online running stage, the system continuously adds the latest interaction data to the experience pool and periodically updates the policy, realizes abrasive autonomous selection, processing parameter adaptation and comprehensive optimization of surface quality, processing efficiency and energy consumption; Through this mechanism, the reinforcement learning model can gradually adapt to new workpiece types and processing targets, improve the intelligent level and generalization ability of the system.
[0099] This embodiment realizes the self-learning and self-optimization of polishing parameters through reinforcement learning algorithm, so that the robot can automatically converge to the optimal parameter configuration under different workpieces and environments. Its online self-learning process is as follows:
[0100] ① Initialize the policy model
[0101] The system first trains the policy model in a digital simulation environment. By importing a historical polishing dataset (including force, speed, surface roughness, temperature change, etc.), the PPO algorithm is used to pre-train the Actor and Critic networks, and an initial policy model with basic control capabilities is obtained.
[0102] The model parameters (weights, biases, normalization factors, etc.) are saved in the industrial PC, and this initial policy is loaded as the starting point during online operation. During the initialization phase, key hyperparameters are also set, including learning rate η, discount factor γ, advantage estimation λ, exploration noise σ, etc., to balance policy exploration and convergence stability.
[0103] ②Collecting multi-dimensional signals and surface feedback
[0104] After the system enters online operation, the multi-modal sensor group collects multi-dimensional information in real time during the polishing process. The sampling period can be set to 1-10 ms according to the control accuracy requirements, and the system uses a time synchronization mechanism to ensure the time consistency of multi-channel data. After the original signals are filtered, denoised and normalized by the preprocessing module, they are converted into a high-dimensional state vector that describes the current polishing state, and input into the reinforcement learning decision module for policy calculation.
[0105] In addition, after each polishing cycle, the detection module or personnel assesses the workpiece surface to obtain quality indicators such as surface roughness (R a , R z ), gloss, polishing mark consistency, and residual stress. The above quality feedback serves as the basis for evaluating the reinforcement learning reward function, guiding the optimization and update of subsequent policies.
[0106] ③Decision module generates polishing parameters
[0107] The reinforcement learning model receives the current state vector , calculates the output action through the policy network (Actor Network), which includes continuous and discrete control variables to generate a complete set of polishing parameters:
[0108] ;
[0109] Wherein, the meanings of each parameter are as follows:
[0110] : Polishing speed;
[0111] : Normal force;
[0112] : Tool contact method;
[0113] : feed rate;
[0114] : abrasive type (discrete variable, for autonomous abrasive selection);
[0115] To ensure the feasibility and safety of parameters, the system introduces a parameter constraint mapping function outside the action space:
[0116] ;
[0117] where and are defined by the physical limits of the device or process specifications. For the abrasive type
[0118] To enhance the exploration ability of the strategy, the system can superimpose Gaussian noise on continuous variables during the training phase , and use Softmax sampling for abrasive selection to balance exploration and utilization. The final parameter set is transmitted to the robot control module in real time through the communication interface, realizing adaptive adjustment and online optimization of polishing speed, attitude, and abrasive type.
[0119] ④ Execute the polishing operation and record the feedback
[0120] The robot control module drives the end effector to complete real-time polishing operation according to the optimal polishing parameters output by the reinforcement learning decision model. The execution stage includes force control and path control: the force control unit maintains the polishing contact force within the target pressure range, and compensates for the force fluctuations caused by changes in workpiece curvature or wear in real time; the path control unit adjusts the robot motion trajectory dynamically according to the correction amount output by the decision module to ensure high consistency between the processing path and the target surface topography.
[0121] During the polishing operation, the system continuously records multi-modal state quantities and control signals, including but not limited to force / torque curves, end speed, contact temperature, vibration characteristics, end effector current changes, and surface roughness and glossiness changes. After each polishing cycle, the system organizes the corresponding state, action, reward, and next state data into a complete interaction sample , , , ).
[0122] All interaction samples are stored in the experience replay buffer to support subsequent policy updates and model optimization. This mechanism ensures that the system has adaptive learning ability in continuous polishing tasks, realizing a closed-loop reinforcement learning process that continuously improves the control strategy from actual operation data.
[0123] ⑤Reward calculation and experience storage
[0124] After each polishing cycle, the system calculates the immediate reward based on the multi-dimensional quality feedback (definition method see the above description). The reward function takes into account factors such as surface quality, quality consistency, material removal amount, efficiency, and energy consumption level.
[0125] The system normalizes and clips the calculated reward value to avoid excessive reward gradient during training. Then, the reward value and the corresponding state, action, and next state , , , are stored in the experience replay buffer for subsequent policy update module calls. This mechanism ensures stable reinforcement learning training based on multi-dimensional performance feedback and continuously improves the adaptive ability of the polishing strategy.
[0126] ⑥Periodic policy update
[0127] The system triggers the policy update process every N polishing cycles (or reward convergence threshold). Randomly sample several batches of samples from the experience pool, calculate the advantage function (Advantage Function) using the PPO algorithm, and update the Actor and Critic network parameters based on it.
[0128] During the update process, the clipping objective function is used to prevent policy overfitting and ensure update stability. The clipping objective function is defined as follows:
[0129] L clip = E t [min(r t (θ)A t , clip(r t (θ), 1−ε, 1+ε)A t )]
[0130] After the update is complete, the new policy model immediately replaces the running control strategy, achieving adaptive optimization of polishing parameters. The system simultaneously monitors training loss and reward curves, and if the reward does not increase or fluctuates excessively, it automatically adjusts the learning rate η or stops updating to prevent performance degradation.
[0131] The safety control of the scheme example is as follows:
[0132] To ensure the safety of the system during the reinforcement learning phase, the invention designs a multi-layer protection mechanism:
[0133] 1. Force and temperature monitoring dual-channel protection: When the force value is detected to be out of limit (e.g. exceeding the set threshold ±10%) or the temperature gradient is abnormal (e.g. temperature rising rate >5℃ / s), the system immediately triggers an interrupt signal to suspend the strategy update and control command sending;
[0134] 2. Surface quality anomaly detection: If the detection module identifies that the surface roughness deviates from the target value by more than the tolerance range (e.g. ±15%), the system automatically switches to the safety mode and uses fixed safety parameters (v safe , F safe , θ safe ) to maintain light load grinding and polishing;
[0135] 3. Abnormal data recording and retraining: All state sequences triggering protection are marked as "abnormal samples" and stored in an independent data set for subsequent model retraining and strategy correction.
[0136] In addition, the online control based on the reliable model is as follows:
[0137] When the reinforcement learning model converges to a stable strategy (i.e. the reward function tends to be stable and the strategy variance is lower than the set threshold) after multiple online training, the system enters the "reliable model online control phase".
[0138] At this stage, the model has the generalization ability to different working conditions and can be used as a high-precision parameter planner.
[0139] The control logic is as follows:
[0140] 1. Before starting each new task, the system loads the converged strategy model;
[0141] 2. According to the real-time sensor input, the model directly outputs the grinding and polishing parameters without strategy update;
[0142] 3. The control parameters are sent to the actuator through the robot controller in real time to complete efficient grinding and polishing;
[0143] 4. At the same time, the monitoring and anomaly detection modules are still active to prevent sudden working condition deviation.
[0144] Through the above design, the invention not only realizes online self-optimization driven by reinforcement learning, but also guarantees the safety and controllability of the system in long-term operation, realizing the closed-loop transition from "exploration learning" to "stable autonomous control".
[0145] The foregoing shows and describes the basic principles, main features and advantages of the present application. It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present application.
[0146] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.
Claims
1. A robot grinding and polishing parameter decision system based on deep reinforcement learning, characterized in that, It includes a state awareness module, a feature extraction and modeling module, a reinforcement learning decision-making module, a parameter execution and feedback module, an experience playback and policy update module, and an executor; The state perception module outputs a connection to the feature extraction and modeling module; The output of the feature extraction and modeling module is connected to the reinforcement learning decision module; The output of the reinforcement learning decision module is connected to the parameter execution and feedback module; The parameter execution and feedback module outputs a connection to the state perception module and the actuator. The parameter execution and feedback module outputs a connection to the experience playback and strategy update module. The experience playback and strategy update module outputs a reverse connection to the reinforcement learning and decision-making module, forming a closed-loop learning and optimization process. The state perception module is used to monitor multidimensional physical quantities and surface features in real time during the grinding and polishing process, and to construct a comprehensive characterization of the process state. The feature extraction and modeling module converts the multimodal raw signals acquired by the perception module into high-dimensional state vectors through data preprocessing, feature engineering, feature dimensionality reduction, and multimodal fusion. It provides input for reinforcement learning decision-making and outputs action vectors. ; The reinforcement learning decision-making module achieves autonomous decision-making through a deep reinforcement learning algorithm based on policy gradients; The parameter execution and feedback module sends the grinding and polishing parameters output by the reinforcement learning module to the robot controller in real time, and collects the results for the next round of optimization. The experience playback and strategy update module stores the status, actions, and reward data of each polishing process into the experience pool for subsequent strategy training and adaptive optimization.
2. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 1, characterized in that, The state sensing module collects, but is not limited to, mechanical, thermal, surface quality, stress, and energy consumption characteristics.
3. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 1, characterized in that, The reinforcement learning decision module is based on the Actor-Critic architecture reinforcement learning algorithm. The state vector is used as input, the action output is the grinding and polishing parameters, and the reward function is constructed based on the surface quality target index, surface quality consistency index, grinding removal amount and grinding and polishing efficiency index. It realizes the optimal strategy learning driven by quality and efficiency and has both online training and offline simulation modes.
4. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 3, characterized in that, The reinforcement learning decision module algorithm selection adopts the policy gradient-based PPO algorithm, with state vector... Including the following characteristics: Surface roughness variation: reflects processing quality; Average and fluctuation of polishing force: reflects contact stability; End effector speed: reflects dynamic response characteristics; Abrasive wear degree: reflects the attenuation characteristics of abrasive; Energy consumption level: reflects energy efficiency performance; Processing time characteristics: reflect cycle time efficiency; Removal rate: Used to measure the efficiency of material removal.
5. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 3, characterized in that, The action vector It includes continuously controllable variables: grinding and polishing pressure, grinding and polishing speed, feed rate, tool angle, and abrasive type, enabling autonomous abrasive selection and adaptive adjustment of processing parameters.
6. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 1, characterized in that, The reward function comprehensively considers factors such as processing quality, efficiency, energy consumption, and consistency, and is defined as follows: R t =w1·(−∣R atarget −R areal ∣)+w2·(−F fluct )+w3·(−E energy )+w4·(−C var ) in: R areal Current surface roughness; R atarget Target surface roughness, specified by workpiece type or user requirements, enables the strategy to be dynamically optimized for different workpiece requirements; F fluct Polishing force fluctuation index; E energy Energy consumption indicators; C var Processing consistency index, calculated from the variance or normalized fluctuation of the amount removed; W1, W2, W3, W4: Adjustable weighting coefficients used to balance the impact of various performance indicators.
7. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 1, characterized in that, The parameter execution and feedback module receives control parameters output by the reinforcement learning decision module and realizes real-time grinding and polishing control through the robot controller interface. The control commands include speed adjustment, pressure correction, and path offset correction, and support force control feedback closed loop to achieve flexible grinding and polishing.
8. The robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 1, characterized in that, The experience playback and strategy update module is used to store the state-action-reward sequence during the polishing process, periodically resample the experience pool samples and update the strategy to improve learning efficiency and robustness, support safety constraint training based on model prediction control, and prevent over-polishing.
9. A robot grinding and polishing parameter decision system based on deep reinforcement learning according to claim 1, characterized in that, The experience replay and strategy update module is responsible for optimizing the reinforcement learning model based on the interaction data stored in the experience replay pool, so as to continuously improve the adaptability and robustness of the polishing strategy. Its main features include Actor-Critic network optimization, experience replay and batch update, adaptive learning rate and strategy stability, adaptability to multi-objective and multi-workpiece requirements, and strategy iteration and update.