A reinforcement learning-based intelligent optimization method and system for robot arm grasping

Abstract

The application discloses a kind of mechanical arm grabbing intelligent optimization method and system based on reinforcement learning, specifically relates to mechanical control technical field, for solving the problem that existing mechanical arm is difficult in strategy convergence when operating dynamic deformation object and action safety is insufficient;Dynamic deformation state vector of fusion deformation energy parameter and joint motion state is constructed by real-time acquisition target object dynamic deformation characteristic data and mechanical arm pose information;Quantify deformation energy accumulation risk by combining historical deformation recovery characteristics and real-time deformation rate correlation analysis dynamic correction state vector;Control instruction set of grabbing force is generated using reinforcement learning strategy network, and the safety limiting of force component is realized through jacobian matrix gradient analysis;Based on deformation energy trend and force stability Dynamic adjustment reward function weight, closed-loop iterative optimization until meeting multi-objective convergence condition;Realize the accurate state representation and safe grabbing control of dynamic deformation object.

Description

Technical Field

[0001] This invention relates to the field of mechanical control technology, and more specifically, to a method and system for intelligent optimization of robotic arm grasping based on reinforcement learning. Background Technology

[0002] The development of industrial automation and intelligent robot technology has enabled robotic arms to gradually extend their grasping tasks from structured scenarios to complex unstructured scenarios. In practical applications such as logistics sorting and medical assistance, robotic arms often need to manipulate flexible objects with dynamic deformation characteristics (such as soft packaging materials and biological tissues). Due to the material properties, flexible objects are prone to non-rigid deformation during grasping, making it difficult to stably perceive and model their key state parameters such as pose and contact force. Traditional grasping control methods usually rely on preset rules or static environment assumptions, which are difficult to adapt to the nonlinear response characteristics of dynamically deformed objects, thus limiting the autonomy and robustness of robotic arms in complex scenarios.

[0003] Existing robotic arm grasping methods suffer from a significant problem when dealing with dynamically deformable objects. The increased dimensionality of the state space caused by the deformation and the uncertainty of the physical response lead to sparsity of the reward signal during policy training. Specifically, reinforcement learning models struggle to effectively link the complex coupling relationship between the grasping action and the changes in the deformed object's state through trial and error mechanisms. This causes the policy to converge to a local optimum, making it impossible to generate a stable and efficient grasping trajectory. Consequently, the practical application of robotic arms in tasks involving the manipulation of flexible objects is limited. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a reinforcement learning-based intelligent optimization method and system for robotic arm grasping to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A reinforcement learning-based intelligent optimization method for robotic arm grasping includes the following steps:

[0007] S1. Acquire dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm end effector;

[0008] S2. Generate deformation energy parameters based on dynamic deformation characteristic data and real-time pose information, and combine them with the joint motion parameters of the robotic arm to perform joint encoding to construct a dynamic deformation state vector.

[0009] S3. Determine the correction coefficient based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector.

[0010] S4. Input the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a motion control instruction set including the grasping force component.

[0011] S5. Based on the Jacobian matrix of the dynamic deformation state vector and motion control instruction set, the amplitude of the grasping force component is limited.

[0012] S6. Based on the changing trend of deformation energy parameters and the grasping force component after amplitude limiting processing, dynamically adjust the weight allocation coefficient of the reward function and iteratively optimize the motion control instruction set until the preset convergence condition is met.

[0013] In a preferred embodiment, S1 includes:

[0014] S1-1. Obtain the deformation displacement distribution of the contact area on the surface of the target object through a visual sensor, and obtain the real-time deformation rate distribution corresponding to the deformation displacement distribution through a tactile sensor.

[0015] S1-2. Obtain the real-time position coordinates of the end effector through the joint encoder of the robotic arm, and obtain the real-time attitude angle of the end effector through the inertial measurement unit;

[0016] S1-3. Synchronously align the deformation displacement distribution, real-time deformation rate distribution, real-time position coordinates, and real-time attitude angles according to a preset time window to generate dynamic deformation characteristic data and real-time pose information.

[0017] In a preferred embodiment, S2 includes:

[0018] S2-1. Based on the displacement of each data point in the deformation displacement distribution, generate the corresponding local deformation elastic potential energy parameters using the elastic potential energy calculation formula.

[0019] S2-2. Based on the rate value of each data point in the real-time deformation rate distribution and the displacement of the same data point in the corresponding deformation displacement distribution, the local deformation kinetic energy parameters are generated through the nonlinear damping correction formula.

[0020] S2-3. Based on the statistical variance of the real-time deformation rate distribution, dynamically adjust the superposition weighting coefficient of the local deformation elastic potential energy parameter and the local deformation dynamic energy parameter to obtain the deformation energy parameter.

[0021] S2-4. Map the deformation energy parameters, real-time position coordinates and joint torques in the robot arm joint motion parameters to the same vector space according to the preset coding rules, and construct the dynamic deformation state vector through normalization processing.

[0022] In a preferred embodiment, S3 includes:

[0023] S3-1. Calculate the historical deformation recovery rate distribution based on the difference in deformation displacement distribution between adjacent time windows in the historical deformation recovery data, wherein the historical deformation recovery data is the aligned deformation displacement distribution sequence within the preset time window in step S1.

[0024] S3-2. Calculate the ratio of the real-time deformation rate distribution to the historical deformation recovery rate distribution at the same data point location to generate the deformation rate recovery ratio distribution.

[0025] S3-3. Based on the relationship between the ratio of each data point in the deformation rate recovery ratio distribution and the preset recovery ratio threshold, generate the correction coefficient of the dynamic deformation state vector, where the preset recovery ratio threshold is calibrated based on the stress relaxation characteristics of the target material.

[0026] S3-4. Multiply the correction coefficient with the deformation energy parameter of the corresponding data point in the dynamic deformation state vector to obtain the corrected dynamic deformation state vector.

[0027] In a preferred embodiment, S4 includes:

[0028] S4-1. Expand the dimension of the corrected dynamic deformation state vector, convert the real-time position coordinates and real-time attitude angles into polar coordinate system parameters, and concatenate them with the deformation energy parameters to form an expanded state vector.

[0029] S4-2. Input the extended state vector into the multi-head attention layer of the reinforcement learning policy network, and generate the initial weight distribution of the grasping force component based on the preset action space decoupling rules;

[0030] S4-3. The initial weight distribution of the grasping force component is non-linearly activated by an independent fully connected layer to generate an independent action probability distribution for each component.

[0031] S4-4. Based on the maximum drive threshold of the robotic arm joint motion parameters, perform inverse normalization on the probability distribution of independent actions to generate a motion control instruction set including the gripping force component.

[0032] In a preferred embodiment, an initial weight distribution of translational displacement components and rotational angle components is generated based on a preset motion space decoupling rule; the initial weight distribution of translational displacement components and rotational angle components is further subjected to nonlinear activation processing; the motion control instruction set also includes translational displacement components and rotational angle components.

[0033] In a preferred embodiment, S5 includes:

[0034] S5-1. Calculate the Jacobian matrix of the end effector pose to the joint angle based on the real-time position coordinates and joint torque in the joint motion parameters of the robotic arm.

[0035] S5-2. Based on the distribution of deformation energy parameters in the dynamic deformation state vector, identify deformation-sensitive regions where the deformation energy parameters exceed a preset energy threshold.

[0036] S5-3. Perform a dot product operation between the Jacobian matrix and the deformation energy parameter gradient of the deformation-sensitive region to generate the spatial gradient influence value of the gripping force component on the deformation energy parameter.

[0037] S5-4. Calculate the amplitude limiting threshold of the grasping force component based on the ratio of the spatial gradient influence value to the preset safety factor, where the preset safety factor is calibrated based on the maximum deformation bearing capacity of the target object material.

[0038] S5-5. Truncate the values ​​in the grasping force component that exceed the amplitude limit threshold, and update the truncated grasping force component to the motion control instruction set.

[0039] In a preferred embodiment, S6 includes:

[0040] S6-1. Calculate the exponentially weighted moving average of the deformation energy accumulation rate based on the temporal change rate of the deformation energy parameter in the dynamic deformation state vector.

[0041] S6-2. Based on the variance of the grasping force component after amplitude limiting processing, generate a stability attenuation coefficient that characterizes the fluctuation of grasping stability.

[0042] S6-3. Input the exponentially weighted moving average of the deformation energy accumulation rate and the stability decay coefficient into the preset weight allocation function, and dynamically adjust the weight allocation coefficients of the deformation energy accumulation term and the grasping stability term in the reward function.

[0043] S6-4. Recalculate the reward function value based on the updated weight allocation coefficients, and iteratively optimize the motion control instruction set through the policy gradient algorithm until the joint convergence index of the deformation energy parameter and the grasping force component meets the preset convergence condition.

[0044] On the other hand, the present invention provides a reinforcement learning-based intelligent optimization system for robotic arm grasping, comprising:

[0045] Deformation and pose acquisition module: acquires dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm end effector;

[0046] Energy state encoding module: Generates deformation energy parameters based on dynamic deformation characteristic data and real-time pose information, and performs joint encoding with the joint motion parameters of the robotic arm to construct a dynamic deformation state vector;

[0047] Dynamic state correction module: Determines correction coefficients based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector;

[0048] The strategy instruction generation module inputs the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a set of motion control instructions including the grasping force component.

[0049] Force limiting control module: Based on the Jacobian matrix of dynamic deformation state vector and motion control instruction set, it performs amplitude limiting processing on the grasping force component;

[0050] The dynamic weight optimization module dynamically adjusts the weight allocation coefficient of the reward function based on the changing trend of the deformation energy parameters and the grasping force component after amplitude limiting, and iteratively optimizes the motion control instruction set until the preset convergence condition is met.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. By employing multi-dimensional state representation and a closed-loop feedback mechanism, the autonomous decision-making ability and stability of the robotic arm in grasping dynamically deformable objects are effectively improved. Addressing the nonlinear characteristics of the deformation process of flexible objects, deformation energy parameters are introduced as core state indicators. Combined with the joint encoding mechanism of the robotic arm's joint motion parameters, the complex deformation state and the robotic arm's motion space are mapped into a high-dimensional dynamic vector, significantly enhancing the reinforcement learning model's understanding of the coupling relationship between deformation and motion. By real-time correction of the deformation state vector and dynamic adjustment of the reward function weights, the system can adaptively balance deformation energy control and grasping stability objectives, avoiding the local convergence problem caused by sparse reward signals in traditional methods. Simultaneously, it suppresses excessive interference of the robotic arm's motion on the object's deformation, ensuring efficient execution of the grasping process within a safe threshold.

[0053] 2. Through mechanical gradient analysis and hierarchical optimization strategies, a complete closed-loop logic from perception to control was constructed; based on the force limiting mechanism of the Jacobian matrix, the gradient influence of the grasping action on the deformation energy was accurately quantified, and the force component was safely truncated in real time; the multi-objective dynamic weight allocation and iterative optimization mechanism endowed the system with the ability to adaptively adjust to complex dynamic scenarios, which not only reduced the uncertainty of physical response in the operation of dynamically deformed objects, but also improved the task generalization of the robotic arm in unstructured environments. Attached Figure Description

[0054] Figure 1 This is a flowchart of a reinforcement learning-based intelligent optimization method for robotic arm grasping according to the present invention.

[0055] Figure 2 This is a schematic diagram of the structure of a reinforcement learning-based intelligent optimization system for robotic arm grasping according to the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0057] Example 1: Figure 1 This invention presents a reinforcement learning-based intelligent optimization method for robotic arm grasping, which includes the following steps:

[0058] S1. Acquire dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm end effector;

[0059] S2. Generate deformation energy parameters based on dynamic deformation characteristic data and real-time pose information, and combine them with the joint motion parameters of the robotic arm to perform joint encoding to construct a dynamic deformation state vector.

[0060] S3. Determine the correction coefficient based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector.

[0061] S4. Input the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a motion control instruction set including the grasping force component.

[0062] S5. Based on the Jacobian matrix of the dynamic deformation state vector and motion control instruction set, the amplitude of the grasping force component is limited.

[0063] S6. Based on the changing trend of deformation energy parameters and the grasping force component after amplitude limiting processing, dynamically adjust the weight allocation coefficient of the reward function and iteratively optimize the motion control instruction set until the preset convergence condition is met.

[0064] S1. Acquire dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm's end effector, including:

[0065] S1-1. Obtain the deformation displacement distribution of the contact area on the surface of the target object through a visual sensor, and obtain the real-time deformation rate distribution corresponding to the deformation displacement distribution through a tactile sensor.

[0066] S1-2. Obtain the real-time position coordinates of the end effector through the joint encoder of the robotic arm, and obtain the real-time attitude angle of the end effector through the inertial measurement unit;

[0067] S1-3. Synchronously align the deformation displacement distribution, real-time deformation rate distribution, real-time position coordinates, and real-time attitude angles according to a preset time window to generate dynamic deformation characteristic data and real-time pose information.

[0068] In step S1-1, the deformation displacement distribution of the contact area on the surface of the target object is acquired using a vision sensor. The vision sensor employs an RGB-D camera, whose depth information is used to capture the three-dimensional deformation displacement data of the object's surface. Specifically, preset feature points on the target object's surface are tracked, and the deformation displacement distribution is generated based on the changes in the three-dimensional coordinates of feature points in adjacent frames. Each data point in the deformation displacement distribution represents the displacement amount at the corresponding position in the contact area of ​​the object's surface, in millimeters. Simultaneously, a tactile sensor acquires the real-time deformation rate distribution corresponding to the deformation displacement distribution. The tactile sensor is a distributed pressure sensor array attached to the gripping surface of the robotic arm's end effector, measuring the dynamic pressure distribution changes in the object's contact area in real time, and calculating the real-time deformation rate distribution based on the rate of change of the pressure distribution over time.

[0069] Each data point in the real-time deformation rate distribution represents the pressure change at the corresponding location per unit time, measured in Pascals per second. For example, when an end effector contacts an inflatable packaging bag, a tactile sensor captures the process of the pressure in the contact area gradually increasing from an initial value to a stable value; the rate of change of this pressure is a component of the real-time deformation rate distribution.

[0070] In steps S1-2, the real-time position coordinates of the end effector are obtained through the robotic arm joint encoders. The robotic arm joint encoders are installed at each rotary joint and record the rotation angle of each joint in real time. Based on the forward kinematics model of the robotic arm, the three-dimensional coordinates of the end effector in the base coordinate system, i.e., the real-time position coordinates, are calculated in meters. The real-time position coordinates include components in the X, Y, and Z axes. Simultaneously, the real-time attitude angles of the end effector are obtained through an inertial measurement unit (IMU). The IMU is fixed to the end of the end effector and measures the rotation angles of the end effector around the X, Y, and Z axes in real time, i.e., roll, pitch, and yaw angles, in degrees.

[0071] Real-time attitude angles are used to characterize the spatial orientation of the end effector during the grasping process. For example, when the end effector approaches the target object, the roll angle output by the inertial measurement unit is 5 degrees, the pitch angle is -3 degrees, and the yaw angle is 0 degrees, indicating that the end effector is in a slightly tilted state.

[0072] In steps S1-3, the deformation displacement distribution, real-time deformation rate distribution, real-time position coordinates, and real-time attitude angle are synchronized and aligned according to a preset time window. The length of the preset time window is determined based on the maximum sampling frequency of each sensor. For example, if the sampling frequency of the vision sensor is 30 Hz, the sampling frequency of the tactile sensor is 100 Hz, the sampling frequency of the robotic arm joint encoder is 1 kHz, and the sampling frequency of the inertial measurement unit is 200 Hz, then the preset time window is set to 33 milliseconds to match the data cycle of the sensor with the lowest sampling frequency.

[0073] The specific method of synchronization alignment is as follows: taking the data acquisition time of the vision sensor as the reference timestamp, interpolating the data of the tactile sensor, the robotic arm joint encoder and the inertial measurement unit within a preset time window before and after the reference timestamp to generate time-aligned deformation displacement distribution, real-time deformation rate distribution, real-time position coordinates and real-time attitude angle.

[0074] The synchronized data is integrated into dynamic deformation characteristic data and real-time pose information. The dynamic deformation characteristic data includes deformation displacement distribution and real-time deformation rate distribution, while the real-time pose information includes real-time position coordinates and real-time attitude angles.

[0075] The above implementation process, through multi-sensor data fusion and time synchronization technology, ensures the spatiotemporal consistency of dynamic deformation characteristic data and real-time pose information, providing reliable input for subsequent deformation energy parameter generation and state vector construction. The combined use of visual and tactile sensors can simultaneously capture the geometric features and dynamic response characteristics of deformation, while the collaborative work of the robotic arm joint encoder and inertial measurement unit accurately quantifies the spatial pose state of the end effector. The time window synchronization mechanism solves the time misalignment problem caused by differences in the data acquisition frequencies of heterogeneous sensors. For example, high-frequency pressure data from the tactile sensor is aligned with low-frequency deformation data from the visual sensor through interpolation, avoiding control command oscillations caused by data asynchrony.

[0076] S2. Based on dynamic deformation characteristic data and real-time pose information, deformation energy parameters are generated and jointly encoded with the joint motion parameters of the robotic arm to construct a dynamic deformation state vector, including:

[0077] S2-1. Based on the displacement of each data point in the deformation displacement distribution, generate the corresponding local deformation elastic potential energy parameters using the elastic potential energy calculation formula.

[0078] S2-2. Based on the rate value of each data point in the real-time deformation rate distribution and the displacement of the same data point in the corresponding deformation displacement distribution, the local deformation kinetic energy parameters are generated through the nonlinear damping correction formula.

[0079] S2-3. Based on the statistical variance of the real-time deformation rate distribution, dynamically adjust the superposition weighting coefficient of the local deformation elastic potential energy parameter and the local deformation dynamic energy parameter to obtain the deformation energy parameter.

[0080] S2-4. Map the deformation energy parameters, real-time position coordinates and joint torques in the robot arm joint motion parameters to the same vector space according to the preset coding rules, and construct the dynamic deformation state vector through normalization processing.

[0081] In step S2-1, local deformation elastic potential energy parameters are generated based on the displacement of each data point in the deformation displacement distribution. The deformation displacement distribution originates from the three-dimensional displacement data of the contact area on the object surface acquired by the visual sensor in step S1-1. The elastic potential energy calculation formula is based on Hooke's Law. In practice, the displacement of each data point is multiplied by its corresponding material stiffness coefficient, the square is taken, and then multiplied by a preset elastic coefficient constant to obtain the local deformation elastic potential energy parameters. The material stiffness coefficient is pre-calibrated according to the material type of the target object. For example, when the target object is made of rubber, the material stiffness coefficient is set to a first preset value; when the target object is made of silicone, the material stiffness coefficient is set to a second preset value. The elastic coefficient constant is determined by fitting experimental data. For example, in the test scenario of grasping a soft packaging bag, the elastic coefficient constant is set to 0.5.

[0082] In step S2-2, local deformation kinetic energy parameters are generated based on the rate value of each data point in the real-time deformation rate distribution and the displacement of the corresponding data point in the deformation displacement distribution. The real-time deformation rate distribution originates from the pressure change rate data acquired by the tactile sensor in step S1-1. The specific implementation of the nonlinear damping correction formula is as follows: multiply the square of the rate value by a preset mass coefficient, and then multiply by a correction factor, where the correction factor is 1 minus the ratio of the displacement to the preset maximum displacement threshold.

[0083] The preset maximum displacement threshold is set based on the material properties of the target object; for example, when grasping an inflatable packaging bag, the maximum displacement threshold is set to 10 mm. The preset mass coefficient is calibrated experimentally; for example, in the same test scenario, the mass coefficient is set to 0.3. By introducing a correction factor related to the displacement, the kinetic energy calculation deviation in the large displacement region is suppressed.

[0084] In steps S2-3, the superposition weighting coefficients of the local deformation elastic potential energy parameter and the local deformation kinetic energy parameter are dynamically adjusted based on the statistical variance value of the real-time deformation rate distribution. The statistical variance value is calculated by performing variance calculations on the rate values ​​of all data points in the real-time deformation rate distribution to obtain the statistical variance value characterizing the degree of rate fluctuation. The dynamic adjustment rule for the superposition weighting coefficients is as follows: when the statistical variance value is greater than a preset variance threshold, the weighting coefficient of the local deformation elastic potential energy parameter is increased; when the statistical variance value is less than or equal to the preset variance threshold, the weighting coefficient of the local deformation kinetic energy parameter is increased. The preset variance threshold is determined based on historical data statistics; for example, in multiple capture tests, the variance threshold is set to 5 Pascals per second squared. The adjustment range of the superposition weighting coefficients is limited to 0.3 to 0.7 to ensure a balanced contribution from the two types of energy parameters.

[0085] In steps S2-4, the deformation energy parameters, real-time position coordinates, and joint torques in the robotic arm joint motion parameters are mapped to the same vector space. The specific implementation of the preset encoding rules is as follows: the X, Y, and Z axis components of the real-time position coordinates are converted to radial distance, azimuth angle, and elevation angle in polar coordinates, respectively; the joint torques are converted to per-unit values; and the deformation energy parameters retain their original scalar values. The normalization process is as follows: the radial distance, azimuth angle, and elevation angle in polar coordinates are normalized to their maximum and minimum values, and the per-unit joint torques and deformation energy parameters are standardized using Z-Score to eliminate dimensional differences. All normalized parameters are concatenated into a dynamic deformation state vector in a preset order, such as polar coordinate components, per-unit joint torques, and deformation energy parameters. The dimension of the dynamic deformation state vector is consistent with the number of parameters; for example, when the polar coordinate components contain 3 dimensions, the per-unit joint torques contain 6 dimensions, and the deformation energy parameters contain 1 dimension, the state vector is 10-dimensional.

[0086] The above implementation process constructs a high-dimensional vector that can simultaneously characterize the deformation energy properties of an object and the motion state of a robotic arm through staged energy parameter calculation, dynamic weight adjustment, and multi-source data fusion encoding. The separate calculation of elastic potential energy parameters and kinetic energy parameters avoids the linear deviation problem of energy indices in traditional methods, while the dynamic weight adjustment mechanism adaptively balances energy contributions based on the fluctuation characteristics of the deformation rate. Polar coordinate transformation and normalization ensure the comparability of heterogeneous data in the vector space; for example, the position coordinates of the robotic arm and the deformation energy parameters are on the same numerical order of magnitude after normalization, avoiding convergence difficulties caused by data scale differences during the training of the subsequent reinforcement learning policy network.

[0087] S3. Determine correction coefficients based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector, including:

[0088] S3-1. Calculate the historical deformation recovery rate distribution based on the difference in deformation displacement distribution between adjacent time windows in the historical deformation recovery data, wherein the historical deformation recovery data is the aligned deformation displacement distribution sequence within the preset time window in step S1.

[0089] S3-2. Calculate the ratio of the real-time deformation rate distribution to the historical deformation recovery rate distribution at the same data point location to generate the deformation rate recovery ratio distribution.

[0090] S3-3. Based on the relationship between the ratio of each data point in the deformation rate recovery ratio distribution and the preset recovery ratio threshold, generate the correction coefficient of the dynamic deformation state vector, where the preset recovery ratio threshold is calibrated based on the stress relaxation characteristics of the target material.

[0091] S3-4. Multiply the correction coefficient with the deformation energy parameter of the corresponding data point in the dynamic deformation state vector to obtain the corrected dynamic deformation state vector.

[0092] In step S3-1, the historical deformation recovery rate distribution is calculated based on the difference in deformation displacement distribution between adjacent time windows in the historical deformation recovery data. The historical deformation recovery data originates from the aligned deformation displacement distribution sequence obtained in step S1-3. Specifically, the deformation displacement distribution data of the current time window and the previous time window are taken, and the difference in displacement for each data point is calculated. This difference is then divided by the time window length to obtain the historical deformation recovery rate distribution. For example, if the time window length is 33 milliseconds, and the displacement of a data point in the previous window is 2 millimeters, while the displacement in the current window is 1.5 millimeters, then the historical deformation recovery rate is approximately (2-1.5) / 0.033 ≈ 15.15 millimeters per second. The historical deformation recovery rate distribution reflects the object's natural deformation recovery capability without external intervention.

[0093] In step S3-2, the ratio of the real-time deformation rate distribution to the historical deformation recovery rate distribution is calculated point-by-point at the same data point location. The real-time deformation rate distribution is derived from the real-time deformation rate distribution data acquired by the tactile sensor in step S1-1, and the historical deformation recovery rate distribution is derived from the calculation result of step S3-1. The specific implementation method for calculating the point-by-point ratio is as follows: the real-time deformation rate value of each data point is divided by the historical deformation recovery rate value at the corresponding location to generate the deformation rate recovery ratio distribution. For example, when the real-time deformation rate of a certain data point is 20 mm / s and the historical deformation recovery rate is 15 mm / s, the ratio is 20 / 15≈1.33. The deformation rate recovery ratio distribution characterizes the relative rate difference between real-time deformation and historical natural recovery.

[0094] In step S3-3, a correction coefficient is generated based on the relationship between the ratio of each data point in the deformation rate recovery ratio distribution and the preset recovery ratio threshold. The preset recovery ratio threshold is calibrated based on the stress relaxation characteristics of the target material. In practice, the stress decay curve of the object under fixed deformation is measured through a material tensile test, and the reciprocal of the time required for the stress to decay to 50% of the initial value is used as the recovery ratio threshold. For example, if it takes 5 seconds for the stress of a rubber material to decay to 50% under 10% tensile deformation, then the recovery ratio threshold is set to 1 / 5 = 0.2 seconds. -1 For each data point in the deformation rate recovery ratio distribution, if the ratio is greater than the preset recovery ratio threshold, the correction coefficient is set to 0.8; if the ratio is less than or equal to the preset recovery ratio threshold, the correction coefficient is set to 1.2. The purpose of the correction coefficient is to suppress the accumulation of deformation energy exceeding the natural recovery capacity.

[0095] In steps S3-4, the correction coefficient is multiplied by the deformation energy parameter of the corresponding data point in the dynamic deformation state vector. The dynamic deformation state vector originates from the vector data constructed in steps S2-4, and the deformation energy parameter corresponds to a scalar value in the vector representing the accumulated deformation energy of the object. In practice, each deformation energy parameter value in the dynamic deformation state vector is multiplied by the correction coefficient of the corresponding data point generated in step S3-3, while other parameters (such as real-time position coordinates and joint torques) remain unchanged. For example, if the deformation energy parameter of a certain data point is 50 joules and the correction coefficient is 0.8, then the corrected deformation energy parameter is 50 × 0.8 = 40 joules. The corrected dynamic deformation state vector is input into the reinforcement learning policy network in the subsequent step S4 to ensure that the policy generation process considers the dynamic constraints of deformation recovery capability.

[0096] The above implementation process addresses the strategy overshoot problem caused by traditional methods neglecting the natural recovery characteristics of materials by quantifying the correlation between real-time deformation and historical recovery rates and dynamically adjusting the weight of deformation energy parameters. The historical deformation recovery rate is calculated using a time window difference method, avoiding the introduction of complex models; the calibration of the preset recovery ratio threshold, combined with material mechanics experimental data, ensures the physical rationality of the correction coefficient. The point-by-point multiplication of the correction coefficient and the deformation energy parameter precisely suppresses abnormal energy accumulation in local areas. For example, when grasping an inflatable packaging bag, if the local deformation rate far exceeds the historical recovery capacity, the correction coefficient automatically lowers the corresponding energy parameter, preventing the strategy from generating excessive squeezing actions.

[0097] S4. Input the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a motion control instruction set including the grasping force component, including:

[0098] S4-1. Expand the dimension of the corrected dynamic deformation state vector, convert the real-time position coordinates and real-time attitude angles into polar coordinate system parameters, and concatenate them with the deformation energy parameters to form an expanded state vector.

[0099] S4-2. Input the extended state vector into the multi-head attention layer of the reinforcement learning policy network, and generate the initial weight distribution of translational displacement components, rotational angle components and grasping force components based on the preset action space decoupling rules;

[0100] S4-3. The initial weight distributions of translational displacement components, rotational angle components, and gripping force components are nonlinearly activated through independent fully connected layers to generate independent motion probability distributions for each component.

[0101] S4-4. Based on the maximum drive threshold of the joint motion parameters of the robotic arm, the probability distribution of independent actions is denormalized to generate a motion control instruction set including translational displacement components, rotational angle components, and gripping force components.

[0102] In step S4-1, the corrected dynamic deformation state vector is dimensionally expanded. The corrected dynamic deformation state vector is derived from the correction result of step S3-4, and includes deformation energy parameters, real-time position coordinates, and joint torques. The specific implementation of the dimensional expansion is as follows: the X-axis, Y-axis, and Z-axis components of the real-time position coordinates are converted into radial distance, azimuth angle, and elevation angle in polar coordinates. The conversion rule is calculated based on the mathematical relationship between the three-dimensional rectangular coordinate system and the polar coordinate system. The converted polar coordinate system parameters and deformation energy parameters are concatenated in a preset order, such as radial distance, azimuth angle, elevation angle, and deformation energy parameters, to form the expanded state vector.

[0103] In step S4-2, the extended state vector is input into the multi-head attention layer of the reinforcement learning policy network to generate the initial weight distribution. The specific implementation of the multi-head attention layer is as follows: the extended state vector is split into multiple sub-vectors according to a preset number of heads. Each sub-vector is then multiplied by a preset query vector for translational displacement, rotational angle, and grasping force components to generate attention weights for each action component. A preset action space decoupling rule assigns an independent query vector to each action component; for example, the query vector for the translational displacement component is 3-dimensional, the rotational angle component is 3-dimensional, and the grasping force component is 1-dimensional. The initial weight distribution is obtained by weighted summation of the attention weights of each sub-vector. For example, when the extended state vector is split into four sub-vectors, the initial weight of the translational displacement component is the average of the weights corresponding to the four sub-vectors.

[0104] In step S4-3, the initial weight distribution is nonlinearly activated using independent fully connected layers. Specifically, the independent fully connected layers are implemented as follows: independent fully connected neural network layers are set for the initial weight distributions of the translational displacement component, rotational angle component, and grasping force component. Each layer contains a preset number of hidden neurons; for example, the translational displacement component fully connected layer contains 64 hidden neurons, the rotational angle component fully connected layer contains 64 hidden neurons, and the grasping force component fully connected layer contains 32 hidden neurons. The nonlinear activation process uses the ReLU function to map the output values ​​of the fully connected layers to a non-negative interval, generating independent action probability distributions for each component. For example, the action probability distribution of the translational displacement component is a normalized value in the interval [-1, 1], representing the displacement ratio of the robotic arm in the X, Y, and Z axes.

[0105] In step S4-4, the probability distribution of independent actions is denormalized based on the maximum drive threshold of the robotic arm joint motion parameters. The maximum drive threshold is derived from the physical drive limit of each joint of the robotic arm. For example, the maximum drive threshold for the translational displacement component is ±0.5 m / s, the maximum drive threshold for the rotational angle component is ±30 degrees / s, and the maximum drive threshold for the grasping force component is 50 N. The specific method of denormalization is as follows: the normalized action probability distribution value is multiplied by the maximum drive threshold of the corresponding component to obtain the actual motion control command. For example, if the action probability distribution value of the translational displacement component is 0.5, then the actual translational displacement command is 0.5 × 0.5 = 0.25 m / s. The final generated motion control command set includes the actual control values ​​of the translational displacement component, the rotational angle component, and the grasping force component.

[0106] The above implementation process, through polar coordinate transformation, multi-head attention decoupling, and independent fully connected layer design, ensures that the generation process of motion components retains deformation energy characteristics while conforming to the physical driving constraints of the robotic arm. Polar coordinate transformation enhances the spatial representation capability of position parameters; for example, azimuth and elevation angles more intuitively reflect the motion direction of the end effector. The multi-head attention mechanism improves the policy network's ability to capture complex state features through parallel computation of multiple sub-vectors. Independent fully connected layers avoid parameter coupling between different motion components; for example, the training of the grasping force component is not affected by the gradient interference of the translational displacement component. Denormalization maps the abstract probability distribution to the actual driving range; for example, the 50-Newton upper limit for the grasping force component prevents damage to the robotic arm due to overload.

[0107] S5. Based on the Jacobian matrix of the dynamic deformation state vector and motion control instruction set, amplitude limiting processing is performed on the grasping force component, including:

[0108] S5-1. Calculate the Jacobian matrix of the end effector pose to the joint angle based on the real-time position coordinates and joint torque in the joint motion parameters of the robotic arm.

[0109] S5-2. Based on the distribution of deformation energy parameters in the dynamic deformation state vector, identify deformation-sensitive regions where the deformation energy parameters exceed a preset energy threshold.

[0110] S5-3. Perform a dot product operation between the Jacobian matrix and the deformation energy parameter gradient of the deformation-sensitive region to generate the spatial gradient influence value of the gripping force component on the deformation energy parameter.

[0111] S5-4. Calculate the amplitude limiting threshold of the grasping force component based on the ratio of the spatial gradient influence value to the preset safety factor, where the preset safety factor is calibrated based on the maximum deformation bearing capacity of the target object material.

[0112] S5-5. Truncate the values ​​in the grasping force component that exceed the amplitude limit threshold, and update the truncated grasping force component to the motion control instruction set.

[0113] In step S5-1, the Jacobian matrix of the end effector pose to the joint angle is calculated based on the real-time position coordinates and joint torques in the robot arm joint motion parameters. The real-time position coordinates are obtained from the three-dimensional coordinates of the end effector acquired by the robot arm joint encoder in step S1-2, and the joint torques are obtained from the real-time torque feedback data of the robot arm drive system in step S2-4.

[0114] The Jacobian matrix is ​​calculated as follows: based on the forward kinematics formula of the robotic arm, the partial derivatives of the real-time position coordinates (X-axis, Y-axis, Z-axis) and attitude angles (roll, pitch, yaw) of the end effector with respect to each joint angle are calculated, forming a 6xN Jacobian matrix, where N is the number of robotic arm joints. For example, when the robotic arm has 6 rotary joints, the Jacobian matrix is ​​a 6x6 matrix, with each column corresponding to the degree of influence of a joint angle change on the end effector pose.

[0115] In step S5-2, deformation-sensitive regions are identified based on the distribution of deformation energy parameters in the dynamic deformation state vector. The deformation energy parameters are derived from the scalar values ​​generated in step S2-3, and the dynamic deformation state vector is derived from the correction results in step S3-4. The rule for identifying deformation-sensitive regions is as follows: traverse all deformation energy parameter data points in the dynamic deformation state vector; if the deformation energy parameter value of a certain data point exceeds a preset energy threshold, then the region where that data point is located is determined to be a deformation-sensitive region.

[0116] The preset energy threshold is calibrated based on the yield strength of the target object's material. For example, when grasping a rubber object, the preset energy threshold is set to 80% of the material's yield strength. If the yield strength of rubber is measured to be 10 joules through a tensile test, then the preset energy threshold is 8 joules. The determination results of deformation-sensitive areas are stored in the form of a binary mask, with the location marked as a sensitive area having a value of 1 and the location marked as a non-sensitive area having a value of 0.

[0117] In step S5-3, the Jacobian matrix is ​​multiplied by the deformation energy parameter gradient of the deformation-sensitive region to generate the spatial gradient influence value. The deformation energy parameter gradient originates from the deformation energy parameter distribution after weighting in step S2-3, and the gradient value is obtained by calculating the difference in deformation energy parameters between adjacent data points. The specific implementation of the dot product operation is as follows: for each data point within the deformation-sensitive region, the gradient vector of its deformation energy parameter (containing components in the X, Y, and Z directions) is extracted, and the gradient vector is multiplied by the column vector of the corresponding end effector pose in the Jacobian matrix to obtain the spatial gradient influence value of that data point.

[0118] For example, if the gradient vector of a data point is (0.2, 0.3, 0.1), and the column vectors of the Jacobian matrix corresponding to the X, Y, and Z positions of the end effector are (0.5, 0.6, 0.4), then the dot product result is 0.2×0.5+0.3×0.6+0.1×0.4=0.1+0.18+0.04=0.32. The spatial gradient influence values ​​of all data points in the sensitive area constitute a gradient influence distribution map.

[0119] In step S5-4, the amplitude limiting threshold is calculated based on the ratio of the spatial gradient influence value to the preset safety factor. The preset safety factor is calibrated based on the maximum deformation tolerance of the target object's material. The maximum deformation tolerance is determined by measuring the maximum compressive deformation of the object before fracture through a material compression experiment and converting it into an energy threshold. For example, when the maximum compressive deformation of silicone material is 20 mm, the corresponding energy threshold is 15 joules, and the safety factor is set as the reciprocal of the energy threshold, i.e., 1 / 15 ≈ 0.067 joules. -1 .

[0120] The formula for calculating the amplitude limiting threshold is: safety factor divided by the absolute value of the spatial gradient influence value. When the spatial gradient influence value is 0.32, the amplitude limiting threshold is 0.067 / 0.32 ≈ 0.21 Newtons. -1 This threshold represents the upper limit of the contribution of a unit gripping force to the accumulation of deformation energy.

[0121] In step S5-5, values ​​exceeding the amplitude limiting threshold in the grasping force component are truncated. The grasping force component originates from the force command values ​​in the motion control command set generated in step S4-4. The truncation process is as follows: the command value of each grasping force component is compared with the amplitude limiting threshold. If the command value is greater than the threshold, the command value is set to the threshold; if the command value is less than or equal to the threshold, the original value is retained. For example, when the grasping force command value is 25 N and the amplitude limiting threshold is 20 N, the truncated force command value is 20 N. The truncated grasping force component is updated to the motion control command set to ensure that the subsequent iterative optimization process in step S6 is performed within the safe boundary.

[0122] The above implementation process precisely controls the influence of gripping force on deformation energy through mechanical gradient analysis and dynamic safety threshold setting. The introduction of the Jacobian matrix incorporates the mechanical relationship between the robotic arm joint motion and the end effector pose. For example, when the joint torque of the robotic arm increases, the amplitude of the corresponding column vector of the Jacobian matrix increases, leading to an increase in the spatial gradient influence value, thereby reducing the amplitude limit threshold and suppressing the gripping force command. The identification of deformation-sensitive areas is based on material yield strength calibration, ensuring that force limiting is automatically triggered when approaching the material's deformation limit. The truncation process employs a point-by-point comparison mechanism to avoid a decrease in gripping stability caused by global force suppression; for example, force commands are limited only in high-risk areas, while other areas maintain their original command values.

[0123] S6. Based on the changing trend of deformation energy parameters and the grasping force component after amplitude limiting processing, dynamically adjust the reward function weight allocation coefficient, and iteratively optimize the motion control instruction set until the preset convergence condition is met, including:

[0124] S6-1. Calculate the exponentially weighted moving average of the deformation energy accumulation rate based on the temporal change rate of the deformation energy parameter in the dynamic deformation state vector.

[0125] S6-2. Based on the variance of the grasping force component after amplitude limiting processing, generate a stability attenuation coefficient that characterizes the fluctuation of grasping stability.

[0126] S6-3. Input the exponentially weighted moving average of the deformation energy accumulation rate and the stability decay coefficient into the preset weight allocation function, and dynamically adjust the weight allocation coefficients of the deformation energy accumulation term and the grasping stability term in the reward function.

[0127] S6-4. Recalculate the reward function value based on the updated weight allocation coefficients, and iteratively optimize the motion control instruction set through the policy gradient algorithm until the joint convergence index of the deformation energy parameter and the grasping force component meets the preset convergence condition.

[0128] In step S6-1, the exponentially weighted moving average of the deformation energy accumulation rate is calculated based on the temporal change rate of the deformation energy parameters in the dynamic deformation state vector. The dynamic deformation state vector originates from the correction result of step S3-4 and includes the corrected deformation energy parameters, real-time position coordinates, and joint torques. The temporal change rate is calculated by taking the difference between the deformation energy parameters at the current moment and the previous moment, and dividing it by the time interval. For example, when the time interval is 0.1 seconds, the current deformation energy parameter is 50 joules, and the previous moment's parameter was 48 joules, then the temporal change rate is (50-48) / 0.1 = 20 joules / second. The exponentially weighted moving average is calculated by weighting the historical change rates with an exponential decay factor. The decay factor is calibrated according to the material relaxation characteristics of the target object; for example, the decay factor for rubber is set to 0.9, and for silicone, it is set to 0.85. The exponentially weighted moving average reflects the long-term trend of deformation energy accumulation and suppresses instantaneous fluctuation interference.

[0129] In step S6-2, a stability attenuation coefficient is generated based on the variance of the grasping force component after amplitude limiting processing. The grasping force component after amplitude limiting processing is derived from the truncation result in step S5-5. The variance is calculated by taking the numerical sequence of the grasping force component within a continuous preset time window and calculating its variance. For example, when the time window contains 10 sampling points, the numerical sequence of the grasping force component is [20 N, 21 N, 19 N, 20 N, 22 N], and its variance is calculated as the average of the sum of squared deviations of each value from the mean. The stability attenuation coefficient is generated by dividing the variance by a preset variance benchmark value and then taking the natural logarithm. The stability attenuation coefficient characterizes the degree of stability fluctuation of the grasping action; the larger the coefficient, the worse the stability.

[0130] In step S6-3, the exponentially weighted moving average of the deformation energy accumulation rate and the stability decay coefficient are input into a preset weight allocation function to dynamically adjust the weight allocation coefficients. The specific implementation of the preset weight allocation function is as follows: the exponentially weighted moving average and the stability decay coefficient are linearly combined according to a preset ratio to generate weight allocation coefficients for the deformation energy accumulation term and the stability capture term. For example, when the exponentially weighted moving average is 15 joules / second, the stability decay coefficient is 0.693, and the preset ratio is 60% for the deformation energy accumulation term and 40% for the stability capture term, the weight allocation coefficient for the deformation energy accumulation term is 15 × 0.6 = 9, and the weight allocation coefficient for the stability capture term is 0.693 × 0.4 ≈ 0.277. After normalization, the weight of the deformation energy accumulation term is 9 / (9 + 0.277) ≈ 97%, and the weight of the stability capture term is 3%. The weight allocation coefficients are dynamically updated with each iteration.

[0131] In step S6-4, the reward function value is recalculated based on the updated weight allocation coefficients, and the motion control command set is iteratively optimized using the policy gradient algorithm. The reward function value is calculated by multiplying the values ​​of the deformation energy accumulation term and the grasping stability term by their respective weight allocation coefficients and then summing them. For example, when the deformation energy accumulation term is 50 joules, the weight allocation coefficient is 0.97, the grasping stability term is 80 (dimensionless stability score), and the weight allocation coefficient is 0.03, the reward function value is 50 × 0.97 + 80 × 0.03 = 48.5 + 2.4 = 50.9. The specific implementation of the policy gradient algorithm is as follows: the gradient direction of the policy network is calculated based on the reward function value, and the network parameters are updated along the gradient direction to maximize the reward value. The iterative optimization terminates when the following two preset convergence conditions are simultaneously met: the rate of change of the deformation energy parameter is less than 1 joule / second, and the variance of the grasping force component is less than 1 N². When the joint convergence index meets the above conditions, the final motion control command set is output.

[0132] The above implementation process, through dynamic weight adjustment and multi-objective joint convergence verification, ensures that the strategy optimization process takes into account both deformation energy control and grasping stability. The exponentially weighted moving average suppresses misjudgments of energy accumulation trends due to instantaneous noise; for example, when grasping an inflatable packaging bag, a brief surge in deformation will not trigger a sudden change in weights. The stability decay coefficient quantifies force fluctuations as an adjustable penalty term; for example, it automatically increases the stability weight when the force variance is too large. The iterative optimization of the strategy gradient algorithm is based on normalized weight allocation coefficients, avoiding convergence oscillations caused by objective conflicts. The joint convergence index combines deformation energy and force stability; for example, convergence is determined when the rate of change of deformation energy meets the target and force fluctuations are controllable, ensuring the physical feasibility and reliability of the grasping action.

[0133] Example 2: Figure 2 A schematic diagram of a reinforcement learning-based intelligent optimization system for robotic arm grasping is provided. The system comprises:

[0134] Deformation and pose acquisition module: acquires dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm end effector;

[0135] Energy state encoding module: Generates deformation energy parameters based on dynamic deformation characteristic data and real-time pose information, and performs joint encoding with the joint motion parameters of the robotic arm to construct a dynamic deformation state vector;

[0136] Dynamic state correction module: Determines correction coefficients based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector;

[0137] The strategy instruction generation module inputs the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a set of motion control instructions including the grasping force component.

[0138] Force limiting control module: Based on the Jacobian matrix of dynamic deformation state vector and motion control instruction set, it performs amplitude limiting processing on the grasping force component;

[0139] The dynamic weight optimization module dynamically adjusts the weight allocation coefficient of the reward function based on the changing trend of the deformation energy parameters and the grasping force component after amplitude limiting, and iteratively optimizes the motion control instruction set until the preset convergence condition is met.

[0140] The above formulas are all dimensionless calculations. The formulas are derived from software simulations using a large amount of collected data, and are the closest to the real situation. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0141] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0142] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0143] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0145] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0147] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0149] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent optimization of robotic arm grasping based on reinforcement learning, characterized in that, Includes the following steps: S1. Acquire dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm end effector; S2. Generate deformation energy parameters based on dynamic deformation characteristic data and real-time pose information, and combine them with the joint motion parameters of the robotic arm to perform joint encoding to construct a dynamic deformation state vector. S3. Determine the correction coefficient based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector. S4. Input the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a motion control instruction set including the grasping force component. S5. Based on the Jacobian matrix of the dynamic deformation state vector and motion control instruction set, amplitude limiting processing is performed on the grasping force component, including: S5-1. Calculate the Jacobian matrix of the end effector pose to the joint angle based on the real-time position coordinates and joint torque in the joint motion parameters of the robotic arm. S5-2. Based on the distribution of deformation energy parameters in the dynamic deformation state vector, identify deformation-sensitive regions where the deformation energy parameters exceed a preset energy threshold. S5-3. Perform a dot product operation between the Jacobian matrix and the deformation energy parameter gradient of the deformation-sensitive region to generate the spatial gradient influence value of the gripping force component on the deformation energy parameter. S5-4. Calculate the amplitude limiting threshold of the grasping force component based on the ratio of the spatial gradient influence value to the preset safety factor, where the preset safety factor is calibrated based on the maximum deformation bearing capacity of the target object material. S5-5. Truncate the values ​​in the grasping force component that exceed the amplitude limit threshold, and update the truncated grasping force component to the motion control instruction set. S6. Based on the changing trend of deformation energy parameters and the grasping force component after amplitude limiting processing, dynamically adjust the reward function weight allocation coefficient, and iteratively optimize the motion control instruction set until the preset convergence condition is met, including: S6-1. Calculate the exponentially weighted moving average of the deformation energy accumulation rate based on the temporal change rate of the deformation energy parameter in the dynamic deformation state vector. S6-2. Based on the variance of the grasping force component after amplitude limiting processing, generate a stability attenuation coefficient that characterizes the fluctuation of grasping stability. S6-3. Input the exponentially weighted moving average of the deformation energy accumulation rate and the stability decay coefficient into the preset weight allocation function, and dynamically adjust the weight allocation coefficients of the deformation energy accumulation term and the grasping stability term in the reward function. S6-4. Recalculate the reward function value based on the updated weight allocation coefficients, and iteratively optimize the motion control instruction set through the policy gradient algorithm until the joint convergence index of the deformation energy parameter and the grasping force component meets the preset convergence condition.

2. The intelligent optimization method for robotic arm grasping based on reinforcement learning according to claim 1, characterized in that, S1 includes: S1-1. Obtain the deformation displacement distribution of the contact area on the surface of the target object through a visual sensor, and obtain the real-time deformation rate distribution corresponding to the deformation displacement distribution through a tactile sensor. S1-2. Obtain the real-time position coordinates of the end effector through the joint encoder of the robotic arm, and obtain the real-time attitude angle of the end effector through the inertial measurement unit; S1-3. Synchronously align the deformation displacement distribution, real-time deformation rate distribution, real-time position coordinates, and real-time attitude angle according to a preset time window to generate dynamic deformation characteristic data and real-time pose information.

3. The intelligent optimization method for robotic arm grasping based on reinforcement learning according to claim 1, characterized in that, S2 include: S2-1. Based on the displacement of each data point in the deformation displacement distribution, generate the corresponding local deformation elastic potential energy parameters using the elastic potential energy calculation formula. S2-2. Based on the rate value of each data point in the real-time deformation rate distribution and the displacement of the same data point in the corresponding deformation displacement distribution, the local deformation kinetic energy parameters are generated through the nonlinear damping correction formula. S2-3. Based on the statistical variance of the real-time deformation rate distribution, dynamically adjust the superposition weighting coefficient of the local deformation elastic potential energy parameter and the local deformation dynamic energy parameter to obtain the deformation energy parameter. S2-4. Map the deformation energy parameters, real-time position coordinates and joint torques in the robot arm joint motion parameters to the same vector space according to the preset coding rules, and construct the dynamic deformation state vector through normalization processing.

4. The intelligent optimization method for robotic arm grasping based on reinforcement learning according to claim 1, characterized in that, S3 include: S3-1. Calculate the historical deformation recovery rate distribution based on the difference in deformation displacement distribution between adjacent time windows in the historical deformation recovery data, wherein the historical deformation recovery data is the aligned deformation displacement distribution sequence within the preset time window in step S1. S3-2. Calculate the ratio of the real-time deformation rate distribution to the historical deformation recovery rate distribution at the same data point location to generate the deformation rate recovery ratio distribution. S3-3. Based on the relationship between the ratio of each data point in the deformation rate recovery ratio distribution and the preset recovery ratio threshold, generate the correction coefficient of the dynamic deformation state vector, where the preset recovery ratio threshold is calibrated based on the stress relaxation characteristics of the target material. S3-4. Multiply the correction coefficient with the deformation energy parameter of the corresponding data point in the dynamic deformation state vector to obtain the corrected dynamic deformation state vector.

5. The intelligent optimization method for robotic arm grasping based on reinforcement learning according to claim 1, characterized in that, S4 include: S4-1. Expand the dimension of the corrected dynamic deformation state vector, convert the real-time position coordinates and real-time attitude angles into polar coordinate system parameters, and concatenate them with the deformation energy parameters to form an expanded state vector. S4-2. Input the extended state vector into the multi-head attention layer of the reinforcement learning policy network, and generate the initial weight distribution of the grasping force component based on the preset action space decoupling rules; S4-3. The initial weight distribution of the grasping force component is non-linearly activated by an independent fully connected layer to generate an independent action probability distribution for each component. S4-4. Based on the maximum drive threshold of the robotic arm joint motion parameters, perform inverse normalization on the probability distribution of independent actions to generate a motion control instruction set including the gripping force component.

6. The intelligent optimization method for robotic arm grasping based on reinforcement learning according to claim 5, characterized in that, Based on the preset motion space decoupling rules, the initial weight distribution of translational displacement components and rotational angle components is also generated; it also includes nonlinear activation processing of the initial weight distribution of translational displacement components and rotational angle components; the motion control instruction set also includes translational displacement components and rotational angle components.

7. A reinforcement learning-based intelligent optimization system for robotic arm grasping, used to implement the reinforcement learning-based intelligent optimization method for robotic arm grasping as described in any one of claims 1-6, characterized in that, include: Deformation and pose acquisition module: acquires dynamic deformation characteristic data of the target object and real-time pose information of the robotic arm end effector; Energy state encoding module: Generates deformation energy parameters based on dynamic deformation characteristic data and real-time pose information, and performs joint encoding with robotic arm joint motion parameters to construct dynamic deformation state vector; Dynamic state correction module: Determines correction coefficients based on the correlation between historical deformation recovery data and real-time deformation rate distribution to correct the dynamic deformation state vector; The strategy instruction generation module inputs the corrected dynamic deformation state vector into the reinforcement learning policy network to generate a set of motion control instructions including the grasping force component. Force limiting control module: Based on the Jacobian matrix of dynamic deformation state vector and motion control instruction set, it performs amplitude limiting processing on the grasping force component; The dynamic weight optimization module dynamically adjusts the weight allocation coefficient of the reward function based on the changing trend of the deformation energy parameters and the grasping force component after amplitude limiting, and iteratively optimizes the motion control instruction set until the preset convergence condition is met.

Images