Self-adaptive grabbing control method and system of robot

Abstract

The invention relates to the technical field of robot control, and particularly provides a self-adaptive grabbing control method and system for a robot, and the method comprises the steps: collecting the initial environment data of a to-be-grabbed target through a multi-mode sensor; inputting the initial environment data into a pre-trained trend prediction network, and generating an initial grabbing pose and a predefined initial impedance parameter set; constructing a multi-modal space-time fusion model, performing alignment and deep fusion on visual feature flow and force feature flow in the dynamic interaction sequence data, and outputting an environment state representation vector at the current moment; calculating a real-time impedance correction amount at the current moment by using a self-adaptive variable impedance controller, and calculating a target impedance control law in combination with the initial impedance parameter set; mapping the target impedance control law to a joint space to generate a joint torque driving signal; according to the method, the matching degree of the current behavior and the final target can be continuously evaluated, and when the environment dynamically changes, the control instruction and the task chain are optimized in real time.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to an adaptive grasping control method and system for a robot. Background Technology

[0002] Robotic grasping technology is a core component of the automation field. As application scenarios become more complex, robots need to possess the ability to autonomously complete grasping tasks in unstructured, dynamic, and unknown environments. This places extremely high demands on the robot's environmental perception, task understanding, real-time planning, and precise control.

[0003] Prior art 1, Chinese Patent Application No.: CN202411499164.3, discloses a robot behavior planning method, controller, and robot for adaptive grasping tasks. It solves the problem that robots cannot accurately reason and make decisions based on objects, obstacles, and their own physical manipulation capabilities in the surrounding environment, providing a robot behavior planning method, controller, and robot for adaptive grasping tasks. The adaptive grasping task robot behavior planning method includes generating navigation instructions based on a global scene map, the robot's real-time position, and the grasping task; determining whether the robot has the ability to complete the grasping task; if so, controlling the robot to move according to the navigation instructions to execute the grasping task; otherwise, when there are candidate tools matching the grasping task in the global scene map, calculating the score of each candidate tool, selecting the one with the highest score as the grasping tool, and updating the navigation instructions. Based on the updated navigation instructions, the robot's flexibility and adaptability in complex environments are improved. Prior art two, Chinese patent application number: CN202111014867.9, discloses a robot adaptive grasping control method and system based on a spiking neural network. First, an initial neural network model is converted into a spiking neural network model. Second, the presence of an object to be grasped within the grasping range is determined based on the depth image to be predicted. Then, the depth image and the color image to be predicted are fused and input into the spiking neural network model for prediction to obtain the robotic arm's grasping pose. Finally, the robotic arm is manipulated and controlled according to the grasping pose to grasp the object. The spiking neural network model converted in this invention not only retains the accuracy of a fully convolutional neural network to the greatest extent, but also achieves adaptive grasping of objects of different sizes and shapes, and has a high grasping accuracy for objects within the grasping range that were not pre-trained. Prior art three, Chinese patent application number: CN202310778064.3, discloses a method and system for underwater robot cruising, searching, and grasping based on adaptive control. The method includes: acquiring the current actual state information of the underwater robot, and determining the current error of the underwater robot based on the current expected state information corresponding to the current search path and the current actual state information; if the current error is greater than a preset error, optimizing the current control input corresponding to the current actual state information based on the current actual state information and the current expected state information to obtain an optimized current control input; based on the optimized current control input, controlling the underwater robot to cruise and search along the current search path, and grasping the target object when a target object is detected during the cruising and searching process. This invention improves the underwater robot's ability to autonomously search for and operate on target objects in unknown areas. Current technologies 1, 2, and 3 suffer from problems such as fragmented modules, limited adaptive dimensions, and a lack of closed-loop dynamic correction capabilities. Therefore, this invention provides an adaptive grasping control method and system for robots. Summary of the Invention

[0004] To achieve the above objectives, the present invention adopts the following technical solution: In one aspect, the present invention provides an adaptive grasping control method for a robot, comprising the following steps: Initial environmental data of the target to be grasped is collected by a multimodal sensor. The initial environmental data includes visual point cloud data, semantic data of object surface material, and task instruction attribute data. The initial environmental data is input into a pre-trained trend prediction network to generate an initial grasping pose and a predefined set of initial impedance parameters, which includes desired stiffness, desired damping, and desired moment of inertia. The robotic arm is controlled to move toward the target to be grasped according to the initial grasping posture, and dynamic interactive sequence data is collected in real time through force sensors and vision sensors. A multimodal spatiotemporal Transformer fusion model is constructed to align and deeply fuse the visual feature stream and force feature stream in the dynamic interaction sequence data, and output the environmental state representation vector at the current moment. Based on the environmental state characterization vector, the real-time impedance correction amount at the current moment is calculated using an adaptive variable impedance controller, and the target impedance control law is calculated by combining the initial impedance parameter set. The target impedance control law is mapped to the joint space to generate a joint torque drive signal, which controls the robotic arm to perform an adaptive grasping action. The network weights of the trend prediction network and the adaptive variable impedance controller are updated based on the grasping result.

[0005] In one optional implementation, the specific process of generating a predefined set of initial impedance parameters includes: Extract the material feature vector from the semantic data of the object's surface material, and the risk level label from the task instruction attribute data; The material feature vector is concatenated with the risk level label and then input into the material-impedance mapping neural network. The material-impedance mapping neural network adopts a multilayer perceptron structure and outputs a reference stiffness value and a reference damping value that match the current material and the risk of the task. Based on the current load state of the robotic arm, the reference stiffness value and reference damping value are dynamically constrained and verified to obtain the initial impedance parameter set.

[0006] In one optional implementation, the processing of the multimodal spatiotemporal Transformer fusion model includes: Embedding encoding is performed on the visual feature stream and the force feature stream in the time dimension to obtain the visual embedding sequence and the force embedding sequence; The cross-attention mechanism is used to calculate the association weight matrix between the visual embedding sequence and the force embedding sequence, capturing force compensation information under visual occlusion and visual verification information under force mutation. The fused feature sequence is modeled using a multi-head self-attention mechanism, and a six-dimensional environmental state representation vector is output, which includes the environmental contact state, the object sliding trend, and the degree of object deformation.

[0007] In one optional implementation, the specific steps for calculating the real-time impedance correction using an adaptive variable impedance controller include: Construct a state observer based on Kalman filtering, and estimate the contact stiffness and environmental deformation of the current contact point according to the environmental state characterization vector; Calculate the stiffness error between the current estimated contact stiffness and the expected contact stiffness, as well as the current tracking speed error of the end effector; An error cost function based on the Lyapunov stability criterion is designed. The error cost function is minimized by the gradient descent algorithm, and the expected stiffness correction coefficient and expected damping correction coefficient for the current moment are obtained.

[0008] In one optional implementation, the process of updating network weights based on the crawling results includes: The detection function checks whether the final position of the object remains within a predetermined threshold range after the grasping action is completed, and whether the force sensor detects an impact force exceeding the safety threshold. If the object's position remains stable and there is no impact force, the current sample is marked as a successful sample, and the trend prediction network is fine-tuned using the backpropagation algorithm. If a slip or impact occurs, it is marked as a failure sample. The residual between the actual output torque and the ideal torque is calculated, and the policy parameters of the adaptive variable impedance controller are updated penalizedly using a reinforcement learning algorithm.

[0009] In one optional implementation, an unexpected interference handling step is also included: During the calculation of the real-time impedance correction at the current moment, the mutation rate of the environmental state characterization vector is monitored in real time. When the rate of change of the environmental state representation vector exceeds the preset disturbance threshold, it is determined that external interference has occurred. Immediately activate the instantaneous protection mode, lock the current joint angle, and switch the desired stiffness in the target impedance control law to the maximum preset safety stiffness to resist external impacts until the mutation rate returns to the normal range.

[0010] In one optional implementation, the method for acquiring dynamic interactive sequence data includes: Before the end effector of the robotic arm makes contact with the target to be grasped, high-resolution visual point cloud data is acquired at the first sampling frequency. At the instant the end effector contacts the target to be grasped, the force sensor is simultaneously triggered to collect six-dimensional force / torque data at a second sampling frequency, where the second sampling frequency is higher than the first sampling frequency. Background removal and point cloud registration are performed on the acquired visual point cloud data, and gravity compensation and filtering noise reduction are performed on the force / torque six-dimensional data to form a synchronized timestamp aligned data stream.

[0011] In another aspect, the present invention provides an adaptive grasping control system for a robot, comprising: The perception module is used to collect initial environmental data and dynamic interaction sequence data of the target to be grasped through multimodal sensors; The decision reasoning module integrates a trend prediction network and a multimodal spatiotemporal Transformer fusion model to generate an initial grasping pose, an initial impedance parameter set, and an environmental state representation vector based on environmental data. The adaptive control module integrates an adaptive variable impedance controller, which is used to calculate the real-time impedance correction based on the environmental state characterization vector and generate the target impedance control law. The execution drive module is used to convert the target impedance control law into a joint torque drive signal to control the robotic arm to perform grasping actions; The learning optimization module is used to update the algorithm parameters in the decision reasoning module and the adaptive control module based on the feedback of the captured results.

[0012] In another aspect, the present invention provides an electronic device comprising: At least one memory stores computer-executable instructions non-transiently; At least one processor, configured to run the computer-executable instructions, The computer-executable instructions are executed by the processor to implement the aforementioned adaptive grasping control method for a robot.

[0013] In another aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by at least one processor, implement the above-described adaptive grasping control method for a robot.

[0014] This invention acquires initial environmental data of the target to be grasped using multimodal sensors. This initial environmental data includes visual point cloud data, semantic data of object surface material, and task instruction attribute data. The initial environmental data is input into a pre-trained trend prediction network to generate an initial grasping pose and a predefined set of initial impedance parameters, including desired stiffness, desired damping, and desired moment of inertia. Based on the initial grasping pose, a robotic arm is controlled to move towards the target, and dynamic interaction sequence data is acquired in real time using force and vision sensors. A multimodal spatiotemporal Transformer fusion model is constructed to integrate the dynamic interaction sequence data... The visual feature stream and the force feature stream are aligned and deeply fused to output the environmental state representation vector at the current moment. Based on the environmental state representation vector, the real-time impedance correction amount at the current moment is calculated using an adaptive variable impedance controller, and the target impedance control law is calculated by combining the initial impedance parameter set. The target impedance control law is mapped to the joint space to generate joint torque drive signals, which control the robotic arm to perform adaptive grasping actions. The network weights of the trend prediction network and the adaptive variable impedance controller are updated based on the grasping results. This invention can continuously evaluate the matching degree between the current behavior and the final goal, and optimize the control commands and task chains in real time when the environment changes dynamically. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an adaptive grasping control method for a robot provided in Embodiment 1 of the present invention; Figure 2 This is a framework diagram of an adaptive grasping control system for a robot provided in Embodiment 3 of the present invention; Figure 3 This is a block diagram of the electronic device provided in Embodiment 4 of the present invention; Figure 4 This is a block diagram of a computer-readable storage medium provided in Embodiment 4 of the present invention. Detailed Implementation

[0016] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0017] Hereinafter, the terms "first," "second," etc., are used for descriptive convenience only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0018] In this invention, unless otherwise explicitly specified and limited, the term "connection" should be interpreted broadly. For example, "connection" can be a fixed mechanical connection, a detachable mechanical connection, or an integral part; or, "connection" can be a direct connection or an indirect connection through an intermediate medium. Furthermore, unless otherwise explicitly specified and limited, the term "coupling" should be interpreted broadly. For example, "coupling" can be a direct electrical connection, such as physical contact and electrical conduction between two components; it can also be understood as an electrical connection between different components in a circuit structure through physical lines capable of transmitting electrical signals, such as copper foil or wires on a printed circuit board (PCB), to transmit electrical signals; or, "coupling" can be an indirect electrical connection between two components through an intermediate medium; or, "coupling" can be an electrical connection between two components in a non-contact manner, such as an electrical connection between two components using capacitive coupling to transmit electrical signals.

[0019] In this embodiment of the invention, directional terms such as "up," "down," "left," and "right" may be defined relative to the orientation of the components shown in the accompanying drawings. It should be understood that these directional terms can be relative concepts, used for relative description and clarification, and can change accordingly depending on the orientation of the components in the accompanying drawings.

[0020] Example 1: like Figure 1 As shown, this embodiment of the invention provides an adaptive grasping control method for a robot, comprising the following steps: Step S100: Collect initial environmental data of the target to be grasped through a multimodal sensor. The initial environmental data includes visual point cloud data, semantic data of object surface material, and task instruction attribute data. Step S200: Input the initial environmental data into the pre-trained trend prediction network to generate the initial grasping pose and a predefined set of initial impedance parameters, the initial impedance parameter set including the desired stiffness, desired damping and desired moment of inertia; Step S300: Control the robotic arm to move toward the target to be grasped according to the initial grasping posture, and collect dynamic interactive sequence data in real time through force sensor and vision sensor; Step S400: Construct a multimodal spatiotemporal Transformer fusion model, align and deeply fuse the visual feature stream and force feature stream in the dynamic interaction sequence data, and output the environmental state representation vector at the current moment; Step S500: Based on the environmental state characterization vector, the real-time impedance correction amount at the current moment is calculated using an adaptive variable impedance controller, and the target impedance control law is calculated by combining the initial impedance parameter set. Step S600: Map the target impedance control law to the joint space, generate a joint torque drive signal, control the robotic arm to perform an adaptive grasping action, and update the network weights of the trend prediction network and the adaptive variable impedance controller based on the grasping result.

[0021] The above embodiments break away from the fragmented planning, perception, and control modules of traditional technologies, constructing a unified, layered, and collaborative architecture. The system can directly map real-time control feedback from the lower layers (such as force perception and pose error) to the upper-layer visual perception and task planning modules. This enables the robot to not only "execute the plan" but also perceive feedback from the physical world in real time during execution. For example, when the lower-layer control detects abnormal grasping resistance, the upper-layer planning can immediately perceive and trigger visual re-examination or pose fine-tuning, greatly improving the accuracy and success rate of the grasping action.

[0022] A multi-dimensional anomaly assessment mechanism is introduced, capable of distinguishing between perception layer failures, planning layer conflicts, and execution layer disturbances. Based on a pre-set decision tree or AI model, the robot autonomously selects a response strategy—whether to adjust underlying parameters, replan local paths, or roll back the entire task. Compared to the single-dimensional adaptability of existing technologies, this invention endows the robot with the intelligence to cope with "unexpected" situations. When encountering complex situations such as object displacement, slippage during grasping, or sudden obstacles, the robot, like a human, does not rigidly adhere to the original plan but dynamically adjusts its strategy (such as changing the grasping angle or seeking auxiliary support), significantly improving its survivability and robustness in unstructured environments.

[0023] By combining real-time environmental data streams with global task objectives, the system not only focuses on the current state (such as in Technology 3) but also considers long-term task goals (such as in Technology 1). The system can continuously evaluate the matching degree between the current behavior and the final goal, and optimize control commands and task chains in real time when the environment undergoes dynamic changes (not only physical errors but also changes in scene semantics).

[0024] This makes the robots particularly suitable for complex and varied environments such as underwater, outer space, or home services. The robots no longer need frequent shutdowns to await human intervention; instead, they can autonomously, continuously, and flexibly complete a series of complex actions such as patrolling and searching, precise positioning, and adaptive grasping, significantly improving operational efficiency and intelligence.

[0025] Example 2: like Figure 1 As shown, based on Embodiment 1, the specific process of generating a predefined set of initial impedance parameters provided in this embodiment of the invention includes: Extract the material feature vector from the semantic data of the object's surface material, and the risk level label from the task instruction attribute data; The material feature vector is concatenated with the risk level label and then input into the material-impedance mapping neural network. The material-impedance mapping neural network adopts a multilayer perceptron structure and outputs a reference stiffness value and a reference damping value that match the current material and the risk of the task. Based on the current load state of the robotic arm, the reference stiffness value and reference damping value are dynamically constrained and verified to obtain the initial impedance parameter set.

[0026] In one optional implementation, the processing of the multimodal spatiotemporal Transformer fusion model includes: Embedding encoding is performed on the visual feature stream and the force feature stream in the time dimension to obtain the visual embedding sequence and the force embedding sequence; The cross-attention mechanism is used to calculate the association weight matrix between the visual embedding sequence and the force embedding sequence, capturing force compensation information under visual occlusion and visual verification information under force mutation. The fused feature sequence is modeled using a multi-head self-attention mechanism, and a six-dimensional environmental state representation vector is output, which includes the environmental contact state, the object sliding trend, and the degree of object deformation.

[0027] In one optional implementation, the specific steps for calculating the real-time impedance correction using an adaptive variable impedance controller include: Construct a state observer based on Kalman filtering, and estimate the contact stiffness and environmental deformation of the current contact point according to the environmental state characterization vector; Calculate the stiffness error between the current estimated contact stiffness and the expected contact stiffness, as well as the current tracking speed error of the end effector; An error cost function based on the Lyapunov stability criterion is designed. The error cost function is minimized by the gradient descent algorithm, and the expected stiffness correction coefficient and expected damping correction coefficient for the current moment are obtained.

[0028] In one optional implementation, the process of updating network weights based on the crawling results includes: The detection function checks whether the final position of the object remains within a predetermined threshold range after the grasping action is completed, and whether the force sensor detects an impact force exceeding the safety threshold. If the object's position remains stable and there is no impact force, the current sample is marked as a successful sample, and the trend prediction network is fine-tuned using the backpropagation algorithm. If a slip or impact occurs, it is marked as a failure sample. The residual between the actual output torque and the ideal torque is calculated, and the policy parameters of the adaptive variable impedance controller are updated penalizedly using a reinforcement learning algorithm.

[0029] In one optional implementation, an unexpected interference handling step is also included: During the calculation of the real-time impedance correction at the current moment, the mutation rate of the environmental state characterization vector is monitored in real time. When the rate of change of the environmental state representation vector exceeds the preset disturbance threshold, it is determined that external interference has occurred. Immediately activate the instantaneous protection mode, lock the current joint angle, and switch the desired stiffness in the target impedance control law to the maximum preset safety stiffness to resist external impacts until the mutation rate returns to the normal range.

[0030] In one optional implementation, the method for acquiring dynamic interactive sequence data includes: Before the end effector of the robotic arm makes contact with the target to be grasped, high-resolution visual point cloud data is acquired at the first sampling frequency. At the instant the end effector contacts the target to be grasped, the force sensor is simultaneously triggered to collect six-dimensional force / torque data at a second sampling frequency, where the second sampling frequency is higher than the first sampling frequency. Background removal and point cloud registration are performed on the acquired visual point cloud data, and gravity compensation and filtering noise reduction are performed on the force / torque six-dimensional data to form a synchronized timestamp aligned data stream.

[0031] In the above embodiments, the multimodal perception module acquires environmental data of the robot's operating environment and the robot's own state data in real time. Based on the environmental and state data, the dynamic task understanding and planning module generates the task execution link for the current grasping task. The task execution link includes one or more ordered sub-task nodes and their associated context constraints. The multimodal control module executes the task execution link, wherein the multimodal control module calls the corresponding dedicated controller according to the type of the currently activated sub-task node. During task execution, the adaptive evaluation and replanning module continuously monitors the execution effect of the multimodal control module and changes in environmental data, and calculates the task execution deviation. When the task execution deviation exceeds a first preset threshold, the current sub-task is determined to have failed, triggering local replanning, generating an alternative execution strategy for the current sub-task, and updating the task execution link. When local replanning fails or the task execution deviation exceeds a higher second preset threshold, global replanning is triggered, and the dynamic task understanding and planning module regenerates a new task execution link starting from the current state based on the latest environmental and state data.

[0032] The multimodal control module includes: a navigation controller for handling the subtask of the robot moving to the target area; an attitude adjustment controller for handling the subtask of adjusting the robot's end effector or body to a preset grasping pose; and a grasping operation controller for handling the subtask of performing grasping actions and applying interactive forces. The corresponding controller is automatically switched and activated based on the sequence of subtask nodes in the task execution chain. Task execution deviation is calculated, specifically: for the navigation controller, the deviation is calculated based on the deviation between the real-time position and the planned path, as well as the threat level of dynamic obstacles; for the attitude adjustment controller, the deviation is calculated based on the error between the current end effector pose and the target grasping pose, as well as the stability of the visual target features; and for the grasping operation controller, the deviation is calculated based on the difference between the actual interactive force and the expected force, as well as the displacement of the target object after grasping.

[0033] When generating the task execution chain, a set of alternative strategies corresponding to each sub-task node is generated simultaneously; alternative execution strategies are generated, with priority given to selecting from the set of alternative strategies; if no strategy is available in the set of alternative strategies, a new strategy is generated online based on the current environment and status data. Environmental data includes, but is not limited to, RGB-D images, laser point clouds, and sonar data; status data includes, but is not limited to, joint angles, end-effector pose, chassis odometer readings, and force / torque sensor readings.

[0034] Step S1: Multimodal environment perception and initial feature extraction. The robot scans the environment using an RGB-D camera and a six-dimensional force / torque sensor mounted on the end effector.

[0035] Specifically, it acquires color images, depth images, and point cloud data of the target to be captured.

[0036] Pre-trained convolutional neural networks (such as Mask R-CNN) are used to extract the target's category label (such as "apple" or "metal cup"), geometric dimensions, and initial pose.

[0037] Meanwhile, the semantic segmentation network is used to analyze the surface texture of objects and predict their physical property parameters (such as friction coefficient μest and elastic modulus Eest).

[0038] Step S2: Generate initial grasping pose and initial control parameters. Input the information extracted in step S1 into the grasping strategy generation network.

[0039] The network outputs the initial grasping pose (x0, y0, z0, α0, β0, γ0).

[0040] Simultaneously, based on the predicted physical property parameters, the initial expected impedance parameters, including the target stiffness, are calculated by looking up a table. and target damping For example, for "apples" predicted to be fragile, set a lower [value]. To maintain smoothness; for "metal cups", set higher. To ensure crawling stability.

[0041] Step S3: Construct a multimodal spatiotemporal feature fusion model. During the process of the robotic arm approaching the object according to the initial pose, real-time visual flow data (changes in continuous frame images) and force flow data (changes in force / torque sensor readings) are collected.

[0042] Using the Transformer encoder-decoder architecture, visual flow features are... Characteristics of force flow Perform feature alignment.

[0043] The correlation matrix between visual and force features is calculated using a cross-attention mechanism. For example, when the visual sensor detects a slight displacement of an object, but the force sensor does not detect any contact force, the model determines this as a "slippage" risk and incorporates it into the fused feature vector analysis. The weight of this risk signal will be increased.

[0044] Step S4: Adaptive parameter adjustment based on variable impedance control will fuse the feature vector generated in step S3. The input is fed into the dynamic impedance controller.

[0045] The controller employs a Long Short-Term Memory (LSTM) network structure to predict the optimal impedance adjustment at the current moment.

[0046] The calculation formula is: Where η is the learning rate.

[0047] Key point: When the fusion feature shows "sudden change in contact force" or "object slippage", the controller rapidly increases the damping Bnew to suppress oscillation and dynamically adjusts the stiffness Knew according to the slippage direction to increase gripping force.

[0048] Step S5: Combining dynamic feedforward compensation and precise control with adaptively adjusted impedance parameters, construct the robot's dynamic equations: in, The environmental contact torque is used. To address unknown disturbances in underwater or complex environments, a radial basis function (RBF) neural network is introduced as a compensator for uncertainties. The network approximates the nonlinear error terms in the dynamic model in real time, generating the compensating torque. .

[0049] The final control torque output is: ,in The desired torque is calculated based on variable impedance control.

[0050] Step S6: Online learning and parameter update. After the grabbing action is completed, construct the reward function R based on the grabbing result (success / failure / slip).

[0051] The parameters of the dynamic impedance controller and the RBF neural network are updated online using reinforcement learning algorithms (such as the PPO algorithm).

[0052] If the capture is successful and the process is smooth, the weights of the current policy network are increased; if slippage occurs, the penalty intensity is increased for the state characteristics that caused the slippage (such as low friction characteristics), forcing the network to optimize the impedance adjustment strategy for the next time.

[0053] Example 3: like Figure 3 As shown, based on Embodiment 1, this embodiment of the invention provides an adaptive grasping control system for a robot, comprising: The perception module is used to collect initial environmental data and dynamic interaction sequence data of the target to be grasped through multimodal sensors; The decision reasoning module integrates a trend prediction network and a multimodal spatiotemporal Transformer fusion model to generate an initial grasping pose, an initial impedance parameter set, and an environmental state representation vector based on environmental data. The adaptive control module integrates an adaptive variable impedance controller, which is used to calculate the real-time impedance correction based on the environmental state characterization vector and generate the target impedance control law. The execution drive module is used to convert the target impedance control law into a joint torque drive signal to control the robotic arm to perform grasping actions; The learning optimization module is used to update the algorithm parameters in the decision reasoning module and the adaptive control module based on the feedback of the captured results.

[0054] The adaptive control module also includes: The dynamic decoupling unit is used to eliminate the nonlinear coupling effects between the joints of the robotic arm and decouple the target impedance control law into control commands for each independent joint. The safety monitoring unit is used to monitor joint torque, joint speed and motor temperature in real time. When the monitored data exceeds the preset safety boundary, it forcibly cuts off the output of the execution drive module and triggers the robot arm to stop and brake. The system is equipped with edge computing nodes, and the decision reasoning module and adaptive control module are deployed on these edge computing nodes to ensure real-time data processing. Example 4: Figure 3 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present invention is shown.

[0055] The electronic device may include a central processing unit / microprocessor / main control chip, etc. 4; and a storage medium 5, coupled to the central processing unit / microprocessor / main control chip, etc. 4, and storing computer-executable instructions therein for performing the steps of various methods of embodiments of the present invention when executed by the processor.

[0056] The central processing unit / microprocessor / main control chip, etc., can include, but are not limited to, one or more processors or microprocessors.

[0057] Storage medium 5 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CDROM, DVDROM, Blu-ray disc, etc.).

[0058] In addition, the electronic device may also include (but is not limited to) a data bus 6, an input / output bus / external bus / device bus 7, a display 8, and input / output devices 9 (e.g., keyboard, mouse, speaker, etc.).

[0059] The central processing unit / microprocessor / main control chip, etc. 4 can communicate with external devices (8, 9, etc.) via I / O bus 7 through wired or wireless network (not shown).

[0060] Storage medium 5 may also store at least one computer-executable instruction for performing the steps of various functions and / or methods in the embodiments described herein when the central processing unit / microprocessor / main control chip, etc., 4 is running.

[0061] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0062] Figure 4 A schematic diagram of a computer-readable storage medium according to an embodiment of the present invention is shown.

[0063] like Figure 4 As shown, the non-transitory computer-readable storage medium 11 stores instructions, such as computer-readable instructions 10. When the computer-readable instructions 10 are executed by a processor, the various methods described above can be performed. The non-transitory computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-transitory non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the non-transitory computer-readable storage medium 11 can be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions 10 stored on the computer-readable storage medium 11, the various methods described above can be performed.

[0064] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0065] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0066] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0067] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods of the various embodiments of this invention through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive grasping control method for a robot, characterized in that, Includes the following steps: Initial environmental data of the target to be grasped is collected by a multimodal sensor. The initial environmental data includes visual point cloud data, semantic data of object surface material, and task instruction attribute data. The initial environmental data is input into a pre-trained trend prediction network to generate an initial grasping pose and a predefined set of initial impedance parameters, which includes desired stiffness, desired damping, and desired moment of inertia. The robotic arm is controlled to move toward the target to be grasped according to the initial grasping posture, and dynamic interactive sequence data is collected in real time through force sensors and vision sensors. A multimodal spatiotemporal Transformer fusion model is constructed to align and deeply fuse the visual feature stream and force feature stream in the dynamic interaction sequence data, and output the environmental state representation vector at the current moment. Based on the environmental state characterization vector, the real-time impedance correction amount at the current moment is calculated using an adaptive variable impedance controller, and the target impedance control law is calculated by combining the initial impedance parameter set. The target impedance control law is mapped to the joint space to generate a joint torque drive signal, which controls the robotic arm to perform an adaptive grasping action. The network weights of the trend prediction network and the adaptive variable impedance controller are updated based on the grasping result.

2. The adaptive grasping control method for a robot as described in claim 1, characterized in that, The specific process of generating a predefined set of initial impedance parameters includes: Extract the material feature vector from the semantic data of the object's surface material, and the risk level label from the task instruction attribute data; The material feature vector is concatenated with the risk level label and then input into the material-impedance mapping neural network. The material-impedance mapping neural network adopts a multilayer perceptron structure and outputs a reference stiffness value and a reference damping value that match the current material and the risk of the task. Based on the current load state of the robotic arm, the reference stiffness value and reference damping value are dynamically constrained and verified to obtain the initial impedance parameter set.

3. The adaptive grasping control method for a robot as described in claim 1, characterized in that, The processing steps of the multimodal spatiotemporal Transformer fusion model include: Embedding encoding is performed on the visual feature stream and the force feature stream in the time dimension to obtain the visual embedding sequence and the force embedding sequence; The cross-attention mechanism is used to calculate the association weight matrix between the visual embedding sequence and the force embedding sequence, capturing force compensation information under visual occlusion and visual verification information under force mutation. The fused feature sequence is modeled using a multi-head self-attention mechanism, and a six-dimensional environmental state representation vector is output, which includes the environmental contact state, the object sliding trend, and the degree of object deformation.

4. The adaptive grasping control method for a robot as described in claim 1, characterized in that, The specific steps for calculating the real-time impedance correction using an adaptive variable impedance controller include: Construct a state observer based on Kalman filtering, and estimate the contact stiffness and environmental deformation of the current contact point according to the environmental state characterization vector; Calculate the stiffness error between the current estimated contact stiffness and the expected contact stiffness, as well as the current tracking speed error of the end effector; An error cost function based on the Lyapunov stability criterion is designed. The error cost function is minimized by the gradient descent algorithm, and the expected stiffness correction coefficient and expected damping correction coefficient for the current moment are obtained.

5. The adaptive grasping control method for a robot as described in claim 1, characterized in that, The process of updating network weights based on crawling results includes: The detection function checks whether the final position of the object remains within a predetermined threshold range after the grasping action is completed, and whether the force sensor detects an impact force exceeding the safety threshold. If the object's position remains stable and there is no impact force, the current sample is marked as a successful sample, and the trend prediction network is fine-tuned using the backpropagation algorithm. If a slip or impact occurs, it is marked as a failure sample. The residual between the actual output torque and the ideal torque is calculated, and the policy parameters of the adaptive variable impedance controller are updated penalizedly using a reinforcement learning algorithm.

6. The adaptive grasping control method for a robot as described in claim 1, characterized in that, It also includes steps for handling unexpected interference: During the calculation of the real-time impedance correction at the current moment, the mutation rate of the environmental state characterization vector is monitored in real time. When the rate of change of the environmental state representation vector exceeds the preset disturbance threshold, it is determined that external interference has occurred. Immediately activate the instantaneous protection mode, lock the current joint angle, and switch the desired stiffness in the target impedance control law to the maximum preset safety stiffness to resist external impacts until the mutation rate returns to the normal range.

7. The adaptive grasping control method for a robot as described in claim 1, characterized in that, The methods for acquiring dynamic interactive sequence data include: Before the end effector of the robotic arm makes contact with the target to be grasped, high-resolution visual point cloud data is acquired at the first sampling frequency. At the instant the end effector contacts the target to be grasped, the force sensor is simultaneously triggered to collect six-dimensional force / torque data at a second sampling frequency, where the second sampling frequency is higher than the first sampling frequency. Background removal and point cloud registration are performed on the acquired visual point cloud data, and gravity compensation and filtering noise reduction are performed on the force / torque six-dimensional data to form a synchronized timestamp aligned data stream.

8. An adaptive grasping control system for a robot according to any one of claims 1 to 7, characterized in that, include: The perception module is used to collect initial environmental data and dynamic interaction sequence data of the target to be grasped through multimodal sensors; The decision reasoning module integrates a trend prediction network and a multimodal spatiotemporal Transformer fusion model to generate an initial grasping pose, an initial impedance parameter set, and an environmental state representation vector based on environmental data. The adaptive control module integrates an adaptive variable impedance controller, which is used to calculate the real-time impedance correction based on the environmental state characterization vector and generate the target impedance control law. The execution drive module is used to convert the target impedance control law into a joint torque drive signal to control the robotic arm to perform grasping actions; The learning optimization module is used to update the algorithm parameters in the decision reasoning module and the adaptive control module based on the feedback of the captured results.

9. An electronic device, comprising: At least one memory stores computer-executable instructions non-transiently; At least one processor, configured to run the computer-executable instructions, The computer-executable instructions are executed by the processor to implement the adaptive grasping control method for a robot according to any one of claims 1-7.

10. A computer-readable storage medium, wherein, The computer-readable storage medium stores computer-executable instructions, which, when executed by at least one processor, implement an adaptive grasping control method for a robot according to any one of claims 1-7.

Images