A few-sample precise plug-in skill learning method based on vision and force sensation

Abstract

This application relates to the technical field of robot intelligence, and particularly to a method for learning precise insertion skills using few samples based on vision and force perception. The method includes constructing multimodal observations; generating a desired motion sequence for the robot's end effector using a policy model conditioned on the multimodal observations; training a procedural reward model; training the policy model using a training dataset containing successful and failed / backoff samples; and during deployment, the policy model outputs the desired motion sequence of the end effector to perform insertion, and the procedural reward model continuously evaluates the insertion progress, determining successful insertion when the evaluation score exceeds a threshold. This application has the advantage of enabling the robot to simultaneously learn tactile feedback and failure-backoff capability with a very small amount of teaching data, without relying on large-scale data or manual force control rules, and maintaining a high success rate at sub-millimeter alignment.

Description

Technical Field

[0001] This application relates to the technical field of robot intelligence, and in particular to a method for learning precise interlocking skills using few samples based on vision and force perception. Background Technology

[0002] Precision insertion is a high-frequency and highly complex operation in industrial assembly and service robots, widely used in electronic component assembly, consumer electronics manufacturing, and electric vehicle charging. With the development of intelligent manufacturing and automated assembly, the demand for precision insertion skills with sub-millimeter alignment accuracy is increasing, especially in scenarios such as memory module insertion, USB connection, and charging gun docking, where the alignment tolerance is typically less than 1mm.

[0003] In existing technologies, precision mating mainly relies on the following approaches. First, end-to-end imitation learning or reinforcement learning methods based on big data require hundreds to thousands of teaching trajectories, resulting in high acquisition costs and long cycles. Furthermore, the pursuit of generalization weakens the specialization towards a single high-precision interface, leading to unstable sub-millimeter alignment success rates. Second, methods combining visual servoing with manual force control rules rely heavily on expert parameter tuning. These rules are strongly bound to specific interfaces, making it difficult to express continuous tactile feedback and exhibiting poor robustness to unmodeled failure states. Moreover, existing end-to-end solutions based on imitation learning cannot autonomously determine and retry when encountering mating errors, leading to a sharp drop in success rate or hardware damage in real-world conditions. Summary of the Invention

[0004] This application provides a few-sample precision interlocking skill learning method based on vision and force perception, which solves the technical problem of enabling robots to simultaneously learn tactile sensation and failure recovery ability with a very small amount of teaching data without relying on large-scale data or manual force control rules, and maintaining a high success rate in sub-millimeter alignment.

[0005] This application provides a few-sample precision interpolation skill learning method based on vision and force perception, which adopts the following technical solution: A few-sample precision interpolation skill learning method based on vision and force perception includes: Construct a multimodal observation, which is formed by concatenating a pre-trained visual feature vector and a normalized force perception vector along the feature dimension; Based on the aforementioned multimodal observations, the desired motion sequence of the robot's end effector is generated through a policy model; A process reward model is trained, which is trained with artificial binary labels as supervision signals and outputs continuous scores representing the insertion progress. During the training phase, the process reward model serves as a process reward signal to drive the policy model to learn contact adjustment behaviors related to force perception, and during the deployment phase, the process reward model serves as a threshold criterion for successful insertion. The policy model is trained using a training dataset containing successful samples and failure-backup samples, wherein the failure-backup samples demonstrate the action sequence of retreating, realigning, and re-inserting when the contact force changes abruptly. During deployment, the desired motion sequence at the end of the strategy model is used to perform the insertion, and the procedural reward model continuously evaluates the insertion progress. When the evaluation score exceeds the threshold, the insertion is considered successful.

[0006] In one embodiment: the step of constructing multimodal observations includes: The image data captured by the robot's end-effector camera is acquired and input into a pre-trained visual feature extractor, which outputs the pre-trained visual feature vector. A six-dimensional force vector consisting of three-dimensional force and three-dimensional torque is acquired from a six-dimensional force sensor on the wrist. The six-dimensional force vector is normalized according to a calibrated range or statistical mean and variance to obtain the normalized force vector. The pre-trained visual feature vector and the normalized force perception vector are concatenated along the feature dimension to obtain the multimodal observation.

[0007] In one embodiment: the strategy model is a flow matching generation model, which is used to characterize the ordinary differential equation that continuously transforms the prior distribution of Gaussian noise into the target action distribution.

[0008] In one embodiment: the step of generating the desired motion sequence of the robot's end effector through a policy model, conditioned on the multimodal observations, includes: An initial noise sequence is obtained by sampling from a Gaussian prior distribution, and the initial noise sequence is input into the policy model along with the current multimodal observations. The initial noise sequence is transformed into an action distribution by numerically integrating the ordinary differential equation along the velocity field learned by the policy model. The terminal velocity sequence of multiple future time steps is output from the motion distribution in a motion block manner as the desired motion sequence.

[0009] In one embodiment: the step of training a procedural reward model includes: Acquire plug-in trajectory data including success, near success, failure, or far from success status; The state of each frame or segment in the interpolated trajectory data is manually binary-labeled by the annotator. Using the artificial binary annotation as a supervision signal, the procedural reward model is trained under supervision using the binary cross-entropy loss function, so that the procedural reward model outputs a continuous score in the interval between 0 and 1.

[0010] In one embodiment: the step of training the policy model using a training dataset containing successful samples and failure-backoff samples includes: The policy model is trained using an off-policy reinforcement learning algorithm; During training, the continuous scores output by the procedural reward model are used as the reward signal for each step. The strategy model is driven to learn to fine-tune the contact state based on the normalized force perception vector, with the goal of maximizing the cumulative process reward.

[0011] In one embodiment: the process of obtaining the training dataset includes: The robot is controlled to autonomously explore and plug in using the current strategy model; The uncertainty of the strategy model output is calculated in real time during the exploration process; When the uncertainty exceeds the preset intervention threshold, a human operator is triggered to take over and demonstrate the action sequence of retraction, realignment and re-insertion corresponding to the failed-retrograde sample. The generated takeover teaching data is used as the failure-rollback samples to supplement the training dataset for iterative model fine-tuning.

[0012] In one embodiment: the step of performing the insertion of the desired motion sequence output from the end of the strategy model includes: The system refreshes the action blocks output by the strategy model at a preset frequency, and sequentially sends the end-effector velocities in the action blocks to the robot execution layer; In the robot execution layer, the end effector velocity is converted into robot joint command execution through damped least squares inverse kinematics calculation; Meanwhile, the procedural reward model calculates the continuous score of the current state in real time. If the continuous score reaches the threshold, a stop command is sent to the execution layer to end the action.

[0013] In one embodiment: before inputting the initial noise sequence and the current multimodal observations into the policy model, a step of utilizing the force-sensory gradient for potential spatial perturbation is included, specifically comprising: Calculate the time difference gradient of the normalized force perception vector with respect to time; When the time difference gradient exceeds a preset force perception abrupt change threshold, a directional potential spatial perturbation vector is generated using the time difference gradient. The potential spatial perturbation vector is superimposed on the initial noise sequence to obtain the corrected noise sequence; The step of inputting the initial noise sequence and the current multimodal observations into the policy model is replaced by inputting the corrected noise sequence and the current multimodal observations into the policy model.

[0014] In one embodiment: the step of generating a directional latent spatial perturbation vector using the time difference gradient includes: Extract the opposite direction of the increase in drag in the time difference gradient as the perturbation direction; The perturbation step size is calculated based on the magnitude of the time difference gradient; The potential spatial perturbation vector is generated based on the perturbation direction and the perturbation step size, so that the strategy model avoids the current high drag direction when generating the end velocity sequence for multiple future time steps, thereby enabling the robot to escape the local deadlock state during the precision insertion process.

[0015] In summary, this application has the following beneficial effects: 1. By splicing visual features with normalized force vectors, the high sensitivity of force signals to contact states is preserved to the maximum extent. 2. A flow matching generation model is adopted, and the desired motion sequence is generated by action block generation. This reduces the inference frequency and suppresses high-frequency jitter, thereby improving the timing consistency of the robot in sub-millimeter level operations. 3. A process-based reward model was creatively designed. This same model is used as a reward signal to drive contact behavior learning during the training phase and as a threshold standard for success determination during the deployment phase, replacing the complex and fragile manual force-controlled state machine. 4. Actively introduce datasets containing failure-back-off samples into the training data so that the policy model not only learns how to insert, but also learns the ability to autonomously back off and realign when there are sudden changes in contact force. 5. A potential spatial perturbation mechanism guided by force gradient is introduced. When encountering hard contact or abrupt changes, the physical resistance feedback is directly mapped to the directional perturbation at the input end of the generated model. This allows the robot to naturally avoid high resistance directions without modifying the model structure, greatly improving its ability to escape local deadlock. Attached Figure Description

[0016] Figure 1 This is a flowchart of the few-sample precision interpolation skill learning method based on vision and force perception in this embodiment; Figure 2This is a schematic diagram of the failure-back-off mechanism in the few-sample precision interlocking skill learning method based on vision and force perception in this embodiment. Detailed Implementation

[0017] To better understand the purpose, technical solutions, and advantages of this application, it has been described and illustrated below with reference to the accompanying drawings and embodiments. However, those skilled in the art should understand that this application can be implemented without these details. In some cases, to avoid obscuring various aspects of this application due to unnecessary description, well-known methods, processes, systems, components, and / or circuits already described at a higher level will not be elaborated upon. It will be apparent to those skilled in the art that various modifications can be made to the embodiments disclosed in this application, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the illustrated embodiments, but conforms to the broadest scope consistent with the scope of protection claimed in this application.

[0018] It should be noted that the descriptions of these embodiments are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0019] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0020] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0021] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication and program execution functions, such as a personal computer, server, etc., or a robot control system capable of realizing the above functions.

[0022] A few-sample precision interpolation skill learning method based on vision and force perception, such as Figure 1 As shown, the method includes: S100, Construct a multimodal observation, which is formed by concatenating a pre-trained visual feature vector and a normalized force perception vector along the feature dimension.

[0023] It should be noted that multimodal observation refers to a comprehensive state representation that integrates data from two different sensor modalities: visual and force. The pre-trained visual feature vector is a general image feature representation automatically extracted by a deep neural network. The normalized force vector is three-dimensional force and torque data that has undergone range or mean-variance normalization.

[0024] Understandably, by concatenating visual features with a normalized force vector—in this embodiment, concatenating the visual feature vector and the normalized force vector end-to-end along the feature dimension—the high sensitivity of the force signal to contact states can be preserved to the greatest extent, providing rich and accurate environmental perception information for subsequent insertion strategies.

[0025] In the specific implementation, the system at each moment Construct a multimodal observation The multimodal observations serve as the common input to the subsequent policy model and reward model, without applying any other weighting or projection transformations to avoid disrupting the original contact force distribution characteristics.

[0026] S200, based on the multimodal observations, the desired motion sequence of the robot end effector is generated through a strategy model.

[0027] It should be noted that the policy model is a deep learning model used to determine the robot's next action based on the current observation state. The expected motion sequence refers to the expected velocity or displacement trajectory of the robot's end effector over multiple future time steps.

[0028] Understandably, generating the desired motion sequence based on multimodal observations enables the robot to dynamically adjust its motion direction in real time according to the relative position of the interface seen visually and the contact resistance felt by force, thereby achieving high-precision alignment and insertion.

[0029] In its implementation, the strategy model receives the concatenated multimodal observation vectors, performs feature mapping and calculations through its internal network structure, and finally outputs the expected end velocity of six degrees of freedom.

[0030] S300, train a process reward model. The process reward model is trained with artificial binary labels as supervision signals and outputs continuous scores representing the insertion progress. During the training phase, the process reward model serves as a process reward signal to drive the policy model to learn contact adjustment behaviors related to force perception. During the deployment phase, the process reward model serves as a threshold criterion for determining whether the insertion is successful.

[0031] It should be noted that the process-based reward model is a neural network evaluator specifically designed to assess the progress of the current insertion task. Manual binary labeling involves the operator assigning discrete "0" (failure) or "1" (success) markers to trajectory segments.

[0032] Understandably, by creatively designing this procedural reward model, it can provide dense, continuous reward signals during the training phase, accelerating the convergence of reinforcement learning; and during the deployment phase, it can naturally be transformed into a judgment criterion for successful insertion, thus completely replacing the complex and fragile manual force-controlled state machine in traditional solutions.

[0033] In the specific implementation, the reward model outputs a scalar score in the range of 0 to 1. The closer the score is to 1, the closer it is to a successful connection.

[0034] S400, the policy model is trained using a training dataset containing successful samples and failure-backup samples, wherein the failure-backup samples demonstrate the action sequence of retreating, realigning, and re-inserting when the contact force changes abruptly.

[0035] It should be noted that failure-backup samples refer to trajectory data actively included in the teaching data that simulates the robot's exit and retry operation when encountering jamming or misalignment during the insertion process.

[0036] Understandably, actively introducing failure-backup samples into the training data enables the policy model not only to learn how to insert smoothly, but also to acquire the ability to autonomously correct errors, back off, and realign when contact forces undergo abnormal changes, thus significantly improving the model's robustness in real unstructured environments.

[0037] In its implementation, the system employs an off-policy reinforcement learning algorithm combined with takeover teaching data in the replay buffer to continuously and alternately update the parameters of the policy model and the reward model.

[0038] S500, during deployment, the expected motion sequence at the end of the strategy model is used to perform insertion, and the procedural reward model continuously evaluates the insertion progress. When the evaluation score exceeds the threshold, the insertion is determined to be successful.

[0039] It should be noted that deployment refers to the stage where the model training is completed and online inference execution is performed on the actual robot hardware.

[0040] Understandably, the closed-loop architecture, where actions are generated by the strategy model and the stop is determined by the reward model, enables the entire precision insertion process to be fully autonomous and has clear task termination conditions.

[0041] In practice, if the continuous score output by the reward model exceeds a preset threshold, the system determines that the connection is successful and stops the movement.

[0042] In one possible implementation, the step of constructing multimodal observations in step S100 above includes: S101, acquire image data captured by the robot's end-effector camera, input it into a pre-trained visual feature extractor, and output the pre-trained visual feature vector; S102, acquire the six-dimensional force vector composed of three-dimensional force and three-dimensional torque collected by the wrist six-dimensional force sensor, and normalize the six-dimensional force vector according to the calibrated range or statistical mean and variance to obtain the normalized force vector; S103, the pre-trained visual feature vector and the normalized force perception vector are concatenated along the feature dimension to obtain the multimodal observation.

[0043] In the embodiments described in this specification, the visual branch uses an end-effector camera to acquire RGB or RGB-D images. Input to a pre-trained visual feature extractor (visual encoder). Output visual feature vector The output visual feature vector is a general image feature representation automatically extracted by a deep neural network, and the visual feature extractor can use a convolutional backbone (such as ResNet) or a self-supervised pre-trained backbone (such as DINO). The force feedback branch handles six-dimensional force / torque signals. Range normalization is performed to obtain Finally, the two are spliced ​​together to obtain multimodal observations:

[0044] Furthermore, the strategy model is a flow matching generation model, which is used to characterize the ordinary differential equation that continuously transforms the prior distribution of Gaussian noise into the target action distribution.

[0045] In the embodiments described in this specification, flow matching is a method for generating continuous normalized flows: during training, a time-based algorithm is learned. Parameterized velocity field This allows the sample to be continuously transformed from the prior distribution to the target action distribution along the ordinary differential equation (ODE). Given the target action in the teaching... Construction made of noise arrive Interpolation path:

[0046] Its corresponding target speed is Its conditional flow matching loss is:

[0047] in, For conditional flow matching loss, For model parameters, For the desired operation, For time parameters, The initial Gaussian noise, For the target action, Data taken from teaching or exploration. The velocity field learned by the policy model. The above training supervised by "imitating the target action" corresponds to the role of behavior cloning—the flow matching loss makes the generated distribution approximate the taught action distribution.

[0048] Optionally, step S200 may specifically include: S201. Obtain an initial noise sequence generated by sampling from a Gaussian prior distribution, and input the initial noise sequence and the current multimodal observation into the policy model; S202. Numerically integrate the ordinary differential equation along the velocity field learned by the strategy model to transform the initial noise sequence into an action distribution; S203. Output the end velocity sequence of multiple future time steps from the motion distribution in the manner of motion block division, as the desired motion sequence.

[0049] In the embodiments described in this specification, the inference stage uses Gaussian noise. Let's begin by using Euler's method for numerical integration:

[0050] in, , , For the first The action sequence state of the step, The integration step size is... The total number of steps for integration. For the first The time parameter of the step.

[0051] Final score Get action blocks That is, the future End velocity sequence of the step This is known as ActionChunking, which reduces inference frequency and suppresses high-frequency jitter. The system generates or refreshes action chunks at approximately 20Hz, and then sequentially distributes the end-point velocities within them to the execution layer.

[0052] Furthermore, before inputting the initial noise sequence and the current multimodal observations into the policy model, the method further includes a step of using force-sensory gradients to perform potential spatial perturbations: Calculate the time difference gradient of the normalized force perception vector with respect to time; When the time difference gradient exceeds a preset force perception abrupt change threshold, a directional potential spatial perturbation vector is generated using the time difference gradient. The potential spatial perturbation vector is superimposed on the initial noise sequence to obtain the corrected noise sequence; The step of inputting the initial noise sequence and the current multimodal observations into the policy model is replaced by inputting the corrected noise sequence and the current multimodal observations into the policy model.

[0053] In the embodiments of this specification, a potential spatial perturbation mechanism guided by force gradient is introduced, which directly maps the physical resistance feedback into a directional perturbation at the input of the generation model when encountering hard contact or abrupt changes.

[0054] Specifically, let the normalized force perception vector be... The system calculates its time-difference gradient with respect to time in each control cycle:

[0055] in, For time difference gradient, The normalized force perception vector at the current moment. The normalized force perception vector from the previous moment. For time step.

[0056] Further, the step of generating a directional potential spatial perturbation vector using the time difference gradient includes: Extract the opposite direction of the increase in drag in the time difference gradient as the perturbation direction; The perturbation step size is calculated based on the magnitude of the time difference gradient; The potential spatial perturbation vector is generated based on the perturbation direction and the perturbation step size, so that the strategy model avoids the current high drag direction when generating the end velocity sequence for multiple future time steps, thereby enabling the robot to escape the local deadlock state during the precision insertion process.

[0057] In the embodiments described in this specification, the perturbation mechanism enables the robot to naturally avoid high-resistance directions without modifying the model network structure, greatly improving its ability to escape local deadlock states.

[0058] In specific implementation, when (in When the force perception abrupt change threshold is set, the opposite direction of the increase in resistance is extracted as the perturbation direction:

[0059] The perturbation step size is calculated based on the gradient magnitude. ( (This is a scaling factor). This generates the potential spatial perturbation vector:

[0060] Subsequently, the perturbation vector is superimposed onto the initial Gaussian noise sequence. The corrected noise sequence is obtained as follows:

[0061] Strategy model with Starting with numerical integration of the ordinary differential equation, the resulting action blocks are obtained. The natural deviation from the high-resistance direction guides the end effector to actively retract or slide, breaking the local deadlock caused by hard contact. The force-sensor time-difference gradient is a six-dimensional force / torque vector in the end effector coordinate system, corresponding one-to-one with the six-dimensional velocity components of the end effector. The resulting six-dimensional perturbation direction is applied along each time step of the action block, thus obtaining a perturbation vector of the same dimension as the initial noise sequence.

[0062] In one possible implementation, step S300 may specifically include: S301. Obtain plug-in trajectory data containing successful, near-successful, failed, or far-from-successful states; S302, The annotator performs manual binary annotation on the state of each frame or segment in the interpolated trajectory data; S303. Using the artificial binary label as a supervision signal, the process reward model is trained under supervision using the cross-entropy loss function, so that the process reward model outputs a continuous score in the interval between 0 and 1.

[0063] In the embodiments described in this specification, the reward model Training using binary cross-entropy loss:

[0064] in , To reward the model's cross-entropy loss, For reward model parameters, For the desired operation, For multimodal observation, For manually labeled binary signals, This is a process-oriented reward model. Through extensive manual discrete annotation, the reward model is forced to learn the continuous evolution of the insertion process. After training... It can output continuous progress scores for new, unlabeled states. .

[0065] Further, step S400 may specifically include: S401. The policy model is trained using an off-policy reinforcement learning algorithm; S402. During the training process, the continuous scores output by the procedural reward model are used as the reward signal for each step. S403. Drive the strategy model to learn to fine-tune the contact state based on the normalized force perception vector under the objective of maximizing the cumulative process reward.

[0066] In the embodiments described in this specification, the reinforcement learning objective is to maximize the cumulative process reward:

[0067] in, For the desired operation, As a discount factor, For time steps, For the first The step-by-step process-based reward score effectively avoids the exploration difficulties caused by sparse rewards.

[0068] In practice, the cumulative process reward can be maximized by using the process reward (or its advantage value) as the sample weight to weight the flow matching loss within the flow matching training framework.

[0069] Optionally, the process of obtaining the training dataset includes: Initial teaching involves a human operator demonstrating approximately the first 10 trajectories via a teleoperated device (such as a 3D mouse) and storing them in a replay buffer to provide initial data for the policy and reward model. In this embodiment, the initial data consists of approximately 50 trajectories, including approximately 5 to 10 failure-rollback samples. The robot is controlled to autonomously explore and plug in using the current strategy model; During the exploration process, a guided exploration mechanism related to the uncertainty or entropy of the exploration data is introduced. Randomness modulated by the uncertainty of the strategy distribution is introduced into the action generation, so that the system strengthens the exploration near the state with high information content. During the exploration process, the uncertainty of the output of the strategy model is calculated in real time. The uncertainty can be obtained by sampling the initial noise of the same observation multiple times and statistically analyzing the dispersion (variance or entropy) of the generated action blocks.

[0070] When the uncertainty exceeds the preset intervention threshold (or when failure is imminent or hardware damage is possible), a human operator is triggered to take over and demonstrate the action sequence of retreat, realignment and reconnection corresponding to the failed-retracement sample. The resulting takeover teaching data is used as failure-backup samples to supplement the training dataset for iterative model fine-tuning. The takeover teaching data generated by intervention is processed in a basically consistent manner with the autonomous exploration data and stored in the same replay buffer as expert samples for policy imitation; the reward model is still binary labeled according to the actual insertion progress of each frame.

[0071] like Figure 2 As shown, among the approximately 50 data points mentioned above, there are about 5 to 10 samples that intentionally collide with the wrong hole / misaligned position, demonstrating the process of "returning - realigning - trying again"; the force signal changes significantly upon contact, and the model learns to use this force change as a cue to autonomously determine that the insertion is incorrect and to back off and try again.

[0072] Finally, the policy model (flow matching loss + process reward driven) and the reward model (human 0 / 1 labeled cross-entropy) are updated alternately in a disjoint policy manner until the policy stably reaches the success threshold in verification plugging.

[0073] In the embodiments described in this specification, the human-in-the-loop mechanism is used to determine the takeover timing based on the uncertainty of the model output, thus ensuring the collection of high-quality positive samples and failure-back-off samples.

[0074] Furthermore, Figure 2 The principle of the failure-backoff mechanism is explained in detail: In actual precision insertion scenarios, due to visual blind spots or system errors, robots are prone to misalignment or incorrect insertion, leading to jamming. This embodiment introduces the aforementioned active failure-backoff samples into the training set, enabling the strategy model to establish a mapping relationship between "force perception mutation" and "spatial retreat".

[0075] Specifically, when the robot's end effector encounters hard contact, the normalized force vector in its multimodal observations exhibits significant high-frequency abrupt changes. Upon receiving this multimodal observation containing these abrupt changes, the policy model's internally learned ordinary differential equation velocity field naturally guides the initial noise sequence integral towards a motion distribution of "end effector reverse movement (i.e., retreat)." Subsequently, when the force signal returns to the free-space level, the model reassesses the relative pose based on visual features, generating the desired motion sequence for realignment and re-attempting insertion. This mechanism completely breaks the limitations of traditional end-to-end models that rely on a single approach, endowing the robot with a human-like tactile error-correction instinct.

[0076] In one possible implementation, step S500 may specifically include: S501. The system refreshes the action blocks output by the strategy model at a preset frequency, and sequentially sends the end velocity in the action block to the robot execution layer. S502. In the robot execution layer, the end velocity is converted into robot joint command execution through damped least squares inverse kinematics calculation. S503. Simultaneously, the process reward model calculates the continuous score of the current state in real time. If the continuous score reaches the threshold, a stop command is sent to the execution layer to end the action.

[0077] In the embodiments described in this specification, the strategy model outputs the desired terminal velocity. (Including linear velocity and angular velocity). The execution layer needs to calculate it into the joint velocities of the robotic arm. The integral is then used to generate joint commands. The relationship between the end-effector velocity and the joint velocity is given by the Jacobian matrix. Given:

[0078] To avoid joint velocity divergence caused by singular configurations and to improve the success rate of inverse kinematics, damped least squares (DLS) is used to solve for joint velocities:

[0079] in The damping coefficient is... For Jacobian matrices, This is the transpose of the Jacobian matrix. It is the identity matrix. For the desired terminal velocity, For joint velocity. If (like If the connection is successful, the connection is considered complete. Whether or not joint limiting constraints are introduced in this embodiment does not affect the implementation of this solution; joint limiting constraints can be handled according to actual needs. (The result is...) The joint position command is then obtained by integrating the control cycle. The frequency is sent to the robotic arm driver at approximately 500~1000Hz to complete high-frequency closed-loop control.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for learning precise interlocking skills using few samples based on vision and force perception, characterized in that, include: Construct a multimodal observation, which is formed by concatenating a pre-trained visual feature vector and a normalized force perception vector along the feature dimension; Based on the aforementioned multimodal observations, the desired motion sequence of the robot's end effector is generated through a policy model; A process reward model is trained, which is trained with artificial binary labels as supervision signals and outputs continuous scores representing the insertion progress. During the training phase, the process reward model serves as a process reward signal to drive the policy model to learn contact adjustment behaviors related to force perception, and during the deployment phase, the process reward model serves as a threshold criterion for successful insertion. The policy model is trained using a training dataset containing successful samples and failure-backup samples, wherein the failure-backup samples demonstrate the action sequence of retreating, realigning, and re-inserting when the contact force changes abruptly. During deployment, the desired motion sequence at the end of the strategy model is used to perform the insertion, and the procedural reward model continuously evaluates the insertion progress. When the evaluation score exceeds the threshold, the insertion is considered successful.

2. The method for learning precise interlocking skills based on vision and force perception using few samples, as described in claim 1, is characterized in that... The steps for constructing multimodal observations include: The image data captured by the robot's end-effector camera is acquired and input into a pre-trained visual feature extractor, which outputs the pre-trained visual feature vector. A six-dimensional force vector consisting of three-dimensional force and three-dimensional torque is acquired from a six-dimensional force sensor on the wrist. The six-dimensional force vector is normalized according to a calibrated range or statistical mean and variance to obtain the normalized force vector. The pre-trained visual feature vector and the normalized force perception vector are concatenated along the feature dimension to obtain the multimodal observation.

3. The method for learning precise interlocking skills using few samples based on vision and force perception according to claim 2, characterized in that, The strategy model is a flow matching generation model, which is used to characterize the ordinary differential equations that continuously transform the prior distribution of Gaussian noise into the target action distribution.

4. The method for learning precise interlocking skills based on vision and force perception using few samples, as described in claim 3, is characterized in that... The step of generating the desired motion sequence of the robot's end effector through a policy model, based on the multimodal observations, includes: An initial noise sequence is obtained by sampling from a Gaussian prior distribution, and the initial noise sequence is input into the policy model along with the current multimodal observations. The initial noise sequence is transformed into an action distribution by numerically integrating the ordinary differential equation along the velocity field learned from the policy model. The terminal velocity sequence for multiple future time steps is output from the motion distribution in a motion block manner as the desired motion sequence.

5. The method for learning precise interlocking skills based on vision and force perception using a small number of samples, as described in claim 1, is characterized in that... The steps for training the procedural reward model include: Acquire plug-in trajectory data including success, near success, failure, or far from success status; The state of each frame or segment in the interpolated trajectory data is manually binary-labeled by the annotator. Using the artificial binary annotation as a supervision signal, the procedural reward model is trained under supervision using the binary cross-entropy loss function, so that the procedural reward model outputs a continuous score in the interval between 0 and 1.

6. The method for learning precise interlocking skills based on vision and force perception using few samples, as described in claim 1, is characterized in that... The step of training the policy model using a training dataset containing successful samples and failure-backup samples includes: The policy model is trained using an off-policy reinforcement learning algorithm; During training, the continuous scores output by the procedural reward model are used as the reward signal for each step. The strategy model is driven to learn to fine-tune the contact state based on the normalized force perception vector, with the goal of maximizing the cumulative process reward.

7. The method for learning precise interlocking skills based on vision and force perception using few samples according to claim 1, characterized in that... The process of obtaining the training dataset includes: The robot is controlled to autonomously explore and plug in using the current strategy model; The uncertainty of the strategy model output is calculated in real time during the exploration process; When the uncertainty exceeds the preset intervention threshold, a human operator is triggered to take over and demonstrate the action sequence of retraction, realignment and re-insertion corresponding to the failed-retrograde sample. The generated takeover teaching data is used as the failure-rollback samples to supplement the training dataset for iterative model fine-tuning.

8. The method for learning precise interlocking skills based on vision and force perception using few samples according to claim 1, characterized in that... The step of inserting the desired motion sequence output from the end of the strategy model includes: The system refreshes the action blocks output by the strategy model at a preset frequency, and sequentially sends the end-effector velocities in the action blocks to the robot execution layer; In the robot execution layer, the end effector velocity is converted into robot joint command execution through damped least squares inverse kinematics calculation; Meanwhile, the procedural reward model calculates the continuous score of the current state in real time. If the continuous score reaches the threshold, a stop command is sent to the execution layer to end the action.

9. The method for learning precise interlocking skills using few samples based on vision and force perception according to claim 4, characterized in that, Before inputting the initial noise sequence and the current multimodal observations into the policy model, the method further includes a step of using force-sensory gradients to perform potential spatial perturbations, specifically including: Calculate the time difference gradient of the normalized force perception vector with respect to time; When the time difference gradient exceeds a preset force perception abrupt change threshold, a directional potential spatial perturbation vector is generated using the time difference gradient. The potential spatial perturbation vector is superimposed on the initial noise sequence to obtain the corrected noise sequence; The step of inputting the initial noise sequence and the current multimodal observations into the policy model is replaced by inputting the corrected noise sequence and the current multimodal observations into the policy model.

10. The method for learning precise interlocking skills based on vision and force perception using few samples, as described in claim 9, is characterized in that... The step of generating a directional potential spatial perturbation vector using the time difference gradient includes: Extract the opposite direction of the increase in drag in the time difference gradient as the perturbation direction; The perturbation step size is calculated based on the magnitude of the time difference gradient; The potential spatial perturbation vector is generated based on the perturbation direction and the perturbation step size, so that the strategy model avoids the current high drag direction when generating the end velocity sequence for multiple future time steps, thereby enabling the robot to escape the local deadlock state during the precision insertion process.

Images