A robot control method and system based on contact phase scheduling

Abstract

The application provides a robot control method and system based on contact phase scheduling, and relates to the technical field of robot control. The application outputs contact probability and phase belief through a contact perception phase predictor, so that the reference action block generated by a low-frequency diffusion planner and the pose residual generated by a high-frequency residual corrector can work together, the low-frequency diffusion planner can focus on maintaining the coherence of the overall intention and visual semantics of a task, and the high-frequency residual corrector can correct according to the contact probability and phase belief and exert the correction driven by force feedback, so that the fast closed-loop response to the contact state in the task execution process is realized on the premise of maintaining the visual dominant task semantics, and the success rate, robustness and adaptability of the robot in the contact-rich operation task are improved as a whole.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a robot control method and system based on contact phase scheduling. Background Technology

[0002] Visual imitation learning and diffusion strategies perform well in vision-driven tasks such as grasping and handling, but for contact-rich operations like insertion, wiping, and drawer opening, success depends on millimeter-level geometric alignment and closed-loop responses to short-term force / torque transients. Images alone often fail to distinguish between insertion in place and jamming at the orifice, or tool disengagement and overpressure contact. Existing force perception strategies typically concatenate force history with visual tokens or fuse them through attention, outputting actions based on the action block update frequency; while some slow-fast strategies increase intra-block responses, they do not explicitly differentiate task stages or clearly define the degrees of freedom that can be modified at different stages. Specific shortcomings include: The effective timescale of force / torque is higher than the action block update frequency of generative strategies, making it difficult to utilize transients such as micro-collisions, stick-slip, and early jamming in a timely manner.

[0003] Continuous and indiscriminate injection of force signals will mix sensor noise into visual semantics in free space, affecting task intent planning.

[0004] Different stages require mutually exclusive or orthogonal correction degrees of freedom (e.g., plane search and normal insertion). High-speed correction without phase scheduling is prone to generating irrelevant corrections and inducing jamming.

[0005] When only a reference action block is generated without a control frequency residual closed loop, the adaptability to contact geometry offset and height offset is insufficient. Summary of the Invention

[0006] In view of this, the core technical problem to be solved by the present invention is: how to improve the success rate, robustness and adaptability of robots in contact with rich operation tasks, so as to achieve the coordination of low-frequency motion planning and high-frequency force feedback correction while maintaining the vision-dominated task semantics, and adaptively adjust the force / torque history injection intensity and correction degree of freedom according to the task stage.

[0007] On one hand, this invention provides a robot control method based on contact phase scheduling, comprising the following steps: S1, acquiring multimodal perception data of the robot, including multi-view images, force / torque history, and body state; S2, inputting the multimodal perception data into a contact perception phase predictor, which outputs contact probability and phase belief; wherein, the contact probability is used to characterize the likelihood of contact occurring within a future time window, and the phase belief is used to characterize the probability distribution of being in different preset task phases at a future preset offset time; S3, processing the multi-view images and force / torque history at a first frequency using a low-frequency diffusion planner. S4. The force / torque history, body state, contact probability, phase belief, and reference action block are processed by a high-frequency residual corrector at a second frequency higher than the first frequency to generate pose residuals. The high-frequency residual corrector dynamically adjusts the influence of the force / torque history on the pose residual generation process according to the contact probability, and selects or weights and combines multiple preset correction subspaces according to the phase belief to form an effective correction subspace, and limits the pose residuals within the effective correction subspace. S5. The pose residuals are composited with the reference action block to obtain the execution action, and the execution action is sent to the robot for execution.

[0008] Optionally, in step S2, the contact-aware phase predictor includes a visual encoder and a force / torque encoder. The visual encoder is used to extract visual tokens from multi-view images, and the force / torque encoder is used to extract force history tokens from force / torque history.

[0009] Optionally, in step S3, the low-frequency diffusion planner includes a multi-head cross-attention unit. The multi-head cross-attention unit uses the visual token as the query and the force history token as the key and value to perform multi-head cross-attention calculation. The phase belief is converted into a per-attention head gating vector through a head-level gating network, and the outputs of each attention head in the multi-head cross-attention calculation are weighted.

[0010] Optionally, the output of the multi-head cross-attention unit is orthogonalized relative to the visual token to obtain orthogonal residuals, and the orthogonal residuals are combined with the visual token to form a fusion token; the low-frequency diffusion planner generates a baseline action block based on the fusion token and the ontology state.

[0011] Optionally, multiple correction subspaces are defined by diagonal selection matrices, which are used to limit the combination of degrees of freedom that can be corrected under the corresponding task phase. The high-frequency residual corrector obtains an effective correction subspace by weighted summation of multiple diagonal selection matrices through phase belief, and performs overall gating of the pose residuals through contact probability.

[0012] Optionally, the formula for calculating the pose residual is: ;where p t c Let p be the contact probability at time t. t (k) represents the probability that the task belongs to the k-th task phase at time t, B k Let c be the diagonal selection matrix corresponding to the k-th task phase, where K is the total number of task phases. t The original candidate residuals generated for the high-frequency residual corrector.

[0013] Optionally, the method further includes the following steps: S6, using a physical prior model to automatically generate the target pose residual based on the force / torque history and phase belief; training the high-frequency residual corrector with the goal of minimizing the difference between the pose residual output by the high-frequency residual corrector and the target pose residual.

[0014] Optionally, a contact-aware phase predictor can be trained using future contact labels and future phase labels automatically generated from task phase and deterministic event rules.

[0015] Optionally, in step S5, the pose residual is composited with the reference action block, specifically by performing a composite operation between the reference action block and the pose residual in the pose space of the robot's end effector.

[0016] On the other hand, the present invention provides a robot control system, comprising: a multimodal data acquisition module for acquiring multimodal perception data, the multimodal perception data including multi-view images, force / torque history, and body state; a contact perception phase predictor, communicatively connected to the multimodal data acquisition module, for receiving the multimodal perception data and outputting contact probability and phase belief, the contact probability being used to characterize the likelihood of contact occurring within a future time window, and the phase belief being used to characterize the probability distribution of being in different preset task phases at a future preset offset time; and a low-frequency diffusion planner, communicatively connected to the contact perception phase predictor, for processing the multi-view images, force / torque history, and body state at a first frequency to generate a reference action block; A high-frequency residual corrector, communicatively connected to a low-frequency diffusion planner, processes force / torque history, body state, contact probability, phase belief, and reference action block at a second frequency higher than the first frequency to generate pose residuals. The high-frequency residual corrector dynamically adjusts the influence of force / torque history on the pose residual generation process based on the contact probability, and selects or weights an effective correction subspace from multiple preset correction subspaces based on the phase belief, confining the pose residuals within the effective correction subspace. An action compositing unit, communicatively connected to the high-frequency residual corrector, composites the pose residuals to the reference action block and outputs the executed action.

[0017] Implementing this invention offers the following advantages: This invention outputs contact probability and phase belief through a contact-aware phase predictor, enabling the reference action block generated by the low-frequency diffusion planner and the pose residual generated by the high-frequency residual corrector to work collaboratively. Specifically, this invention accurately predicts contact probability and phase belief through the contact-aware phase predictor, providing a basis for subsequent scheduling; it generates reference actions that maintain visual semantic coherence through the low-frequency diffusion planner, ensuring that the task intent does not drift; and it applies force feedback correction at appropriate times and degrees of freedom based on the contact probability and phase belief, achieving a rapid and accurate closed-loop response to contact states. Through the synergistic effect of these technical features, the low-frequency diffusion planner can focus on maintaining the overall task intent and visual semantic coherence, while the high-frequency residual corrector can apply force feedback-driven correction based on the contact probability and phase belief. This achieves a rapid closed-loop response to contact states during task execution while maintaining visual-dominated task semantics, thus improving the robot's success rate, robustness, and adaptability in contact-rich operation tasks. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the architecture of the contact phase scheduling vision-force slow-fast cooperative control of the present invention in one embodiment; Figure 2 This is a schematic diagram of the dual-gated vision-force fusion and orthogonal residual injection process in one embodiment; Figure 3 In one embodiment, the contact gating and phase routing correction subspace of the high-frequency residual corrector; Figure 4 This is a schematic diagram of the slow planning and fast correction closed-loop execution process of contact phase scheduling in one embodiment. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.

[0020] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0021] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0022] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0023] Example 1 In this embodiment, as Figure 1 The robot control method based on contact phase scheduling, as shown, includes the following steps: S1. Acquire multimodal perception data of the robot; multimodal perception data includes multi-view images, force / torque history and body state; multi-view images are acquired by a multi-view RGB camera, force / torque history (force history and torque history) is acquired by a six-dimensional force / torque sensor, and body state is acquired by the robot body state acquisition module.

[0024] S2. Input the multimodal sensing data into the contact sensing phase predictor, and the contact sensing phase predictor outputs the contact probability and phase belief; where the contact probability is used to characterize the possibility of contact occurring within a future time window, and the phase belief is used to characterize the probability distribution of the future preset offset time being in different preset task phases. S3. The multimodal sensing data is processed at the first frequency through a low-frequency diffusion planner (hereinafter also referred to as slow) to generate a baseline action block; S4. The force / torque history, body state, contact probability, phase belief, and reference action block in the multimodal sensing data are processed by a high-frequency residual corrector (hereinafter also referred to as Fast) at a second frequency higher than the first frequency to generate pose residuals. Among them, the high-frequency residual corrector dynamically adjusts the influence intensity of the force / torque history on the pose residual generation process according to the contact probability, and selects or weights and combines an effective correction subspace from multiple preset correction subspaces according to the phase belief, and limits the pose residuals within the effective correction subspace. S5. Composite the pose residual to the reference action block to obtain the action to be executed, and send the action to the robot for execution.

[0025] In this embodiment, the robot control system includes a multimodal data acquisition module, a contact-aware phase predictor (CAP), a low-frequency diffusion planner, a high-frequency residual corrector, and a robot control interface, all interconnected. Specifically, the multimodal data acquisition module includes a multi-view RGB (Red-Green-Blue) camera, a six-dimensional force / torque sensor, and a robot body state acquisition module. The low-frequency diffusion planner processes multimodal information including images acquired by the multi-view RGB camera, force / torque history acquired by the six-dimensional force / torque sensor, and body state acquired by the robot body state acquisition module; the high-frequency residual corrector only processes force / torque history, body state, and recent baseline action history.

[0026] In this embodiment, in step S2, the contact sensing phase predictor includes a visual encoder and a force / torque encoder. The visual encoder is used to extract visual tokens from multi-view images, and the force / torque encoder is used to extract force history tokens from force / torque history.

[0027] Specifically, the visual encoder can use ResNet with independent viewpoints, while the force / torque encoder uses a dilated one-dimensional convolutional temporal network with residual connections to jointly encode sudden force / torque changes and short-term dependencies at different time scales, thereby obtaining multi-scale transient features and further forming force history tokens.

[0028] Specifically, the low-frequency diffusion planner generates the reference action block at a first frequency, and the high-frequency residual corrector and CAP operate at a second frequency, with the first frequency being lower than the second frequency. The action executed at time t is obtained by combining the reference action block and the pose residual on a three-dimensional special Euclidean group SE(3).

[0029] In one specific embodiment, when the robot is a gripper-equipped robot, each action in the reference action block includes the position and orientation of the tool center point (TCP) and the gripper opening / closing amount. The TCP orientation can be represented by a unit quaternion, and the gripper opening / closing amount can be represented by the gripper width or an opening / closing command. For tasks without grippers or requiring no gripping or releasing, the gripper opening / closing amount can be omitted. The high-frequency residual corrector only corrects the six-dimensional pose components of the TCP, without changing the gripper opening / closing amount.

[0030] In this embodiment, in step S3, the low-frequency diffusion planner includes a multi-head cross-attention unit. This unit performs multi-head cross-attention computation using a visual token as the query and a force history token as the key and value. The phase belief is converted into a per-attention-head gating vector through a head-level gating network, which weights the outputs of each attention head in the multi-head cross-attention computation. The output of the multi-head cross-attention unit is orthogonalized relative to the visual token to obtain orthogonal residuals. These orthogonal residuals are combined with the visual token to form a fusion token. The low-frequency diffusion planner generates a baseline action block based on the fusion token and the ontology state.

[0031] In this embodiment, K task phases are predefined for each task. For example, a plug-in / plug-out task may include approach, search, recovery, insertion, and completion. CAP also includes a contact prediction head and a phase prediction head. After encoding the multimodal sensing data, CAP outputs the continuous contact probability p through the contact prediction head. c In the range [0,1], the vision-force fusion module globally gates the vision-force fusion intensity and high-frequency pose residuals based on the continuous contact probability of the dual-gated fusion units. The phase prediction head outputs a K-dimensional phase belief vector p, which is used for phase routing of the cross-attention head and the correction subspace. CAP employs predictive supervision; the contact label indicates whether contact will occur within the next Kf control steps, and the phase label indicates the task phase after a future offset of δ. The contact prediction head uses binary cross-entropy loss, while the phase prediction head uses multi-class cross-entropy loss. The phase label can be automatically generated from the force / torque history and TCP pose according to the deterministic event rules described later.

[0032] Specifically, such as Figure 2 As shown, the visual encoder of the low-frequency diffusion planner encodes each viewpoint image of the multi-view image and concatenates them into a visual token. The force / torque encoder of the low-frequency diffusion planner encodes the force / torque history into a force history token sequence. Multi-head cross-attention (MHCA) is performed using the visual token as the query and the force history token as the key and value. The phase belief is converted into a per-attention head gating vector through an MLP (Multi-Layer Perceptron) to weight the force information outputs of different heads; the contact probability p c As a global injection gate, it suppresses noise force information in free space.

[0033] In this embodiment, to avoid the force history token overriding the semantics of the visual task, the orthogonal component of the MHCA output Δ relative to the visual token v is calculated: Formula (1); Where <·,·> represent vector dot product, ε is a positive number set to prevent the denominator from being zero, and α is the fusion coefficient for controlling the strength of the historical token injection. This forms the fused token. Subsequently, the diffusion action head, conditioned on v′ and the body state, progressively denoises and generates an executable action block of length Ha in the real action space.

[0034] In this embodiment, multiple correction subspaces are defined by diagonal selection matrices, which are used to limit the combination of degrees of freedom that can be corrected under the corresponding task phase. The high-frequency residual corrector performs a weighted summation of multiple diagonal selection matrices by phase belief to obtain an effective correction subspace, and performs overall gating of the pose residual by contact probability.

[0035] In this embodiment, step S5, which involves combining the pose residual with the reference action block, specifically includes: This is achieved by performing composite calculations on the reference motion block and the pose residual in the pose space of the robot's end effector.

[0036] Specifically, such as Figure 3 As shown, the high-frequency residual corrector generates a six-dimensional original candidate residual c in each control cycle based on the force / torque history, the body state, and the recent low-frequency diffusion planner output reference action block. t Then use the contact probability p t c and phase belief p t Gating and phase routing are performed. The original candidate residuals are represented in coordinates using the Lie algebra se(3), and its specific expression is as follows: Formula (2).

[0037] Among them, c x c y and c z Let c represent the candidate translation components along the x, y, and z axes in the TCP local coordinate system, respectively. rx c rry and c rz These represent the candidate components of the rotation vector around the x, y, and z axes, respectively. For each task phase k, a pre-defined six-dimensional diagonal binary selection matrix B is used. k ∈{0,1} 6x6 This limits the degrees of freedom for phase correction allowed in this task. The final pose residual is: Formula (3).

[0038] Where, p t (k) represents the probability that the task belongs to the k-th phase at time t. p t cLet be the contact probability at time t. ∈R 6 The effective pose residual is obtained after gating and phase routing. In Equation (3), the process of weighting and combining multiple selection matrices with phase belief constitutes phase soft routing, which enables a smooth transition between adjacent task phases; the contact probability is used as an overall gating to suppress meaningless corrections in the non-contact state.

[0039] For example, the search phase of a plug-in / plug-out task can activate x, y, and r. z Channels, inserting phase can activate z and r z Channel; drawer pull phase can be combined with translation and r x r y Compliant channel; wiping the phase enables the normal z-channel. Where r x r y and r z These represent the components of the rotation vector about the corresponding coordinate axes. The supervisory target of the high-frequency residual corrector can be automatically constructed from physical priors: in the plane search, a target residual is generated based on the tangential force and the torque about the z-axis to reduce friction and alleviate jamming; in the wiping process, a z-axis conforming target is generated based on the difference between the target normal force and the measured normal force.

[0040] In this embodiment, let the phase belief be p. t ∈R K The top-level gating network is: Formula (4); Where W1, W2, b1, and b2 are the trainable weights and biases of the head-gated network, ReLU and sigmoid are the activation functions, and H is the number of attention heads; g t head (h) is the h-th scalar gate, which is directly multiplied by the h-th cross-attention head output O. h Contact probability p t c The region ∈[0,1] serves as a global gate, which is multiplied by the force residual formed by concatenating and projecting the outputs of each head. This structure eliminates the need for softmax normalization of each head, allowing multiple heads associated with the current phase to be activated simultaneously.

[0041] In this embodiment, orthogonal residual injection is performed at the visual token level. For the j-th visual token v j and the corresponding cross-attention output Δ at the same position j Calculate the predicted cross-attention output Δ j per： Formula (5); And obtained Where j is the sequence number of the visual token. and The meaning is the same as that of formula (1).

[0042] In some embodiments, if only one global visual token is retained, then v j With Δ j These are the results of global pooling; both implementations guarantee that the two vectors involved in the projection have the same dimension.

[0043] The physical priors of the high-frequency residual corrector are used to deterministically generate target residuals corresponding to each preset task phase from the force / torque history and the bulk state. Each target residual corresponds one-to-one with the phase belief component output by the CAP, and is used as training supervision after being weighted according to the phase probability; for example, when the CAP outputs a high probability for the search phase, the following search phase target is mainly used. The target residual for the search phase is: Formula (6); Insertion / wipe normal phase: Formula (7).

[0044] The drawer pull phase can be augmented with a gain k in the pull direction. pull Furthermore, reverse damping terms are applied to the lateral force and the moments about the x and y axes. In the formula, F... x F y and F z For the measured force components, To measure the torque about the z-axis, The target normal force; sat is the amplitude limiting function; k x k y k z k rz and k pull F is the admittance gain determined based on the maximum allowable displacement, maximum allowable rotation angle, and control period. x0 F y0 F z0 and The sensor is calibrated using a specific scale. The target residuals for each phase are calculated according to p... t (k) weighted and by p t c Gating, resulting in a single supervised target, is as follows: Formula (8).

[0045] in, This represents the target pose residual generated based on the physical prior model at time t in the k-th task phase; This represents the final target pose residual obtained by weighting the probabilities of each task phase and gating it with the contact probability at time t. , The meaning of K is the same as that of formula (3).

[0046] During training, the aforementioned physical prior objectives are converted into Lie algebra residuals and supervised by Fast using L1 or Huber loss; Slow trains the diffusion denoising objective using synchronous teaching action blocks; CAP uses future contact and future phase labels automatically generated by the rules.

[0047] In this embodiment, the robot control method based on contact phase scheduling further includes the following steps: S6. Using a physical prior model, automatically generate the target pose residual based on force / torque history and phase belief; train the high-frequency residual corrector with the goal of minimizing the difference between the pose residual output by the high-frequency residual corrector and the target pose residual; train the contact sensing phase predictor using future contact labels and future phase labels automatically generated by task phase and deterministic event rules.

[0048] Specifically, during training, the CAP minimizes the contact prediction loss and the phase prediction loss; the low-frequency diffusion planner uses diffusion denoising loss to learn action blocks; and the high-frequency residual corrector minimizes the L1 loss between its output residual and the phase-weighted physical prior residual. The robot's CAP, low-frequency diffusion planner, and high-frequency residual corrector can be trained jointly or in stages.

[0049] In this embodiment, as Figure 4 As shown, the robot's closed-loop execution process is as follows: S601. Collect multi-view images of the robot's current state, force / torque history, and the robot's body state; S602, using CAP to predict contact probability and phase belief; S603, Generates a baseline action block using a low-frequency diffusion planner; S604. Run the high-frequency residual corrector in each control cycle to generate the pose residual after phase routing; S605. Composite the pose residual onto the current reference action on SE(3) to obtain the action to be executed and send it to the robot's low-level controller. S606, Update multimodal observations; if the task is not completed, repeat steps S601 to S606.

[0050] In one possible embodiment, the inference frequency of the low-frequency diffusion planner is 6 Hz, the control frequency of the CAP and the high-frequency residual corrector is 24 Hz, and the action block length H... aThe force / torque history window is 36 control steps. Diffusion training can use 100 time steps, and inference uses 10 denoising steps. The robot's control system can send interpolation commands to the lower-level controller at frequencies higher than 500 Hz. The above values ​​are merely examples and do not constitute a limitation on the scope of protection.

[0051] Formula (2) The six-dimensional original candidate residuals are directly generated by the high-frequency residual corrector, where cρ, t and c , t These are the candidate translation components and candidate rotation vector components in the TCP local coordinate system, respectively. The original candidate residuals are then subjected to contact gating and phase routing according to formula (3) to obtain the effective pose residual. ,in For the TCP local coordinate system translation increment, The vector is a rotation vector. An incremental transformation is obtained using an exponential mapping. ,in for The 4×4 matrix obtained by the se(3) hat operator; in this embodiment, the pose is performed by right multiplication composition, as follows: Formula (9); Among them, T t slow∈SE(3) represents the TCP reference pose transformation output by the low-frequency diffusion planner at time t; This represents the incremental pose transformation obtained by exponential mapping of high-frequency pose residuals. This indicates the TCP target pose transformation that is sent and executed after the two are combined.

[0052] T t exec Right multiplication indicates that the residual is applied in the current TCP local coordinate system.

[0053] In other embodiments, left-multiplication composition may also be used: However, the residuals should be expressed in the robot's base coordinate system accordingly.

[0054] In this embodiment, the phase tag is automatically generated using a method that combines "task-defined phase with deterministic event rules". Taking a plug-in / plug-out task as an example: When the relative pose distance of the target is greater than the threshold d app Furthermore, the normal force within the future window is less than F. c The time marker is near; When the distance is not greater than d app Tangential force norm exceeds F xy And the insertion depth is less than d ins The time marker is used for searching; It is marked as recovered when the force / moment norm or its first difference exceeds the anomaly threshold and the TCP moves in the retreat direction; when the insertion axis displacement exceeds d ins And the normal force is in [F min ,F max When the position is set to "insertion" and the gripper / contact conditions are met, it is marked as "completion". The threshold is determined by the quantile of the taught trajectory or the normalized value of the sensor range, and can be calibrated according to the task.

[0055] The drawer and wiping tasks use the same "geometric progress + force / torque event + end motion direction" rule to generate their respective stage labels.

[0056] In this embodiment, training data is acquired by human teleoperation or a validated expert planner. Each timestamp records synchronized multi-view images, TCP pose, joint states, gripper states, six-dimensional force / torque, and expert control commands; all sensors are aligned with a uniform timestamp, forces / torques are expressed in the TCP coordinate system, and a sliding window of length H is used to construct the data. w The history of teaching control commands by pressing H consecutively. a Each control step slice forms the baseline action supervision for the low-frequency diffusion planner.

[0057] Example 2 Based on Embodiment 1, the present invention provides a robot control system.

[0058] In this embodiment, the robot control system includes: The multimodal data acquisition module is used to acquire multimodal sensing data, which includes multi-view images, force / torque history, and body state. The contact sensing phase predictor is communicatively connected to the multimodal data acquisition module to receive multimodal sensing data and output contact probability and phase belief. The contact probability is used to characterize the likelihood of contact occurring within a future time window, and the phase belief is used to characterize the probability distribution of being in different preset task phases at a future preset offset time. A low-frequency diffusion planner, communicatively connected to a contact-sensing phase predictor, is used to process multi-view images, force / torque history, and body state at a first frequency to generate a baseline action block. A high-frequency residual corrector is communicatively connected to a low-frequency diffusion planner. It processes force / torque history, body state, contact probability, phase belief, and reference action block at a second frequency higher than the first frequency to generate pose residuals. The high-frequency residual corrector dynamically adjusts the influence of force / torque history on the pose residual generation process based on the contact probability, and selects or weights an effective correction subspace from multiple preset correction subspaces based on the phase belief, and confines the pose residuals within the effective correction subspace. The motion composite unit is communicatively connected to the high-frequency residual corrector, used to composite the pose residual to the reference motion block and output the executed motion.

[0059] Specifically, the multimodal data acquisition module includes a multi-view RGB camera, a six-dimensional force / torque sensor, and a robot body state acquisition module; the multi-view RGB camera is used to acquire multi-view images, the six-dimensional force / torque sensor is used to acquire force / torque history, and the robot body state acquisition module is used to acquire the robot's body state.

[0060] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A robot control method based on contact phase scheduling, characterized in that, Includes the following steps: S1. Acquire multimodal perception data of the robot, including multi-view images, force / torque history, and body state; S2. Input the multimodal sensing data into the contact sensing phase predictor, and output the contact probability and phase belief from the contact sensing phase predictor; wherein, the contact probability is used to characterize the possibility of contact occurring within a future time window, and the phase belief is used to characterize the probability distribution of being in different preset task phases at a future preset offset time. S3. Process the multi-view image, force / torque history and body state at a first frequency using a low-frequency diffusion planner to generate a reference action block; S4. The force / torque history, the body state, the contact probability, the phase belief, and the reference action block are processed by a high-frequency residual corrector at a second frequency higher than the first frequency to generate a pose residual; wherein, the high-frequency residual corrector dynamically adjusts the influence intensity of the force / torque history on the pose residual generation process according to the contact probability, and selects or weights and combines an effective correction subspace from a plurality of preset correction subspaces according to the phase belief, and confines the pose residual within the effective correction subspace; S5. The pose residual is combined with the reference action block to obtain the action to be executed, and the action is sent to the robot for execution.

2. The robot control method as described in claim 1, characterized in that, In step S2, the contact sensing phase predictor includes a visual encoder and a force / torque encoder. The visual encoder is used to extract a visual token from the multi-view image, and the force / torque encoder is used to extract a force history token from the force / torque history.

3. The robot control method as described in claim 2, characterized in that, In step S3, the low-frequency diffusion planner includes a multi-head cross-attention unit. The multi-head cross-attention unit performs multi-head cross-attention calculations using the visual token as the query and the force history token as the key and value. The phase belief is converted into a per-attention head gating vector through a head-level gating network, and the outputs of each attention head calculated by the multi-head cross-attention are weighted.

4. The robot control method as described in claim 3, characterized in that, The output of the multi-head cross-attention unit is orthogonalized relative to the visual token to obtain an orthogonal residual, which is then combined with the visual token to form a fusion token. The low-frequency diffusion planner generates the baseline action block based on the fusion token and the ontology state.

5. The robot control method as described in claim 1, characterized in that, Each of the multiple correction subspaces is defined by a diagonal selection matrix, which is used to limit the combination of degrees of freedom that can be corrected under the corresponding task phase; the high-frequency residual corrector performs a weighted summation of the multiple diagonal selection matrices by the phase belief to obtain the effective correction subspace, and performs overall gating of the pose residual by the contact probability.

6. The robot control method as described in claim 5, characterized in that, The formula for calculating the pose residual is: ； Where p t c Let p be the contact probability at time t. t (k) represents the probability that the task belongs to the k-th task phase at time t, B k Let c be the diagonal selection matrix corresponding to the k-th task phase, where K is the total number of task phases. t The original candidate residuals generated for the high-frequency residual corrector.

7. The robot control method as described in claim 1, characterized in that, It also includes the following steps: S6. Using a physical prior model, automatically generate the target pose residual based on the force / torque history and the phase belief; train the high-frequency residual corrector with the goal of minimizing the difference between the pose residual output by the high-frequency residual corrector and the target pose residual.

8. The robot control method as described in claim 7, characterized in that, The contact-aware phase predictor is trained using future contact labels and future phase labels automatically generated from task phase and deterministic event rules.

9. The robot control method as described in claim 1, characterized in that, In step S5, the pose residual is composited to the reference action block, specifically as follows: This is achieved by performing a composite operation between the reference motion block and the pose residual in the pose space of the robot's end effector.

10. A robot control system, characterized in that, include: A multimodal data acquisition module is used to acquire multimodal sensing data, which includes multi-view images, force / torque history, and body state. A contact-sensing phase predictor is communicatively connected to the multimodal data acquisition module. It is used to receive multimodal sensing data and output contact probability and phase belief. The contact probability is used to characterize the likelihood of contact occurring within a future time window, and the phase belief is used to characterize the probability distribution of being in different preset task phases at a future preset offset time. A low-frequency diffusion planner, communicatively connected to the contact-sensing phase predictor, is used to process the multi-view image, force / torque history, and body state at a first frequency to generate a reference action block. A high-frequency residual corrector, communicatively connected to the low-frequency diffusion planner, is used to process the force / torque history, the body state, the contact probability, the phase belief, and the reference action block at a second frequency higher than the first frequency to generate pose residuals. The high-frequency residual corrector dynamically adjusts the influence intensity of the force / torque history on the pose residual generation process based on the contact probability, and selects or weights an effective correction subspace from a plurality of preset correction subspaces based on the phase belief, confining the pose residuals within the effective correction subspace. The motion composite unit is communicatively connected to the high-frequency residual corrector and is used to composite the pose residual to the reference motion block and output the executed motion.

Images