Robot reasoning method and device, electronic device, computer readable storage medium

Abstract

The present disclosure provides a robot reasoning method and device, electronic equipment and computer readable storage medium. The method comprises: at a low frequency level, mapping the latent variable to a coarse action draft covering different modes by a generator using the full history state condition; at a high frequency level, receiving the coarse action draft and combining real-time feedback for closed-loop refinement to output refined actions for driving the robot; wherein the real-time feedback comprises real-time force feedback and ontology feedback; the operating frequency of the high frequency level is higher than that of the low frequency level; the high frequency level and the low frequency level operate on different clocks. The present disclosure solves the technical problem that the existing scheme cannot meet the two conflicting demands of decisive decision-making at the low frequency level and high-frequency closed-loop response at the high frequency level in the actual task of robot action generation.

Description

Technical Field

[0001] This disclosure relates to the field of robot control technology, and more specifically, to a robot reasoning method and apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] Robot motion generation faces two conflicting demands in practical tasks: Decision-making must be "decisive": Many operational tasks inherently have multiple equally feasible solutions (e.g., an object has two equally valid grab points, or there are two paths to bypass an obstacle). The strategy should decisively select one of these feasible patterns and stick to it, rather than averaging multiple correct patterns. Averaging results in a compromise action that is neither pattern A nor pattern B (e.g., "starting from the exact middle of the two correct grab points"), often falling into the failure zone between the two correct solutions.

[0003] Execution must be "real-time": When a robot comes into contact with its environment, the contact force, friction, and external disturbances change rapidly. Actions need to be performed with a high-frequency (e.g., 100 Hz or higher) closed-loop response in order to stably complete contact-intensive operations.

[0004] The existing solutions are insufficient to satisfy both of the above requirements at the same time. Summary of the Invention

[0005] This disclosure provides a robot reasoning method and apparatus, electronic device, and computer-readable storage medium to solve the technical problem that existing solutions cannot meet the conflicting demands of robot motion generation in actual tasks, which require decisive decision-making at low frequencies and high-frequency closed-loop response at high frequencies.

[0006] According to one aspect of the present disclosure, a robot reasoning method is provided, the method comprising: at a low-frequency level, using a generator to map latent variables to coarse motion drafts covering different modes using full-history state conditions; at a high-frequency level, receiving the coarse motion drafts and performing closed-loop refinement in conjunction with real-time feedback, and outputting refined motions for driving the robot; wherein the real-time feedback includes real-time force feedback and body feedback; the high-frequency level operates at a higher frequency than the low-frequency level; and the high-frequency level and the low-frequency level operate on different clocks.

[0007] Optionally, the above method further includes: conditionally encoding the output of the full-history state propagator at the current moment and the current multimodal observation to obtain the full-history state conditions.

[0008] Optionally, the generator is trained using implicit maximum likelihood estimation. The training method includes the following steps: randomly selecting multiple latent variables for each teaching sample to generate multiple candidate drafts; wherein, the teaching sample includes task conditions and expert actions; selecting the candidate draft that is closest to the teaching sample from the multiple candidate drafts; and bringing the candidate draft that is closest to the teaching sample closer to the teaching sample.

[0009] Optionally, the above method further includes performing continuous trajectory evolution on the latent variables on a unit sphere to ensure mode consistency across control steps.

[0010] Optionally, continuous trajectory evolution of the latent variables is performed on the unit sphere, including: using the inertia coefficient to control the time correlation of the tangential velocity of the latent variables; using the evolution angle to control the velocity of the latent variables evolving along the sphere, and normalizing the result and projecting it onto the unit sphere; wherein the inertia coefficient and the evolution angle are configurable parameters or learnable parameters.

[0011] Optionally, a coarse motion draft is received and closed-loop refinement is performed in conjunction with real-time feedback, including: treating the robot's motion trajectory as the state of a second-order dynamic system including position and velocity; using a second-order physics refiner to provide acceleration in conjunction with the coarse motion draft and real-time feedback, and updating position and velocity with Sinusoidal integral; wherein the number of refinement iterations for each control step at the high-frequency level is adaptively adjusted according to the contact intensity or disturbance amplitude.

[0012] Optionally, at the low-frequency level, the latent variables are mapped to coarse action drafts covering different modes by a generator using full-history state conditions, including: running once every configurable number of control steps, or being triggered by at least one of the following events: task phase switching, target change, or high-frequency level refinement residual exceeding a threshold.

[0013] Optionally, the above method further includes applying a pattern consistency regularization to the draft differences of adjacent control steps during training.

[0014] According to another aspect of the present disclosure, a robot reasoning device is provided, comprising: a first processing module for mapping latent variables to coarse motion drafts covering different modes using a generator at a low-frequency level, utilizing full-history state conditions; and a second processing module for receiving the coarse motion drafts at a high-frequency level, performing closed-loop refinement in conjunction with real-time feedback, and outputting refined motions for driving the robot; wherein the real-time feedback includes real-time force feedback and body feedback; the operating frequency of the high-frequency level is higher than that of the low-frequency level; and the high-frequency level and the low-frequency level operate on different clocks.

[0015] According to another aspect of the present disclosure, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described method.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method.

[0018] The beneficial effects of the technical solutions provided in this disclosure are: The proposed solution, at the low-frequency level, starts from the full historical state conditions and uses a generator to map latent variables into coarse motion drafts covering different modes, achieving "decisive selection of a solution." At the high-frequency level, it receives the drafts and uses a second-order physics refiner combined with real-time force feedback or body feedback for high-frequency closed-loop refinement. Furthermore, the two levels operate on different clocks without blocking each other. This achieves the technical effect of simultaneously satisfying both decisive decision-making and real-time execution in robot motion generation for practical tasks, decoupling these two conflicting requirements. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0020] Figure 1 This is a flowchart illustrating a robot reasoning method provided in a disclosed embodiment; Figure 2 This is a diagram of a two-level asynchronous decoupled inference system architecture for robot inference provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a robot reasoning device provided in the disclosed embodiments; Figure 4 This is a schematic diagram of the structure of an electronic device provided in a disclosed embodiment. Detailed Implementation

[0021] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0022] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term, for example, “A and / or B” or “A, B” indicates implementation as “A,” or implementation as “B,” or implementation as “A and B.”

[0023] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0024] As mentioned above, robot motion generation faces two conflicting demands in practical tasks: the need for "decisive" decision-making and the need for "real-time" execution. Existing solutions struggle to simultaneously meet both demands, exhibiting the following technical shortcomings: Drawback 1: Generative strategies that can make multimodal decisions have synchronous decision-making and execution, and are dragged down by the slower end.

[0025] One type of scheme (denoted as D1) employs a generative strategy to sample actions from a multimodal distribution, possessing a certain ability to "choose a solution." However, its decision network and action output operate on the same clock, requiring a heavy multimodal generation process (such as multi-step iterative sampling) for each control step, resulting in high inference latency and failing to meet high-frequency closed-loop requirements. If the generation frequency is reduced to increase speed, the ability to respond to contact / disturbance at high frequencies is lost. Decision-making and execution are tied to the same frequency, dragged down by the slower end (regeneration).

[0026] Defect 2: Solutions capable of high-frequency asynchronous execution are often only single-level, lack multimodal decision-making, and are prone to compromise and failure.

[0027] Another type of solution (denoted as D2) prioritizes real-time performance by employing single-stage high-frequency action generation or high-frequency refinement, enabling rapid response to disturbances. However, its action generation is typically deterministic or single-modal regression. When faced with tasks containing multiple correct patterns, the regression objective averages out these correct patterns, resulting in compromise actions that fall into the failure zone (i.e., the pattern collapse / averaging problem). This type of solution lacks a dedicated step responsible for "decisively selecting one from multiple solutions."

[0028] Defect 3: Even if you try to divide it into two levels, if the latent variables are independently resampled at each step, it will cause the mode to jump.

[0029] If we simply divide decision-making and execution into two levels and repeatedly call low-frequency decisions in high-frequency execution, we will also encounter the problem of pattern consistency: if the latent variables representing "which mode to choose" are independently resampled in each control step, then adjacent steps may sample different modes (this step tends to grab point A, the next step tends to grab point B), and the action jumps back and forth between the two modes, which disrupts the continuity of execution.

[0030] Limitations of existing solutions: 1. Multimodal decision-making and real-time execution are tied together, but are hampered by slow endpoints (D1). 2. High-frequency asynchronous execution is single-level and lacks multimodal decision-making, making it prone to failure due to average compromise (D2). 3. Lack of mode consistency guarantees across control steps; independent resampling of latent variables leads to mode hopping. 4. No scheme was found that separates "multimodal mode selection" and "high-frequency closed-loop refinement" into two asynchronous stages and adds time-related latent variables.

[0031] Therefore, there is an urgent need for an inference method that separates multimodal mode selection and high-frequency closed-loop refinement into two-stage asynchronous pipelines running on different clocks and uses time-dependent latent variable sampling to ensure the consistency of step mode, so as to simultaneously satisfy decisive decision-making and real-time execution.

[0032] The robot reasoning method, apparatus, electronic device, and computer-readable storage medium disclosed herein are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0033] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0034] This disclosure provides a robot reasoning method, such as... Figure 1 As shown, the method includes the following steps: Step S101: At the low-frequency level, the latent variables are mapped to coarse action drafts covering different modes by using the generator with full history state conditions.

[0035] In robot control architecture, the "low-frequency level" refers to the system level responsible for macro-planning, logical reasoning, and global decision-making. It usually corresponds to "slow thinking" in cognitive science and is equivalent to the robot's "brain".

[0036] According to some optional embodiments of this disclosure, before performing step S101, it is also necessary to conditionally encode the output of the full history state propagator at the current moment and the current multimodal observation to obtain the full history state conditions.

[0037] "Full historical state conditions" are the core input for low-frequency decision-making. It refers to the complete record of all observations and bodily states of the robot from the start moment to the current moment during the execution of a task.

[0038] Suppose that the robot needs to output an action (or action block) at control time t. The system maintains a full-history state propagator (such as a selective state-space model), whose output at time t is denoted as... (Carrying historical information from the start of the mission to the present), the current multimodal observation is denoted as... Define the conditions for the full history state: ; in, For conditional encoding (such as concatenation followed by projection / cross attention). Low-frequency levels are provided as generation conditions.

[0039] "Multimodal observation" refers to the multi-dimensional comprehensive data collected by a robot through various sensors at a certain moment during the execution of a task, which describes the "current state of itself" and the "state of the external environment".

[0040] "Hidden variables" are "abstract thinking units" within a robot used to encode and select different action strategies. They are a key bridge connecting macroscopic logical reasoning (low-frequency level) and specific physical actions (high-frequency level).

[0041] Figure 2 This is a diagram of a two-level asynchronous decoupled inference system architecture for robot inference provided in this disclosure embodiment, such as... Figure 2 As shown, the low-frequency stage is responsible for "decisively selecting one from multiple feasible solutions." It uses an implicit maximum likelihood estimation (IMLE) generator to map latent variables into coarse-action drafts: ; in, As a latent variable, For all historical conditions, For generators (a draft is produced in a single forward pass, without the need for multi-step iterative sampling).

[0042] In step S102, at the high-frequency level, a coarse motion draft is received and refined in a closed loop by combining real-time feedback, and the refined motion for driving the robot is output; the real-time feedback includes real-time force feedback and body feedback.

[0043] In robot control architecture, the "high-frequency level" is equivalent to the robot's "cerebellum" or "spinal reflex," operating at extremely high frequencies, typically 50Hz or even hundreds of hertz (executing tens to hundreds of control commands per second). It is responsible for real-time execution, closed-loop refinement, and safety fallback.

[0044] like Figure 2 As shown, the coarse motion draft of the high-frequency stage receiving the low-frequency stage output. A second-order physical refiner combined with real-time feedback (real-time force feedback / proprioceptive feedback) is used for high-frequency closed-loop refinement, enabling rapid response to contact / disturbance. "Proprioceptive feedback" is equivalent to human "proprioception," allowing a robot to know the degree of its arm bend or the distance its leg has traveled, even with its eyes closed. For a robot, this is its ability to perceive its own internal state. "Real-time force feedback" is equivalent to human "touch," allowing it to feel the weight of a glass of water and immediately sense the impact of spilled water on the back of its hand.

[0045] At the high-frequency level, the robot receives "coarse motion drafts" from the low-frequency level and, combined with real-time sensor feedback (such as vision, force, and body posture), performs millisecond-level fine-tuning. It utilizes a second-order physics refiner to ensure that the robot's movements are not only "correctly planned" but also "stable." If it encounters external interference or environmental changes, the high-frequency level can react instantly (such as emergency stopping or adjusting gripping force) without waiting for the "brain" to replan.

[0046] In some embodiments, the high-frequency stage operates at a higher frequency than the low-frequency stage; the high-frequency stage and the low-frequency stage operate on different clocks.

[0047] In the embodiments of this disclosure, the high-frequency stage operates at a higher frequency than the low-frequency stage, and the low-frequency stage is decoupled from the high-frequency stage, operating on different clocks without blocking each other.

[0048] For example, the low-frequency stage runs once every N control steps (or is triggered by events such as task phase switching) to generate / update coarse motion drafts. The high-frequency stage runs once for each control step, based on the most recent draft. A second-order physical closed-loop refinement is performed for the target. The two communicate via a shared draft buffer: low-frequency asynchronous writes. High-frequency reading of the latest data per step Low-frequency recomputation does not hinder high-frequency real-time refinement. With this scheme, the slow end (multimodal generation) no longer drags down the fast end (high-frequency response), and the whole retains both decisive multimodal decision-making and meets the requirements of high-frequency real-time execution.

[0049] Multimodal tasks have multiple correct solutions, and directly averaging multiple solutions can lead to compromise failure. Furthermore, the requirements of "decisively selecting a solution" (low-frequency, requiring full historical conditions) and "real-time response to contact / disturbance" (high-frequency closed-loop) are conflicting demands, with the slower stage hindering the faster stage during simultaneous processing. The technical solution proposed in this disclosure addresses this: the low-frequency stage starts from full historical state conditions, and the IMLE generator maps latent variables to coarse-grained action drafts covering different modes, achieving "decisive selection of a solution"; the high-frequency stage receives the coarse-grained action drafts and uses a second-order physics refiner combined with real-time force feedback / body feedback for high-frequency closed-loop refinement; the two stages operate on different clocks, with the low-frequency stage running only once every few steps, and the high-frequency stage running once per control step, without blocking each other. This achieves both decisive decision-making and real-time execution, decoupling and simultaneously satisfying the two conflicting demands; the slower stage no longer hinders the faster stage, and the overall high-frequency real-time performance is met.

[0050] In some optional embodiments of this disclosure, the generator is trained using implicit maximum likelihood estimation. The training method includes the following steps: randomly selecting multiple latent variables for each teaching sample to generate multiple candidate drafts; wherein the teaching sample includes task conditions and expert actions; selecting the candidate draft that is closest to the teaching sample from the multiple candidate drafts; and merging the candidate draft that is closest to the teaching sample into the teaching sample.

[0051] In some embodiments, "task conditions" are equivalent to "exam questions" or "current situations" given to the robot, telling the robot "what environment you are in now and what goal you need to achieve." "Expert actions" are equivalent to the "standard answers" to these exam questions; they are real action data recorded by advanced algorithms when they perfectly complete the task in a real environment.

[0052] In the embodiments of this disclosure, the training objective of IMLE is to avoid pattern collapse / averaging. For each teaching sample (task condition) Expert Actions ), using K independent latent variables { ,..., }, generate K candidate drafts { ( , Only the most recent candidate draft was brought closer by the expert's action: ; ; These two formulas together constitute the core training mechanism of IMLE, in which, An index to the candidate draft that best approximates the expert's action. It's an index that finds the candidate draft that is closest to the expert's actions. For distance metrics (such as L2 or weighted distance in action space).

[0053] This represents the "loss function" used during model training. This means calculating the expectation (i.e., the average) of all training samples (condition and expert action pairs).

[0054] The goal of "only the nearest ones are pulled closer" ensures that, when faced with teaching multiple correct patterns under the same conditions, different latent variables are pushed towards the neighborhoods of different patterns, rather than all latent variables being pulled towards the average of all teachings. This allows different latent variables to cover different patterns, fundamentally avoiding the compromise failures caused by averaging in regression (such as "starting from the exact middle of two correct capture points"). A single forward pass at the low-frequency level outputs a draft, thus determining "which solution to choose."

[0055] In robot learning, the same task often has multiple correct solutions (i.e., multimodal learning). For example, there's a cup on a table, and an expert demonstrates two ways to grab it: from the left or from the right. Traditional models, during training, attempt to fit both solutions simultaneously. As a result, the model learns to "grab it from the middle." In the physical world, grabbing from the middle not only fails to grab the cup but also knocks it over. This is a compromise failure caused by "mode collapse."

[0056] To avoid the aforementioned tragedy, IMLE employs a very clever training strategy, specifically the formula logic mentioned above: Step 1: Collect K latent variables Facing the same task conditions The correct actions of experts Instead of generating a direct answer, IMLE randomly selects K different latent variables, which is equivalent to letting the robot instantly conceive K different "rough drafts" in its mind.

[0057] Step 2: Select the optimal option Among these K drafts, the algorithm calculates which draft most closely resembles the expert's actual actions. This closest draft is the "winner" of this round of training.

[0058] Step 3: Only the nearest one is pulled closer. The algorithm only penalizes and pulls the "winner" closer to the correct action of the expert; while the drafts of the remaining K-1 "losers" are completely unaffected in this round of training and remain unchanged.

[0059] Results: The latent variables of "natural division of labor" and "decisive decision-making" Because only the "winner" is taught each time, over time, these K latent variables will automatically differentiate. For example, Specially responsible for remembering "grab from the left". Each variable is specifically responsible for remembering "grab from the right." They each occupy the territory of the correct solution, and none will learn the incorrect "right middle." Because the hidden variables have clearly defined roles, during the robot's actual operation (reasoning), the low-frequency level only needs to randomly select one hidden variable (or choose the most suitable one based on the current environment), perform a forward calculation, and directly output an extremely clear and unambiguous "rough draft." This perfectly achieves "decisively choosing a solution."

[0060] Through the above technical solution, low-frequency multimodal pattern selection is obtained through IMLE training. IMLE prompts different latent variables to map to coarse action drafts covering different patterns by having generated samples match the nearest real samples (rather than having real samples match generated samples). This allows for "decisive selection from multiple correct solutions" during inference, rather than averaging. This fundamentally avoids the compromise failures caused by pattern collapse and averaging, improving the success rate of multimodal tasks. Different latent variables correspond to different solutions, providing clear and self-consistent pattern drafts for high-frequency patterns.

[0061] According to some optional embodiments of this disclosure, the method further includes: performing continuous trajectory evolution of latent variables on a unit sphere to ensure mode consistency across control steps.

[0062] Low-frequency implicit variables are independently resampled in each control step. Adjacent steps may fall into different modes, causing actions to jump between modes (e.g., selecting "turn around from the left" in one step and "turn around from the right" in the next step). This results in high-frequency stages receiving contradictory drafts, disjointed actions, or even failures. A latent variable sampling mechanism is needed that maintains mode diversity while ensuring the continuity of selected modes across steps.

[0063] The technical solution proposed in this disclosure constrains the time-related latent variables to undergo continuous orbit evolution on a unit sphere. The latent variables move slowly over time along a smooth orbit on the sphere. After selecting a certain mode, they are smoothly maintained within several control steps and do not suddenly jump between modes. The inertia / evolution speed, the two-level frequency ratio, and the number of high-frequency refinement steps of the latent variables can all be configured.

[0064] In some embodiments, performing continuous trajectory evolution of latent variables on a unit sphere includes: controlling the time correlation of the tangential velocity of the latent variables using an inertia coefficient; controlling the velocity of the latent variables evolving along the sphere using an evolution angle, and normalizing and projecting the result onto the unit sphere; wherein the inertia coefficient and the evolution angle are configurable parameters or learnable parameters.

[0065] This disclosure ensures that latent variables are not independently resampled, but rather undergo continuous orbital evolution on a unit sphere, guaranteeing smooth retention after the selected mode is selected. Tangential velocity with inertia in, The tangential velocity of the latent variable at the current moment. ∈[0,1) is the inertia coefficient. Let be the tangential velocity of the latent variable at the previous time step. This is the standard function used to calculate the arithmetic square root of non-negative real numbers. It is random noise (such as standard normal and projected onto the tangent space).

[0066] Rotate along the sphere by the angle theta in, The rate at which the hidden variable evolves along the sphere. and For spherical rotation operators, For evolution angle, These are the latent variables from the previous time step.

[0067] Projected back onto the unit sphere in, For the current time, it is a hidden variable. This is the normalized projection function.

[0068] Controlling tangential speed Time correlation: The larger the value, the smoother the movement of latent variables and the more "sticky" the pattern. Controlling the rate at which hidden variables evolve along the sphere: The smaller, relatively The smaller the change, the more stable the pattern.

[0069] because A continuous, slow movement on the sphere maps to a rough draft of motion. It also changes continuously, and the selected mode is smoothly maintained within a few control steps without suddenly jumping to another mode; at the same time, it retains the ability to slowly explore / switch modes when the environment or task stage changes.

[0070] According to some optional embodiments of this disclosure, step S102 receives a coarse motion draft and performs closed-loop refinement in conjunction with real-time feedback, including: treating the robot's motion trajectory as the state of a second-order dynamic system including position and velocity; using a second-order physics refiner to combine the coarse motion draft and real-time feedback to provide acceleration, and updating position and velocity with Sinusoidal integral; wherein the number of refinement iterations for each control step at the high-frequency level is adaptively adjusted according to the contact intensity or disturbance amplitude.

[0071] In embodiments of this disclosure, the high-frequency stage receives the coarse motion draft output by the low-frequency stage. It uses a second-order physical refiner combined with real-time feedback to perform high-frequency closed-loop refinement, and responds quickly to contact / disturbance.

[0072] The motion trajectory is treated as the state (position q and velocity v) of a second-order dynamic system. The acceleration term is given by the physics refiner (which may include target guidance terms from the rough motion draft, correction terms from real-time feedback, and physics constraint / damping terms), and is advanced by Sinusoidal integral (semi-implicit Euler). The second-order physics refiner provides acceleration. First, update the speed (Cynola). Update the position again using the new speed. in, For real-time force / body feedback. Sinusoidal integrals offer better energy stability compared to explicit Euler methods, making them suitable for contact-intensive operations in high-frequency closed-loop systems. The high-frequency stage runs once per control step, quickly converging the coarse motion draft into a refined motion that matches the current contact state, and immediately correcting for disturbances. The number of refinement steps can be adaptively adjusted (e.g., more iterations during intense contact).

[0073] This disclosure provides an optional implementation in which, at a low frequency level, latent variables are mapped to coarse action drafts covering different modes using a generator based on full-history state conditions, including: running once every configurable number of control steps, or being triggered by at least one of the following events: task phase switching, target change, or high-frequency level refinement residual exceeding a threshold.

[0074] In traditional architectures, low-frequency stages typically run at fixed time intervals (e.g., 5 times per second). However, this often results in wasted computing power or untimely responses.

[0075] In embodiments of this disclosure, the low-frequency stage can be triggered by events such as configurable control steps, task stage switching, target changes, or excessive fine-tuning residuals in the high-frequency stage.

[0076] For example, once the robot has completed the "finding the cup" phase, it needs to seamlessly switch to the "grabbing the cup" phase. This change in macro-level intent must be immediately recognized and replanned by the low-frequency level, as the high-frequency level's physical fine-tuning is no longer sufficient to handle this logical leap across phases.

[0077] For example, a robot is executing a plan to "hand me the red cup," but a human suddenly changes their mind and says, "Put it down and get the blue cup." This fundamental change in the external environment requires the low-frequency actions to be immediately "rewritten," generating a new rough draft of the action.

[0078] "Refinement residuals" refers to the situation where the high-frequency stage, while diligently executing the "rough draft" provided by the low-frequency stage, discovers that the error between the actual physical state and the draft target is increasing, even exceeding the correction limit of the high-frequency stage. For example, the low-frequency stage plans a straight trajectory, but the robot suddenly finds a rock blocking its path. The high-frequency stage frantically adjusts its posture to try and go around it, but finds that it will collide with the rock no matter what it does, causing the error (residual) to skyrocket. At this point, the high-frequency stage will send a "distress signal (interrupt)" to the low-frequency stage.

[0079] The "low-frequency event triggering" mechanism breaks the rigid limitations of fixed frequencies, turning the low-frequency level into an "on-demand" intelligent brain. It doesn't need to operate frequently under normal circumstances to save computing power, but it will be instantly awakened and re-planned once it encounters one of three situations: "logic change," "instruction change," or "high-frequency execution failure." This greatly improves the robot's adaptability and computing efficiency.

[0080] In some embodiments, a pattern consistency regularization is applied to the draft differences of adjacent control steps during training.

[0081] In embodiments of this disclosure, regularization is applied to the differences between adjacent step drafts during training to further encourage coherent stride patterns; or the evolution angle of latent variables is adjusted. Apply constraints.

[0082] During training, the algorithm not only calculates the difference between the generated draft and the expert's actual actions, but also specifically calculates the difference between the "draft generated in the current step" and the "draft generated in the previous step." If the model provides a macroscopic draft with too large a difference between two adjacent control steps (for example, planning to move left in the previous step and suddenly planning to move right in the next step), the regularization term will give a large "penalty score." This forces the model to actively learn to "maintain action continuity" during training, eliminating temporal jumps at the source.

[0083] The evolution angle determines the step size of the latent variable sliding on the sphere. The larger the evolution angle, the farther the latent variable "slides" between adjacent steps, meaning the robot's macroscopic planning changes faster. Constraining the evolution angle sets an upper limit on this "sliding speed," preventing the model from "taking shortcuts" during training.

[0084] In summary, "pattern consistency regularization" forces robots to develop good habits of "coherent actions and smooth planning" during the training phase by penalizing differences between adjacent actions and limiting the evolution speed of latent variables.

[0085] This disclosure provides an alternative implementation method. In addition to the IMLE generator, low-frequency multimodal pattern selection can also be achieved using other single-forward implicit generative models, as long as their training objective enables different latent variables to cover different patterns and avoids averaging.

[0086] This disclosure provides an optional implementation where the number of IMLE candidates K and the distance metric are configurable. A larger K results in more comprehensive pattern coverage; the distance metric can be action-space weighted distance or a task-related metric.

[0087] This disclosure provides an optional implementation method, the hidden variable inertia coefficient. With evolution angle It can learn or adapt.

[0088] The robot reasoning method proposed in this disclosure can achieve the following technical effects: 1. Achieving both decisive decision-making and real-time execution: Decisive selection of mode at low frequency (no compromise), and high-frequency closed-loop (fast response) at high frequency, thus simultaneously satisfying the two conflicting demands after decoupling; 2. Avoid compromise failure: IMLE training allows different latent variables to cover different modes, selecting one from multiple correct solutions instead of taking the average, thus improving the success rate of multimodal tasks; 3. Two-stage non-blocking: The low-frequency stage runs every few steps, while the high-frequency stage runs every step, so the slow end no longer drags down the fast end, and the overall high-frequency real-time requirements are met. 4. Mode transition is coherent and without jumps: The time-related latent variables evolve continuously on the unit sphere, and are smoothly maintained after the mode is selected, without jumping back and forth between modes; 5. Real-time response to contact / disturbance: The high-frequency stage uses a second-order physical refiner combined with force / body feedback to form a closed loop, stably completing contact-intensive operations; 6. Modular and configurable: The two-level frequency ratio, refinement steps, and latent variable inertia / evolution speed are all adjustable to adapt to different tasks and computing power.

[0089] This disclosure provides a robotic reasoning device, such as... Figure 3 As shown, the robot reasoning device 30 may include: The first processing module 301 is used to map latent variables into coarse action drafts covering different modes by using the generator with full historical state conditions at the low frequency level.

[0090] The second processing module 302 is used to receive coarse motion drafts at the high-frequency level, and perform closed-loop refinement in combination with real-time feedback, and output refined motions for driving the robot; wherein, the real-time feedback includes real-time force feedback and body feedback; the operating frequency of the high-frequency level is higher than that of the low-frequency level; the high-frequency level and the low-frequency level operate on different clocks.

[0091] The apparatus of this disclosure embodiment can execute the method provided in this disclosure embodiment, and its implementation principle is similar, and it has corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of this disclosure correspond to the steps in the method of each embodiment of this disclosure. For a detailed functional description of each module of the apparatus, please refer to the description in the corresponding method shown above, and it will not be repeated here.

[0092] This disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this disclosure. Compared with the prior art, the solution proposed in this disclosure starts from the full historical state conditions at the low-frequency level, and the generator maps the latent variables into coarse motion drafts covering different modes, achieving "decisive selection of a solution"; at the high-frequency level, it receives the drafts and uses a second-order physics refiner combined with real-time force feedback or body feedback to perform high-frequency closed-loop refinement; and the two levels operate on different clocks without blocking each other. This achieves the technical effect of simultaneously satisfying both decisive decision-making and real-time execution in robot motion generation in actual tasks, decoupling the two conflicting requirements.

[0093] In one alternative embodiment, an electronic device is provided, such as Figure 4 As shown, Figure 4 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0094] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure herein. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0095] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0096] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0097] The memory 4003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 4001 to execute them. The processor 4001 is used to execute the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0098] Electronic devices include, but are not limited to, robots.

[0099] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0100] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0101] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0102] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A robot reasoning method, characterized in that, include: At the low-frequency level, the generator maps the latent variables to coarse action drafts covering different modes by utilizing the full history state conditions. At the high-frequency level, the coarse motion draft is received and refined in a closed loop using real-time feedback, outputting the refined motion to drive the robot; wherein, the real-time feedback includes real-time force feedback and body feedback; the operating frequency of the high-frequency level is higher than that of the low-frequency level; the high-frequency level and the low-frequency level operate on different clocks; The latent variables are continuously trajectoried on a unit sphere to ensure mode consistency across control steps.

2. The method according to claim 1, characterized in that, The method further includes: The full-history state conditions are obtained by conditionally encoding the output of the full-history state propagator at the current moment and the current multimodal observation.

3. The method according to claim 1, characterized in that, The generator is trained using implicit maximum likelihood estimation, and the training method includes the following steps: For each teaching sample, multiple latent variables are randomly selected to generate multiple candidate drafts; wherein, the teaching sample includes task conditions and expert actions; Select the candidate draft that is closest to the teaching sample from the plurality of candidate drafts; The candidate draft that is closest to the teaching sample is pulled closer to the teaching sample.

4. The method according to claim 1, characterized in that, The continuous orbital evolution of the latent variables on a unit sphere includes: The time correlation of the tangential velocity of the hidden variable is controlled by the inertia coefficient; The evolution angle is used to control the speed at which the latent variables evolve along the sphere, and the result is normalized and projected onto a unit sphere; The inertia coefficient and the evolution angle are configurable or learnable parameters.

5. The method according to claim 1, characterized in that, The process of receiving the rough draft and performing closed-loop refinement based on real-time feedback includes: The robot's trajectory is considered as the state of a second-order dynamic system, including position and velocity. The acceleration is given by combining the coarse motion draft with the real-time feedback using a second-order physics refiner, and the position and velocity are updated using the Sinusoidal integral. The number of refinement iterations for each control step of the high-frequency stage is adaptively adjusted according to the contact intensity or disturbance amplitude.

6. The method according to claim 1, characterized in that, At the low-frequency level, the generator maps latent variables to coarse action drafts covering different modes using full-history state conditions, including: It runs once every configurable number of control steps, or is triggered by at least one of the following events: task phase switching, target change, or high-frequency fine-tuning residual exceeding a threshold.

7. The method according to claim 1, characterized in that, The method further includes: During training, a pattern consistency regularization is applied to the draft differences between adjacent control steps.

8. A robotic reasoning device, characterized in that, include: The first processing module is used at the low-frequency level to map latent variables into coarse action drafts covering different modes by using the generator with full historical state conditions. The second processing module is used to receive the coarse motion draft at the high-frequency level, and perform closed-loop refinement in combination with real-time feedback to output the refined motion for driving the robot; wherein, the real-time feedback includes real-time force feedback and body feedback; the operating frequency of the high-frequency level is higher than the operating frequency of the low-frequency level; the high-frequency level and the low-frequency level operate on different clocks; The aforementioned device is also used to perform continuous trajectory evolution of the latent variables on a unit sphere to ensure mode consistency across control steps.

9. An electronic device, comprising: A memory, a processor, and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images