Robot control method and device, equipment, storage medium and program product
Through the combination of vision planner and encoder decoder, depth and RGB information in the robot environment are extracted and processed, and the problem of low robot control accuracy is solved, achieving high accuracy and robust control in complex environments.
Patent Information
- Application Number
- CN202411027786.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, robots have low control accuracy and are difficult to effectively deploy in long-term tasks, especially in complex and dynamic environments.
The trained vision planner determines the future frame image based on the current frame image, combines depth information and RGB information, and uses a state encoder and action decoder to extract the deviations of the target state characteristics and the expected state characteristics, and then controls the robot to perform the corresponding actions. Improve control accuracy.
It significantly improves the control accuracy and adaptability of the robot, can maintain stable performance in complex environments, and enhances robustness and adaptability in long-term tasks.
Smart Images

Figure CN118906057A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a robot control method, device, equipment, storage medium and program product. Background Art
[0002] In recent years, with the successful advancement of representation learning in computer vision, visual generation technology, natural language understanding and large-scale language models, and policy learning, robotics and embodied general intelligence have made great achievements. Robots have demonstrated excellent behavioral intelligence in various complex single-task environments, from picking up Lego blocks to solving Rubik's cubes. However, deploying robots to long-term tasks remains a long-term challenge.
[0003] In the existing technology, vision is used as a universal medium to develop an embodied agent that can plan various tasks through imagination and execution. This method involves a generative model to predict future videos or target images, and then a target-conditioned strategy to transform the visual plan into actual actions. Although it has achieved certain success, it still follows the open loop paradigm, resulting in the problem of low robot control accuracy. Summary of the invention
[0004] The present invention provides a robot control method, device, equipment, storage medium and program product, which can improve the control accuracy of the robot.
[0005] In a first aspect, an embodiment of the present disclosure provides a robot control method, comprising: determining a future frame image based on a current frame image through a trained visual planner; wherein the image includes depth information and RGB information; encoding and extracting features of the current frame image and the future frame image in sequence through a trained state encoder to obtain current target state features and expected target state features; determining a deviation between the current target state features and the expected target state features; decoding the deviation through a trained action decoder to obtain an action feature; and controlling the robot according to the action feature so that the robot performs an action corresponding to the action feature.
[0006] In the second aspect, the embodiments of the present disclosure also provide a robot control device, including: a future frame image determination module, used to determine the future frame image according to the current frame image through a trained visual planner; wherein the image includes depth information and RGB information; an encoding and feature extraction module, used to encode and extract features of the current frame image and the future frame image in sequence through a trained state encoder to obtain current target state features and expected target state features; a deviation determination module, used to determine the deviation between the current target state features and the expected target state features; a decoding module, used to decode the deviation through a trained action decoder to obtain action features; and a control module, used to control the robot according to the action features so that the robot performs the action corresponding to the action features.
[0007] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the robot control method as described in the embodiment of the present disclosure.
[0008] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the robot control method as described in the embodiments of the present disclosure.
[0009] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, implements the robot control method as described in the embodiment of the present disclosure.
[0010] The technical solution of the disclosed embodiment is to determine the future frame image according to the current frame image through a trained visual planner; wherein the image includes depth information and RGB information; the current frame image and the future frame image are sequentially encoded and feature extracted through a trained state encoder to obtain current target state features and expected target state features; the deviation of the current target state features and the expected target state features is determined; the deviation is decoded through a trained action decoder to obtain action features; the robot is controlled according to the action features so that the robot performs the action corresponding to the action features. The disclosed embodiment is to determine the future frame image according to the current frame image through a trained visual planner, encode the current frame image and the future frame image sequentially and feature extract through a trained state encoder to obtain current target state features and expected target state features, decode the deviation of the current target state features and the expected target state features through a trained action decoder to obtain action features, and control the robot according to the action features, which can improve the control accuracy of the robot. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0012] Figure 1 A schematic flow chart of a robot control method provided by an embodiment of the present invention;
[0013] Figure 2 A schematic flow chart of another robot control method provided by an embodiment of the present invention;
[0014] Figure 3 A schematic diagram of the structure of a robot control device provided by an embodiment of the present invention;
[0015] Figure 4 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0017] It should be understood that the various steps recorded in the method implementation of the present disclosure can be performed in different orders and / or in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect. The term "including" and its variations used herein are open inclusions, that is, "including but not limited to". It should be noted that the concepts of "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise clearly stated in the context, it should be understood as "one or more". It is understandable that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of the corresponding laws, regulations and relevant provisions.
[0018] Figure 1 The following is a flow chart of a robot control method provided by an embodiment of the present invention. The embodiment of the present disclosure is applicable to the case of controlling a robot. The method can be executed by a robot control device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server. Figure 1 As shown, the method includes:
[0019] S110, determining a future frame image according to the current frame image through a trained visual planner.
[0020] The image includes depth information and RGB information.
[0021] Among them, the visual planner can be understood as a diffusion model, and the diffusion model (Diffusion Models) can be understood as a data generation technology that simulates common diffusion processes in nature to synthesize new data. In this embodiment, the diffusion model can be used to predict future frame images based on the current frame image, that is, the diffusion model can generate future videos spanning a predetermined time frame (starting from the current frame, the frame in a certain time period in the future, the time end is pre-set, which can be a few seconds or minutes). Among them, the current frame image can be understood as an RGB (red, green, and blue) image and a depth map. The future frame image can also be understood as an RGB image and a depth map, that is, the current frame image and the future frame image can both include depth information and RGB information.
[0022] Optionally, determining a future frame image based on the current frame image through a trained visual planner includes: acquiring the current frame image from the perspective of the robot; and predicting multiple future frame images based on the current frame image.
[0023] In this embodiment, the current frame image (that is, the image corresponding to the current moment) from the perspective of the robot can be obtained through a camera installed on the robot, and the future multiple frames of images can be predicted based on the current frame image through a trained visual planner, that is, through a trained diffusion model. In this embodiment, there is no limit on the number of future multiple frames of images, for example, it can be 3 frames, 5 frames, etc., and there can be an interval between each frame, for example, the current frame is the first frame, and the future multiple frames take 3 frames as an example, and the future multiple frames can be the third frame, the fifth frame, and the seventh frame respectively.
[0024] The training method of the visual planner includes: collecting images from historical video samples at set intervals to obtain multiple frames of image samples; using the first image sample of the multiple image samples as an image training sample, wherein the first image sample is equivalent to the current frame image in the prediction stage (or the actual application stage); using the remaining image samples of the multiple image samples as multiple actual image samples; inputting the image training samples into the visual planner to output multiple predicted images; inputting the multiple predicted images and the multiple actual image samples into the target loss function to output the loss value; iteratively training the visual planner based on the loss value until the corresponding iterative training stop condition is met to obtain the trained visual planner.
[0025] The set interval may be one frame per second or one frame per two seconds, and this embodiment does not limit the specific set interval. For example, assuming that the set interval is 2, the historical video sample has 30 frames in total, and the multi-frame image samples are 4, then the 4 multi-frame image samples are the 5th frame, 10th frame, 15th frame, and 20th frame in the historical video sample. The 5th frame in the historical video sample can be used as an image training sample, and the 10th frame, 15th frame, and 20th frame in the historical video sample are used as multiple actual image samples.
[0026] The iterative training stopping condition may be that the number of iterations is greater than or equal to the iteration number threshold, or the loss value is less than or equal to the set loss threshold.
[0027] In this embodiment, images are collected from historical video samples at set intervals, and the collected multiple frame image samples are divided, and the first image sample among the multiple image samples is used as an image training sample for training a visual planner. The remaining image samples are used as multiple actual image samples for comparison with the prediction results of the visual planner to evaluate its performance. The selected image training samples are input into the visual planner, and the visual planner generates a series of predicted images based on the input training samples. Multiple predicted images can represent the prediction or planning of future video frames by the visual planner. The multiple predicted images output by the visual planner and the corresponding multiple actual image samples are input into a target loss function. The loss function calculates the loss value between the predicted image and the actual image, and the visual planner is iteratively trained based on the calculated loss value. In each iteration, the visual planner parameters are adjusted by the back propagation algorithm to minimize the loss value. When the iterative training stop condition is met, the training process ends and the trained visual planner is obtained. Among them, the visual planner can be a diffusion model.
[0028] The target loss function is composed of a set loss function and a set optical flow regularization; wherein the set optical flow regularization is composed of optical flow features and the multiple frame image samples.
[0029] In this embodiment, there is no limitation on the specific loss function, which may be, for example, mean square error, mean absolute error, etc.
[0030] In this embodiment, the robot manipulation task has interactive dynamic characteristics, namely the moving robot arm and the interactive objects in the environment. In this embodiment, optical flow is used as an explicit regularization term to further promote the application of diffusion models in manipulation tasks. Specifically, following the end-to-end optical flow estimation framework, a pixel-level correspondence map is first established between the diffusion latent spaces of two image frames. Then, the optical flow estimate is iteratively refined through search and update operations using this map. Based on the final estimate, the optical flow-based regularization term is formulated as: That is, the optical flow regularization formula is set as:
[0031]
[0032] Among them, L reg To set the value of optical flow regularization, is the estimated optical flow, W represents the mapping function, and {O k ,O k+1} are two consecutive image frames in the multi-frame image samples, where k is a variable and can represent the index value of the frame.
[0033] S120, encoding and extracting features of the current frame image and the future frame image in sequence through the trained state encoder to obtain current target state features and expected target state features.
[0034] In this embodiment, the future frame image includes multiple future frame images. Each time, a future frame image is extracted from the multiple future frame images in time sequence, and is combined with the current frame image to form an image pair. The current frame image and the future frame image are sequentially encoded and feature extracted by the trained state encoder to obtain the current state feature corresponding to the current frame image and the expected state feature corresponding to the future frame image. If the similarity difference between the current state feature and the expected state feature is greater than or equal to the set similarity threshold, the corresponding current state feature is used as the current target state feature, and the corresponding expected state feature is used as the expected target state feature. If the similarity difference between the current state feature and the expected state feature is less than the set similarity threshold, the image pair is reselected, for example, the next future frame image is extracted from the multiple future frame images, and is combined with the current frame image to form a new image pair, and the steps of sequentially encoding and feature extraction of the current frame image and the future frame image by the trained state encoder are re-executed until the similarity difference between the current state feature and the expected state feature is greater than or equal to the set similarity threshold, and the current target state feature and the expected target state feature are obtained.
[0035] Optionally, the current frame image and the future frame image are sequentially encoded and feature extracted by a trained state encoder to obtain current target state features and expected target state features, including: extracting the Kth future frame image from the future multiple frame images; wherein K is a positive integer, and initially, K is 1; taking the current frame image and the Kth future frame image as an image pair; sequentially encoding and feature extracting the images by a trained state encoder to obtain current state features and expected state features; determining the similarity difference between the current state features and the expected state features; if the similarity difference is greater than or equal to a set similarity difference threshold, taking K+1 as K, returning to the step of extracting the Kth future frame image from the future multiple frame images, and continuing to execute; if the similarity difference is less than a set similarity difference threshold, taking the current state features as the current target state features, and taking the expected state features as the expected target state features.
[0036] Exemplarily, K is a positive integer, and K is 1 initially; K is 3 as an example. Extract the first future frame image from the future multi-frame images; and take the current frame image and the first future frame image as an image pair. Encode and extract features from the images in sequence through the trained state encoder to obtain the current state features corresponding to the current frame image and the expected state features corresponding to the first future frame image; determine the cosine similarity between the current state features and the expected state features, and then subtract the cosine similarity from 1 to obtain the similarity difference; if the similarity difference is less than the set similarity difference threshold, take the current state features as the current target state features, and take the expected state features as the expected target state features. If the similarity difference is greater than or equal to the set similarity difference threshold, extract the second future frame image from the future multi-frame images; take the current frame image and the second future frame image as an image pair; encode and extract features from the images in sequence through the trained state encoder to obtain the current state features corresponding to the current frame image and the expected state features corresponding to the second future frame image; and determine the similarity difference between the current state features and the expected state features. It is determined again whether the similarity difference is less than the set similarity difference threshold. Assuming that the similarity difference at this time is less than the set similarity difference threshold, the current state feature corresponding to the previous frame image is used as the current target state feature, and the expected state feature corresponding to the second future frame image is used as the expected target state feature.
[0037] In this embodiment, there is no limitation on the similarity algorithm used, for example, it may be cosine similarity. There is no limitation on the specific similarity difference threshold, for example, it may be 10%, 5%, etc.
[0038] In this embodiment, by calculating the cosine similarity between the current target state feature and the desired target state feature, and judging whether the similarity difference between the current state feature and the desired state feature is less than a set similarity difference threshold, state transfer can be achieved and future frame images can be replanned, that is, the input of the state encoder can be flexibly adjusted. Adaptive transition between different future frame images is achieved through cosine similarity.
[0039] The state encoder includes a multimodal encoder and a feature aggregator. The multimodal encoder includes an RGB image encoder and a depth image encoder.
[0040] Among them, the multimodal encoder can convert raw pixels into rich visual representations. The state encoder can include an RGB image encoder, a depth image encoder, and a multimodal feature fusion module. Multimodal features can be understood as RGB features and depth features.
[0041] Optionally, the image is encoded and feature extracted in sequence through a trained state encoder to obtain current state features and expected state features, including: respectively encoding the RGB information of the current frame image and the RGB information of the K-th future frame image through the RGB image encoder to obtain current RGB features and future RGB features; respectively encoding the depth information of the current frame image and the depth information of the K-th future frame image through the depth image encoder to obtain current depth features and future depth features; fusing the current RGB features and the current depth features to obtain current fused features; fusing the future RGB features and the future depth features to obtain future fused features; and extracting features of the current fused features and the future fused features through the feature aggregator to obtain current state features and expected state features.
[0042] Exemplarily, the RGB information of the current frame image and the RGB information of the Kth future frame image are respectively encoded by the RGB image encoder to obtain the current RGB feature and the future RGB feature; the depth information of the current frame image and the depth information of the Kth future frame image are respectively encoded by the depth image encoder to obtain the current depth feature and the future depth feature, and the current RGB feature and the current depth feature are fused by the multimodal feature fusion module to obtain the current fused feature; the future RGB feature and the future depth feature are fused by the multimodal feature fusion module to obtain the future fused feature. In the feature fusion process, a squeeze-and-excitation module is specifically used for channel integration and selection. The feature aggregator extracts the current fused feature and the future fused feature to obtain the current state feature and the expected state feature. In the feature aggregator, the key information related to control can be adaptively selected from the current fused feature and the future fused feature, and compressed into a compact state embedding, that is, the current state feature and the expected state feature.
[0043] Among them, the feature aggregator includes a multi-head attention pooling unit and a multi-layer perceptron.
[0044] Optionally, the feature aggregator performs feature extraction on the current fusion feature and the future fusion feature to obtain current state features and expected state features, including: performing feature extraction on the current fusion feature and the future fusion feature through the multi-head attention pooling unit to obtain current state sub-features and expected state sub-features; performing nonlinear projection on the current state sub-features and the expected state sub-features through the multi-layer perceptron to obtain current state features and expected state features.
[0045] Exemplarily, the specific process of the feature aggregator is as follows:
[0046]
[0047] in, represents the state embedding, where Indicates the current state characteristics, represents the expected state feature, MultiHeadAttn() represents the multi-head attention pooling unit. MLP represents the multi-layer perceptron. Q = query, K = key, V = value, Q, K, V are the three inputs of the multi-head attention pooling unit. E∈{E cur ,E goal}, where E cur represents the current fusion feature, E goal represents the future fusion feature. And q∈{q cur ,q goal}Query vector for extracting visual features, q cur and q goal Each is a learnable query that can extract information from E. Shared weights are used to encode the two states in parallel, except that two separately initialized queries are used to extract each state embedding. The state encoder forms an information bottleneck, preferentially encoding features related to control operations while filtering out irrelevant background details to ensure the amount of information.
[0048] S130: Determine the deviation between the current target state characteristic and the expected target state characteristic.
[0049] In this embodiment, the difference between the current target state feature and the desired target state feature can be explicitly modeled by performing element-wise subtraction on the current target state feature and the desired target state feature.
[0050] S140 , decoding the deviation through a trained action decoder to obtain action features.
[0051] In this embodiment, the motion decoder can use a multi-layer perceptron (MLP) to decode the error and obtain a series of motion features. The motion features can correspond to the specific actions or instructions that the robot needs to perform. Among them, the motion features can be understood as 7 degrees of freedom. The 7 degrees of freedom action can include the position adjustment (x, y, z) of the robot end effector in three-dimensional space, posture adjustment (roll, pitch, yaw) and the opening and closing state of the gripper.
[0052] S150: Control the robot according to the action feature so that the robot performs the action corresponding to the action feature.
[0053] In this embodiment, the robot is controlled in real time according to the motion features output by the motion decoder, so that the robot performs corresponding actions.
[0054] Optionally, after controlling the robot according to the action features, it also includes: acquiring a new current frame image from the perspective of the robot; predicting new future multi-frame images based on the new current frame image; if K is equal to the number of future multi-frame images, taking the new current frame image as the current frame image, and the new future multi-frame images as future multi-frame images, returning to the step of encoding and extracting features of the current frame image and future frame images in sequence through the trained state encoder to obtain current target state features and expected target state features, and continuing to execute until the number of times the robot performs an action is greater than or equal to a set action number threshold, and the execution is terminated; if K is less than the number of future multi-frame images, taking K+1 as K, returning to the step of extracting the Kth future frame image from the future multi-frame images, and continuing to execute.
[0055] Exemplarily, still taking K as 2 as an example, the current state feature corresponding to the previous frame image is used as the current target state feature, and the expected state feature corresponding to the second future frame image is used as the expected target state feature. The deviation between the current target state feature and the expected target state feature is determined, and the deviation is decoded by a trained action decoder to obtain an action feature, and the robot is controlled according to the action feature so that the robot performs the action corresponding to the action feature.
[0056] Next, a new current frame image from the perspective of the robot can be obtained again; new future multi-frame images are predicted based on the new current frame image; since K is 2, the number of future multi-frame images is 3, that is, if K is less than the number of future multi-frame images, K+1 is taken as K, and the process returns to the step of extracting the Kth future frame image from the future multi-frame images and continues to execute. That is, the third future frame image is extracted from the future multi-frame images; the current frame image and the third future frame image are taken as an image pair; the images are sequentially encoded and feature extracted through the trained state encoder to obtain current state features and expected state features; and the similarity difference between the current state features and the expected state features is determined. Since K is 3 at this time, which is equal to the number of future multi-frame images, the new current frame image is used as the current frame image, and the new future multi-frame images are used as future multi-frame images. Assuming that the similarity difference is greater than the set similarity difference threshold at this time, K+1 is used as K, and the process returns to the step of extracting the Kth future frame image from the future multi-frame images and continues to execute; that is, K is 4 at this time, and the fourth future frame image is the first future frame image in the new future multi-frame images, and for the future multi-frame images, it can be understood as the fourth future frame image.
[0057] Similarly, if the number of times the robot performs an action is greater than or equal to the set action number threshold, the entire process ends. In this embodiment, there is no restriction on the specific set action number threshold, for example, it can be 20 times, 30 times, etc.
[0058] In this embodiment, the state encoder and the action decoder can be trained simultaneously during the training phase. Random sampling of historical video samples is performed to obtain training samples of the encoder-decoder (for one training sample, two image samples are included, one image sample is equivalent to the current frame image in the actual application phase, and the other image sample is equivalent to the future frame image in the actual application phase), the action decoder outputs the predicted action feature (that is, the action with 7 degrees of freedom), the loss value of the predicted action feature and the real action feature is determined based on the loss function (such as the mean square error), and the state encoder and the action decoder are iteratively trained based on the loss value until the iterative training stop condition is met, and the trained state encoder and the trained action decoder are obtained.
[0059] The technical solution of the disclosed embodiment is to determine the future frame image according to the current frame image through a trained visual planner; wherein the image includes depth information and RGB information; the current frame image and the future frame image are sequentially encoded and feature extracted through a trained state encoder to obtain current target state features and expected target state features; the deviation of the current target state features and the expected target state features is determined; the deviation is decoded through a trained action decoder to obtain action features; the robot is controlled according to the action features so that the robot performs the action corresponding to the action features. The disclosed embodiment is to determine the future frame image according to the current frame image through a trained visual planner, encode the current frame image and the future frame image sequentially and feature extract through a trained state encoder to obtain current target state features and expected target state features, decode the deviation of the current target state features and the expected target state features through a trained action decoder to obtain action features, and control the robot according to the action features, which can improve the control accuracy of the robot.
[0060] Figure 2 A flow chart of another robot control method provided by an embodiment of the present invention. The method provided by an embodiment of the present invention specifically comprises the following steps:
[0061] S201, obtaining a current frame image from the robot's perspective.
[0062] S202: predict multiple future image frames based on the current image frame.
[0063] S203: extracting the Kth future frame image from the future multiple frame images.
[0064] S204 , respectively encode the RGB information of the current frame image and the RGB information of the Kth future frame image through an RGB image encoder to obtain current RGB features and future RGB features.
[0065] S205 . Encode the depth information of the current frame image and the depth information of the Kth future frame image respectively through a depth image encoder to obtain a current depth feature and a future depth feature.
[0066] S206: Fuse the current RGB feature and the current depth feature to obtain a current fused feature.
[0067] S207: Fuse the future RGB features and the future depth features to obtain future fusion features.
[0068] S208. Extract the current fusion features and the future fusion features through a feature aggregator to obtain current state features and expected state features.
[0069] S209, extracting features of the current fusion feature and the future fusion feature through the multi-head attention pooling unit to obtain the current state sub-feature and the expected state sub-feature.
[0070] S210, performing nonlinear projection on the current state sub-feature and the expected state sub-feature respectively through a multi-layer perceptron to obtain a current state feature and an expected state feature.
[0071] S211: Determine the similarity difference between the current state feature and the expected state feature.
[0072] S212: If the similarity difference is greater than or equal to the set similarity difference threshold, K+1 is used as K and the process returns to S203.
[0073] S213: If the similarity difference is less than the set similarity difference threshold, the current state feature is used as the current target state feature, and the expected state feature is used as the expected target state feature.
[0074] S214: Determine the deviation between the current target state characteristics and the expected target state characteristics.
[0075] S215. Decode the deviation through the trained action decoder to obtain action features.
[0076] S216: Control the robot according to the action feature so that the robot performs the action corresponding to the action feature.
[0077] S217: Obtain a new current frame image from the robot's perspective.
[0078] S218: Predict new future multiple frame images based on the new current frame image.
[0079] S219. If K is equal to the number of future multi-frame images, the new current frame image is used as the current frame image, and the new future multi-frame images are used as future multi-frame images, and the process returns to S203 and continues to execute until the number of times the robot performs an action is greater than or equal to the set action number threshold, and the execution ends.
[0080] S220: If K is less than the number of future multi-frame images, K+1 is used as K, and the process returns to S203 to continue the process.
[0081] In this embodiment, the future multi-frame images are predicted by the visual planner, so that the robot can perceive and evaluate the changes in the future environment in advance, significantly enhancing its adaptability in complex or dynamic environments. The selection and gradual adjustment of the Kth future frame image (by judging the similarity difference) further improves the accuracy of the prediction. The RGB information and depth information are encoded separately, and these features are fused, so that the robot can understand the current and future environmental states more comprehensively. This fusion of multimodal information improves the richness and accuracy of feature representation. Through the processing of feature aggregators and multi-layer perceptrons, the robot can accurately evaluate the difference between the current state and the desired state, providing a reliable basis for subsequent decision-making. Deviation-based action decoding ensures that the robot can perform actions corresponding to the difference between the current state and the target state, thereby achieving precise control. By continuously acquiring new current frame images and predicting new future frame images, the robot can continuously perform state evaluation, decision-making and control, forming a closed-loop visual motion control framework. This continuous control cycle enables the robot to maintain stable performance in a constantly changing environment.
[0082] In this embodiment, the current frame image and the future frame image are encoded and feature extracted in sequence through the trained state encoder to obtain the current target state feature and the expected target state feature; the deviation between the current target state feature and the expected target state feature is determined; the deviation is decoded by the trained action decoder to obtain the action feature. These three parts can constitute a feedback-driven strategy network, which can also be called a feedback mechanism.
[0083] The technical solution provided in this embodiment can be called CLOVER. Compared with the prior art, the embodiment of the present invention integrates the closed-loop control philosophy into the robot visual motion control, introduces a feedback mechanism to improve adaptability, so that the errors in the execution process can be monitored and adjusted in real time, so as to more effectively control the behavior of the robot in a dynamic environment. In the embodiment of the present invention, by measuring the state (deviation) of the potential embedding (current target state characteristics and expected target state characteristics) and clearly quantifying the feedback error, an execution process is established, which effectively solves the uncertainty challenges in image generation and task planning, so that the execution state can be more accurately evaluated and the action plan can be adjusted in time, thereby improving the robustness and adaptability of the robot in long-term tasks. In the embodiment of the present invention, the instability of the diffusion model can be detected and resolved, and adaptive future frame image transition can be achieved according to the similarity difference. Compared with the traditional method, the embodiment of the present invention is not limited by a specific data set, can flexibly adapt to different visual planners and task execution environments, and improves the performance and scalability of the system. In summary, the embodiments of the present invention effectively solve the challenges of the prior art in long-term mission control and dynamic environment adaptability through the application of key technical solutions such as closed-loop control, error quantization and real-time feedback. It has higher intelligence and adaptive performance, a wider range of applications, and can better meet actual application needs.
[0084] For experimental tasks, most of the experiments are mainly conducted using CALVIN, an evaluation benchmark designed for long-term tasks and language-conditioned operations. CALVIN includes four simulated environments (labeled A, B, C, and D), which differ in texture and object position. Each environment includes a Franka Emika Panda robot arm located next to a table, equipped with a variety of operable objects. In this embodiment of the present invention, the policy model is trained on demonstration videos collected in environments A, B, and C, and zero-shot testing is performed in environment D. The evaluation protocol involves evaluating the model performance on a set of tasks containing 1,000 unique command chains, each containing five different tasks. The CALVIN benchmark provides a large dataset paired with natural language annotations, which helps to train a general and reliable visual planner. Real machine experiments, actual robot experiments are conducted on the assembled MobileALOHA platform.
[0085] The main experimental results show that the method provided by the embodiment of the present invention has achieved state-of-the-art performance on CALVIN, which is significantly better than the previous method using a similar "planner + executor" process. Without using gripper view images and body signals, the method provided by the embodiment of the present invention surpasses methods that use generative pre-trained (GPT) style transformers with billions of parameters, such as RoboFlamingo and GR-1 visual language manipulation framework. It should be noted that all previous methods follow the standard evaluation protocol of CALVIN, in which the simulator returns a signal marking the completion of the current task. However, such a task completion signal cannot be obtained in a real environment. In fact, by utilizing the method provided by the embodiment of the present invention, the completion of a task can be determined autonomously in the absence of a signal.
[0086] Figure 3 A schematic diagram of the structure of a robot control device provided by an embodiment of the present disclosure is shown in FIG. Figure 3 As shown, the device includes: a future frame image determination module 310, a coding and feature extraction module 320, a deviation determination module 330, a decoding module 340 and a control module 350;
[0087] A future frame image determination module 310, configured to determine a future frame image according to a current frame image through a trained visual planner; wherein the image includes depth information and RGB information;
[0088] The encoding and feature extraction module 320 is used to encode and extract features of the current frame image and the future frame image in sequence through the trained state encoder to obtain the current target state feature and the expected target state feature;
[0089] A deviation determination module 330, configured to determine a deviation between the current target state feature and the desired target state feature;
[0090] A decoding module 340, configured to decode the deviation using a trained motion decoder to obtain motion features;
[0091] The control module 350 is used to control the robot according to the action feature so that the robot performs the action corresponding to the action feature.
[0092] The technical solution of the disclosed embodiment is that the future frame image determination module determines the future frame image according to the current frame image through the trained visual planner; wherein the image includes depth information and RGB information; the encoding and feature extraction module sequentially encodes and extracts features from the current frame image and the future frame image through the trained state encoder to obtain the current target state feature and the expected target state feature; the deviation determination module determines the deviation of the current target state feature and the expected target state feature; the decoding module decodes the deviation through the trained action decoder to obtain the action feature; the control module controls the robot according to the action feature so that the robot performs the action corresponding to the action feature. In the disclosed embodiment, the future frame image is determined according to the current frame image by the trained visual planner, the current frame image and the future frame image are sequentially encoded and feature extracted through the trained state encoder to obtain the current target state feature and the expected target state feature, the deviation of the current target state feature and the expected target state feature is decoded through the trained action decoder to obtain the action feature, and the robot is controlled according to the action feature, which can improve the control accuracy of the robot.
[0093] Optionally, the future frame image determination module is specifically used to: obtain a current frame image from the perspective of the robot; and predict future multiple frame images based on the current frame image.
[0094] Optionally, the encoding and feature extraction module is specifically used to: extract the Kth future frame image from the future multi-frame images; wherein K is a positive integer, and initially, K is 1; take the current frame image and the Kth future frame image as an image pair; encode and extract features from the images in sequence through a trained state encoder to obtain current state features and expected state features; determine the similarity difference between the current state features and the expected state features; if the similarity difference is greater than or equal to a set similarity difference threshold, take K+1 as K, return to the step of extracting the Kth future frame image from the future multi-frame images, and continue executing; if the similarity difference is less than a set similarity difference threshold, take the current state features as current target state features, and take the expected state features as expected target state features.
[0095] Wherein, the state encoder includes a multimodal encoder and a feature aggregator; the multimodal encoder includes an RGB image encoder and a depth image encoder.
[0096] Optionally, the encoding and feature extraction module is specifically used to: respectively encode the RGB information of the current frame image and the RGB information of the K-th future frame image through the RGB image encoder to obtain current RGB features and future RGB features; respectively encode the depth information of the current frame image and the depth information of the K-th future frame image through the depth image encoder to obtain current depth features and future depth features; fuse the current RGB features and the current depth features to obtain current fused features; fuse the future RGB features and the future depth features to obtain future fused features; perform feature extraction on the current fused features and the future fused features through the feature aggregator to obtain current state features and expected state features.
[0097] Among them, the feature aggregator includes a multi-head attention pooling unit and a multi-layer perceptron.
[0098] Optionally, the encoding and feature extraction module is also used to: perform feature extraction on the current fusion feature and the future fusion feature through the multi-head attention pooling unit to obtain current state sub-features and expected state sub-features; perform nonlinear projection on the current state sub-features and the expected state sub-features through the multi-layer perceptron to obtain current state features and expected state features.
[0099] Optionally, the encoding and feature extraction module is also used to: obtain a new current frame image from the perspective of the robot; predict new future multi-frame images based on the new current frame image; if K is equal to the number of future multi-frame images, then use the new current frame image as the current frame image, and the new future multi-frame images as future multi-frame images, return to the step of encoding and feature extracting the current frame image and future frame images in sequence through the trained state encoder to obtain current target state features and expected target state features, and continue to execute until the number of times the robot performs an action is greater than or equal to a set action number threshold, and then terminate the execution; if K is less than the number of future multi-frame images, then use K+1 as K, return to the step of extracting the Kth future frame image from the future multi-frame images, and continue to execute.
[0100] Optionally, the above-mentioned device also includes a training module, which is specifically used to: collect images from historical video samples at set intervals to obtain multiple frames of image samples; use the first image sample among the multiple image samples as an image training sample, and use the remaining image samples among the multiple image samples as multiple actual image samples; input the image training sample into the visual planner and output multiple predicted images; input the multiple predicted images and the multiple actual image samples into a target loss function and output a loss value; iteratively train the visual planner based on the loss value until the corresponding iterative training stop condition is met, so as to obtain a trained visual planner.
[0101] The target loss function is composed of a set loss function and a set optical flow regularization; wherein the set optical flow regularization is composed of optical flow features and the multiple frame image samples.
[0102] The robot control device provided in the embodiments of the present disclosure can execute the robot control method provided in any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.
[0103] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0104] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0105] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0106] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as the control of the method robot.
[0107] In some embodiments, the control of the method robot can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the control of the method robot described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the control of the method robot in any other suitable manner (for example, by means of firmware).
[0108] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0109] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0110] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0111] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0112] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0113] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0114] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the robot control method provided in any embodiment of the present application.
[0115] In the process of implementation, the computer program product can be written in one or more programming languages or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).
[0116] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. A robot control method, characterized in that: include: Determine a future frame image according to the current frame image through a trained visual planner; wherein the image includes depth information and RGB information; The current frame image and the future frame image are sequentially encoded and feature extracted by a trained state encoder to obtain current target state features and expected target state features; Determining a deviation between the current target state characteristic and the desired target state characteristic; Decoding the deviation through a trained action decoder to obtain action features; The robot is controlled according to the action feature so that the robot performs the action corresponding to the action feature.
2. The method according to claim 1, characterized in that The trained visual planner determines the future frame image based on the current frame image, including: Acquire the current frame image from the perspective of the robot; Predicting multiple future image frames based on the current image frame.
3. The method according to claim 2, characterized in that The current frame image and the future frame image are sequentially encoded and feature extracted by the trained state encoder to obtain current target state features and expected target state features, including: Extracting the Kth future frame image from the future multiple frame images; wherein K is a positive integer, and initially, K is 1; Taking the current frame image and the Kth future frame image as an image pair; The image is encoded and feature extracted in sequence by a trained state encoder to obtain current state features and expected state features; Determining a similarity difference between the current state feature and the expected state feature; If the similarity difference is greater than or equal to the set similarity difference threshold, K+1 is used as K, and the process returns to the step of extracting the Kth future frame image from the future multiple frame images and continues to execute; If the similarity difference is less than a set similarity difference threshold, the current state feature is used as the current target state feature, and the expected state feature is used as the expected target state feature.
4. The method according to claim 3, characterized in that in, The state encoder includes a multimodal encoder and a feature aggregator; the multimodal encoder includes an RGB image encoder and a depth image encoder; The image is encoded and feature extracted in sequence by the trained state encoder to obtain current state features and expected state features, including: Encoding the RGB information of the current frame image and the RGB information of the Kth future frame image respectively through the RGB image encoder to obtain current RGB features and future RGB features; Encoding the depth information of the current frame image and the depth information of the Kth future frame image respectively by the depth image encoder to obtain a current depth feature and a future depth feature; Fusing the current RGB feature and the current depth feature to obtain a current fused feature; Fusing the future RGB feature and the future depth feature to obtain a future fused feature; The feature aggregator is used to extract features from the current fusion feature and the future fusion feature to obtain current state features and expected state features.
5. The method according to claim 4, characterized in that in, The feature aggregator includes a multi-head attention pooling unit and a multi-layer perceptron; the feature aggregator extracts the current fusion feature and the future fusion feature to obtain the current state feature and the expected state feature, including: The multi-head attention pooling unit performs feature extraction on the current fusion feature and the future fusion feature respectively to obtain a current state sub-feature and an expected state sub-feature; The current state feature and the expected state feature are obtained by performing nonlinear projection on the current state sub-feature and the expected state sub-feature respectively through the multilayer perceptron.
6. The method according to claim 3, characterized in that After controlling the robot according to the action characteristics, the method further includes: Acquire a new current frame image from the perspective of the robot; Predicting new future multiple frame images according to the new current frame image; If K is equal to the number of the future multi-frame images, the new current frame image is used as the current frame image, and the new future multi-frame images are used as the future multi-frame images, and the process returns to the step of encoding and extracting features of the current frame image and the future frame image in sequence through the trained state encoder to obtain the current target state feature and the expected target state feature step, and the process continues until the number of times the robot performs an action is greater than or equal to the set action number threshold, and the execution ends; If K is less than the number of the future multiple-frame images, K+1 is taken as K, and the process returns to the step of extracting the Kth future frame image from the future multiple-frame images and continues to execute.
7. The method according to claim 1, characterized in that in, The training method of the visual planner includes: Collect images from historical video samples at set intervals to obtain multiple frames of image samples; Using the first image sample among the multiple image samples as an image training sample, and using the remaining image samples among the multiple image samples as multiple actual image samples; Inputting the image training samples into the visual planner and outputting a plurality of predicted images; Input the multiple predicted images and the multiple actual image samples into a target loss function, and output a loss value; The visual planner is iteratively trained based on the loss value until a corresponding iterative training stop condition is met, thereby obtaining a trained visual planner.
8. The method according to claim 7, characterized in that in, The target loss function is composed of a set loss function and a set optical flow regularization; wherein the set optical flow regularization is composed of optical flow features and the multiple frame image samples.
9. A robot control device, characterized in that: include: A future frame image determination module, used to determine a future frame image according to a current frame image through a trained visual planner; wherein the image includes depth information and RGB information; An encoding and feature extraction module, used to encode and extract features of the current frame image and the future frame image in sequence through a trained state encoder to obtain current target state features and expected target state features; A deviation determination module, used to determine the deviation between the current target state characteristic and the desired target state characteristic; A decoding module, used for decoding the deviation through a trained action decoder to obtain action features; The control module is used to control the robot according to the action feature so that the robot performs the action corresponding to the action feature.
10. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the robot control method as described in any one of claims 1-8.
11. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute the robot control method according to any one of claims 1 to 8 when executed by a computer processor.
12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the robot control method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Image analysis method based on self-adaptive regularization
CN107464228A
Target following method and robot
CN114550209A
Depth estimation model training method and device, electronic equipment and storage medium
CN116152694A
Method and device for determining placement scheme of object and computer readable storage medium
CN118046372A
Robot planning method and system based on multi-mode large model predictive control
CN118061186A
