Robot strategy model training method and electronic device

Abstract

The application provides a robot strategy model training method and an electronic device, and the method comprises the following steps: obtaining an initial self-centered visual angle teaching data set and a target new visual angle parameter containing plane translation and yaw rotation, and performing reorientation on an initial joint action sequence to generate an action sequence adapted to the target new visual angle; combining an observation video, a depth map, a reoriented action, and a pre-trained video repair model to generate a matched observation video under the new visual angle; after constructing a new visual angle paired teaching data set, a strategy model is trained by mixing original data. The application can automatically generate multi-view teaching data with strictly aligned vision and action without collecting real data under a new visual angle, significantly improves the generalization ability of the model to visual angle offset, ensures the simultaneous improvement of the operation success rate of the standard visual angle and the new visual angle, and enhances the stability and robustness of robot deployment.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and more specifically, to a method for training a robot strategy model and an electronic device. Background Technology

[0002] In the field of robotics, imitation learning trains robots to directly map visual observations to motion control using human-taught data. Currently, vision-motion strategies have made significant progress with large-scale, diverse data training. However, in practical deployments, robots often employ an egocentric perspective. This perspective is mechanically coupled to the robot's chassis, making the model highly sensitive to changes in viewpoint and struggling to maintain operational success rates when the viewpoint shifts.

[0003] Existing solutions are mainly divided into three categories: geometry-driven, visual synthesis, and motion repositioning. Geometric methods use point clouds or NeRF to synthesize images from new perspectives and control the robot's movements from these new perspectives; visual synthesis methods use generative models to generate realistic images; and motion repositioning methods are mainly used to handle changes in object pose.

[0004] However, existing solutions generate only visual data without adjusting the motion, which leads to inconsistency between vision and motion. Secondly, third-person motion retargeting methods ignore the characteristic of the camera moving with the chassis in an egocentric viewpoint. Finally, simple image overlay produces blur artifacts, and the lack of a high-quality paired data generation framework results in poor generalization ability of the strategy in new perspectives. Summary of the Invention

[0005] The purpose of this application is to provide a robot strategy model training method and electronic device to address the shortcomings of the prior art, thereby solving the problem of poor generalization ability of strategies under new perspectives in the prior art.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows: Firstly, this application provides a method for training a robot policy model, the method comprising: The robot acquires an initial teaching dataset from its egocentric perspective and new perspective parameters from the target new perspective. The initial teaching dataset includes multiple sets of initial data, each set of initial data including: a first observation video, a first joint motion sequence, and a depth map sequence. The new perspective parameters include: planar translation parameters and yaw rotation parameters. Based on the new perspective parameters, the first joint motion sequence of each group of initial data in the initial teaching dataset is redirected to obtain the second joint motion sequence of each group of target data under the new target perspective. Based on the first observation video, the depth map sequence, the second joint motion sequence, the new perspective parameters, and the pre-trained video restoration model, generate a second observation video of each set of target data under the new perspective of the target; Based on the second observation video, the second joint motion sequence, and the new target perspective, a target teaching dataset is constructed. The initial policy model is trained using the target teaching dataset and the initial teaching dataset to obtain the target policy model.

[0007] Optionally, the step of redirecting the first joint motion sequence of each group of initial data in the initial teaching dataset according to the new perspective parameters to obtain the second joint motion sequence of each group of target data under the new target perspective includes: Determine the coordinate transformation matrix from the source base coordinate system to the target base coordinate system based on the new perspective parameters; The first joint motion sequence of each group of initial data in the initial teaching dataset is redirected according to the coordinate transformation matrix to obtain the second joint motion sequence of each group of initial data under the new target perspective.

[0008] Optionally, the step of redirecting the first joint motion sequences of each group of initial data in the initial teaching dataset according to the coordinate transformation matrix to obtain the second joint motion sequences of each group of initial data under the new target perspective includes: Extract the target pose of the robot's end effector in the source base coordinate system from the first joint motion sequence; The target pose is transformed according to the coordinate transformation matrix to obtain the desired pose of the robot end effector in the target base coordinate system. Based on the desired pose, the joint angles of the robot are solved using an inverse kinematics algorithm to generate a second joint motion sequence under the new target perspective.

[0009] Optionally, the step of solving the robot's joint angles using an inverse kinematics algorithm based on the desired pose to generate a second joint motion sequence from the new target perspective includes: If the robot is a two-armed robot, then based on the desired pose, the first joint angle of the first joint arm and the second joint angle of the second joint arm of the robot are solved by inverse kinematics algorithm. The first joint angle and the second joint angle are spliced ​​together to obtain the second joint action sequence.

[0010] Optionally, generating a second observation video of the robot target from a new perspective based on the first observation video, the depth map sequence, the second joint motion sequence, the new viewpoint parameters, and a pre-trained video restoration model includes: Based on the first observation video and the depth map sequence of each set of initial data, a scene video is generated; Generate robot video based on the second joint motion sequence and the new perspective parameters; The scene video and the robot video are input as dual-stream conditions into the video restoration model, and the video restoration model synthesizes a second observation video from the new perspective of the target.

[0011] Optionally, generating a scene video based on the first observation video and the depth map sequence of each set of initial data includes: The pixel mapping relationship from the egocentric viewpoint to the target new viewpoint is constructed using the new viewpoint parameters; Based on the pixel mapping relationship and the depth map sequence, the source frame pixels in the first observed video are reprojected onto the target view plane to generate an initial scene frame; The occluded and hollow areas in the initial scene frame are inpainted to obtain the scene video.

[0012] Optionally, the video restoration model includes: a scene video encoder, a robot video encoder, a temporal denoising network, and a decoding module; The scene video and the robot video are input as dual-stream conditions into the video restoration model, which then synthesizes a second observation video from the new perspective of the target, including: The scene video is input into the scene video encoder for encoding processing to obtain a scene feature map, and the robot video is input into the robot video encoder to obtain a robot feature map; The scene feature map and the robot feature map are input into the temporal denoising network, and the new viewpoint parameters are also input into the temporal denoising network as conditional vectors. The temporal denoising network performs denoising processing to generate an initial observation video, and the initial observation video is input into the decoder. The decoder decodes the initial observation video to obtain the second observation video.

[0013] Optionally, the temporal denoising network includes: multiple DiT blocks, each of which includes self-attention and cross-attention mechanisms; The process by which the temporal denoising network performs denoising processing to generate the initial observation video includes: Through the self-attention and cross-attention mechanisms of each DiT block, the background artifacts of the scene video are repaired simultaneously during the denoising process, and the robot video is merged into the background. The output of the last DiT block is used as the initial observation video.

[0014] Optionally, the training process of the video restoration model includes: Construct training sample pairs, which include simulated scene videos with artifacts and corresponding real source videos. The simulated scene videos with artifacts are generated by reprojecting and filling holes in randomly sampled source videos with simulated viewpoint transformations. The training diffusion model learns the mapping distribution from simulated scene videos containing artifacts to real source videos based on the training samples, thus obtaining the video restoration model.

[0015] Secondly, this application provides an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the robot strategy model training method described in the first aspect above.

[0016] The beneficial effects of this application are as follows: By acquiring initial teaching data and new perspective parameters, joint movements adapted to the new perspective are generated through retargeting. Then, corresponding observation videos are synthesized using a video restoration model to construct a new perspective teaching dataset. Finally, the original data is mixed to train the strategy model. This method can generate consistent multi-view teaching data without the need for additional collection of real data from the new perspective. This allows the trained model to improve the success rate of operations under both standard egocentric and new offset perspectives, effectively solving the problem of insufficient generalization ability caused by single-view training. At the same time, it ensures the alignment and matching of vision and action, improving the robustness of model deployment. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart of a robot strategy model training method provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating a method for determining a second joint motion sequence according to an embodiment of this application is shown; Figure 3This document illustrates a flowchart of another method for determining a second joint motion sequence, as provided in an embodiment of this application. Figure 4 This document illustrates a flowchart of a method for generating a second observation video, as provided in an embodiment of this application. Figure 5 This document illustrates a flowchart of a method for generating scene videos according to an embodiment of this application. Figure 6 This illustration shows a structural diagram of a video restoration model provided in an embodiment of this application; Figure 7 This application provides a flowchart illustrating yet another method for generating a second observation video. Figure 8 The flowchart of a training video restoration model provided in an embodiment of this application is shown; Figure 9 This paper shows a schematic diagram of the structure of a robot strategy model training device provided in an embodiment of this application; Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0020] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0022] In existing technologies, vision-motion strategies based on imitation learning are the mainstream approach for robot manipulation. Egocentric perspective is widely used for teaching data acquisition because it aligns with robot perception. Existing data augmentation methods mainly fall into three categories: geometrically driven methods synthesize new perspective observations through point cloud rendering and 3D reconstruction, but only update the vision without adjusting the action; visual synthesis methods generate realistic observations using generative models, but lack corresponding action annotations; and motion retargeting methods adapt to action trajectories, but are only oriented towards a third-person perspective. Meanwhile, video generation models in the robotics field mostly rely on video decoding of actions, prediction followed by execution, or action-conditional video generation, none of which explicitly address the egocentric perspective shift problem caused by robot motion.

[0023] Specifically, existing teaching data collected from a single egocentric perspective makes the model highly sensitive to perspective shifts, and simply adding data from the same perspective cannot solve the generalization bottleneck; the separation between visual synthesis and action annotation from new perspectives leads to observation-action mismatch, interfering with policy learning; action retargeting does not consider the coupling characteristics between the egocentric perspective and the chassis mechanical characteristics, and cannot adapt to the action changes caused by perspective shifts; naive video synthesis methods have problems such as blurring and flickering, resulting in poor visual quality and temporal coherence; and the overall lack of a unified framework makes it impossible to generate effective teaching data with consistent visual and action pairings under new perspectives.

[0024] Based on this, this application proposes a robot policy model training method. First, it obtains an initial teaching dataset with an egocentric perspective and target new perspective parameters including planar translation and yaw rotation. Then, it redirects joint action sequences based on the new perspective parameters to obtain a second joint action sequence adapted to the new perspective. Next, it combines a first observation video, a depth map sequence, the new action sequence, and a pre-trained video repair model to generate a second observation video under the new perspective. Subsequently, it constructs a target teaching dataset using the observation video paired with the new perspective and the action sequence. Finally, it trains the model by mixing the initial and target teaching datasets. This application can automatically generate visual-action paired teaching data under the new perspective without the need for additional real new perspective data collection. This significantly improves the generalization ability of the policy model to egocentric perspective shifts, while ensuring that the success rate of standard and new perspective operations increases simultaneously, enhancing the robustness of robot deployment.

[0025] Next, combine Figure 1 This paper describes the robot strategy model training method of this application. The execution subject of this method can be a computer device or a robot controller with data processing and model training capabilities, such as... Figure 1 As shown, the method includes: S101. Obtain the initial teaching dataset from the robot's egocentric perspective and the new perspective parameters of the target new perspective. The initial teaching dataset includes multiple sets of initial data, each set of initial data including: the first observation video, the first joint action sequence, and the depth map sequence. The new perspective parameters include: plane translation parameters and yaw rotation parameters.

[0026] Here, egocentric viewpoint refers to the first-person observation perspective of the robot's built-in camera, fixed to the base or the robot's head, moving synchronously with the robot and directly aligned with the robot's perception. Initial teaching dataset refers to the raw teaching data collected under a single standard egocentric viewpoint, used for basic training of the policy model. New viewpoint parameters refer to the parameters describing the robot's chassis planar motion, including planar translation parameters (Δx, Δy) and yaw rotation parameters Δθ. Planar translation parameters refer to the displacement of the robot base along the x and y axes in the horizontal plane, while yaw rotation parameters refer to the rotation angle of the robot base around the vertical axis.

[0027] The first observation video refers to the RGB (Red, Green, Blue, three primary color model) visual video acquired by the robot from its initial egocentric perspective, used to record the task execution scene. The first joint motion sequence refers to the timing data of the joint angles of the robot as it completes the task from the initial perspective, used to control the movement of the robotic arm. The depth map sequence refers to the depth information aligned with the first observation video, used to characterize the spatial distance of objects in the scene.

[0028] Optionally, the robot can acquire initial observation videos using its camera, acquire initial joint motion sequences using its joint encoder, and acquire depth map sequences using its depth sensor. Planar translation, yaw, and rotation parameters for the new target perspective can be manually set or randomly sampled. Simultaneously, the acquired RGB video, joint motion data, and depth maps from the robot's egocentric perspective are paired by timestamp to form a single set of initial data, and multiple sets of data are aggregated into an initial teaching dataset.

[0029] The target new perspective can be a pre-set perspective, or multiple perspectives can be pre-set. For each perspective, observation videos and action sequences are generated separately, thereby obtaining teaching data for that perspective.

[0030] S102. Based on the new perspective parameters, redirect the first joint motion sequence of each group of initial data in the initial teaching dataset to obtain the second joint motion sequence of each group of target data under the new perspective.

[0031] The second joint motion sequence refers to the timing data of robot joint angles adapted to the new perspective of the target.

[0032] After an egocentric viewpoint shift, the original joint movements become insufficient to complete the task, requiring redirection to ensure movement effectiveness. Therefore, the first joint movement sequence can be redirected, transforming the initial viewpoint's joint movements into kinematically feasible joint movements from the target's new viewpoint.

[0033] In one possible implementation, the initial viewpoint end effector pose can be calculated using forward kinematics, the new viewpoint pose can be obtained through coordinate transformation, and then the second joint motion sequence can be generated by inverse kinematics.

[0034] S103. Based on the first observation video, depth map sequence, second joint motion sequence, new perspective parameters, and pre-trained video restoration model, generate the second observation video of each group of target data under the new perspective of the target.

[0035] The video restoration model can be a generative model fine-tuned based on a diffusion model, used to synthesize high-fidelity observation videos from new perspectives. The second observation video refers to an RGB observation video aligned with the second joint action sequence from the new perspective of the target.

[0036] Optionally, the initial vision, depth data, and new perspective actions can be used as conditions to first process and generate scene videos and robot videos. Then, the two can be used as dual-stream conditional inputs to the video inpainting model to iteratively denoise and generate a second observation video under the new perspective, ensuring vision-action alignment.

[0037] S104. Based on the second observation video, second joint action sequence, and new target perspective of each group of target data, construct a target teaching dataset.

[0038] The target teaching dataset refers to the synthetic teaching data consisting of a second observation video paired with a second joint motion sequence from a new perspective.

[0039] In one possible implementation, each set of second observation videos and second joint action sequences can be matched by timestamp, and the corresponding new target perspective can be labeled to obtain multiple sets of target data. The obtained sets of target data can then be combined to form a target teaching dataset.

[0040] S105. Train the initial policy model based on the target teaching dataset and the initial teaching dataset to obtain the target policy model.

[0041] The initial policy model refers to the untrained visual-motor policy basic model, while the target policy model refers to the final robot policy model with viewpoint generalization ability after training on mixed data.

[0042] In one possible implementation, the data from the target teaching dataset and the initial teaching dataset can be mixed proportionally, and the mixed dataset can be input into the initial policy model and trained using supervised learning until convergence to obtain the target policy model.

[0043] In this embodiment, initial teaching data and new perspective parameters are acquired, joint movements adapted to the new perspective are retargeted and generated, and then the corresponding observation video is synthesized using a video restoration model to construct a new perspective teaching dataset. Finally, the original data is mixed to train the strategy model. This method can generate consistent multi-view teaching data without the need for additional collection of real data from the new perspective. This allows the trained model to improve the success rate of operations under both standard egocentric perspectives and new offset perspectives, effectively solving the problem of insufficient generalization ability caused by single-view training. At the same time, it ensures the alignment and matching of vision and action, improving the robustness of model deployment.

[0044] The following is a further explanation of how the first joint motion sequences of each group of initial data in the initial teaching dataset are redirected based on the new perspective parameters to obtain the second joint motion sequences of each group of target data under the new perspective. Figure 2 As shown, step S102 above includes: S201. Determine the coordinate transformation matrix from the source base coordinate system to the target base coordinate system based on the new perspective parameters.

[0045] The source base coordinate system refers to the original coordinate system in which the robot base is located when the robot collects the initial teaching data. The target base coordinate system refers to the new coordinate system of the robot base after transformation, corresponding to the new perspective of the target.

[0046] The new perspective is generated by the translation and yaw rotation of the base plane. A transformation matrix is ​​used to uniformly quantify the perspective transformation relationship, providing a spatial transformation basis for motion redirection. In one possible implementation, a homogeneous coordinate transformation matrix can be constructed based on the plane translation and yaw rotation parameters in the new perspective parameters; alternatively, the coordinate transformation matrix can be constructed through matrix multiplication, Euler angle rotation matrix, etc.

[0047] S202. Based on the coordinate transformation matrix, the first joint motion sequence of each group of initial data in the initial teaching dataset is redirected to obtain the second joint motion sequence of each group of initial data under the new perspective of the target.

[0048] The coordinate transformation matrix can map the source view action to the new view coordinate system, ensuring that the action is executable under the new view and the task logic remains unchanged.

[0049] As one possible implementation, the end effector pose corresponding to the first joint motion sequence can be multiplied with the coordinate transformation matrix to complete the spatial pose transformation; then, a second joint motion sequence adapted to the new perspective can be generated based on the transformed pose.

[0050] This application determines the coordinate transformation matrix through new perspective parameters, and then uses the matrix to retarget the joint motion sequence. This accurately establishes the spatial mapping relationship between the source base and the target base coordinate systems, ensuring that the joint motion adapts to the new perspective. Furthermore, it can quickly convert the original motion into effective motion under the new perspective, avoiding motion failure caused by perspective shift. This provides reliable motion data for subsequent observation video generation and model training, improving the accuracy and feasibility of motion retargeting.

[0051] The following is a further explanation of how the first joint motion sequences of each group of initial data in the initial teaching dataset are redirected based on the coordinate transformation matrix to obtain the second joint motion sequences of each group of initial data from the new target perspective. Figure 3 As shown, the above step S202 includes: S301. Extract the target pose of the robot's end effector in the source base coordinate system from the first joint motion sequence.

[0052] Optionally, for each robotic arm ∈ { , }, at each time step First, the target pose of the end effector in the source base coordinate system is calculated using forward kinematics. For example, it can be calculated using the following formula (1): (1) in ∈ For the source joint angle, For positive kinematic mapping.

[0053] S302. Perform coordinate transformation on the target pose according to the coordinate transformation matrix to obtain the desired pose of the robot end effector in the target base coordinate system.

[0054] Optionally, the target pose can be... The desired pose is obtained by transforming the target base coordinate system defined by the new target perspective using the following equation (2). .

[0055] (2) in, The coordinate transformation matrix from the source base coordinate system to the target base coordinate system is given by... = (Δ , Δ ,Δ ) Sure.

[0056] S303. Based on the desired pose, the robot's joint angles are solved using an inverse kinematics algorithm to generate a second joint motion sequence from the new target perspective.

[0057] Optionally, the robot joint angles in the target base coordinate system can be obtained by inverse kinematics. For example, it can be calculated using the following formula (3): (3) in, This represents the inverse kinematics algorithm, which calculates the first joint motion sequence step by step to obtain the continuous joint angles, and then splices the continuous joint angles according to the time steps to form the second joint motion sequence.

[0058] Furthermore, when the robot is a dual-arm robot, the steps described above—based on the desired pose, solving the robot's joint angles using inverse kinematics algorithms, and generating the second joint motion sequence from the new target perspective—include: If the robot is a two-armed robot, then based on the desired pose, the first joint angle of the robot's first joint arm and the second joint angle of the robot's second joint arm are solved by inverse kinematics algorithm. By splicing the first joint angle and the second joint angle, the second joint motion sequence is obtained.

[0059] Optionally, a dual-arm robot refers to a robot system comprising a left robotic arm and a right robotic arm, denoted as L and R, respectively. The first joint angle and the second joint angle are the joint angles of the left and right arms in the target base coordinate system, respectively, denoted as [insert angles here]. , .

[0060] In one possible implementation, it can be done first by... The joint angles of the left arm under the new target perspective are obtained by solving the time-step problem, and then... The joint angles of the right arm under the new target perspective are calculated step by step over time. Then, the joint angles of the left and right arms are stitched together frame by frame over time to obtain the second joint motion sequence. The stitching can be performed using the following formula (4): (4) in, This indicates the sequence of movements of the second joint.

[0061] The following explains the steps for generating a second observation video of the robot target from a new perspective, based on the first observation video, depth map sequence, second joint motion sequence, new perspective parameters, and a pre-trained video restoration model. Figure 4 As shown, the above step S103 includes: S401. Generate scene videos based on the first observation video and depth map sequence of each group of initial data.

[0062] Optionally, scene video refers to a new perspective video of the target object, with the robot removed and only the environment retained. Since the robot's joint states differ in the new perspective, the robot can be removed from the original frames first to isolate the scene content. As one possible implementation, the robot and background can be separated first, then the scene can be reprojected onto the new perspective and the gaps filled to obtain a scene video with clean scene conditions.

[0063] S402. Generate robot video based on the second joint motion sequence and new perspective parameters.

[0064] Optionally, robot videos can provide precise geometric information about the robot's motion state from the new perspective of the target, serving as additional conditional signals for video inpainting. The robot's motion morphology from the new perspective can be obtained by rendering the new perspective parameters and the second joint motion sequence step-by-step over time.

[0065] S403. The scene video and robot video are used as dual-stream conditional inputs to the video restoration model, and the video restoration model synthesizes a second observation video from a new perspective of the target.

[0066] The video inpainting model can be a two-stream conditional generative video inpainting model. Scene video and robot video can be input into the model, which uses the scene video as the background and the robot video as the geometric constraint to fuse and inpaint a second observation video from a new perspective of the target.

[0067] The following is a further explanation of how scene videos are generated based on the first observation video and depth map sequences of each set of initial data, such as... Figure 5 As shown, the above S401 step includes: S501. Construct a pixel mapping relationship from the egocentric viewpoint to the target new viewpoint using the new viewpoint parameters.

[0068] Pixel mapping refers to the spatial correspondence between pixels in the source view and pixels in the new view, determined by depth and camera intrinsic and extrinsic parameters. The target's new view offset is caused by base motion; pixel-level reprojection mapping can be established using the new view parameters to achieve scene geometric transfer.

[0069] Optionally, the projection correspondence from the source pixel to the new viewpoint pixel can be established based on the depth map and the camera projection model, combined with the new viewpoint extrinsic transformation.

[0070] S502. Based on the pixel mapping relationship and depth map sequence, the source frame pixels in the first observation video are reprojected onto the target view plane to generate the initial scene frame.

[0071] Wherein, the source frame pixel refers to the original image pixels of the first observed video, denoted as... A depth map sequence refers to depth information aligned with an image, denoted as... The initial scene frame refers to the reprojected image of the scene from a new perspective, containing holes, and is denoted as... .

[0072] For each pixel, it can be back-projected to a 3D point based on the depth information, and then projected onto a new image plane using the new viewpoint parameters to obtain the initial scene frame.

[0073] S503. Perform image restoration processing on the occluded and hole areas in the initial scene frame to obtain the scene video.

[0074] It should be understood that reprojection will produce holes, so the holes need to be filled first in order to obtain a complete scene video from a new perspective of the target without holes, thus providing complete conditions for the subsequent video restoration model.

[0075] In one possible implementation, the robot can be removed from the original frame first to isolate the scene content. The robot segmentation mask is then rendered for each frame. The pure scene image is obtained by the following formula (5): (5) scene-depth pairs , The image is reprojected onto the target's new viewpoint, generating an initial scene frame that includes the occluded holes. Then, the missing areas are filled to obtain a complete scene frame. Finally, all frames are aggregated to form a scene video. .

[0076] This application embodiment generates a complete and geometrically accurate new perspective scene video through pixel mapping, depth reprojection, and hole filling. It can effectively eliminate reprojection holes and artifacts, providing high-quality, missing scene conditions for subsequent dual-stream video restoration and ensuring the realism and coherence of the synthesized observation video.

[0077] like Figure 6 The image shown is a structural diagram of a video restoration model. (Refer to...) Figure 6 The video restoration model includes: a scene video encoder, a robot video encoder, a temporal denoising network, and a decoding module.

[0078] The above steps involve using scene video and robot video as dual-stream conditional inputs to a video inpainting model, and then using the video inpainting model to synthesize a second observation video from a new perspective of the target. Figure 7 As shown, it includes: S701. Input the scene video into the scene video encoder for encoding processing to obtain the scene feature map, and input the robot video into the robot video encoder to obtain the robot feature map.

[0079] The scene video encoder encodes the scene video, extracting background features to obtain a scene feature map. The robot video encoder encodes the robot video, extracting the robot's robotic arm features to obtain a robot feature map. After encoding the two video streams separately, they can be stitched together along the channel dimension to form a dual-stream conditional feature map.

[0080] S702. Input the scene feature map and robot feature map into the temporal denoising network, and also input the new viewpoint parameters as conditional vectors into the temporal denoising network. The temporal denoising network performs denoising processing to generate the initial observation video, and then inputs the initial observation video into the decoder.

[0081] The temporal denoising network incorporates self-attention and cross-attention mechanisms. It can use the concatenated features of the scene feature map and the robot feature map as conditions for progressive denoising. At the same time, it uses new perspective parameters as conditions to simultaneously repair the background and fuse the robot, obtaining a denoised latent space video vector as the initial observation video.

[0082] S703. The decoder decodes the initial observation video to obtain the second observation video.

[0083] Optionally, the decoder can decode the initial observation video to restore it to RGB video, thus obtaining the second observation video.

[0084] The process of video restoration using the video restoration model can be expressed as follows (6):

[0085] in, Represents a scene video, This refers to robot videos. This indicates the second observation video. This represents a video restoration model.

[0086] Continue to refer to Figure 6 The temporal denoising network consists of multiple DiT blocks, each of which includes self-attention and cross-attention mechanisms.

[0087] The DiT block (Diffusion Transformer module) is the fundamental unit of the temporal denoising network, used for temporal modeling and conditional denoising of the video latent space. Self-attention mechanism refers to the attention mechanism within the DiT block that models the temporal and spatial correlation of the video's own features. Cross-attention mechanism refers to the attention mechanism within the DiT block that injects external conditional features into the denoising process.

[0088] The process of generating the initial observation video by the aforementioned temporal denoising network includes: By employing the self-attention and cross-attention mechanisms of each DiT block, background artifacts in the scene video are simultaneously repaired during the denoising process, and the robot video is merged into the background. The output of the last DiT block is then used as the initial observation video.

[0089] Background artifacts refer to geometric distortions such as blurring, holes, and misalignment caused by reprojection. Each layer of DiT blocks can model the temporal dependencies between video frames, eliminate flicker and jitter, ensure temporal continuity, and use scene features and robot features as conditional inputs to simultaneously denoise and repair background artifacts. The output of the last layer serves as the initial observation video.

[0090] This application embodiment utilizes the collaborative work of self-attention and cross-attention of multi-layer DiT blocks to simultaneously repair background artifacts and naturally integrate robots during the denoising process. This results in flicker-free, misaligned, visually realistic, and temporally coherent generated videos, significantly improving the quality of new perspective observation videos and providing high-quality visual signals for policy model training.

[0091] The training process of the video restoration model will be explained next. The video restoration model can be trained using a self-supervised fine-tuning strategy, relying solely on teaching data from the robot's egocentric perspective, without requiring data from a new target perspective. For example... Figure 8 As shown, the training process of the video restoration model includes: S801. Construct training sample pairs, which include simulated scene videos with artifacts and corresponding real source videos. The simulated scene videos with artifacts are generated by reprojecting and filling holes in randomly sampled source videos to simulate viewpoint transformations.

[0092] For each frame of data in the real source video ( , This paper simulates geometric distortion and occlusion artifacts in novel perspective synthesis through a double reprojection process. First, the source frame is reprojected onto a randomly sampled novel perspective. ( ), to obtain a new perspective frame ( The new viewpoint frame is then reverse-projected back to the source viewpoint to obtain a simulated scene video containing artifacts. This addresses the challenge of simulating incomplete scene information in new perspective synthesis.

[0093] S802. Based on the training samples, the training diffusion model learns the mapping distribution from the simulated scene video containing artifacts to the real source video, thus obtaining the video restoration model.

[0094] Optionally, it can be used for scene videos containing artifacts. Hole filling was performed to eliminate the influence of the robot's shape black hole, and scene condition video was obtained. Model learning with Heyuan Robot Rendering Video As a condition, reconstruct the original source video. Following the standard diffusion training paradigm, minimizing the denoising objective function can be expressed as equation (7): (7) in For diffusion time step, (0, () represents Gaussian noise. This is the video after noise has been added. , For noise dispatch coefficient, This is a noise prediction network.

[0095] Based on the same inventive concept, this application also provides a robot strategy model training device corresponding to the robot strategy model training method. Since the principle of the device in this application is similar to the robot strategy model training method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0096] Reference Figure 9 The diagram shown is a schematic of a robot strategy model training device provided in an embodiment of this application. The device includes: an acquisition module 901, a redirection module 902, a generation module 903, a construction module 904, and a training module 905.

[0097] The acquisition module 901 is used to acquire the initial teaching dataset under the robot's egocentric perspective and the new perspective parameters of the target new perspective. The initial teaching dataset includes multiple sets of initial data, each set of initial data including: the first observation video, the first joint action sequence, and the depth map sequence. The new perspective parameters include: planar translation parameters and yaw rotation parameters. The redirection module 902 is used to redirect the first joint motion sequence of each group of initial data in the initial teaching dataset according to the new perspective parameters, so as to obtain the second joint motion sequence of each group of target data under the new perspective of the target. The generation module 903 is used to generate a second observation video of each set of target data under the new perspective of the target based on the first observation video, the depth map sequence, the second joint action sequence, the new perspective parameters, and the pre-trained video restoration model. Module 904 is used to construct a target teaching dataset based on the second observation video, the second joint action sequence, and the new target perspective of each set of target data under the new target perspective. The training module 905 is used to train the initial policy model based on the target teaching dataset and the initial teaching dataset to obtain the target policy model.

[0098] Optionally, the redirection module 902 is specifically used for: Determine the coordinate transformation matrix from the source base coordinate system to the target base coordinate system based on the new perspective parameters; Based on the coordinate transformation matrix, the first joint motion sequences of each group of initial data in the initial teaching dataset are redirected to obtain the second joint motion sequences of each group of initial data under the new target perspective.

[0099] Optionally, the redirection module 902 is specifically used for: Extract the target pose of the robot's end effector in the source base coordinate system from the first joint motion sequence; The target pose is transformed according to the coordinate transformation matrix to obtain the desired pose of the robot end effector in the target base coordinate system. Based on the desired pose, the robot's joint angles are solved using an inverse kinematics algorithm to generate a second joint motion sequence from a new perspective of the target.

[0100] Optionally, the redirection module 902 is specifically used for: If the robot is a two-armed robot, then based on the desired pose, the first joint angle of the robot's first joint arm and the second joint angle of the robot's second joint arm are solved by inverse kinematics algorithm. By splicing the first joint angle and the second joint angle, the second joint motion sequence is obtained.

[0101] Optionally, the generation module 903 is specifically used for: Based on the first observation video and depth map sequence of each set of initial data, a scene video is generated; Generate robot video based on the second joint motion sequence and new perspective parameters; The scene video and robot video are used as dual-stream conditional inputs to the video restoration model, which then synthesizes a second observation video from a new perspective of the target.

[0102] Optionally, the generation module 903 is specifically used for: Construct a pixel mapping relationship from the egocentric viewpoint to the target new viewpoint using new perspective parameters; Based on pixel mapping relationships and depth map sequences, the source frame pixels in the first observation video are reprojected onto the target view plane to generate the initial scene frame; Image inpainting is performed on the occluded and hole areas in the initial scene frame to obtain the scene video.

[0103] Optionally, the video restoration model includes: a scene video encoder, a robot video encoder, a temporal denoising network, and a decoding module; Module 903 is specifically used for: The scene video is input into the scene video encoder for encoding processing to obtain the scene feature map, and the robot video is input into the robot video encoder to obtain the robot feature map; The scene feature map and robot feature map are input into the temporal denoising network, and the new viewpoint parameters are also input into the temporal denoising network as conditional vectors. The temporal denoising network performs denoising processing to generate the initial observation video, and the initial observation video is input into the decoder. The decoder decodes the initial observation video to obtain the second observation video.

[0104] Optionally, the temporal denoising network includes: multiple DiT blocks, each of which includes self-attention and cross-attention mechanisms; Module 903 is specifically used for: By employing the self-attention and cross-attention mechanisms of each DiT block, background artifacts in the scene video are simultaneously repaired during the denoising process, and the robot video is merged into the background. The output of the last DiT block is then used as the initial observation video.

[0105] Optionally, training module 905 is specifically used for: Construct training sample pairs, which include simulated scene videos with artifacts and corresponding real source videos. The simulated scene videos with artifacts are generated by reprojecting and filling holes in randomly sampled source videos to simulate viewpoint transformations. The training diffusion model learns the mapping distribution from simulated scene videos containing artifacts to real source videos based on the training samples, thus obtaining the video restoration model.

[0106] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0107] This application's embodiments acquire initial teaching data and new perspective parameters, redirect and generate joint movements adapted to the new perspective, then use a video restoration model to synthesize corresponding observation videos to construct a new perspective teaching dataset, and finally mix the original data to train the strategy model. This method generates consistent multi-view teaching data without requiring additional collection of real-world new perspective data. This allows the trained model to improve operational success rates under both standard egocentric and new offset perspectives, effectively solving the problem of insufficient generalization ability caused by single-view training, while ensuring alignment between vision and action, thus improving model deployment robustness.

[0108] This application also provides an electronic device, such as... Figure 10 The diagram shown is a schematic representation of an electronic device structure provided in an embodiment of this application, including a processor 101, a memory 102, and a bus. The memory 102 stores machine-readable instructions executable by the processor 101. When the computer device is running, the processor 101 and the memory 102 communicate via the bus. When the machine-readable instructions are executed by the processor 101, the aforementioned robot strategy model training method is performed.

[0109] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the robot strategy model training method described above.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0111] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0112] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for training a robot policy model, characterized in that, include: The robot acquires an initial teaching dataset from its egocentric perspective and new perspective parameters from the target new perspective. The initial teaching dataset includes multiple sets of initial data, each set of initial data including: a first observation video, a first joint motion sequence, and a depth map sequence. The new perspective parameters include: planar translation parameters and yaw rotation parameters. Based on the new perspective parameters, the first joint motion sequence of each group of initial data in the initial teaching dataset is redirected to obtain the second joint motion sequence of each group of target data under the new target perspective. Based on the first observation video, the depth map sequence, the second joint motion sequence, the new perspective parameters, and the pre-trained video restoration model, generate a second observation video of each set of target data under the new perspective of the target; Based on the second observation video, the second joint motion sequence, and the new target perspective, a target teaching dataset is constructed. The initial policy model is trained using the target teaching dataset and the initial teaching dataset to obtain the target policy model.

2. The method according to claim 1, characterized in that, The step of redirecting the first joint motion sequences of each group of initial data in the initial teaching dataset according to the new perspective parameters to obtain the second joint motion sequences of each group of target data under the new target perspective includes: Determine the coordinate transformation matrix from the source base coordinate system to the target base coordinate system based on the new perspective parameters; The first joint motion sequence of each group of initial data in the initial teaching dataset is redirected according to the coordinate transformation matrix to obtain the second joint motion sequence of each group of initial data under the new target perspective.

3. The method according to claim 2, characterized in that, The step of redirecting the first joint motion sequences of each group of initial data in the initial teaching dataset according to the coordinate transformation matrix to obtain the second joint motion sequences of each group of initial data under the new target perspective includes: Extract the target pose of the robot's end effector in the source base coordinate system from the first joint motion sequence; The target pose is transformed according to the coordinate transformation matrix to obtain the desired pose of the robot end effector in the target base coordinate system. Based on the desired pose, the joint angles of the robot are solved using an inverse kinematics algorithm to generate a second joint motion sequence under the new target perspective.

4. The method according to claim 3, characterized in that, The step of solving the robot's joint angles using an inverse kinematics algorithm based on the desired pose to generate a second joint motion sequence under the new target perspective includes: If the robot is a two-armed robot, then based on the desired pose, the first joint angle of the first joint arm and the second joint angle of the second joint arm of the robot are solved by inverse kinematics algorithm. By splicing the first joint angle and the second joint angle, the second joint action sequence is obtained.

5. The method according to claim 1, characterized in that, The step of generating a second observation video of the robot target from a new perspective based on the first observation video, the depth map sequence, the second joint motion sequence, the new viewpoint parameters, and a pre-trained video restoration model includes: Based on the first observation video and the depth map sequence of each set of initial data, a scene video is generated; Generate robot video based on the second joint motion sequence and the new perspective parameters; The scene video and the robot video are input as dual-stream conditions into the video restoration model, and the video restoration model synthesizes a second observation video from the new perspective of the target.

6. The method according to claim 5, characterized in that, The step of generating scene videos based on the first observation video and the depth map sequence of each set of initial data includes: The pixel mapping relationship from the egocentric viewpoint to the target new viewpoint is constructed using the new viewpoint parameters; Based on the pixel mapping relationship and the depth map sequence, the source frame pixels in the first observed video are reprojected onto the target view plane to generate an initial scene frame; The occluded and hollow areas in the initial scene frame are incised to obtain the scene video.

7. The method according to claim 5, characterized in that, The video restoration model includes: a scene video encoder, a robot video encoder, a temporal denoising network, and a decoding module; The scene video and the robot video are input as dual-stream conditions into the video restoration model, which then synthesizes a second observation video from the new perspective of the target, including: The scene video is input into the scene video encoder for encoding processing to obtain a scene feature map, and the robot video is input into the robot video encoder to obtain a robot feature map; The scene feature map and the robot feature map are input into the temporal denoising network, and the new viewpoint parameters are also input into the temporal denoising network as conditional vectors. The temporal denoising network performs denoising processing to generate an initial observation video, and the initial observation video is input into the robot video decoder. The robot video decoder decodes the initial observation video to obtain the second observation video.

8. The method according to claim 7, characterized in that, The temporal denoising network includes: multiple DiT blocks, each of which includes self-attention and cross-attention mechanisms; The process by which the temporal denoising network performs denoising processing to generate the initial observation video includes: Through the self-attention and cross-attention mechanisms of each DiT block, the background artifacts of the scene video are repaired simultaneously during the denoising process, and the robot video is merged into the background. The output of the last DiT block is used as the initial observation video.

9. The method according to claim 1, characterized in that, The training process of the video restoration model includes: Construct training sample pairs, which include simulated scene videos with artifacts and corresponding real source videos. The simulated scene videos with artifacts are generated by reprojecting and filling holes in randomly sampled source videos with simulated viewpoint transformations. The training diffusion model learns the mapping distribution from simulated scene videos containing artifacts to real source videos based on the training samples, thus obtaining the video restoration model.

10. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the robot policy model training method as described in any one of claims 1-9.

Images