Hand reinforcement learning training data generation method and device, medium and electronic equipment
By generating hand reinforcement learning training data from hand movement videos, the problem of high hardware costs was solved, enabling low-cost, large-scale data collection and improving the generalization ability of hand reinforcement learning.
Patent Information
- Application Number
- CN202511116255.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-08
- Publication Date
- 2025-12-16
AI Technical Summary
In existing technologies, the hardware cost for collecting hand reinforcement learning training data is high, making it difficult to collect diverse data on a large scale, resulting in poor generalization ability.
By acquiring hand movement videos, image frames are split, and each frame is processed using a pre-defined hand pose estimation neural network to generate a hand mesh file, determine key point location information and wrist rotation matrix, and generate target data files for hand reinforcement learning training.
It eliminates the need for expensive optical labeling systems and inertial sensors, reducing hardware costs and making it suitable for large-scale reinforcement learning training scenarios, thus improving the generalization ability of hand reinforcement learning.
Smart Images
Figure CN121148005A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of computer vision, and particularly relates to a hand reinforcement learning training data generation method and device, a computer readable storage medium and an electronic device. BACKGROUND
[0002] In reinforcement learning (RL) training, obtaining accurate human hand operation trajectory data is crucial for training agents to complete fine operation tasks (such as grasping, assembly, gesture control, etc.). In the prior art, the acquisition of hand reinforcement learning training data is mainly performed by optical marking systems (such as Vicon, OptiTrack) and inertial sensors (such as data gloves, etc.), which has a high hardware cost and is difficult to collect large-scale diversified data, thereby restricting the generalization ability of hand reinforcement learning. SUMMARY
[0003] Therefore, the embodiments of the present application provide a hand reinforcement learning training data generation method and device, a computer readable storage medium and an electronic device to solve the problem of high data acquisition hardware cost and poor generalization ability of hand reinforcement learning in the prior art.
[0004] The first aspect of the embodiments of the present application provides a hand reinforcement learning training data generation method, which can include:
[0005] Obtaining a hand movement video and performing image frame splitting on the hand movement video to obtain split frames of images;
[0006] Using a preset hand pose estimation neural network to process each frame of image to obtain a hand mesh file of each frame of image;
[0007] According to the hand mesh file of each frame of image, determining the hand key point position information and the wrist rotation matrix of each frame of image, respectively;
[0008] According to the hand key point position information and the wrist rotation matrix of each frame of image, generating a target data file for hand reinforcement learning training.
[0009] In a specific implementation manner of the first aspect, the determination of the hand key point position information and the wrist rotation matrix of each frame of image according to the hand mesh file of each frame of image can include:
[0010] For a target frame of image, extracting hand key point position information in a camera coordinate system from the hand mesh file; wherein the target frame of image is any one of the frames of images;
[0011] The hand key point position information in the camera coordinate system is converted to obtain hand key point position information in an image coordinate system.
[0012] In an implementation form of the first aspect, the conversion of the hand key point position information in the camera coordinate system to obtain hand key point position information in an image coordinate system can include:
[0013] A camera translation vector corresponding to the target frame image is determined.
[0014] The camera translation vector is superimposed on the hand key point position information in the camera coordinate system to obtain hand key point position information in an image coordinate system.
[0015] In an implementation form of the first aspect, the hand reinforcement learning training data generation method can further include:
[0016] In the case that the hand key point position information of the left hand exists, the hand key point position information of the left hand is mirrored to obtain mirrored hand key point position information.
[0017] In an implementation form of the first aspect, after the target data file for hand reinforcement learning training is generated according to the hand key point position information and the wrist rotation matrix of each frame image, the method can further include:
[0018] The robot is trained for hand reinforcement learning using the target data file to obtain a hand movement strategy of the robot.
[0019] The robot is controlled to perform hand movement based on the hand movement strategy.
[0020] In an implementation form of the first aspect, after the hand mesh file of each frame image is obtained by processing each frame image using a preset hand pose estimation neural network, the method can further include:
[0021] Each frame hand mesh image is generated according to the hand mesh file of each frame image.
[0022] The hand mesh video is obtained by video synthesis of each frame hand mesh image.
[0023] In an implementation form of the first aspect, after the target data file for hand reinforcement learning training is generated according to the hand key point position information and the wrist rotation matrix of each frame image, the method can further include:
[0024] The hand key point position information of each frame image is extracted from the target data file.
[0025] The location information of key hand points in each frame of the image is visualized to obtain the visualization results.
[0026] A second aspect of this application provides a hand reinforcement learning training data generation apparatus, which may include:
[0027] The video segmentation module is used to acquire hand movement video and segment the hand movement video into image frames to obtain the segmented frame images;
[0028] The network processing module is used to process each frame of the image using a preset hand pose estimation neural network to obtain the hand mesh file of each frame of the image;
[0029] The key point determination module is used to determine the position information of the key points of the hand and the wrist rotation matrix of each frame image based on the hand mesh file of each frame image;
[0030] The data file generation module is used to generate target data files for hand reinforcement learning training based on the hand key point position information and wrist rotation matrix of each frame image.
[0031] In one specific implementation of the second aspect, the key point determination module may include:
[0032] The key point location information extraction unit is used to extract the hand key point location information in the camera coordinate system from the hand mesh file for the target frame image; wherein, the target frame image is any one of the frame images;
[0033] The coordinate system transformation processing unit is used to perform coordinate system transformation processing on the hand key point position information in the camera coordinate system to obtain the hand key point position information in the image coordinate system.
[0034] In one specific implementation of the second aspect, the coordinate system transformation processing unit may be specifically used to: determine the camera translation vector corresponding to the target frame image; and superimpose the camera translation vector onto the hand key point position information in the camera coordinate system to obtain the hand key point position information in the image coordinate system.
[0035] In one specific implementation of the second aspect, the key point determination module may further include:
[0036] The mirror processing unit is used to mirror the key point position information of the left hand when the key point position information of the left hand exists, so as to obtain the mirrored key point position information.
[0037] In one specific implementation of the second aspect, the hand reinforcement learning training data generation method may further include:
[0038] The hand motion control module is used to perform hand reinforcement learning training on the robot using the target data file to obtain the robot's hand motion strategy; and to control the robot to perform hand movements based on the hand motion strategy.
[0039] In one specific implementation of the second aspect, the hand reinforcement learning training data generation method may further include:
[0040] The video compositing module is used to generate individual hand grid images based on the hand grid files of each frame; and to perform video compositing on the individual hand grid images to obtain a hand grid video.
[0041] In one specific implementation of the second aspect, the hand reinforcement learning training data generation method may further include:
[0042] The visualization processing module is used to extract the hand key point position information of each frame image from the target data file; and to perform visualization processing on the hand key point position information of each frame image to obtain the visualization processing result.
[0043] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described hand reinforcement learning training data generation methods.
[0044] A fourth aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described hand reinforcement learning training data generation methods.
[0045] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the steps of any of the above-described hand reinforcement learning training data generation methods.
[0046] The beneficial effects of this application embodiment compared with the prior art are as follows: This application embodiment acquires hand movement video and performs image frame segmentation on the hand movement video to obtain segmented frame images; a preset hand pose estimation neural network is used to process each frame image to obtain a hand mesh file for each frame image; based on the hand mesh file of each frame image, the hand key point position information and wrist rotation matrix of each frame image are determined; based on the hand key point position information and wrist rotation matrix of each frame image, a target data file for hand reinforcement learning training is generated. Through this application embodiment, a target data file for hand reinforcement learning training can be directly generated based on hand movement video, without relying on expensive optical labeling systems and inertial sensors, greatly reducing hardware costs, and is applicable to large-scale reinforcement learning training scenarios, which helps to improve the generalization ability of hand reinforcement learning. Attached Figure Description
[0047] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0048] Figure 1 This is a flowchart of one embodiment of a hand reinforcement learning training data generation method in this application.
[0049] Figure 2 A diagram illustrating key points of the hand;
[0050] Figure 3 A schematic diagram of the specific contents of the target data file;
[0051] Figure 4 A schematic diagram illustrating the visualization results of a single frame of data showing outstretched hands.
[0052] Figure 5 A schematic diagram illustrating the visualization results of a single frame of data showing two hands clenched into fists.
[0053] Figure 6 This is a structural diagram of one embodiment of a hand reinforcement learning training data generation device according to the present application.
[0054] Figure 7 This is a schematic block diagram of an electronic device according to an embodiment of this application. Detailed Implementation
[0055] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0056] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0057] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0058] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0059] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."
[0060] Furthermore, in the description of this application, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0061] In reinforcement learning (RL) training, acquiring accurate human hand movement trajectory data is crucial for training agents to perform fine motor tasks (such as grasping, assembly, and gesture control). Current technologies primarily rely on optical labeling systems (such as Vicon and OptiTrack) and inertial sensors (such as data gloves) to collect hand reinforcement learning training data. However, this approach is costly and makes it difficult to collect diverse data on a large scale, thus limiting the generalization ability of hand reinforcement learning.
[0062] In view of this, embodiments of this application provide a method, apparatus, computer-readable storage medium, and electronic device for generating training data for hand reinforcement learning, in order to solve the problems of high data acquisition hardware costs and poor generalization ability of hand reinforcement learning in the prior art.
[0063] In this embodiment, target data files for hand reinforcement learning training can be directly generated based on hand motion videos, without the need for expensive optical labeling systems and inertial sensors, greatly reducing hardware costs. This makes it suitable for large-scale reinforcement learning training scenarios and helps improve the generalization ability of hand reinforcement learning.
[0064] Please see Figure 1 One embodiment of a hand reinforcement learning training data generation method in this application may include:
[0065] Step S101: Acquire hand movement video and perform image frame segmentation on the hand movement video to obtain the segmented frame images.
[0066] The specific frame rate (Frames Per Second, FPS) during the image frame splitting process can be flexibly set according to the actual situation, and this application embodiment does not impose a specific limitation on it. As an example, the frame rate can be set to 30, that is, the video per second is split into 30 frames of images.
[0067] In one specific implementation of this application, the extract_video_frames(video_path,frames_folder) function can be used to split the hand motion video into image frames, where video_path is the storage path of the hand motion video and frames_folder is the folder where the split frame images are stored.
[0068] Step S102: Use a preset hand pose estimation neural network to process each frame of the image to obtain the hand mesh file of each frame of the image.
[0069] The specific hand pose estimation neural network used can be flexibly set according to the actual situation, including but not limited to the Hand Mesh Recovery (HaMeR) network, etc., and this application embodiment does not make specific limitations on this.
[0070] For ease of explanation, the HaMeR network will be used as an example in the following descriptions. The HaMeR network abandons the traditional Convolutional Neural Network (CNN) and adopts a pure Transformer architecture to process image input. This allows it to better capture long-range dependencies between hand joints and improves its ability to model complex hand poses (such as crossed fingers or occlusion). The input to the HaMeR network is a single-frame image, and the output is a hand mesh file based on the Metric-Affine Hand Model (MANO) for that frame. This mesh may include, but is not limited to, the 3D position information of 778 vertices and the wrist rotation matrix.
[0071] Step S103: Based on the hand mesh file of each frame image, determine the hand key point position information and wrist rotation matrix of each frame image respectively.
[0072] Taking any one frame from the various frames as an example, we denote it as the target frame image. For the target frame image, we can extract the hand keypoint position information (pred_keypoints_3d) in the camera coordinate system from its hand mesh file. Figure 2The diagram shows key points of the hand. As shown, the hand can include 21 key points, numbered from 0 to 20: wrist, proximal thumb, middle thumb, distal thumb, thumb tip, proximal index finger, middle index finger, distal index finger, index finger tip, and proximal middle finger. The four fingers are: ximal (middle_intermediate), middle_distal (middle_distal), middle_tip (middle_tip), ring_proximal (ring_proximal), ring_intermediate (ring_intermediate), ring_distal (ring_distal), ring_tip (ring_tip), pinky_proximal (pinky_proximal), pinky_intermediate (pinky_intermediate), pinky_distal (pinky_distal), and pinky_tip (pinky_tip).
[0073] In one specific implementation of this application, the HaMeR network output can be obtained using the function out = model(batch), and the three-dimensional position information of 21 hand keypoints can be extracted into the keypoints_3d variable using the function keypoints_3d = out['pred_keypoints_3d'][n].detach().cpu().numpy().
[0074] Since the extracted hand keypoint location information is in the camera coordinate system and does not include the camera's position information, in the embodiments of the application, the hand keypoint location information in the camera coordinate system can be transformed to obtain the hand keypoint location information in the image coordinate system. Specifically, the camera translation vector (pred_cam_t_full[n]) corresponding to the target frame image can be determined, and the camera translation vector can be superimposed on the hand keypoint location information in the camera coordinate system, that is: keypoints_3d[:,:3]+=pred_cam_t_full[n][:3], thereby obtaining the hand keypoint location information in the image coordinate system.
[0075] The HaMeR network processes the left hand first by mirroring a right-hand model and recording the point positions, then processes the right hand. The left and right hands can be labeled using `batch['right'][n].item()`, where `right=0` represents the left hand and `right=1` represents the right hand. Given the left-hand keypoint position information, this information can be mirrored about the horizontal coordinate using `keypoints_3d[:,0]*=-1`, resulting in the mirrored keypoint positions.
[0076] Step S104: Generate a target data file for hand reinforcement learning training based on the hand key point position information and wrist rotation matrix of each frame image.
[0077] In this embodiment of the application, the hand key point position information and wrist rotation matrix of each frame image can be superimposed and input into the same target data file (pkl file), thereby recording the complete hand operation trajectory of the entire video for use in hand reinforcement learning training.
[0078] Figure 3 The diagram illustrates the specific contents of the target data file. For each frame, information may include, but is not limited to, frame_id, img_fn, right, cam_t, verts, pos, and wrist_rot. Here, frame_id is the frame identifier; img_fn is the image sequence number, with two frames representing the same image. For example, frame_id = 00 and frame_id = 01 represent image img_fn = 00000, i.e., the first image processed; right indicates the left or right hand, with a value of 0 representing the left hand and a value of 1 representing the right hand; cam_t is the 3D camera translation vector; verts contains the 3D position information of 778 vertices in the mesh file; pos contains the 3D position information of 21 hand keypoints; and wrist_rot is the wrist rotation matrix. In MANO or similar hand pose estimation models, batch_size represents the number of samples processed simultaneously. If batch_size = 2, it means the model processes data from both hands or two frames of the same hand simultaneously, thus outputting two wrist rotation matrices.
[0079] In one specific implementation of this application, after generating the target data file, the robot can be trained using hand reinforcement learning to obtain the robot's hand movement strategy. Based on the hand movement strategy, the robot can be controlled to perform hand movements to complete fine operation tasks such as grasping and assembly.
[0080] In one specific implementation of this application, after obtaining the hand mesh files for each frame of images, individual frame hand mesh images can be generated based on these files, and then video composites can be performed on these hand mesh images to obtain a hand mesh video, allowing users to quickly browse and verify it. For example, the function images_to_video(image_folder,output_path,fps) can be used to composite the hand mesh images, where image_folder is the folder containing the hand mesh images, output_path is the storage path for the hand mesh video, and fps is the frame rate of the hand mesh video.
[0081] In one specific implementation of this application, to intuitively analyze whether the data is correct, after generating the target data file, the hand key point position information of each frame image can be extracted from the target data file, and the hand key point position information of each frame image can be visualized to obtain the corresponding visualization results. The specific visualization tool used can be flexibly set according to the actual situation, including but not limited to Matplotlib, mplot3d, etc., and this application embodiment does not specifically limit this.
[0082] Figure 4 The image shown is a visualization of the processing result of a frame of data with outstretched hands. Figure 5 The image shown is a visualization of the processing results of a single frame of data with both hands clenched into fists. The top image displays the key hand points extracted from the target data file, while the bottom image is the mesh file obtained after processing by the HaMeR network. As can be seen, this embodiment successfully extracted the trajectory information of hand movements from the video and converted it into a universal format file that can be used for reinforcement learning training.
[0083] In summary, this application embodiment acquires hand movement video and performs image frame segmentation on the hand movement video to obtain segmented frame images; it processes each frame image using a preset hand pose estimation neural network to obtain a hand mesh file for each frame image; based on the hand mesh file of each frame image, it determines the hand keypoint position information and wrist rotation matrix of each frame image; and based on the hand keypoint position information and wrist rotation matrix of each frame image, it generates a target data file for hand reinforcement learning training. Through this application embodiment, a target data file for hand reinforcement learning training can be directly generated based on hand movement video, without relying on expensive optical labeling systems and inertial sensors, greatly reducing hardware costs. It is applicable to large-scale reinforcement learning training scenarios and helps improve the generalization ability of hand reinforcement learning.
[0084] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0085] Corresponding to the hand reinforcement learning training data generation method described in the above embodiments, Figure 6 This illustration shows a structural diagram of an embodiment of a hand reinforcement learning training data generation device provided in this application.
[0086] In this embodiment, a hand reinforcement learning training data generation device may include:
[0087] The video splitting module 601 is used to acquire a hand movement video and split the hand movement video into image frames to obtain the split frame images.
[0088] The network processing module 602 is used to process each frame of the image using a preset hand pose estimation neural network to obtain the hand mesh file of each frame of the image.
[0089] The key point determination module 603 is used to determine the hand key point position information and wrist rotation matrix of each frame image based on the hand mesh file of each frame image;
[0090] The data file generation module 604 is used to generate target data files for hand reinforcement learning training based on the hand key point position information and wrist rotation matrix of each frame image.
[0091] In one specific implementation of this application embodiment, the key point determination module may include:
[0092] The key point location information extraction unit is used to extract the hand key point location information in the camera coordinate system from the hand mesh file for the target frame image; wherein, the target frame image is any one of the frame images;
[0093] The coordinate system transformation processing unit is used to perform coordinate system transformation processing on the hand key point position information in the camera coordinate system to obtain the hand key point position information in the image coordinate system.
[0094] In one specific implementation of this application, the coordinate system transformation processing unit can be specifically used to: determine the camera translation vector corresponding to the target frame image; and superimpose the camera translation vector onto the hand key point position information in the camera coordinate system to obtain the hand key point position information in the image coordinate system.
[0095] In one specific implementation of this application embodiment, the key point determination module may further include:
[0096] The mirror processing unit is used to mirror the key point position information of the left hand when the key point position information of the left hand exists, so as to obtain the mirrored key point position information.
[0097] In one specific implementation of this application embodiment, the hand reinforcement learning training data generation method may further include:
[0098] The hand motion control module is used to perform hand reinforcement learning training on the robot using the target data file to obtain the robot's hand motion strategy; and to control the robot to perform hand movements based on the hand motion strategy.
[0099] In one specific implementation of this application embodiment, the hand reinforcement learning training data generation method may further include:
[0100] The video compositing module is used to generate individual hand grid images based on the hand grid files of each frame; and to perform video compositing on the individual hand grid images to obtain a hand grid video.
[0101] In one specific implementation of this application embodiment, the hand reinforcement learning training data generation method may further include:
[0102] The visualization processing module is used to extract the hand key point position information of each frame image from the target data file; and to perform visualization processing on the hand key point position information of each frame image to obtain the visualization processing result.
[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0104] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0105] Figure 7 A schematic block diagram of an electronic device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.
[0106] like Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, it implements the steps in the various hand reinforcement learning training data generation method embodiments described above, for example...Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of modules 601 to 604 are shown.
[0107] For example, the computer program 72 may be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 72 in the electronic device 7.
[0108] The electronic device 7 may include, but is not limited to, computing devices such as mobile phones, tablets, desktop computers, laptops, handheld computers, robots, and servers. Those skilled in the art will understand that... Figure 7 This is merely an example of electronic device 7 and does not constitute a limitation on electronic device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 7 may also include input / output devices, network access devices, buses, etc.
[0109] The processor 70 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0110] The memory 71 can be an internal storage unit of the electronic device 7, such as a hard disk or memory. The memory 71 can also be an external storage device of the electronic device 7, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 71 can include both internal and external storage units of the electronic device 7. The memory 71 is used to store the computer program and other programs and data required by the electronic device 7. The memory 71 can also be used to temporarily store data that has been output or will be output.
[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0112] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0114] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0115] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0116] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0117] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.
[0118] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A method for generating training data for hand reinforcement learning, characterized in that, include: Acquire a video of hand movements, and then split the video of hand movements into image frames to obtain the split frame images; Each frame of the image is processed using a pre-defined hand pose estimation neural network to obtain a hand mesh file for each frame. Based on the hand mesh file of each frame image, determine the hand key point position information and wrist rotation matrix of each frame image respectively; Based on the hand key point location information and wrist rotation matrix of each frame image, a target data file for hand reinforcement learning training is generated.
2. The hand reinforcement learning training data generation method according to claim 1, characterized in that, The step of determining the hand key point position information and wrist rotation matrix for each frame of the image based on the hand mesh file includes: For the target frame image, extract the hand key point position information in the camera coordinate system from the hand mesh file; wherein, the target frame image is any one of the frame images; The hand key point position information in the camera coordinate system is transformed to obtain the hand key point position information in the image coordinate system.
3. The hand reinforcement learning training data generation method according to claim 2, characterized in that, The process of transforming the hand keypoint position information in the camera coordinate system to obtain the hand keypoint position information in the image coordinate system includes: Determine the camera translation vector corresponding to the target frame image; The camera translation vector is superimposed onto the hand key point position information in the camera coordinate system to obtain the hand key point position information in the image coordinate system.
4. The hand reinforcement learning training data generation method according to claim 1, characterized in that, Also includes: Given the location information of key points on the left hand, the location information of key points on the left hand is mirrored to obtain the mirrored location information of key points on the left hand.
5. The hand reinforcement learning training data generation method according to claim 1, characterized in that, After generating the target data file for hand reinforcement learning training based on the hand keypoint location information and wrist rotation matrix of each frame image, the following is also included: The robot's hand reinforcement learning training is performed using the target data file to obtain the robot's hand movement strategy; The robot is controlled to perform hand movements based on the aforementioned hand movement strategy.
6. The hand reinforcement learning training data generation method according to claim 1, characterized in that, After processing each frame of the image using a pre-defined hand pose estimation neural network to obtain the hand mesh file for each frame, the process also includes: Generate hand mesh images for each frame based on the hand mesh files of each frame image; The hand grid images from each frame are combined to obtain a hand grid video.
7. The method for generating hand reinforcement learning training data according to any one of claims 1 to 6, characterized in that, After generating the target data file for hand reinforcement learning training based on the hand keypoint location information and wrist rotation matrix of each frame image, the following is also included: Extract the hand key point location information from each frame of the target data file; The location information of key hand points in each frame of the image is visualized to obtain the visualization results.
8. A hand reinforcement learning training data generation device, characterized in that, include: The video segmentation module is used to acquire hand movement video and segment the hand movement video into image frames to obtain the segmented frame images; The network processing module is used to process each frame of the image using a preset hand pose estimation neural network to obtain the hand mesh file of each frame of the image; The key point determination module is used to determine the position information of the key points of the hand and the wrist rotation matrix of each frame image based on the hand mesh file of each frame image; The data file generation module is used to generate target data files for hand reinforcement learning training based on the hand key point position information and wrist rotation matrix of each frame image.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the hand reinforcement learning training data generation method as described in any one of claims 1 to 7.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the hand reinforcement learning training data generation method as described in any one of claims 1 to 7.