Data acquisition method and device

Abstract

The invention provides a data acquisition method and device, and relates to the technical field of robots. According to the specific implementation scheme, the method comprises the steps that the object pose is obtained, the object pose is the pose of a target object interacting with the robot, the interaction pose of the robot under the interaction action is determined according to the object pose, and the interaction pose comprises the initial pose when the robot starts to execute the interaction action and the target pose when the robot completes the interaction action; and on the basis of the created multiple threads, the motion trajectory of the robot moving from the initial pose to the target pose is collected in parallel, so that rapid collection of the motion trajectory is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of robotics technology, and in particular to a data acquisition method and device. Background Art

[0002] In order to enable robots to achieve smooth and efficient interactive actions in daily tasks, it can be achieved based on the interactive ability training of robots. Among them, the interactive ability training process of robots requires a large amount of rich and high-quality data as support.

[0003] However, in some data collection processes, there is a phenomenon that the data collection speed is relatively low. Summary of the invention

[0004] The present disclosure provides a data collection method and device, electronic equipment and medium.

[0005] According to one aspect of the present disclosure, a data collection method is provided, the method comprising: acquiring an object posture, wherein the object posture is the posture of a target object interacting with a robot; determining an interaction posture of the robot under an interaction action according to the object posture, the interaction posture comprising an initial posture when the robot starts to perform the interaction action, and a target posture when the robot completes the interaction action; based on the created multiple threads, collecting in parallel the motion trajectory of the robot moving from the initial posture to the target posture.

[0006] By determining the initial and target poses of the robot for interactive actions based on the pose of the target object, and collecting the robot's motion trajectory from the initial pose to the target pose in parallel based on multiple threads, the motion trajectory can be quickly collected. Especially for large-scale mobile robot scenarios, the use of multiple threads to collect motion trajectories in parallel can significantly improve the speed and efficiency of data collection.

[0007] In some embodiments, based on the created multiple threads, the motion trajectory of the robot moving from the initial posture to the target posture is collected in parallel, including: based on the multiple threads, parallel inverse kinematics solving of the interaction posture in parallel to obtain an initial joint space information set of the robot in the initial posture, and a target joint space information set of the robot in the target posture; based on the multiple threads, the initial joint space information set and the target joint space information set are solved in parallel to obtain the motion trajectory.

[0008] In some embodiments, based on the multiple threads, the interactive postures are solved in parallel by inverse kinematics to obtain an initial joint space information set of the robot in the initial posture and a target joint space information set of the robot in the target posture, including: for each thread in the multiple threads, batch processing is performed on a plurality of solution tasks corresponding to the inverse kinematics solution of each thread to obtain a solution batch processing result; based on each thread, the solution batch processing results are processed in parallel to obtain the initial joint space information set and the target joint space information set.

[0009] In some embodiments, based on the multiple threads, the initial joint space information set and the target joint space information set are solved in parallel to obtain the motion trajectory, including: for each thread in the multiple threads, batch processing is performed on the multiple trajectory tasks corresponding to each thread constructed based on the initial joint space information set and the target joint space information set to obtain a trajectory batch processing result; based on each thread, the trajectory batch processing results are processed in parallel to obtain the motion trajectory.

[0010] In some embodiments, the method further includes: respectively constructing ball fitting models corresponding to the target object and the robot; and connecting the corresponding ball fitting models to obtain a joint model of the robot and the target object.

[0011] In some embodiments, the connecting the corresponding ball fitting models to obtain the joint model of the robot and the target object includes: connecting the corresponding ball fitting models through virtual kinematic chain modeling technology to obtain the joint model.

[0012] In some embodiments, constructing the spherical fitting model of the target object includes: voxelizing the geometric model of the target object to obtain a voxelization result; and performing sphere fitting on the target object based on the voxelization result to obtain the spherical fitting model of the target object.

[0013] In some embodiments, the method further includes: controlling the execution of the action trajectory in the constructed virtual scene, and obtaining and storing the execution result.

[0014] In some embodiments, the method further includes: acquiring and storing a motion image of executing the motion trajectory.

[0015] According to another aspect of the present disclosure, a data acquisition device is provided, including: a first acquisition module, used to acquire an object posture, wherein the object posture is the posture of a target object interacting with a robot; a determination module, used to determine the interaction posture of the robot under an interaction action according to the object posture, wherein the interaction posture includes an initial posture when the robot starts to perform the interaction action, and a target posture when the robot completes the interaction action; an acquisition module, used to collect in parallel the motion trajectory of the robot moving from the initial posture to the target posture based on multiple threads created.

[0016] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned data acquisition method.

[0017] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the above-mentioned data collection method.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-mentioned data collection method when executed by a processor.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. Figure 1 A schematic diagram of an application environment involved in a data collection method according to an embodiment of the present disclosure is schematically shown; Figure 2 A flow chart of a data collection method according to an embodiment of the present disclosure is schematically shown; Figure 3 A schematic diagram schematically shows a data collection process according to an embodiment of the present disclosure; Figure 4 A block diagram of a data acquisition device according to an embodiment of the present disclosure is schematically shown; Figure 5 A block diagram of an electronic device for executing a data collection method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0023] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0024] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0025] In order to enable the robot to achieve smooth and efficient interactive actions in daily tasks, this can be achieved through the robot's interactive ability training process. The robot's interactive ability training process requires a large amount of rich and high-quality data as support. Therefore, it is necessary to collect the data required in the interactive ability process. However, in some data collection processes, there is a phenomenon that the data collection speed is low.

[0026] The embodiments of the present disclosure provide a data collection method and device, an electronic device and a medium. The data collection device can be integrated into an electronic device, and the electronic device can be a terminal device or a server. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0027] According to a data collection method of an embodiment of the present disclosure, the object posture of a target object interacting with a robot can be obtained. Then, the initial posture of the robot in contact with the target object when it starts to perform an interactive action, and the target posture of the robot in contact with the target object when the robot completes the interactive action can be determined based on the object posture. Next, based on the multiple threads created, the possible movement trajectories (i.e., action trajectories) of the robot from the initial posture to the target posture are collected in parallel. Data (such as action trajectories) are collected in parallel through multiple threads, thereby improving the speed and efficiency of data collection. In particular, for scenarios with large-scale mobile robots, the use of multiple threads to collect action trajectories in parallel can significantly improve the speed and efficiency of data collection.

[0028] This embodiment does not limit the interactive action and the target object. Exemplarily, the interactive action can be any action that the robot and the target object may interact with. For example, the interactive action includes but is not limited to: opening the target object and closing the target object. The target object can be any object that interacts with the robot. For example, the target object can be various rigid bodies (such as cups, etc.) or articulated objects (such as drawers, cabinets, etc.).

[0029] Figure 1 The following schematically shows an application environment diagram of a data collection method according to an embodiment of the present disclosure. Figure 1 As shown, the robot 101 can interact with the target object 102 through interactive actions.

[0030] The interactive action can be understood as the action performed by the robot 101 to complete the operation task for the target object 102. Different operation tasks correspond to different interactive actions. The target object 102 can be understood as an object that interacts with the robot, and is an object that can change to different states when the robot performs the interactive action.

[0031] like Figure 1 As shown, the target object 102 may be a microwave oven. The interactive action may be an action corresponding to the operation task of opening the microwave oven. Accordingly, when the robot performs the interactive action, the door of the microwave oven changes from a closed state to an open state.

[0032] Figure 1 The following schematically shows an application environment diagram of a data collection method according to an embodiment of the present disclosure. It should be noted that: Figure 1 The examples shown are only examples of application environments in which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but do not mean to limit the application environments, target objects, interactive actions, etc. of the embodiments of the present disclosure. For example, in other embodiments, the target object may also be a refrigerator; the interactive action may also be closing a door, etc., which are not listed here one by one.

[0033] The present disclosure provides a data collection method. Figure 1 For an example of an application environment, refer to Figure 2~Figure 5 A data collection method according to an exemplary embodiment of the present disclosure is described below.

[0034] Figure 2 The flowchart of a data collection method according to an embodiment of the present disclosure is schematically shown.

[0035] like Figure 2 As shown, the data collection method 200 of the embodiment of the present disclosure may include, for example, operations S210 to S230.

[0036] In operation S210 , an object posture is acquired, where the object posture is a posture of a target object interacting with the robot.

[0037] In operation S220, an interactive posture of the robot in an interactive action is determined according to the posture of the object, where the interactive posture includes an initial posture when the robot starts to perform the interactive action and a target posture when the robot completes the interactive action.

[0038] In operation S230 , based on the created multiple threads, a motion trajectory of the robot moving from an initial posture to a target posture is collected in parallel.

[0039] The implementation process of each operation is schematically illustrated below by way of example.

[0040] In operation S210 , an object posture is acquired, where the object posture is a posture of a target object interacting with the robot.

[0041] Exemplarily, the object pose can be understood as the position and attitude (such as rotation) of the target object in the interactive scene (or interactive space). The object pose can be described by 6 degrees of freedom: 3 for position (x, y, z), and 3 for attitude (roll, pitch, yaw or quaternion). For the understanding of other poses in the following text, please refer to the description here, which will not be repeated later.

[0042] For example, combined with Figure 1 , and the above Figure 1 Based on the description of the target object, such as a microwave oven, in the application environment (i.e., the interaction scene) (i.e., the object pose) can be obtained before the robot performs the interaction action.

[0043] In some embodiments, in combination Figure 3It can be seen that the object pose can be randomly generated. For example, before randomly generating the object pose, the robot, the target object, and the interaction scene can be imported. The interaction scene can be an application environment such as opening the door of the microwave oven described in the above example. Accordingly, the pose of the target object in the interaction scene (i.e., the object pose) can be randomly generated.

[0044] The robot, target object, and interaction scene can be replaced based on demand, historical records, experiments, etc. That is, by importing the robot, target object, and interaction scene, one or more of the robot, target object, and interaction scene can be automatically replaced.

[0045] For example, robots of different types can be imported to achieve the replacement of robots, so that different robots can be used to perform the same interactive action on the same target object in the same interactive scene. For another example, target objects of different types can be imported to achieve the replacement of target objects, so that the same robot can be used to perform the same interactive action on different target objects in the same interactive scene. For another example, interactive scenes of different types can be imported to achieve the replacement of interactive scenes, so that the same robot can be used to perform the same interactive action on the same target object in different interactive scenes.

[0046] It should be understood that the above examples are only exemplary descriptions of a single object among the interactive scene, the robot, and the target object as a replacement object. In other embodiments, at least part of the three can be combined to obtain a new example, and the implementation principle can be referred to the above examples, which will not be repeated here.

[0047] By importing robots, target objects, and interaction scenes, it is possible to automatically change at least one of the different robots, target objects, and interaction scenes. Relatively speaking, the automation and efficiency of data collection are greatly improved, the continuity and integrity of the collected data are ensured, and a solid data foundation is provided for the robot's interactive ability training. Moreover, the action trajectory required (such as interaction with daily objects) can be continuously collected.

[0048] Continue reading Figure 3 ,After importing the robot, target object, and interaction scene, the object pose of the imported target object can be randomly generated.

[0049] As an optional method, the object pose of the target object can also be generated based on demand, historical records, and experiments.

[0050] In operation S220, an interactive posture of the robot in an interactive action is determined according to the posture of the object, where the interactive posture includes an initial posture when the robot starts to perform the interactive action and a target posture when the robot completes the interactive action.

[0051] For example, the interactive posture can be understood as the posture of the robot interacting with the target object when performing an interactive action. The interactive posture can include two dimensions: the posture of contact with the target object when the interactive action starts (i.e., the initial posture) and the posture of contact with the target object when the interactive action is completed (i.e., the target posture).

[0052] For example, in order for the robot to complete its interactive action with the target object, when the object posture is determined, the posture of the robot in contact with the target object when it starts to perform the interactive action (i.e., the initial posture) and the posture of the robot in contact with the target object when it ends the interactive action (i.e., the target posture) can be determined based on the determined object posture.

[0053] Combination Figure 1 , and the above Figure 1 , after randomly generating the object pose of the microwave oven in the interactive scene, in order to enable the robot to open the door of the microwave oven, the initial pose of the robot in contact with the microwave oven when the robot starts to open the door of the microwave oven can be determined, and the target pose of the robot in contact with the microwave oven after the robot opens the door of the microwave oven can also be determined.

[0054] As an optional manner, the number of the initial pose and / or the target pose may be one or more.

[0055] Exemplarily, the number of initial postures may be one or more, and the number of target postures may be one or more.

[0056] For example, taking the initial posture as an example, when the robot performs a door opening operation (i.e., an interactive action), it can select at least one position to start the door opening operation. If the number of positions is one, the number of initial postures is one; if the number of positions is multiple, the number of initial postures is multiple.

[0057] Similarly, taking the target posture as an example, when the robot completes the door opening operation, the robot can have at least one contact position with the microwave oven. If it is one, the number of target postures is one; if it is multiple, the number of target postures is multiple.

[0058] In operation S230 , based on the created multiple threads, a motion trajectory of the robot moving from an initial posture to a target posture is collected in parallel.

[0059] For example, the initial posture may be one or more. The target posture may be one or more. To facilitate the reader's understanding and description, this embodiment is mainly described by taking one initial posture and one target posture as an example.

[0060] For example, the robot can move from the initial position to the target position in a variety of ways. That is, there may be multiple motion paths for the robot to move from the initial position to the target position to obtain the corresponding motion trajectory.

[0061] In this embodiment, multiple threads can be created in the processor CPU, so that the multiple threads can be executed in parallel, that is, the robot's motion trajectory can be collected in parallel between the multiple threads, and the multiple threads do not interfere with each other. By creating multiple threads in the CPU, each thread can independently execute different tasks, thereby making full use of the CPU's computing power.

[0062] For example, combined with Figure 1 , and the above Figure 1 As described in the example, when the initial and target positions are determined, the robot can have multiple motion trajectories to open the door of the microwave oven. In this embodiment, multiple possible motion trajectories are collected by using multiple threads in parallel. The efficiency of data collection can be improved by collecting motion trajectories in parallel using multiple threads. Especially for the scenario of large-scale mobile robots, the speed and efficiency of data collection can be significantly improved by using multiple threads in parallel to collect motion trajectories.

[0063] This embodiment does not limit the specific number of multiple threads, which can be determined based on demand, historical records, experiments, application environment, hardware resources, task type, task dependency, system load, context switching overhead, task granularity, etc.

[0064] For example, taking demand as an example, for scenarios with relatively high data acquisition speed requirements, the number of threads can be relatively large; conversely, for scenarios with relatively low data acquisition speed requirements, the number of threads can be relatively small. For another example, taking hardware resources as an example, the number of threads should match the number of CPU cores. For hardware resources with relatively more CPU cores, the number of threads can be relatively large; conversely, for hardware resources with relatively fewer CPU cores, the number of threads can be relatively small.

[0065] It is worth noting that for each of the multiple threads created, the number of tasks of the thread is greater than or equal to 1. That is, a thread may correspond to one task or multiple tasks. If a thread corresponds to one task, each thread executes the corresponding task in parallel. If a thread corresponds to multiple tasks, each thread executes the corresponding tasks in parallel, and for each thread, the multiple tasks corresponding to the thread are executed sequentially within the thread. In other words, when a thread corresponds to multiple tasks, the threads are executed in parallel, and the threads are executed serially.

[0066] As an optional method, operation S230 may include: based on multiple threads, performing parallel inverse kinematics solutions for the interactive postures to obtain an initial joint space information set of the robot in the initial posture and a target joint space information set of the robot in the target posture; based on multiple threads, performing parallel solutions for the initial joint space information set and the target joint space information set to obtain the motion trajectory.

[0067] The following is an illustrative illustration of "based on multiple threads, performing parallel inverse kinematics solutions for interactive postures to obtain an initial joint space information set of the robot in the initial posture, and a target joint space information set of the robot in the target posture."

[0068] For example, combining the above examples and Figure 3 It can be seen that after randomly generating the object pose, the initial pose and target pose of the robot can be determined. Then, the inverse kinematics solution can be performed in parallel based on multiple threads to obtain the initial joint space information set and the target joint space information set.

[0069] For example, the initial pose can be understood as the pose of the robot's end gripper in contact with the microwave oven when the robot starts to open the door (i.e., the initial pose). The target pose can be understood as the pose of the robot's end gripper in contact with the microwave oven when the robot completes opening the door (i.e., the target pose). The initial joint space information set can be understood as the set of multiple angles of the robot's joint group corresponding to the initial pose. The target joint space information set can be understood as the set of multiple angles of the robot's joint group corresponding to the target pose.

[0070] Among them, for an initial posture, there may be multiple angles of the corresponding joint group. For a target posture, there may be multiple angles of the corresponding multi-joint group. In this embodiment, the inverse kinematics solution is performed in parallel by multiple threads, such as based on an initial posture, multiple threads solve in parallel to obtain the angles of multiple joint groups, or based on a target posture, multiple threads solve in parallel to obtain the angles of multiple joint groups, thereby improving the solution efficiency and speed, that is, the initial joint space information set and the target joint space information set can be quickly obtained.

[0071] As an optional method, for each thread among multiple threads, batch processing is performed on multiple solution tasks corresponding to each thread in the inverse kinematics solution to obtain a solution batch processing result; the solution batch processing results are processed in parallel based on each thread to obtain an initial joint space information set and a target joint space information set.

[0072] Exemplarily, inverse kinematics can be solved by combining multithreading and batch processing techniques. One thread usually processes multiple inverse kinematics solving tasks. An inverse kinematics solving task can be understood as, for example, obtaining at least one initial joint space information based on an initial posture solution.

[0073] Taking a thread and an initial pose as an example, the thread can batch process multiple inverse kinematics solution tasks corresponding to it, and convert multiple low-dimensional vector operations (i.e., multiple low-dimensional solution tasks) into a high-dimensional matrix operation. If at least part of the multiple inverse kinematics solution tasks can be converted into at least one high-dimensional matrix operation, the graphics processor GPU can calculate the converted high-dimensional matrix. For the solution tasks that cannot be converted into high-dimensional matrices, they are processed by the thread. The thread determines the corresponding initial joint space information based on the results of its processing of the inverse kinematics solution tasks and the results of the GPU processing of the high-dimensional matrix.

[0074] For example, an inverse kinematics solution task of a certain thread is an inverse kinematics solution calculation, which includes multiple steps, such as five steps, namely five low-dimensional vector operations. Among the five low-dimensional vector operations, three low-dimensional vector operations can be converted into high-dimensional matrix operations. Based on batch processing, the three low-dimensional vector operations can be converted into high-dimensional matrix operations, and the GPU processes the high-dimensional matrix operations to obtain corresponding results. The thread jointly determines the result of the inverse kinematics solution task based on the processing results of the GPU on the high-dimensional matrix operation and the processing results of the two low-dimensional matrix operations that cannot be converted into high-dimensional matrix operations among the five low-dimensional vector operations.

[0075] Based on batch processing technology, multiple solving tasks can be integrated into a high-dimensional matrix operation, which can effectively utilize the parallel processing capabilities of the GPU and further improve the processing efficiency of the solving tasks. Therefore, the efficiency of data acquisition can be further improved.

[0076] The following is a schematic illustration of “based on multiple threads, solving the initial joint space information set and the target joint space information set in parallel to obtain the motion trajectory”.

[0077] For example, combining the above examples and Figure 3 It can be seen that after solving the inverse kinematics based on multiple threads in parallel, the trajectory of the interactive action (i.e., the action trajectory) can be solved based on multiple threads in parallel. Solving the action trajectory by using multiple threads in parallel can improve the solving efficiency and speed, that is, the action trajectory can be obtained quickly.

[0078] As an alternative, Figure 3 As shown, the motion trajectory can be solved based on the pre-configured constraint conditions. Among them, the constraint conditions can be understood as the corresponding constraints that the motion trajectory needs to meet so that the robot can complete the motion trajectory under the constraint. The constraint conditions can be configured from the dimensions of collision, combined with the display scene, etc., and this embodiment does not limit it.

[0079] Exemplarily, in the process of solving the motion trajectory, constraints need to be met so as to constrain the robot's motion trajectory through the constraints, so that the machine can execute the motion trajectory safely and reliably, thereby improving the reliability and authenticity of solving the motion trajectory.

[0080] As an optional method, for each thread in the multiple threads, batch processing is performed on multiple trajectory tasks corresponding to each thread constructed based on the initial joint space information set and the target joint space information set to obtain a trajectory batch processing result; the trajectory batch processing results are processed in parallel based on each thread to obtain an action trajectory.

[0081] Exemplarily, the motion trajectory can be solved by combining multithreading and batch processing technology. One thread usually processes multiple motion trajectory solving tasks. One motion trajectory solving task can be understood as solving and obtaining at least one motion trajectory based on an initial joint space information and a target joint space information.

[0082] Taking a thread as an example, the thread can batch process the corresponding multiple motion trajectory solving tasks, and convert multiple low-dimensional vector operations (i.e., multiple low-dimensional solving tasks) into a high-dimensional matrix operation. If at least part of the multiple motion trajectory solving tasks can be converted into at least one high-dimensional matrix operation, the GPU can calculate the converted high-dimensional matrix. The solving tasks that cannot be converted into high-dimensional matrices are processed by the thread. The thread determines the corresponding initial joint space information based on the results of its processing of the motion trajectory solving tasks and the results of the GPU processing of the high-dimensional matrix.

[0083] Similarly, a motion trajectory solving task of a certain thread is a motion trajectory solving calculation, which includes multiple steps, such as five steps, namely five low-dimensional vector operations. Among the five low-dimensional vector operations, three low-dimensional vector operations can be converted into high-dimensional matrix operations. Based on batch processing, the three low-dimensional vector operations can be converted into high-dimensional matrix operations, and the GPU processes the high-dimensional matrix operations to obtain corresponding results. The thread jointly determines the result of the motion trajectory solving task based on the processing results of the GPU on the high-dimensional matrix operation and the processing results of the two low-dimensional matrix operations that cannot be converted into high-dimensional matrix operations among the five low-dimensional vector operations.

[0084] Based on batch processing technology, multiple solving tasks can be integrated into a high-dimensional matrix operation, which can effectively utilize the parallel processing capabilities of the GPU and further improve the processing efficiency of the solving tasks. Therefore, the efficiency of data acquisition can be further improved.

[0085] As an optional manner, the data collection method may further include: constructing ball fitting models corresponding to the target object and the robot respectively; and connecting the corresponding ball fitting models to obtain a joint model of the robot and the target object.

[0086] The sphere fitting model is to approximate the geometric shape of the target object, robot (especially its end effector or key joints) as one or more spheres. The radius and center position of each sphere can be determined by an optimization algorithm so that the sphere wraps the original geometric shape as tightly as possible.

[0087] The joint model can also be called the connected ball fitting model, which can be understood as "connecting" the ball fitting models corresponding to the target object and the robot in some way to form an overall model. The joint model can be used to characterize the relative position and posture relationship between the robot and the target object.

[0088] Exemplarily, a sphere fitting model of the target object and a sphere fitting model of the robot may be constructed. After two sphere fitting models are constructed, the two sphere fitting models may be connected to obtain a joint model.

[0089] For example, combined with Figure 3 As in the above example, after importing the robot, target object, and interaction scene, a joint model of the robot and target object can be constructed to randomly generate object poses based on this.

[0090] The sphere fitting model can improve data collection efficiency by simplifying geometric calculations. In addition, sphere fitting has good robustness to noise and irregular shapes, and can tolerate noise and incomplete parts in point cloud data to a certain extent. Moreover, by connecting the sphere fitting models corresponding to the robot and the target object to obtain a joint model, the robot can better support interactive actions.

[0091] As an optional way, the corresponding ball fitting models are connected to obtain a joint model of the robot and the target object, including: connecting the corresponding ball fitting models through a virtual kinematic chain (VKC) modeling technology to obtain a joint model.

[0092] For example, a virtual kinematic chain can be understood as decomposing the motion of a complex mechanical system or object into a combination of multiple virtual joints and connecting rods. These virtual joints and connecting rods are not actual physical components, but abstract concepts introduced to facilitate the description and analysis of system motion. In this way, a complex multi-degree-of-freedom system can be simplified into a series of simple single-degree-of-freedom motion units, making it easier to perform kinematic and dynamic analysis.

[0093] In combination with the above examples, the target joint space information set can be understood as a set of multiple angles of the robot's joint group corresponding to the target posture. That is, the target joint space information set is a set of joint angles at the "group" granularity. For example, a group joint includes multiple joints, such as 7 (or more or less) joints. The number of joints is determined based on the virtual kinematic chain modeling technology.

[0094] Through the virtual kinematic chain modeling technology, the motion of the robot and the target object can be decomposed into a combination of multiple virtual joints and links. This simplifies the complexity of joint model generation and provides a basis for subsequent processing. Moreover, combined with the virtual kinematic chain modeling technology, the robot can efficiently collect the motion trajectory of interacting with the target object, such as various rigid bodies (such as cups) or articulated objects (such as drawers, cabinets, etc.) in the virtual environment.

[0095] As an optional method, constructing a spherical fitting model of the target object includes: voxelizing the geometric model of the target object to obtain a voxelization result; and performing sphere fitting on the target object based on the voxelization result to obtain a spherical fitting model of the target object.

[0096] The voxelization process can be understood as, first, converting the original geometric model of the target object (such as polygonal mesh, surface, etc.) into a voxel representation. Each voxel represents a small part of the target object, and the size of the voxel can be adjusted as needed. The voxelized target object can be more easily subjected to geometric analysis and physical simulation.

[0097] Exemplarily, the geometric model of the target object can be converted into a voxel representation to convert the geometric model of the target object into a plurality of voxels representing a small portion of the target object. Based on the voxelization process, an algorithm such as automatic fitting can be used to calculate a set of relatively good spheres so that they can cover the surface of the target object as accurately as possible. In the fitting process, optimization can be performed based on the geometric features of the target object (such as boundaries, vertices, curvature, etc.) to ensure the fitting accuracy.

[0098] As an optional manner, the data collection method further includes: controlling the execution of the action trajectory in the constructed virtual scene, and obtaining and storing the execution result.

[0099] For example, combining the above examples and Figure 3 , under the constraint conditions, after solving the action trajectory, the action trajectory can be executed in the virtual scene to obtain the execution result of the action trajectory, and the execution result can be stored.

[0100] For example, after the motion trajectory is obtained through the above example, a virtual scene can be pre-built to simulate the robot's execution of the motion trajectory through the virtual scene to determine the situation when the obtained motion trajectory is executed. By executing the motion trajectory in the virtual scene, a large amount of rich and high-quality data support can be provided for the interactive ability training process to ensure that the robot can achieve smooth and efficient interactive actions in daily tasks.

[0101] As an optional manner, the data collection method may further include: acquiring and storing a motion image of the execution motion trajectory.

[0102] For example, combining the above examples and Figure 3 ,When the action trajectory is executed in the virtual scene, the corresponding ,motion image can be captured and saved.

[0103] For example, when the action trajectory is executed in the virtual scene, the corresponding motion image can be obtained through an image acquisition device, etc. The execution result and the motion image can be used as training data. That is, the training data includes both the corresponding trajectory data (execution result) and the corresponding image data (motion image), thereby improving the diversity and richness of the training data. In this way, a large amount of rich and high-quality data is needed to support the robot's interactive ability training process. In turn, the robot can perform more smooth and efficient interactive actions in daily tasks.

[0104] As an optional method, after completing data collection, the collected data (including trajectory data and image data) can be used for interaction capability training so that the robot can perform more smooth and efficient interaction actions.

[0105] Figure 4 A block diagram of a data acquisition device according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 4 As shown, the data acquisition device 400 of the embodiment of the present disclosure includes, for example, a first acquisition module 410 , a determination module 420 , and a collection module 430 .

[0107] The first acquisition module 410 is used to acquire the object posture, wherein the object posture is the posture of the target object interacting with the robot. The determination module 420 is used to determine the interaction posture of the robot under the interaction action according to the object posture, wherein the interaction posture includes the initial posture when the robot starts to perform the interaction action, and the target posture when the robot completes the interaction action. The acquisition module 430 is used to collect the motion trajectory of the robot from the initial posture to the target posture in parallel based on the created multiple threads.

[0108] In some embodiments, the first acquisition module includes: a first solving submodule, which is used to perform inverse kinematics solutions for the interactive posture in parallel based on multiple threads to obtain an initial joint space information set of the robot in the initial posture and a target joint space information set of the robot in the target posture; a second solving submodule, which is used to solve the initial joint space information set and the target joint space information set in parallel based on multiple threads to obtain the motion trajectory.

[0109] In some embodiments, the first solution submodule includes: a first batch processing unit, which is used to batch process multiple solution tasks in the inverse kinematics solution corresponding to each thread in a plurality of threads to obtain a solution batch processing result; and a first parallel processing unit, which is used to parallel process the solution batch processing results based on each thread to obtain the initial joint space information set and the target joint space information set.

[0110] In some embodiments, the second solving submodule includes: a second batch processing unit, which is used to batch process multiple trajectory tasks corresponding to each thread in multiple threads, constructed based on the initial joint space information set and the target joint space information set, to obtain trajectory batch processing results; a second parallel processing unit, which is used to parallel process the trajectory batch processing results based on each thread to obtain the action trajectory.

[0111] In some embodiments, the data acquisition device further includes: a construction module for respectively constructing the ball fitting models corresponding to the target object and the robot; and a connection module for connecting the corresponding ball fitting models to obtain a joint model of the robot and the target object.

[0112] In some embodiments, the connection module is used to connect the corresponding ball fitting models through virtual kinematic chain modeling technology to obtain a joint model.

[0113] In some embodiments, the construction module includes: a voxelization processing submodule, which is used to voxelize the geometric model of the target object to obtain a voxelization result; and a sphere fitting submodule, which is used to perform sphere fitting on the target object based on the voxelization result to obtain a sphere fitting model of the target object.

[0114] In some embodiments, the data acquisition device further includes: a control module, used to control the execution of the action trajectory in the constructed virtual scene to obtain the execution result; and a first storage module, used to store the execution result.

[0115] In some embodiments, the data acquisition device further includes: a second acquisition module for acquiring the execution motion trajectory; and a second storage module for storing motion images.

[0116] It should be noted that the information collection, storage, use, processing, transmission, provision and disclosure involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0118] Figure 5 A block diagram of an electronic device for executing a data collection method according to an embodiment of the present disclosure is schematically shown.

[0119] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device 500 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0120] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0121] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the data acquisition method. For example, in some embodiments, the data acquisition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the data acquisition method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the data acquisition method in any other appropriate manner (e.g., by means of firmware).

[0123] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data acquisition device, so that the program code, when executed by the processor or controller, causes the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0125] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] To provide interaction with an object, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the object; and a keyboard and pointing device (e.g., a mouse or trackball) through which the object can provide input to the computer. Other types of devices can also be used to provide interaction with an object; for example, the feedback provided to the object can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the object can be received in any form (including acoustic input, voice input, or tactile input).

[0127] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., an object computer having a graphical object interface or a web browser through which the object can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0128] A computer system may include a terminal and a server. The terminal and the server are generally remote from each other and usually interact through a communication network. The relationship between the terminal and the server is generated by computer programs running on the corresponding computers and having a terminal-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0129] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0130] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A data collection method, characterized in that: The method comprises: Acquire an object pose, where the object pose is a pose of a target object interacting with the robot; Determining an interactive posture of the robot under an interactive action according to the object posture, the interactive posture comprising an initial posture when the robot starts to perform the interactive action, and a target posture when the robot completes the interactive action; Based on the created multiple threads, the motion trajectory of the robot moving from the initial posture to the target posture is collected in parallel.

2. The method according to claim 1, characterized in that The step of collecting the motion trajectory of the robot from the initial posture to the target posture in parallel based on the created multiple threads includes: Based on the multiple threads, performing inverse kinematics solutions on the interactive postures in parallel to obtain an initial joint space information set of the robot in the initial posture and a target joint space information set of the robot in the target posture; Based on the multiple threads, the initial joint space information set and the target joint space information set are solved in parallel to obtain the motion trajectory.

3. The method according to claim 2, characterized in that The performing inverse kinematics solutions on the interactive postures in parallel based on the multiple threads includes: For each of the multiple threads, batch processing is performed on multiple solution tasks in the inverse kinematics solution corresponding to each thread to obtain a solution batch processing result; The batch processing results are processed in parallel based on each thread to obtain the action trajectory.

4. The method according to claim 2, characterized in that: The step of solving the initial joint space information set and the target joint space information set in parallel based on the multiple threads to obtain the motion trajectory includes: For each of the multiple threads, batch processing is performed on a plurality of trajectory tasks corresponding to each thread and constructed based on the initial joint space information set and the target joint space information set to obtain a trajectory batch processing result; The trajectory batch processing results are processed in parallel based on each thread to obtain the action trajectory.

5. The method according to claim 1, characterized in that The method further comprises: Constructing ball fitting models corresponding to the target object and the robot respectively; The corresponding spherical fitting models are connected to obtain a joint model of the robot and the target object.

6. The method according to claim 5, characterized in that The step of connecting the corresponding spherical fitting models to obtain a joint model of the robot and the target object includes: The corresponding spherical fitting models are connected by virtual kinematic chain modeling technology to obtain the joint model.

7. The method according to claim 5, characterized in that Constructing a spherical fitting model of the target object, comprising: Performing voxelization processing on the geometric model of the target object to obtain a voxelization result; A sphere fitting is performed on the target object based on the voxelization result to obtain a sphere fitting model of the target object.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: The execution of the action track is controlled in the constructed virtual scene, and the execution result is obtained and stored.

9. The method according to claim 8, characterized in that The method further comprises: A motion image of executing the motion trajectory is acquired and stored.

10. A data acquisition device, characterized in that: include: A first acquisition module is used to acquire an object posture, wherein the object posture is a posture of a target object interacting with the robot; A determination module, configured to determine an interactive posture of the robot under an interactive action according to the object posture, wherein the interactive posture includes an initial posture when the robot starts to perform the interactive action and a target posture when the robot completes the interactive action; The acquisition module is used to collect the motion trajectory of the robot from the initial posture to the target posture in parallel based on the created multiple threads.

11. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data acquisition method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the data collection method according to any one of claims 1 to 9.

Images