Path planning method and device based on intelligent mechanical arm with body

Abstract

The invention provides a mechanical arm path planning method and device based on body intelligence. The method comprises the steps that an environment three-dimensional model and planning data are obtained; obtaining a trained strategy model; the strategy model is obtained based on expert tracks, first training conducted through behavior cloning and second training conducted through adversarial imitation learning; based on the environment three-dimensional model and planning data, actions are executed step by step through a strategy model, and a planning path of the mechanical arm is generated; and controlling the mechanical arm to execute the planned path. According to the method and the device, the strategy model is trained through the expert track to absorb past expert experience, and then the planning path is generated through the strategy track, so that actual planning of the mechanical arm path is realized by utilizing the intelligent robot.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and more specifically, to a method and device for robotic arm path planning based on embodied intelligence. Background Art

[0002] Embodied intelligence refers to intelligent entities that have bodies and support interaction with the physical world, such as robots and driverless cars. By processing multiple sensory data inputs, a control center such as a large model generates motion instructions to drive the intelligent entity, replacing the traditional motion-driven method based on rules or mathematical formulas, and realizing the deep integration of virtual and reality.

[0003] Surgical robots can serve as a perfect carrier for embodied intelligence, enabling the functional realization of surgical robots based on embodied intelligence. During the specific implementation process, past experience with surgical robots can be summarized, and then a new control center can be generated based on this experience.

[0004] However, there is still a lack of corresponding solutions on how to use embodied intelligence to realize the specific planning of the robot arm's path. Summary of the Invention

[0005] The problem addressed by this application is that current path planning is difficult to utilize embodied intelligence.

[0006] To solve the above problems, the first aspect of the present application provides a robotic arm path planning method based on embodied intelligence, which includes:

[0007] Obtain environmental 3D models and planning data;

[0008] Obtaining a trained policy model; the policy model is obtained based on expert trajectories through a first training performed by behavior cloning and a second training performed by adversarial imitation learning;

[0009] Based on the 3D model of the environment and planning data, the strategy model is used to execute actions step by step to generate the planned path of the robot arm.

[0010] Control the robotic arm to execute the planned path.

[0011] A second aspect of the present application provides a manufacturing system for a robotic arm path planning method based on embodied intelligence, comprising:

[0012] A data acquisition module, which is used to obtain a three-dimensional model of the environment and planning data;

[0013] A strategy training module, which is used to obtain a trained strategy model; the strategy model is obtained based on expert trajectories through a first training method using behavior cloning and a second training method using adversarial imitation learning;

[0014] The path planning module is used to generate the planned path of the robot arm by gradually executing actions through the policy model based on the 3D model of the environment and planning data;

[0015] A robotic arm control module is used to control the robotic arm to execute the planned path.

[0016] A third aspect of the present application provides an electronic device, comprising: a memory and a processor; the memory may be configured to store a program; the processor is coupled to the memory and is configured to execute the program in the memory, so as to:

[0017] Obtain environmental 3D models and planning data;

[0018] Obtaining a trained policy model; the policy model is obtained based on expert trajectories through a first training performed by behavior cloning and a second training performed by adversarial imitation learning;

[0019] Based on the 3D model of the environment and planning data, the strategy model is used to execute actions step by step to generate the planned path of the robot arm.

[0020] Control the robotic arm to execute the planned path.

[0021] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the aforementioned method for robotic arm path planning based on embodied intelligence.

[0022] In this application, the policy model is trained through expert trajectories to absorb past expert experience, and then the planning path is generated through the policy trajectory, thereby using embodied intelligence to realize the actual planning of the robot arm path. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flowchart of a robotic arm path planning method based on embodied intelligence according to an embodiment of the present application;

[0024] Figure 2 2 is an architectural diagram of a robotic arm path planning device based on embodied intelligence according to an embodiment of the present application;

[0025] Figure 3 2 is a diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, specific embodiments of the present application are described in detail below with reference to the accompanying drawings. Although the accompanying drawings show exemplary embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0027] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art in the field of this application.

[0028] The embodiment of the present application provides the above-mentioned robot arm path planning method based on embodied intelligence, and the specific scheme of the method is as follows: Figure 1 As shown, the method can be performed by a robot arm path planning device based on embodied intelligence, and the robot arm path planning device based on embodied intelligence can be integrated into electronic devices such as computers, servers, computers, server clusters, and data centers. Figure 1 As shown, the robotic arm path planning method based on embodied intelligence includes:

[0029] S101, obtaining a three-dimensional model of the environment and planning data;

[0030] In this application, sensors (such as RGB-D cameras, lidar, IMU, etc.) are used to collect environmental information in real time, including obstacle positions, target object shapes, and the initial state of the robotic arm; and point cloud processing algorithms (such as ICP and PCL libraries) or deep learning models (such as PointNet) are used to build a three-dimensional model of the environment.

[0031] In this application, the planning data includes the target position of the manipulator, task constraints (such as obstacle avoidance requirements), and dynamic factors in the environment (such as moving obstacles).

[0032] S102, obtaining a trained policy model; the policy model is obtained based on expert trajectories through a first training method using behavior cloning and a second training method using adversarial imitation learning;

[0033] S103, based on the three-dimensional model of the environment and the planning data, the action is executed step by step through the strategy model to generate a planned path for the robotic arm;

[0034] In this application, the three-dimensional model of the environment and planning data are used as input to extract the current state of the robotic arm (such as joint angles and end effector positions); the trained strategy model is used to predict the next action based on the current state; and the above actions are repeated to gradually generate a complete planning path.

[0035] Preferably, during the generation process, the trajectory is adjusted in real time in combination with environmental constraints (such as obstacle avoidance requirements) and mission objectives (such as the shortest path).

[0036] S104: Control the robotic arm to execute the planned path.

[0037] In this application, the optimized planned path is deployed on a physical robotic arm, and the robotic arm is controlled to perform actions step by step; the path is dynamically adjusted according to the actual operating conditions to form a closed-loop control.

[0038] In this application, the policy model is trained through expert trajectories to absorb past expert experience, and then the planning path is generated through the policy trajectory, thereby using embodied intelligence to realize the actual planning of the robot arm path.

[0039] In this application, the first training is performed through behavioral cloning, and the result of the first training is used as the initial generator for the second training through adversarial imitation learning, thereby greatly reducing the amount of calculation and improving planning efficiency while achieving high-precision path planning.

[0040] Preferably, a digital twin space of the robotic arm is constructed; during operation, the state data of the robotic arm is collected in real time and synchronized to the digital twin space, and the path is dynamically adjusted according to the actual operation conditions to form a closed-loop control.

[0041] In this way, the strategy model is trained through behavioral cloning and adversarial imitation learning, the robot arm path is generated using expert experience, and digital twin technology is combined to achieve dynamic environment adaptation and closed-loop optimization, thereby completing the robot arm path planning task efficiently and robustly.

[0042] In a specific embodiment, the training process of the policy model includes:

[0043] Access to historical environmental models and expert trajectories;

[0044] Build a strategy model;

[0045] Based on behavior cloning, the policy model is first trained through expert trajectories;

[0046] Based on adversarial imitation learning, the policy model is trained for the second time through expert trajectories.

[0047] In a specific embodiment, the first training of the policy model using expert trajectories based on behavior cloning includes:

[0048] Generate the current state of the robot arm based on the expert trajectory and historical environment model;

[0049] Input the current state of the robot arm into the policy model to obtain the predicted action of the robot arm;

[0050] Determine the expert action of the robot arm based on the expert trajectory;

[0051] Calculate the overall loss based on the expert actions and the predicted actions;

[0052] The policy model is iterated based on the overall loss until convergence.

[0053] In this application, the expert trajectory includes the state sequence (such as joint angles, end effector positions) and action sequence (such as speed instructions) of the robot arm.

[0054] In this application, the historical environment model provides environmental information, including the location of obstacles, the shape of target objects, etc.

[0055] In this application, the strategy model structure is as follows: a neural network π is defined, the input is the current state, and the output is the corresponding action.

[0056] In this application, the network structure can be selected according to the complexity of the task: multi-layer perceptron (MLP), convolutional neural network (CNN) or recurrent neural network (RNN) or Transformer.

[0057] In this application, based on the action sequence in the expert trajectory, the expert action corresponding to the current state is extracted as the target value of supervised learning.

[0058] In this application, the mean square error (MSE) is used as the loss function to measure the difference between the predicted action and the expert action; the error between the predicted action and the expert action is calculated for each sample, and the average is taken as the overall loss.

[0059] In this application, the first training phase of behavioral cloning uses expert trajectories and historical environment models to generate the current state of the robot arm, and then optimizes the policy model using supervised learning. By iteratively calculating losses and updating model parameters, a preliminary policy model is ultimately generated that can mimic the expert's actions.

[0060] In a specific embodiment, the current state of the robotic arm includes the joint angles, joint velocities, end effector positions, end effector postures, and environmental information of the robotic arm.

[0061] Joint angle: describes the rotation angle of each joint of the robotic arm.

[0062] Joint velocity: describes the rotation speed of each joint of the robotic arm.

[0063] End effector position: describes the position coordinates of the end of the robotic arm.

[0064] End effector pose: describes the pose of the end of the robotic arm (such as direction vector or quaternion).

[0065] Environmental information: describes the dynamic environment around the robotic arm (such as obstacle locations, target object locations).

[0066] In a specific embodiment, the predicted motion of the robotic arm includes a change in joint angle, a joint acceleration, an end effector velocity, and an end effector direction vector.

[0067] Change in joint angle: describes the angle change of each joint of the robotic arm.

[0068] Joint acceleration: describes the acceleration of each joint of the robotic arm.

[0069] End effector speed: describes the speed of the end of the robot arm.

[0070] End effector direction vector: describes the movement direction of the end of the robotic arm.

[0071] In a specific embodiment, the second training of the policy model using expert trajectories based on adversarial imitation learning includes:

[0072] Use the first trained policy model as the initial parameters of the generator;

[0073] Access to expert data and historical environmental models;

[0074] Based on the generator, generate an action sequence corresponding to the current state of the robot arm and determine the generated trajectory;

[0075] Fixed the generator parameters and trained the discriminator based on the expert data and generated trajectories until convergence;

[0076] Fixed the discriminator parameters and trained the generator based on the expert data and generated trajectory until convergence;

[0077] The generator and the discriminator are trained alternately until the preset conditions are met.

[0078] In this application, the preliminary strategy model obtained by the first training of behavioral cloning is used as the initial parameters of the generator.

[0079] In this application, expert data: collects high-quality expert trajectory data, including the state and action sequence of the robotic arm.

[0080] In this application, an action sequence is generated: the current state of the robot arm is input into the generator to generate the corresponding action; the generated actions are executed step by step, the state of the robot arm is updated, and a complete trajectory is generated.

[0081] In this application, the discriminator is a binary classification neural network, the input is the motion trajectory, and the output is a probability value indicating whether the trajectory comes from expert demonstration data.

[0082] In this application, the generator parameters are fixed and only the discriminator parameters are updated. The discriminator is trained using the expert trajectory data and the generator trajectory data so that it can accurately distinguish between the two types of trajectories.

[0083] In this application, the goal of the generator is to maximize the perplexity of the discriminator. Preferably, a reward function can be combined to encourage the generator to generate better trajectories (such as obstacle avoidance, shortest path, etc.).

[0084] In this application, the discriminator parameters are fixed, and only the generator parameters are updated. The generator gradually generates trajectories through the simulated environment, and attempts to generate trajectories that are difficult for the discriminator to distinguish. This progresses until the trajectories generated by the generator gradually approach the expert trajectories, and the discriminator cannot distinguish between the two types of trajectories.

[0085] In this application, in each iteration, the discriminator is first trained to distinguish between the expert trajectory and the generator trajectory, and then the generator is trained to generate action sequences that are closer to the expert trajectory. The above process is repeated until one of the following preset conditions is met:

[0086] The trajectory generated by the generator is indistinguishable from the expert trajectory. The classification accuracy of the discriminator is close to random guessing (e.g., 0.5). The loss function value reaches the preset threshold.

[0087] In this application, the second training phase of adversarial imitation learning optimizes the policy model by alternating between training the generator and the discriminator. The generator generates trajectories, while the discriminator distinguishes between expert trajectories and generated trajectories. The two compete with each other, and eventually the trajectories generated by the generator gradually approach the expert trajectories, thereby improving the generalization and robustness of the policy model.

[0088] It should be noted that in this application, the generator generates an action, and the generator is executed cyclically to obtain a complete trajectory; the complete trajectory is directly input into the discriminator to obtain the discrimination result.

[0089] In a specific embodiment, obtaining the historical environment model and expert trajectory includes:

[0090] Build a digital twin space for the robotic arm;

[0091] Obtain expert trajectories and the historical environment model corresponding to the expert trajectories;

[0092] Mapping historical environment models and expert trajectories into the digital twin space;

[0093] Set multiple optimization goals;

[0094] Generate multiple expert trajectories in a historical environment model based on the optimization objective.

[0095] In this application, the dynamics and kinematics models of the robotic arm are constructed in the digital twin space. These include: A kinematic model that describes the forward and inverse kinematics of the robotic arm. A dynamic model that simulates the real-world behavior of the robotic arm, taking into account factors such as friction, inertia, and gravity.

[0096] In this application, the historical environment model and expert trajectory are mapped into the digital twin space to form a virtual scene. The working scene of the robot arm, including obstacles, target objects, etc., is reproduced in the digital twin space.

[0097] In this application, multiple optimization goals are set according to task requirements, such as:

[0098] Shortest path length: Minimize the distance the robot moves from the starting point to the end point.

[0099] Minimum energy consumption: Reduce the energy consumed during the operation of the robot arm.

[0100] Obstacle avoidance capability: Ensures that the robotic arm does not collide with obstacles during operation.

[0101] Time efficiency: Reduce the time required for the robotic arm to complete a task.

[0102] In the digital twin space, multiple different expert trajectories are generated based on the historical environment model and optimization objectives. Each optimization objective or a combination of multiple optimization objectives can generate a corresponding trajectory as the expert trajectory.

[0103] Preferably, the generated expert trajectory is randomly perturbed (such as fine-tuning trajectory points, changing obstacle positions) to generate more diverse training data.

[0104] In this application, by constructing a digital twin space of the robotic arm, obtaining historical environment models and expert trajectories, and combining optimization objectives to generate diverse expert trajectories, high-quality data support can be provided for the training of the strategy model.

[0105] In one embodiment, the historical environment model is a depth image; after obtaining the historical environment model, the historical environment model is also updated; the specific process of the update includes:

[0106] Divide the historical environment model into blocks to obtain independent blocks;

[0107] For each independent block, obtain a first neighboring block and a second neighboring block with different spacings;

[0108] generating a first feature block based on the independent block and the first neighborhood block;

[0109] generating a second feature block based on the independent block and the second neighborhood block;

[0110] Performing feature compression on the first feature block and the second feature block to obtain a compressed block;

[0111] Iterate over all independent blocks and generate an updated historical environment model based on the resulting compressed blocks.

[0112] In this application, the historical environment model is divided into blocks, that is, the historical environment model is divided into corresponding image blocks through a checkerboard grid; wherein, the image block can be at the pixel level (that is, each pixel is an image block) or at other levels, and the specific division shall be based on the actual processing situation.

[0113] In this application, a sliding window or a fixed step size is used to divide the image into blocks of the same size.

[0114] It should be noted that the historical environment model is a three-dimensional image, so one surface is selected for chessboard division, and each grid is a strip grid with a lot of depth (the depth is the depth of the three-dimensional image), and the strip grid is an image block.

[0115] Preferably, in the present application, each image block is 1001000 pixels, so that more feature calculations between local areas can be performed on the basis of ensuring generation accuracy and reducing the amount of calculation.

[0116] In this application, an image block is selected as an independent block. The adjacent image blocks above, below, to the left, and to the right of the independent block are the first neighboring blocks. The image blocks above, below, to the left, and to the right of the independent block, separated by one grid, are the second neighboring blocks. The first and second neighboring blocks have different spacing from the independent block.

[0117] In this application, the neighborhood information of each independent block is extracted to capture the local structure.

[0118] In the present application, the first feature block is generated to generate a local feature representation using an independent block and its first neighborhood block. Specifically, the independent block and the first neighborhood block are processed by a convolution layer and an attention layer to obtain the first feature block.

[0119] In this application, the specific structure and specific parameters of the convolutional layer and the attention layer can be obtained according to the training data or determined according to the actual situation.

[0120] It should be noted that, in this application, there are four first neighborhood blocks and multiple first feature blocks.

[0121] In this application, the independent block and the first neighborhood block are processed by the convolution layer and the attention layer to obtain the first feature block. The specific process is: the independent block and the four neighborhood blocks are spliced ​​together to form a multi-channel input, and the convolution layer is used to extract features from the spliced ​​block; the self-attention mechanism or the channel attention mechanism is used to enhance important features, calculate the attention weight, and weight the convolution layer output to enhance important features; the output of the attention layer is split into multiple feature blocks, each feature block corresponds to the processing results of the independent block and at least one neighborhood block.

[0122] In this application, a second feature block is generated to generate a broader local feature representation using an independent block and its second neighborhood block. The specific generation process is the same as that of the first feature block, except that the parameters of the convolution layer and the attention layer are different.

[0123] In this application, the generated feature blocks are compressed into a more compact representation to reduce the amount of computation and retain key information. Feature compression is performed using pooling operations (such as maximum pooling or average pooling) or fully connected layers.

[0124] In this way, through compression, the first feature blocks and the second feature blocks are compressed into a compressed block, which has the same size and position as the independent block and is used to replace the independent block. All image blocks are replaced by the compressed block to obtain an updated historical environment model.

[0125] In this application, each image block of the historical environment model is traversed by traversal to obtain the corresponding compressed block.

[0126] In this application, for image blocks / independent blocks near edges, their first and second neighboring blocks are incomplete. In this case, they are completed by copying the first and second neighboring blocks at relatively similar positions. For example, if the first neighboring block above the independent block does not exist, the first neighboring block below is copied and used as the block above.

[0127] In this application, the processing accuracy of the adjacent image blocks is greatly improved by completion.

[0128] In this application, the similarity relationship between local regions is captured through an adaptive adjustment module, thereby enhancing feature representation.

[0129] In this application, through updating, on the one hand, the features in the historical environment model are enhanced, and on the other hand, the edge sharpness of the historical environment model is blunted, thereby increasing the difficulty of recognizing the historical environment model, thereby adding training difficulty to the first training and second training processes of the entire strategy model, so as to reversely improve the accuracy of the trained strategy model (the real environment model has higher clarity than the historical environment model).

[0130] An embodiment of the present application provides a robotic arm path planning device based on embodied intelligence, which is used to execute the robotic arm path planning method based on embodied intelligence described above in the present application. The robotic arm path planning device based on embodied intelligence is described in detail below.

[0131] like Figure 2 As shown, the robot arm path planning device based on embodied intelligence includes:

[0132] Data acquisition module 101, which is used to obtain the three-dimensional model of the environment and planning data;

[0133] A strategy training module 102 is configured to obtain a trained strategy model; the strategy model is obtained based on expert trajectories through a first training step using behavior cloning and a second training step using adversarial imitation learning;

[0134] A path planning module 103 is used to generate a planned path for the robotic arm by executing actions step by step through a strategy model based on the three-dimensional model of the environment and planning data;

[0135] The robot arm control module 104 is used to control the robot arm to execute the planned path.

[0136] In one embodiment, the policy training module 102 is further configured to:

[0137] Obtain historical environment models and expert trajectories; build a policy model; based on behavior cloning, perform the first training of the policy model through expert trajectories; based on adversarial imitation learning, perform the second training of the policy model through expert trajectories.

[0138] In one embodiment, the policy training module 102 is further configured to:

[0139] Based on the expert trajectory and the historical environment model, the current state of the robot arm is generated; the current state of the robot arm is input into the policy model to obtain the predicted action of the robot arm; the expert action of the robot arm is determined based on the expert trajectory; based on the expert action and the predicted action, the overall loss is calculated; and based on the overall loss, the policy model is iterated until convergence.

[0140] In one embodiment, the current state of the robotic arm includes joint angles, joint velocities, end effector positions, end effector postures, and environmental information of the robotic arm.

[0141] In one embodiment, the predicted motion of the robotic arm includes a change in joint angle, joint acceleration, end effector velocity, and end effector direction vector.

[0142] In one embodiment, the policy training module 102 is further configured to:

[0143] The first trained policy model is used as the initial parameters of the generator; expert data and a historical environment model are obtained; based on the generator, an action sequence corresponding to the current state of the robot arm is generated, and a generation trajectory is determined; the generator parameters are fixed, and the discriminator is trained based on the expert data and the generation trajectory until convergence; the discriminator parameters are fixed, and the generator is trained based on the expert data and the generation trajectory until convergence; the generator and the discriminator are trained alternately until preset conditions are met.

[0144] In one embodiment, the data acquisition module 101 is further configured to:

[0145] Build a digital twin space for the robotic arm; generate a historical environment model and corresponding expert trajectory in the digital twin space.

[0146] The robotic arm path planning device based on embodied intelligence provided in the above-mentioned embodiment of the present application has a corresponding relationship with the robotic arm path planning method based on embodied intelligence provided in the embodiment of the present application. Therefore, the specific content in the system has a corresponding relationship with the robotic arm path planning method based on embodied intelligence. The specific content can refer to the records in the robotic arm path planning method based on embodied intelligence, and will not be repeated in this application.

[0147] The robotic arm path planning device based on embodied intelligence provided in the above-mentioned embodiments of the present application and the robotic arm path planning method based on embodied intelligence provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0148] The above describes the internal functions and structure of the robot path planning device based on embodied intelligence, such as Figure 3 As shown, in practice, the robot arm path planning device based on embodied intelligence can be implemented as an electronic device, including: a memory 301 and a processor 303.

[0149] The memory 301 may be configured to store programs.

[0150] In addition, the memory 301 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0151] The memory 301 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The processor 303 is coupled to the memory 301 and is used to execute the program in the memory 301 to:

[0152] Obtain environmental 3D models and planning data;

[0153] Obtaining a trained policy model; the policy model is obtained based on expert trajectories through a first training performed by behavior cloning and a second training performed by adversarial imitation learning;

[0154] Based on the 3D model of the environment and planning data, the strategy model is used to execute actions step by step to generate the planned path of the robot arm.

[0155] Control the robotic arm to execute the planned path.

[0156] In one embodiment, the processor 303 is further configured to:

[0157] Obtain historical environment models and expert trajectories; build a policy model; based on behavior cloning, perform the first training of the policy model through expert trajectories; based on adversarial imitation learning, perform the second training of the policy model through expert trajectories.

[0158] In one embodiment, the processor 303 is further configured to:

[0159] Based on the expert trajectory and the historical environment model, the current state of the robot arm is generated; the current state of the robot arm is input into the policy model to obtain the predicted action of the robot arm; the expert action of the robot arm is determined based on the expert trajectory; based on the expert action and the predicted action, the overall loss is calculated; and based on the overall loss, the policy model is iterated until convergence.

[0160] In one embodiment, the current state of the robotic arm includes joint angles, joint velocities, end effector positions, end effector postures, and environmental information of the robotic arm.

[0161] In one embodiment, the predicted motion of the robotic arm includes a change in joint angle, joint acceleration, end effector velocity, and end effector direction vector.

[0162] In one embodiment, the processor 303 is further configured to:

[0163] The first trained policy model is used as the initial parameters of the generator; expert data and a historical environment model are obtained; based on the generator, an action sequence corresponding to the current state of the robot arm is generated, and a generation trajectory is determined; the generator parameters are fixed, and the discriminator is trained based on the expert data and the generation trajectory until convergence; the discriminator parameters are fixed, and the generator is trained based on the expert data and the generation trajectory until convergence; the generator and the discriminator are trained alternately until preset conditions are met.

[0164] In one embodiment, the processor 303 is further configured to:

[0165] Build a digital twin space for the robotic arm; generate a historical environment model and corresponding expert trajectory in the digital twin space.

[0166] In this application, Figure 3 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 3 Components shown.

[0167] The electronic device provided in this embodiment is based on the same inventive concept as the robotic arm path planning method based on embodied intelligence provided in the embodiment of the present application, and has the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0168] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CDROM, optical storage, etc.) that contain computer-usable program code.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0171] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-permanent storage in a computer-readable medium, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory. Memory is an example of a computer-readable medium.

[0172] The present application also provides a computer-readable storage medium corresponding to the robotic arm path planning method based on embodied intelligence provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the interactive image analysis auxiliary method for 3D aerial imaging provided in any of the aforementioned embodiments.

[0173] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CDROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0174] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the interactive image analysis auxiliary method for 3D aerial imaging provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0175] It should be noted that, in the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.

[0176] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0177] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included within the scope of the claims of the present application.

Claims

1. A robotic arm path planning method based on embodied intelligence, characterized in that: include: Obtain environmental 3D models and planning data; Get the trained policy model; The policy model is based on expert trajectories and is obtained through a first training by behavior cloning and a second training by adversarial imitation learning; Based on the 3D model of the environment and planning data, the strategy model is used to execute actions step by step to generate the planned path of the robot arm. Control the robotic arm to execute the planned path.

2. The robot arm path planning method based on embodied intelligence according to claim 1, characterized in that: The training process of the strategy model includes: Access to historical environmental models and expert trajectories; Build a strategy model; Based on behavior cloning, the policy model is first trained through expert trajectories; Based on adversarial imitation learning, the policy model is trained for the second time through expert trajectories.

3. The method for robot arm path planning based on embodied intelligence according to claim 2, characterized in that: The first training of the strategy model based on behavior cloning and expert trajectory includes: Generate the current state of the robot arm based on the expert trajectory and historical environment model; Input the current state of the robot arm into the policy model to obtain the predicted action of the robot arm; Determine the expert action of the robot arm based on the expert trajectory; Calculate the overall loss based on the expert actions and the predicted actions; The policy model is iterated based on the overall loss until convergence.

4. The method for robot arm path planning based on embodied intelligence according to claim 3, characterized in that: The current state of the robotic arm includes the joint angles, joint velocities, end effector positions, end effector postures, and environmental information of the robotic arm.

5. The method for robotic arm path planning based on embodied intelligence according to claim 3, characterized in that: The predicted motion of the robotic arm includes the change in joint angle, joint acceleration, end effector velocity and end effector direction vector.

6. The method for robot arm path planning based on embodied intelligence according to any one of claims 2 to 5, characterized in that: The second training of the strategy model based on adversarial imitation learning through expert trajectory includes: Use the first trained policy model as the initial parameters of the generator; Access to expert data and historical environmental models; Based on the generator, generate an action sequence corresponding to the current state of the robot arm and determine the generated trajectory; Fixed the generator parameters and trained the discriminator based on the expert data and generated trajectories until convergence; Fixed the discriminator parameters and trained the generator based on the expert data and generated trajectory until convergence; The generator and the discriminator are trained alternately until the preset conditions are met.

7. The method for robot arm path planning based on embodied intelligence according to any one of claims 1 to 5, characterized in that: The acquisition of the historical environment model and expert trajectory includes: Build a digital twin space for the robotic arm; Generate historical environment models and corresponding expert trajectories in the digital twin space.

8. A robotic arm path planning device based on embodied intelligence, characterized in that: include: A data acquisition module, which is used to obtain a three-dimensional model of the environment and planning data; A strategy training module, which is used to obtain a trained strategy model; the strategy model is obtained based on expert trajectories through a first training method using behavior cloning and a second training method using adversarial imitation learning; The path planning module is used to generate the planned path of the robot arm by gradually executing actions through the policy model based on the 3D model of the environment and planning data; A robotic arm control module is used to control the robotic arm to execute the planned path.

9. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program to: Obtain environmental 3D models and planning data; Obtaining a trained policy model; the policy model is obtained based on expert trajectories through a first training step using behavioral cloning and a second training step using adversarial imitation learning; Based on the 3D model of the environment and planning data, the strategy model is used to execute actions step by step to generate the planned path of the robot arm. Control the robotic arm to execute the planned path.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the robotic arm path planning method based on embodied intelligence as described in any one of claims 1 to 7.

Images