Multi-degree-of-freedom double-arm robot filling path planning method, medium and equipment

Abstract

According to the multi-degree-of-freedom double-arm robot filling path planning method, the 2D image of the surface of the current workpiece is obtained and input into the trained recognition network model to obtain the marked image, so that the marked image is obtained under the acquisition equipment coordinate system and the robot base coordinate system; according to the central coordinates of the filling port area and the exhaust port area and the normal vectors of the central coordinates, the corresponding poses of the tail end of the robot at any joint angle under the robot base coordinate system and the joint space paths between the poses of the central coordinates under the robot base coordinate system are obtained; coordinates of each joint space path point in the joint space path and the corresponding movement speed are obtained and substituted into a preset model, the positions and speeds of all joints are obtained when the robot is located at the joint space path points, and path planning is completed. The problems that in the prior art, when path planning is conducted on a multi-degree-of-freedom double-arm robot, the redundancy of the robot cannot be fully utilized, efficiency is too low, and intelligence is insufficient are solved.

Description

Technical Field

[0001] The present invention relates to the field of path planning, and in particular to a method, medium and equipment for planning a filling path for a multi-degree-of-freedom dual-arm robot. Background Art

[0002] Robots are a type of high-tech automated production equipment developed in the past thirty years. Manipulators are an important branch of robots. They can be programmed to complete various expected tasks. They have the advantages of both humans and machines in terms of structure and performance, especially in terms of human intelligence and adaptability, and the accuracy of machine operations and the ability to complete tasks in various environments.

[0003] Dual-arm robots are a new bright spot in the industrial robot market. Dual-arm robots have become a solution to reduce labor costs, improve production efficiency and make up for labor shortages. The collaborative operation of multiple robots is an inevitable requirement for the development of the manufacturing industry, and dual-arm robots are a new type of robot developed to meet this requirement. Compared with single-arm robots, it can greatly enhance the adaptability of robots to complex assembly tasks, while improving the efficiency of workspace utilization. The redundancy and coordinated operation of dual-arm robots greatly improve their operating performance, making them more practical than two-arm coordinated robots. Their high self-discipline and learning ability can adapt to many complex environments, making them widely used in industrial production, hazardous handling, national defense, aerospace, etc. However, since the redundancy of dual-arm robots is much higher than that of single-arm robots, the complexity of their path planning methods is also much higher than that of single-arm robots.

[0004] Therefore, how to improve the existing technology for path planning of dual-arm robots with multiple degrees of freedom, the inability to fully utilize the robot's redundancy, and the low efficiency and insufficient intelligence are technical issues that need to be urgently addressed in this field. Summary of the invention

[0005] Based on this, the purpose of this application is to provide a multi-degree-of-freedom dual-arm robot filling path planning method, medium and equipment to solve at least one technical problem mentioned in the above background technology.

[0006] In a first aspect, the present application provides a multi-degree-of-freedom dual-arm robot filling path planning method, comprising:

[0007] Build and train a recognition network model that takes 2D images as input and annotated images as output;

[0008] Control the robot to move in sequence according to the set step length, obtain the 2D image of the current workpiece surface, and input the trained recognition network model to obtain the annotated image; the annotated image includes the filling port area and the exhaust port area;

[0009] Obtain the current workpiece surface point cloud with the current robot end position as the center and the set distance as the radius, and obtain the center coordinates of the filling port area and the exhaust port area and the normal vector of each center coordinate in the acquisition device coordinate system according to the current workpiece surface point cloud;

[0010] Obtain the center coordinates of the filling port area and the exhaust port area and the normal vectors of each center coordinate in the robot base coordinate system;

[0011] According to the current coordinates in the robot base coordinate system and the center coordinates and corresponding normal vectors of the filling port area and the exhaust port area, the position and posture of each center coordinate in the robot base coordinate system are obtained;

[0012] Get any joint angle of the robot in space and obtain the corresponding position and posture of the robot end in the robot base coordinate system;

[0013] Obtain the joint space path between the corresponding pose of any joint angle of the robot end in the robot base coordinate system and the pose of each center coordinate in the robot base coordinate system, and obtain the coordinates of each joint space path point in the joint space path and the corresponding motion speed;

[0014] Substitute the coordinates of each joint space path point in the joint space path and the corresponding movement speed into the preset model to obtain the position and speed of all joints when the robot is located at each joint space path point, and complete the path planning.

[0015] Furthermore, the specific structure of the network model is identified, including:

[0016] Input layer, used to input the acquired 2D image;

[0017] The convolution layer is connected to the input layer and is used to receive the input 2D image and extract the features of the 2D image to obtain the initial feature map;

[0018] The pooling layer is connected to the convolutional layer and is used to downsample the initial feature map to obtain an optimized initial feature map;

[0019] The activation layer is connected to the pooling layer and is used to add nonlinear activation functions after the convolution and fully connected layers to enhance the nonlinear expression ability of the network;

[0020] The fully connected layer is connected to the activation layer to fuse and process the extracted features, annotate the feature parts, and generate annotated images;

[0021] The output layer, connected to the fully connected layer, is used to output the generated labeled image.

[0022] Furthermore, the steps of constructing and training a recognition network model that takes a 2D image as input and annotated images as output include:

[0023] Constructing an initial recognition network model;

[0024] Selecting several 2D images including the filling port, and annotating the area covered by the filling port to obtain an annotated image;

[0025] Rotate each annotated image by a predetermined angle several times to obtain subsets of annotated images at different angles;

[0026] Construct a training data set based on the annotated images and annotated image subsets;

[0027] The training data set is input into the initial recognition network model to optimize the parameters of the initial recognition network model and obtain the final recognition network model.

[0028] Further, the robot is controlled to move sequentially according to the set step length, a 2D image of the current workpiece surface is obtained, and the trained recognition network model is input to obtain the labeled image step, including:

[0029] S21: Control the robot to move according to a first set step length to obtain a 2D image of the current workpiece surface;

[0030] S22: Input the 2D image of the current workpiece surface into the recognition network model to determine whether the filling port area appears in the 2D image. If so, mark the filling port area to obtain an initial marked image and execute step S23. If not, return to step S21.

[0031] S23: Obtain the center point of the filling port area and the initial annotated image, and determine whether the distance between each center point is greater than a set threshold. If so, move toward the center point of the filling port area according to a second set step length until the distance between each center point is no greater than the set threshold. If not, execute step S24;

[0032] S24: Acquire a 2D image of the current workpiece surface, which is the finalized annotation image.

[0033] Furthermore, the first step length and the second step length are specifically:

[0034] The first step length is the diameter of the scanning range of the scanning device; the second step length is the radius of the prior filling port area; the prior filling port area is pre-marked by a technician in this field and / or obtained based on the workpiece model and / or pre-set by a technician in this field.

[0035] Furthermore, the step of obtaining the center coordinates of the filling port area and the exhaust port area and the normal vector of each center coordinate in the acquisition device coordinate system includes:

[0036] Obtain a priori regions of interest on the workpiece surface, and determine point clouds of the filling port region and the exhaust port region;

[0037] Obtain and calculate the average value of the coordinates of all points in the point cloud of each area in the acquisition device coordinate system to obtain the coordinates of the center point of each area;

[0038] Get the normal vector corresponding to the coordinates of the center point of each area.

[0039] Furthermore, the step of obtaining the position and posture of each center coordinate in the robot base coordinate system includes:

[0040] Construct a right-handed coordinate system with the center coordinate as the origin;

[0041] Get the x-axis direction vector and the y-axis direction vector in the right-handed coordinate system;

[0042] Construct a rotation matrix between the right-hand coordinate system and the robot base coordinate system according to the x-axis direction vector and the y-axis direction vector;

[0043] The angle of the center coordinate in the robot base coordinate system is obtained according to the rotation matrix, and the position and posture of the center coordinate in the robot base coordinate system are obtained.

[0044] Furthermore, the steps of obtaining the corresponding posture of any joint angle of the robot end in the robot base coordinate system and the joint space path between the postures of each center coordinate in the robot base coordinate system, and obtaining the coordinates of each joint space path point in the joint space path and the corresponding movement speed include:

[0045] Generate a smooth joint angle trajectory between any pose of the robot end in the robot base coordinate system and the pose of each center coordinate in the robot base coordinate system;

[0046] Obtain the corresponding joint angle trajectory of each joint at each time point, and obtain the corresponding robot end position and posture according to the corresponding joint angle trajectory;

[0047] The joint space path points are obtained by combining the joint angles corresponding to each time point, the robot end posture corresponding to each time point, the joint angular velocity corresponding to each time point, and the joint angular acceleration corresponding to each time point.

[0048] In a second aspect, the present application further provides a computer storage medium storing an executable program code; the executable program code is used to execute the multi-degree-of-freedom dual-arm robot filling path planning method described in any one of the first aspects.

[0049] In a third aspect, the present application also provides a terminal device comprising a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute the multi-degree-of-freedom dual-arm robot filling path planning method described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The present invention is a flowchart of a method for planning a filling path for a multi-degree-of-freedom dual-arm robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., then the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly. In addition, if the embodiments of the present invention involve descriptions of "first, second", "S1, S2", "step one, step two", etc., then such descriptions are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features or indicating the execution order of the method, etc. Those skilled in the art can understand that anything that does not violate the gist of the invention under the technical concept of the invention should be included in the protection scope of the present invention.

[0053] like Figure 1 As shown, the present invention provides a multi-degree-of-freedom dual-arm robot filling path planning method, comprising:

[0054] S1: Build and train a recognition network model that takes 2D images as input and annotated images as output;

[0055] Specifically, the recognition network model can use any image recognition model in the prior art to construct an initial recognition network model; then, by batch collecting 2D images of different workpieces of the same model, the filling ports in the 2D images are marked to construct a training data set: then, according to the training data set, the initial recognition network model is input to train the various parameters of the initial recognition network model to obtain the trained recognition network model. It is worth noting that this step is a preparatory work, which can be constructed and trained in advance, or it can be carried out simultaneously with step S2, as long as it is constructed and trained before the recognition network model is needed for prediction in step S3.

[0056] Preferably, the recognition network model can optionally but not be limited to recognizing the 2D image based on the yolo model, and marking the filler port area therein. For the application of the present invention, since the filler port and the exhaust port always appear side by side at the same time, the recognition accuracy of the filler port area can be improved based on similar features. Therefore, the labeled image preferably includes the filler port area and the exhaust port area.

[0057] More preferably, the specific structure of the recognition network model may include, but is not limited to:

[0058] Input layer, used to input the acquired 2D image;

[0059] The convolutional layer is connected to the input layer and is used to receive the input 2D image and extract the features of the 2D image to obtain the initial feature map.

[0060] The pooling layer is connected to the convolution layer to downsample the initial feature map, reduce the feature dimension, improve the computational efficiency, and obtain the optimized initial feature map;

[0061] The activation layer is connected to the pooling layer and is used to add nonlinear activation functions after the convolution and fully connected layers to enhance the nonlinear expression ability of the network. The nonlinear activation function can be selected but is not limited to common functions such as ReLU and Sigmoid.

[0062] The fully connected layer is connected to the activation layer to fuse and process the extracted features, annotate the feature parts, and generate annotated images;

[0063] The output layer, connected to the fully connected layer, is used to output the generated labeled image.

[0064] Further preferably, the construction and training process of the recognition network model may optionally include but is not limited to:

[0065] S11: construct an initial recognition network model; preferably based on the YOLO model;

[0066] S12: selecting a plurality of 2D images including the filling port, and marking the area covered by the filling port to obtain a marked image;

[0067] S13: rotating each annotated image by a predetermined angle several times to obtain annotated image subsets at different angles;

[0068] S14: construct a training data set based on the annotated images and the annotated image subsets;

[0069] S15: Inputting the training data set into the initial recognition network model to optimize various parameters of the initial recognition network model to obtain the final recognition network model.

[0070] Specifically, the number of rotations, angles, and directions can be arbitrarily set by those skilled in the art.

[0071] Preferably, the annotated image is rotated 60 degrees each time for 5 times to obtain five new subsets at different angles; then the annotated image and the annotated image subsets together constitute the training data set, or a portion of each set is selected to constitute the training data set together.

[0072] For example, a technician in this field can mark the edge of the filling port area in the selected annotation image, and the annotation range is the filling port area, so as to obtain the annotated image. Each annotated image is rotated clockwise to obtain annotated image subsets whose angles differ from the original annotated image by 60°, 120°, 180°, 240°, and 300°, respectively.

[0073] More preferably, step S16 may optionally include but is not limited to:

[0074] The technicians in this field set relevant loss functions, optimization algorithms, etc. to obtain the required recognition network model, input the training data set into the recognition network model for continuous iteration, and finally obtain a recognition network model with 2D images as input and labeled images as output; specifically, optimization algorithms including stochastic gradient descent, Adam, RMSprop, etc. are used to update the recognition network parameters to minimize the loss function.

[0075] S2: Control the robot to move sequentially according to the set step length, obtain the 2D image of the current workpiece surface, and input the trained recognition network model to obtain the labeled image; it is worth noting that since the filling port and the exhaust port always appear side by side at the same time, the labeled image contains images of the filling port area and the exhaust port area.

[0076] Specifically, it is optional but not limited to controlling the robot to use the image acquisition device to scan the current workpiece surface with a set step length each time it moves until the entire workpiece surface is scanned to complete the acquisition of the 2D image, and inputting the 2D image acquired each time into the recognition network model. If the 2D image contains a filling port, the recognition network model marks the filling port area to obtain a marked image; setting the step length is arbitrarily set by technical personnel in this field based on factors such as the surface area of ​​the workpiece, the scanning range of the image acquisition device, the impact of the on-site environment, and process conditions.

[0077] Preferably, the step of controlling the robot to move sequentially according to the set step length, acquiring the 2D image of the current workpiece surface, and inputting the trained recognition network model to obtain the labeled image may be optional but not limited to:

[0078] S21: Control the robot to move according to a first set step length to obtain a 2D image of the current workpiece surface;

[0079] S22: Input the 2D image of the current workpiece surface into the recognition network model to determine whether the filling port area appears in the 2D image. If so, mark the filling port area to obtain an initial marked image and execute step S23. If not, return to step S21.

[0080] S23: Obtain the center point of the filling port area and the initial annotated image, and determine whether the distance between each center point is greater than a set threshold. If so, move toward the center point of the filling port area according to a second set step length until the distance between each center point is no greater than the set threshold. If not, execute step S24;

[0081] S24: Acquire a 2D image of the current workpiece surface, which is the finalized annotation image.

[0082] Preferably, the first step length, the second step length, and the set threshold are arbitrarily set by those skilled in the art.

[0083] More preferably, the first step length is the diameter of the scanning range of the scanning device, and the second step length is the radius of the prior filling port area; the prior filling port area can be optionally but not limited to being pre-marked by technical personnel in this field and / or obtained based on the workpiece model and / or pre-set by technical personnel in this field.

[0084] For example, the first step length and the second step length are set. The robot end carries an image acquisition device to obtain the first 2D image at the lower left corner of the workpiece surface, and then inputs the recognition network model to determine whether the filling port area appears in the 2D image. If it appears, the filling port area is marked to obtain the initial marked image, and then the center point of the filling port area and the initial marked image is obtained to determine whether the distance between the center points is greater than the set threshold. If it is greater, move toward the center point of the filling port area according to the second set step length until the distance between the center points is less than the set threshold. If it is not greater, obtain the 2D image of the current workpiece surface, which is the final marked image; if it does not appear, move to the right side of the workpiece surface by the first step length, continue to obtain 2D images and input the recognition network model. When the robot moves to the rightmost side of the workpiece surface, it moves upward by the first step length, and then moves to the left side of the workpiece surface by the first step length in sequence until the workpiece surface is traversed, and a 2D image of the workpiece surface is obtained each time it moves.

[0085] S3: Obtain the current workpiece surface point cloud with the current robot end position as the center and the set distance as the radius, and obtain the center coordinates of the filling port area and the exhaust port area and the normal vector of each center coordinate in the acquisition device coordinate system according to the current workpiece surface point cloud;

[0086] Specifically, a distance is set as the radius, and the current workpiece surface point cloud within the set distance as the radius is obtained. Then, the point cloud of the filling port area and the point cloud of the exhaust port area are obtained based on the obtained workpiece surface point cloud. The center coordinates of the filling port area and the center coordinates of the exhaust port area are obtained based on the point clouds of each area. Finally, the normal vector corresponding to each center point is calculated based on the specific coordinates of the midpoint of each area. It is worth noting that the radius distance must make the filling port area and the exhaust port area included in the scanning range of the acquisition device.

[0087] Preferably, the step of obtaining the center coordinates of the filling port area and the exhaust port area and the normal vectors of each center coordinate in the acquisition device coordinate system may optionally include but is not limited to:

[0088] S31: Acquire a priori regions of interest on the workpiece surface, and determine a point cloud of a filling port region and a point cloud of an exhaust port region;

[0089] S32: respectively obtain and calculate the average value of the coordinates of all points in the point cloud of each region in the acquisition device coordinate system to obtain the coordinates of the center point of each region;

[0090] S33: Obtain the normal vector corresponding to the coordinates of the center point of each area.

[0091] Specifically, it is optional but not limited to pre-marking by a technician in this field or obtaining a priori areas of interest on the workpiece surface based on the workpiece model or pre-setting by a technician in this field, including the filling port area and the exhaust port area, and then preferably filtering the workpiece surface point cloud by a straight-through filtering method to determine the filling port area point cloud and the exhaust port area point cloud, obtaining the values ​​of all points in the regional point cloud on the x-axis and y-axis respectively, and dividing them by the number of corresponding regional point clouds to obtain the center point coordinates of the corresponding regional point cloud, and finally calculating the normal vector corresponding to each center point coordinate by a point cloud normal vector estimation method; point cloud normal vector estimation methods include commonly used algorithms such as the nearest neighbor method (Nearest Neighbor Method), the least squares plane fitting method (Least Squares Plane Fitting Method), and the normal estimation method (Normal Estimation Method).

[0092] S4: Obtain the center coordinates of the filling port area and the exhaust port area and the normal vectors of each center coordinate in the robot base coordinate system;

[0093] Specifically, it is optional but not limited to converting the center coordinates of the filling port area and the exhaust port area and the normal vectors of each center coordinate in the collection device coordinate system in step S3 to the robot base coordinate system based on the prior hand-eye calibration matrix between the acquisition device coordinate system and the robot base coordinate system to obtain the center coordinates of the filling port area and the exhaust port area and the normal vectors of each center coordinate in the robot base coordinate system; it is worth noting that the prior hand-eye calibration matrix can be pre-set according to the relationship between the image acquisition device and the robot base, and can also be calculated by a technician in this field after fixing the image acquisition device to the end of the robot and before the coordinate conversion step.

[0094] S5: according to the current coordinates in the robot base coordinate system and the center coordinates and corresponding normal vectors of the filling port area and the exhaust port area, the position and posture of each center coordinate in the robot base coordinate system are obtained;

[0095] For example, taking the center coordinates (x, y, z) and the corresponding normal vector (nx, ny, nz) as an example, the specific steps for obtaining the position and posture of the center coordinates in the robot base coordinate system may include but are not limited to:

[0096] S51: Construct a right-handed coordinate system with the center coordinate as the origin;

[0097] S52: Obtaining the x-axis direction vector and the y-axis direction vector in the right-handed coordinate system;

[0098] S53: constructing a rotation matrix between the right-hand coordinate system and the robot base coordinate system according to the x-axis direction vector and the y-axis direction vector;

[0099] S54: Obtain the angle of the center coordinate in the robot base coordinate system according to the rotation matrix, and obtain the position and posture of the center coordinate in the robot base coordinate system.

[0100] Specifically, it is optional but not limited to constructing a right-handed coordinate system with the center coordinate as the origin, and the normal vector (nx, ny, nz) as the unit vector of the z-axis, and then preferably constructing a rotation matrix with the X-axis direction vector as the first column and the Y-axis direction vector as the second column after obtaining the X-axis direction vector by cross product calculation, and preferably calculating the rotation angle of the center coordinate in the robot base coordinate system by inverse trigonometric functions, and combining the position of the center coordinate to obtain the position of the center coordinate in the robot base coordinate system.

[0101] S6: Obtain any joint angle of the robot in space and obtain the corresponding position and posture of the robot end in the robot base coordinate system;

[0102] Specifically, it is optional but not limited to obtaining the joint angles by reading the feedback signals of the servo motors carried by the robot itself, and substituting the joint angles of the robot into the existing forward kinematics equations to obtain the corresponding position and posture of the robot end in the robot base coordinate system.

[0103] S7: Obtain the joint space path between the corresponding posture of any joint angle of the robot end in the robot base coordinate system and the posture of each center coordinate in the robot base coordinate system, and obtain the coordinates of each joint space path point in the joint space path and the corresponding movement speed;

[0104] Specifically, it is optional but not limited to generating a smooth joint angle trajectory between the corresponding posture of any joint angle of the robot end in the robot base coordinate system and the posture of each center coordinate in the robot base coordinate system according to the joint space interpolation method, and setting the boundary conditions of the joint angle and angular velocity to calculate the angle change trajectory of each joint at each time point, and then calculating the corresponding joint angle in the angle change trajectory at each time point, and substituting the joint angle into the existing forward kinematics equation to obtain the corresponding robot end posture, and finally calculating the angular velocity of each joint, and combining the joint angle corresponding to each time point, the robot end posture corresponding to each time point, and the joint angular velocity corresponding to each time point to obtain the joint space path point where the end posture corresponding to any joint angle of the robot moves to each center coordinate posture.

[0105] Preferably, the steps of obtaining the corresponding posture of any joint angle of the robot end in the robot base coordinate system and the joint space path between the postures of each center coordinate in the robot base coordinate system, and obtaining the coordinates of each joint space path point in the joint space path and the corresponding motion speed may optionally include but are not limited to:

[0106] S71: generating a smooth joint angle trajectory between any posture of the robot end in the robot base coordinate system and the posture of each center coordinate in the robot base coordinate system;

[0107] Specifically, it is optional but not limited to generating a smooth joint angle trajectory between any posture of the robot end in the robot base coordinate system and the posture of each center coordinate in the robot base coordinate system based on the joint space interpolation method; the joint space interpolation method is optional but not limited to common methods including linear interpolation, polynomial interpolation, spline interpolation, etc.

[0108] S72: Obtain the corresponding joint angle trajectory of each joint at each time point, and obtain the corresponding robot end position and posture according to the corresponding joint angle trajectory;

[0109] Specifically, it is optional but not limited to calculating the angle change of each joint in the time series according to the selected interpolation method, obtaining the angle trajectory of each joint changing with time, and then substituting the calculated joint angle trajectory into the forward kinematics equation to obtain the corresponding robot end position.

[0110] S73: The joint space path points are obtained by comprehensively calculating the joint angles corresponding to each time point, the robot end position corresponding to each time point, the joint angular velocity corresponding to each time point, and the joint angular acceleration corresponding to each time point.

[0111] Specifically, it is optional but not limited to obtaining the corresponding joint space path points according to the joint angles, robot end postures, and joint angular velocities corresponding to each time point.

[0112] S8: Substitute the coordinates of each joint space path point in the joint space path and the corresponding movement speed into the preset model to obtain the position and speed of all joints when the robot is located at each joint space path point, and complete the path planning.

[0113] Specifically, according to the inverse kinematics model of the existing robot system, all joint space path points are input into the existing inverse kinematics model, and the corresponding joint position and angular velocity of the robot at each time point can be obtained, thereby completing the path planning.

[0114] In this embodiment, a multi-degree-of-freedom dual-arm robot filling path planning method of the present invention is given, by constructing and training a recognition network model with a 2D image as input and a labeled image as output, and then controlling the robot to move sequentially according to a set step size, acquiring a 2D image of the current workpiece surface, and inputting the trained recognition network model to obtain a labeled image, thereby improving the quality of the acquired image and placing the filling area at the center of the image, so that the subsequent steps can directly collect point clouds, improve processing efficiency and intelligence, and then obtain the current workpiece surface point cloud with the current robot end position as the center and the set distance as the radius, and obtain the center coordinates of the filling port area and the exhaust port area in the acquisition device coordinate system and the normal vector of each center coordinate according to the current workpiece surface point cloud, and obtain the center coordinates of the filling port area and the exhaust port area in the robot base coordinate system. and the normal vector of each center coordinate, so as to obtain the position and posture of each center coordinate in the robot base coordinate system according to the center coordinates and corresponding normal vectors of the current coordinates and the filling port area and the exhaust port area in the robot base coordinate system, and then obtain any joint angle of the robot to obtain the corresponding position and posture of the robot end in the robot base coordinate system, so as to obtain the joint space path between the corresponding position and posture of any joint angle of the robot end in the robot base coordinate system and the position and posture of each center coordinate in the robot base coordinate system, and obtain the coordinates of each joint space path point in the joint space path and the corresponding movement speed, substitute the coordinates of each joint space path point in the joint space path and the corresponding movement speed into the preset model, and obtain the position and speed of all joints of the robot when the robot is located at each joint space path point, so as to make full use of the robot redundancy and complete the path planning. The existing technology solves the problems that the robot redundancy cannot be fully utilized when planning the path of a dual-arm robot with multiple degrees of freedom, and the efficiency is too low and the intelligence is insufficient.

[0115] On the other hand, the present invention also provides a computer storage medium storing an executable program code; the executable program code is used to execute any of the above-mentioned multi-degree-of-freedom dual-arm robot filling path planning methods.

[0116] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute any of the above-mentioned multi-degree-of-freedom dual-arm robot filling path planning methods.

[0117] Exemplarily, the program code may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the program code in the terminal device.

[0118] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the terminal device may also include an input / output device, a network access device, a bus, etc.

[0119] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0120] The memory may be an internal storage unit of the terminal device, such as a hard disk or a memory. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Further, the memory may also include both an internal storage unit of the terminal device and an external storage device. The memory is used to store the program code and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0121] The above-mentioned computer storage medium and terminal device are created based on the multi-degree-of-freedom dual-arm robot filling path planning method. Their technical functions and beneficial effects are no longer repeated here. The various technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A multi-degree-of-freedom dual-arm robot filling path planning method, characterized in that: include: Build and train a recognition network model that takes 2D images as input and annotated images as output; Control the robot to move in sequence according to the set step length, obtain the 2D image of the current workpiece surface, and input the trained recognition network model to obtain the annotated image; the annotated image includes the filling port area and the exhaust port area; Obtain the current workpiece surface point cloud with the current robot end position as the center and the set distance as the radius, and obtain the center coordinates of the filling port area and the exhaust port area and the normal vector of each center coordinate in the acquisition device coordinate system according to the current workpiece surface point cloud; Obtain the center coordinates of the filling port area and the exhaust port area and the normal vectors of each center coordinate in the robot base coordinate system; According to the current coordinates in the robot base coordinate system and the center coordinates and corresponding normal vectors of the filling port area and the exhaust port area, the position and posture of each center coordinate in the robot base coordinate system are obtained; Get any joint angle of the robot in space and obtain the corresponding position and posture of the robot end in the robot base coordinate system; Obtain the joint space path between the corresponding pose of any joint angle of the robot end in the robot base coordinate system and the pose of each center coordinate in the robot base coordinate system, and obtain the coordinates of each joint space path point in the joint space path and the corresponding motion speed; Substitute the coordinates of each joint space path point in the joint space path and the corresponding movement speed into the preset model to obtain the position and speed of all joints when the robot is located at each joint space path point, and complete the path planning.

2. The method according to claim 1, characterized in that Identify the specific structure of the network model, including: Input layer, used to input the acquired 2D image; The convolution layer is connected to the input layer and is used to receive the input 2D image and extract the features of the 2D image to obtain the initial feature map; The pooling layer is connected to the convolutional layer and is used to downsample the initial feature map to obtain an optimized initial feature map; The activation layer is connected to the pooling layer and is used to add nonlinear activation functions after the convolution and fully connected layers to enhance the nonlinear expression ability of the network; The fully connected layer is connected to the activation layer to fuse and process the extracted features, annotate the feature parts, and generate annotated images; The output layer, connected to the fully connected layer, is used to output the generated labeled image.

3. The method according to claim 1, characterized in that The steps to build and train a recognition network model that takes 2D images as input and annotated images as output include: Constructing an initial recognition network model; Selecting several 2D images including the filling port, and annotating the area covered by the filling port to obtain an annotated image; Rotate each annotated image by a predetermined angle several times to obtain subsets of annotated images at different angles; Construct a training data set based on the annotated images and annotated image subsets; The training data set is input into the initial recognition network model to optimize the parameters of the initial recognition network model and obtain the final recognition network model.

4. The method according to claim 1, characterized in that: The steps of controlling the robot to move sequentially according to the set step length, obtaining the 2D image of the current workpiece surface, and inputting the trained recognition network model to obtain the labeled image include: S21: Control the robot to move according to a first set step length to obtain a 2D image of the current workpiece surface; S22: Input the 2D image of the current workpiece surface into the recognition network model to determine whether the filling port area appears in the 2D image. If so, mark the filling port area to obtain an initial marked image and execute step S23. If not, return to step S21. S23: Obtain the center point of the filling port area and the initial annotated image, and determine whether the distance between each center point is greater than a set threshold. If so, move toward the center point of the filling port area according to a second set step length until the distance between each center point is no greater than the set threshold. If not, execute step S24; S24: Acquire a 2D image of the current workpiece surface, which is the finalized annotation image.

5. The method according to claim 4, characterized in that The first step length and the second step length are specifically: The first step length is the diameter of the scanning range of the scanning device; the second step length is the radius of the prior filling port area; the prior filling port area is pre-marked by a technician in this field and / or obtained based on the workpiece model and / or pre-set by a technician in this field.

6. The method according to claim 1, characterized in that The step of obtaining the center coordinates of the filling port area and the exhaust port area and the normal vector of each center coordinate in the acquisition device coordinate system includes: Obtain a priori regions of interest on the workpiece surface, and determine point clouds of the filling port region and the exhaust port region; Obtain and calculate the average value of the coordinates of all points in the point cloud of each area in the acquisition device coordinate system to obtain the coordinates of the center point of each area; Get the normal vector corresponding to the coordinates of the center point of each area.

7. The method according to claim 1, characterized in that The steps of obtaining the position and posture of each center coordinate in the robot base coordinate system include: Construct a right-handed coordinate system with the center coordinate as the origin; Get the x-axis direction vector and the y-axis direction vector in the right-handed coordinate system; Construct a rotation matrix between the right-hand coordinate system and the robot base coordinate system according to the x-axis direction vector and the y-axis direction vector; The angle of the center coordinate in the robot base coordinate system is obtained according to the rotation matrix, and the position and posture of the center coordinate in the robot base coordinate system are obtained.

8. The method according to claim 1, characterized in that The steps of obtaining the corresponding pose of any joint angle of the robot end in the robot base coordinate system and the joint space path between the poses of each center coordinate in the robot base coordinate system, and obtaining the coordinates of each joint space path point in the joint space path and the corresponding motion speed include: Generate a smooth joint angle trajectory between any pose of the robot end in the robot base coordinate system and the pose of each center coordinate in the robot base coordinate system; Obtain the corresponding joint angle trajectory of each joint at each time point, and obtain the corresponding robot end position and posture according to the corresponding joint angle trajectory; The joint space path points are obtained by combining the joint angles corresponding to each time point, the robot end posture corresponding to each time point, the joint angular velocity corresponding to each time point, and the joint angular acceleration corresponding to each time point.

9. A computer storage medium, characterized in that An executable program code is stored; the executable program code is used to execute the multi-degree-of-freedom dual-arm robot filling path planning method described in any one of claims 1-8.

10. A terminal device, characterized in that: It comprises a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute the multi-degree-of-freedom dual-arm robot filling path planning method described in any one of claims 1-8.

Images