Automobile stamping part dynamic grabbing and boxing method and system based on flexible manipulator

Abstract

The invention discloses an automobile stamping part dynamic grabbing and boxing method and system based on a flexible manipulator, and the method comprises the steps: obtaining the three-dimensional point cloud data of a target stamping part on a conveyor belt in real time, extracting the key geometric features of the target stamping part, and determining the type, position and posture of the target stamping part; the type, the key geometric features and the posture of the target stamping part are input into a pre-trained contact point prediction model, and the contact point of the flexible manipulator is predicted; according to the position and posture of the target stamping part and the predicted contact point of the flexible manipulator, trajectory planning of the flexible manipulator is conducted, and a planned trajectory is obtained; and target stamping parts are grabbed and boxed according to the planned track of the flexible manipulator. The stamping part grabbing device can flexibly meet the grabbing requirements of stamping parts of different shapes, sizes and positions and has good adaptability to special-shaped stamping parts of complex shapes, and the grabbing precision and the boxing efficiency are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of robot control, and in particular relates to a method and system for dynamically grasping and packing automobile stamping parts based on a flexible manipulator. Background Art

[0002] In the automotive manufacturing industry, stamping parts are an important part of the car body and other key components, and usually need to go through multiple processes for processing and handling. On the production line, grabbing and boxing a large number of stamping parts from the conveyor belt is a key link, which directly affects the efficiency of the entire production line. However, traditional grabbing and boxing methods usually rely on rigid manipulators, which are easily limited by the complexity of part shapes and position changes, resulting in insufficient grasping accuracy, slow speed, and even possible damage to the surface of parts.

[0003] Existing automated packing systems usually use fixed-path manipulators or template matching-based grasping methods, which are often difficult to cope with complex shapes or diverse stamping parts. In addition, the stamping parts on the production line have different shapes and sizes, and their positions and postures on the conveyor belt are difficult to accurately predict, which seriously restricts the grasping efficiency of traditional manipulators. At the same time, with the continuous advancement of automobile manufacturing processes, higher requirements are placed on the automation level and packing efficiency of production lines. In order to meet these challenges, flexible manipulators have gradually become a hot topic in research and application. Flexible manipulators can adapt to the grasping needs of stamping parts of different shapes, sizes and positions to a certain extent through multi-joint structures and flexible end effectors. However, the flexible manipulator systems on the market currently still have deficiencies in dynamic grasping and real-time response, and need to be further optimized to improve the efficiency and adaptability of packing.

[0004] Therefore, there is an urgent need for a method that can efficiently and accurately grab various types of automotive stamping parts from the conveyor belt and quickly pack them to meet the needs of modern automotive manufacturing for automation and efficient production. Summary of the invention

[0005] In view of this, the present invention proposes a method and system for dynamically grasping and packing automobile stamping parts based on a flexible manipulator, which is used to solve the problem of poor packing efficiency of existing manipulators.

[0006] In a first aspect, the present invention discloses a method for dynamically grasping and packing automobile stamping parts based on a flexible manipulator, the method comprising:

[0007] Acquire the three-dimensional point cloud data of the target stamping part on the conveyor belt in real time, extract the key geometric features of the target stamping part, and determine the type, position and posture of the target stamping part;

[0008] The type, key geometric features and posture of the target stamping part are input into the pre-trained contact point prediction model to predict the contact point of the flexible manipulator;

[0009] Perform trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory;

[0010] The target stamping parts are grasped and packed according to the planned trajectory of the flexible manipulator.

[0011] On the basis of the above technical solution, preferably, the determining the type of the target stamping part specifically includes:

[0012] The three-dimensional point cloud data of the body-in-white is acquired, the three-dimensional point cloud data of the target stamping part is matched with the three-dimensional point cloud data of the body-in-white, and the type of the target stamping part is determined according to the point cloud matching result.

[0013] On the basis of the above technical solution, preferably, the pre-trained contact point prediction model is obtained by training the historical captured data of stamping parts, and the training method is:

[0014] Acquire historical grasping data of stamping parts, wherein the historical grasping data includes the type of stamping parts, key geometric features, grasping posture and contact points of the flexible manipulator; the key geometric features are corner points, edge features and normal vectors extracted from the three-dimensional point cloud data of the corresponding stamping parts;

[0015] The type, key geometric features and posture of the stamped parts in the historical grasping data are used as sample attributes, and the contact points of the flexible manipulator are used as sample labels to construct a data set;

[0016] The machine learning model is trained using the data set to obtain a pre-trained contact point prediction model.

[0017] On the basis of the above technical solution, preferably, the complete formula of the contact point prediction model can be expressed as:

[0018] y=W (2) σ(W (1) X+b (1) )+b (2)

[0019] Where X is the input vector, the input vector is the vector corresponding to the key geometric feature, W (2) is the output layer weight matrix, W (1) is the weight matrix, b (1) and b (2) is the bias vector and σ(x) is the activation function.

[0020] On the basis of the above technical solution, preferably, the optimization goal of trajectory planning of the flexible manipulator is: the function value of the cost function is minimized and the grasping error is minimized;

[0021] The expression of the cost function is:

[0022]

[0023] Where: J is the cost function, t represents time, T is the time period of grasping and packing operations, u(t) is the control input, that is, force or torque; q(t) is the acceleration of the flexible manipulator joint, v(t) is the speed of the flexible manipulator end effector, w1, w2, and w3 are weight coefficients respectively.

[0024] On the basis of the above technical solution, preferably, the type, key geometric features and posture of the target stamping part are input into the pre-trained contact point prediction model, after predicting the contact point of the flexible manipulator, the trajectory planning of the flexible manipulator is performed according to the position, posture and predicted contact point of the target stamping part, and before obtaining the planned trajectory, it also includes:

[0025] If the predicted contact points of the flexible manipulator involve weak parts of the vehicle body, the contact points are adjusted so that each contact point is located in a relatively strong part of the structure.

[0026] On the basis of the above technical solution, preferably, during the process of grasping the target stamping part according to the planned trajectory of the flexible manipulator, it is monitored in real time whether the position or posture of the target stamping part is offset. If so, the contact point of the flexible manipulator is re-predicted through the pre-trained contact point prediction model, and the trajectory is re-planned to update the planned trajectory.

[0027] In a second aspect, the present invention discloses a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator, the system comprising:

[0028] Data acquisition module: used to obtain the three-dimensional point cloud data of the target stamping parts on the conveyor belt in real time, extract the key geometric features of the target stamping parts, and determine the type, position and posture of the target stamping parts;

[0029] Contact point prediction module: used to input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact point of the flexible manipulator;

[0030] Trajectory planning module: used for performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact point of the flexible manipulator to obtain a planned trajectory;

[0031] Manipulator control module: used to grasp and pack target stamping parts according to the planned trajectory of the flexible manipulator.

[0032] In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface and a bus;

[0033] Wherein, the processor, memory, and communication interface communicate with each other via the bus;

[0034] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to the first aspect of the present invention.

[0035] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method described in the first aspect of the present invention.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1) The present invention acquires the three-dimensional point cloud data of the target stamping parts on the conveyor belt in real time, extracts the key geometric features of the target stamping parts, determines the type, position and posture of the target stamping parts, and predicts the contact points of the flexible manipulator through a pre-trained contact point prediction model, thereby performing more accurate trajectory planning. It can flexibly respond to the grasping needs of stamping parts of different shapes, sizes and positions, and also has good adaptability to special-shaped stamping parts with complex shapes, thereby improving the grasping accuracy and packing efficiency.

[0038] 2) The present invention constructs a data set based on the historical grasping data of the stamping parts, and trains a machine learning model through the data set to obtain a pre-trained contact point prediction model. The contact point prediction model can predict the optimal contact point of the flexible manipulator according to the type of stamping parts, key geometric features and posture during grasping, thereby avoiding damage to the stamping parts due to improper contact points between the manipulator and the stamping parts and improving the grasping success rate.

[0039] 3) The present invention defines the optimization target through the cost function of trajectory planning, which can make the trajectory smooth while ensuring the grasping accuracy, and can improve the stability of the flexible manipulator's high-speed grasping and packing operations.

[0040] 4) The present invention plans the trajectory of the flexible manipulator according to the position, posture and predicted contact points of the target stamping part, and monitors in real time whether the position or posture of the target stamping part is offset during the movement of the flexible manipulator. If so, the contact points of the flexible manipulator are re-predicted through a pre-trained contact point prediction model, and the trajectory is re-planned. It can adapt to the needs of high-speed production lines and realize real-time detection and dynamic adjustment of grasping actions, thereby significantly improving the automation level of the production line and the overall production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 This is a flow chart of a method for dynamically grabbing and packing automobile stamping parts based on a flexible manipulator of the present invention;

[0043] Figure 2 Shown is a schematic diagram of the operation process of a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator according to the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1 The present invention discloses a method for dynamically grasping and packing automobile stamping parts based on a flexible manipulator, the method comprising:

[0046] S1. Acquire the three-dimensional point cloud data of the target stamping part on the conveyor belt in real time, extract the key geometric features of the target stamping part, and determine the type, position and posture of the target stamping part.

[0047] In an embodiment of the present invention, multiple sensors are installed on the mechanical arm of the industrial robot or on both sides of the conveyor belt, and data is collected from the high-speed conveyor belt through the multiple sensors. For example, the three-dimensional point cloud data of the target stamping part on the conveyor belt is collected in real time through a laser scanner, and the image data of the target stamping part is collected in real time through a high-resolution camera.

[0048] The key geometric features of the three-dimensional point cloud data of the target stamping part, such as corner points, edge features, normal vectors, etc., are extracted. Through these key geometric features, the type of the target stamping part can be determined.

[0049] Specifically, the 3D point cloud data of the body in white is obtained, and based on the key geometric features of the 3D point cloud data of the target stamping part, the 3D point cloud data of the target stamping part is point cloud matched with the 3D point cloud data of the body in white, and the type of the target stamping part is determined according to the point cloud matching result.

[0050] Assume that the 3D point cloud of the body in white is represented as a set, where p i (x i ,y i ,z i ) is the i-th point of the 3D point cloud data of the body in white in 3D space, N is the total number of points, Represents the three-dimensional real number domain. The geometric shape of each stamping part in three-dimensional space can also be represented by a point cloud, recorded as a set where q j (x j ,y j ,z j ) is the jth point of the three-dimensional point cloud data of the stamping part in the three-dimensional space, and M is the total number of points.

[0051] The rigid transformation matrix can be used to match the body in white with the stamping. The rigid transformation matrix includes the rotation matrix R and the translation vector t, so that the distance between the point sets P and Q is minimized. The matching goal is to solve:

[0052]

[0053] where R∈SO(3) is the rotation matrix, is the translation vector, q j ∈Q. This objective function can be solved by the iterative closest point algorithm.

[0054] The position and posture of the target stamping part on the conveyor belt can be further determined by the three-dimensional point cloud data of the target stamping part collected in real time, or the position and posture of the target stamping part on the conveyor belt can be determined by the image data collected by a high-resolution camera.

[0055] S2. Input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator.

[0056] The pre-trained contact point prediction model is trained by historical captured data of stamping parts, and the training method is as follows:

[0057] A. Acquire historical grasping data of stamping parts, wherein the historical grasping data includes the type of stamping parts, key geometric features, grasping posture and contact points of the flexible manipulator; wherein the contact points of the flexible manipulator are the contact points between the flexible manipulator and the target stamping parts;

[0058] B. Constructing a data set using the type, key geometric features, and grasping posture of stamped parts in the historical grasping data as sample attributes and the contact points of the flexible manipulator as sample labels;

[0059] When constructing a data set, in order to prevent insufficient historical data or low sample quality, the data set can be expanded or label optimization can be performed by expanding sample labels.

[0060] Therefore, when making sample labels, the present invention determines the type of the target stamping part, identifies the key parts of the target stamping part, determines the key structural features of the target stamping part, and selects appropriate contact points through a contact point optimization strategy.

[0061] Specifically, take automobile stamping parts (such as door inner panels) as an example. The door inner panels have reinforcing ribs, mounting holes and support areas. When grasping such structures, first identify the key parts of the target stamping parts and determine the key structural features of the target stamping parts. For example, the reinforcing ribs and support areas of the door inner panels are usually relatively strong parts of the structure and are not easy to deform. These areas can be selected when selecting contact points to make labels to avoid damage and deformation of the stamping parts during the grasping process due to improper selection of contact points. Hole areas such as bolt mounting holes or connecting holes are also suitable points for grasping. Through specially designed flexible robotic grippers or suction cups, these positions can be selected as contact points to avoid grasping weak edge areas. In this way, the contact points can be optimized to ensure that the target stamping parts are not deformed or damaged.

[0062] In addition, the center of gravity of automotive stamping parts such as door inner panels is usually unevenly distributed. Therefore, it is necessary to find the most suitable combination of contact points by calculating the geometric shape and weight distribution of the stamping parts to ensure that the robot grasps them in a balanced manner and avoid tilting or instability of the stamping parts due to improper contact point position.

[0063] If the stamping part is large, a multi-point grasping solution can be adopted, that is, multiple contact points are selected and multiple manipulators work together to evenly distribute the tensile force and further reduce the risk of tilting during grasping. For example, the reinforcement ribs on both sides and the mounting holes in the middle can be grasped at the same time to achieve the best traction balance.

[0064] C. Train a machine learning model using the data set to obtain a pre-trained contact point prediction model.

[0065] The present invention trains a machine learning model through a large amount of historical grasping operation data to automatically predict the optimal contact points of different stamping parts. According to the scheme of constructing a data set in step B, the contact point prediction model of the present invention can identify which parts are suitable for grasping (such as areas with strong structures) through training, and adjust the contact points in real time according to the posture of the stamping part.

[0066] Construct a contact point prediction model for a flexible manipulator based on a neural network. The complete neural network calculation formula includes multiple layers of linear transformation, nonlinear activation function, and loss calculation. Set the input data to be the type of stamping part, key geometric features (such as corner points, edge features, and normal vectors), and the posture during grasping. The dimension of the input vector X is n (that is, the total number of all input features):

[0067]

[0068] Here X is a column vector containing the feature data of all samples. The input layer of the neural network receives data describing the type of stamping, key geometric features (such as corners, edges, normal vectors), and the posture when grasping. The input features are represented as vectors as X. Each input feature plays a different role in the entire model, providing the model with important geometric and posture information to help predict the contact points of the flexible manipulator.

[0069] Assume that we use a hidden layer with k neurons, whose calculation is a combination of the input layer after linear transformation, bias addition and nonlinear activation function. The hidden layer output H is:

[0070] H=σ(W (1) X+b (1) )

[0071] Among them, W(1) is a weight matrix of size k×n (weight of the first layer), b(1) is a bias vector of size k×1, and σ(X) is an activation function. Commonly used nonlinear functions include ReLU (rectified linear unit), Sigmoid, Tanh, etc. Specifically, the definition of ReLU nonlinear function is as follows:

[0072] σ(z)=max(0,z)

[0073] The definition of the Sigmoid nonlinear function is as follows:

[0074]

[0075] The Tanh nonlinear function is defined as follows:

[0076]

[0077] Therefore, the output formula of each neuron in the hidden layer is:

[0078]

[0079] This means that the output h of each neuron in the hidden layer j It is the value obtained by weighted summation of the input vector X, adding the bias term, and then calculating it through the activation function.

[0080] The output layer converts the output H of the hidden layer into the predicted value y of the contact point. Assuming that the output layer has m outputs (for example, if you want to predict the contact point position in 3D space, you may need three outputs corresponding to the x, y, and z axes), the calculation formula of the output layer is:

[0081] y=W (2) H+b (2)

[0082] Among them, W (2) is a weight matrix of size m×k, b (2) is a bias vector of size m×1.

[0083] The output formula for each neuron in the output layer is:

[0084]

[0085] Combining the calculation processes of the above layers, the complete formula of the contact point prediction model of the flexible manipulator can be expressed as:

[0086] y=W (2) σ(W (1) X+b (1) )+b (2)

[0087] Among them, X is the input vector, the input vector is the vector corresponding to the key geometric feature, and W (2) is the output layer weight matrix, W (1) is the weight matrix, b (1) and b (2) is the bias vector, σ(x) is the activation function;

[0088] The input layer receives information such as the geometric features and posture of the stamped parts as input to the neural network. Through the linear weighted sum of nonlinear activation functions, information is transmitted layer by layer to extract the complex pattern of input features. The last layer generates the contact point prediction of the flexible manipulator, which may be a continuous coordinate or a probability of the contact point category. The accuracy of the prediction is measured by the loss function, and the back propagation algorithm gradually optimizes the weights and biases of the network so that the model can better predict the contact points. This layer-by-layer calculation and weight adjustment mechanism enables the neural network to learn from the complex features of the input and generate accurate predictions of the contact points.

[0089] In actual operation, the position and posture information of the target stamping parts are captured in real time through sensors. If the target stamping parts undergo slight displacement or posture changes during the transmission process, the contact point position of the robot is adaptively adjusted through the pre-trained contact point prediction model to ensure accurate grasping.

[0090] In addition, if the final predicted contact points of the flexible manipulator still involve weak parts of the car body, the contact points are re-optimized so that each contact point is located in a relatively strong part of the structure, such as a pillar or frame, to ensure that the stamping parts are not deformed or damaged.

[0091] The present invention constructs a data set based on the historical grasping data of stamping parts, and trains a machine learning model through the data set to obtain a pre-trained contact point prediction model. The contact point prediction model can predict the optimal contact point of the flexible manipulator according to the type of stamping parts, key geometric features and posture during grasping, and perform label optimization through a contact point optimization strategy, which can avoid damage to the stamping parts due to improper contact points between the manipulator and the stamping parts and improve the grasping success rate.

[0092] S3. Perform trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory.

[0093] Assuming that the robot used in the present invention has n joints, the forward kinematic equation representing the position of the flexible manipulator is:

[0094] T(θ)=T1(θ1)·T2(θ2)·...·T n (θ n )

[0095] where θ k (k=1,2,…,n) is the angle of the kth joint, T k (θ k ) is the transformation matrix representing the joint pose.

[0096] The position of the flexible manipulator's end effector can be determined based on the position of the flexible manipulator, thereby performing trajectory planning.

[0097] In the embodiment of the present invention, the A* or Dijkstra algorithm is used for trajectory planning. Specifically, after the real-time position information of the target stamping part is identified, a shortest and collision-free path is generated by the A* or Dijkstra algorithm.

[0098] In the embodiment of the present invention, the optimization goal of trajectory planning of the flexible manipulator is: the function value of the cost function is minimized and the grasping error is minimized.

[0099] The expression of the cost function is:

[0100]

[0101] Where: J is the cost function, t represents time, T is the time period of grasping and packing operations, u(t) is the control input, i.e. force or torque; q(t) is the acceleration of the flexible manipulator joint, It can represent the smoothness of the trajectory; v(t) is the velocity of the end effector of the flexible manipulator, and w1, w2, and w3 are weight coefficients respectively.

[0102] The present invention defines the optimization target through the cost function of trajectory planning, which can make the trajectory smooth while ensuring the grasping accuracy, and can improve the stability of high-speed grasping and packing operations of the flexible manipulator.

[0103] Still taking the interior door panel as an example, after the real-time position information of the interior door panel on the conveyor belt is identified, a shortest and collision-free path is generated through the A* or Dijkstra algorithm. When the interior door panel moves on the conveyor belt, the flexible robot needs to grab the reinforcement area from above, and the path needs to avoid collision with other mechanical parts.

[0104] If the position of the door inner panel changes during the transfer process, the planned trajectory of the flexible manipulator also needs to be adjusted dynamically. For example, if the door inner panel is slightly tilted, the trajectory planning algorithm will re-plan the motion trajectory of the flexible manipulator based on sensor feedback to ensure that the manipulator can smoothly grasp the contact point of the door inner panel.

[0105] For stamping parts such as door inner panels, the surface usually has a certain curvature. To ensure smooth grasping, the trajectory of the flexible manipulator can also be planned according to the curvature of the stamping part, so that the flexible manipulator can stay close to the surface of the door inner panel during movement. For example, before grasping, the flexible manipulator can gradually approach from the side of the door instead of directly contacting vertically to avoid uneven force caused by sudden contact.

[0106] The flexible manipulator of the present invention has multiple degrees of freedom, so in path planning, it is necessary to take advantage of its multiple degrees of freedom to perform complex movements. For example, the inner door panel may move quickly on a conveyor belt, and the flexible manipulator can coordinate the movement and position change of the inner door panel through rotation and bending to achieve precise traction.

[0107] In addition, the door inner panel may coexist with other stamping parts or production equipment on the same conveyor belt. During the grasping process of the door inner panel, the grasping trajectory of the manipulator cannot interfere with other parts of the stamping part. In order to avoid collisions during the grasping process, during the trajectory planning process, the flexible manipulator needs to rely on the real-time feedback of the sensor, perform real-time obstacle avoidance detection, and plan an obstacle avoidance path. For example, the position and shape of other workpieces on the conveyor belt can be detected in real time through sensors. Based on this information, trajectory planning is performed, considering the contact angle and movement trajectory of the manipulator when approaching the door inner panel, and instructing the manipulator to bypass these workpieces to avoid collisions and ensure smooth and correct grasping.

[0108] After the grasping is completed, the flexible robot also needs to place the target stamping part in the designated packing position. The trajectory planning in this process also needs to be precisely designed to improve the packing efficiency and the stability of the placement of the target stamping parts.

[0109] Take the door inner panel as an example. The placement of the door inner panel needs to follow a certain order and direction to maximize the use of space in the packing space, so the optimal packing order needs to be determined. For example, the door inner panel can be placed vertically or horizontally according to its shape to reduce the gap during packing. In the multi-layer packing process, the robot needs to dynamically adjust the placement path of the next door inner panel according to the position of the door that has been packed. For example, when several layers of door inner panels have been placed in the box, the robot needs to raise its gripping point and adjust the gripping path to ensure that the next door is safely placed.

[0110] In addition, there is a risk of stampings colliding with each other during the packing process, especially when the packing space is tight. Sensors can monitor the distance between the door inner panel and other workpieces in real time. If an imminent collision is detected, the packing path should be adjusted immediately or the packing sequence should be replanned to ensure that the stampings are not damaged during packing.

[0111] In one possible implementation, a camera is added to the manipulator for visual monitoring and anti-shake and collision avoidance. The stability during movement can be ensured by selecting a high-definition, low-latency camera and a depth sensor, combined with real-time image processing and anti-shake algorithms. The image processing algorithm, for example, performs edge detection and target recognition on the image through OpenCV to ensure that the manipulator can correctly locate the type, position and posture of the target object. Combining IMU (inertial measurement unit) and visual information, the motion trajectory of the manipulator is smoothed through Kalman filter or PID control to reduce the accuracy deviation of the manipulator caused by vibration. The machine learning model is used to predict the jitter that may occur during the movement, and the trajectory is fine-tuned and compensated in advance. Through deep learning models, such as convolutional neural networks (CNN), the dynamic motion trend of the target is predicted, thereby improving the stability of grasping and packing.

[0112] At the same time, with the help of vision and other sensor data fusion (such as IMU, force sensor), obstacles are avoided through collision detection and path planning, providing real-time feedback and adjustment. Combine visual information with data from other sensors such as IMU and force sensors to establish a comprehensive environmental perception system. During the operation of the manipulator, if it is detected that the object to be contacted has displacement or an area that is not suitable for grasping (such as the weak part of the car body), the grasping point should be automatically adjusted. Path planning algorithms such as A* and Dijkstra are used to generate the optimal motion trajectory of the manipulator to avoid collisions. The optimization goal is to minimize the cost function and minimize the grasping error. The cost function can include a comprehensive consideration of the manipulator's terminal velocity, acceleration, and torque input to ensure the smoothness and stability of the trajectory. Through a laser scanner or a depth camera, the environment around the manipulator is monitored in real time, and the path is dynamically updated to avoid sudden obstacles. At the same time, the contact force between the manipulator and the environment is detected through a force sensor to avoid damage to the workpiece due to excessive force.

[0113] By using high-speed communication methods such as industrial Ethernet (such as EtherCAT), the data transmission speed between the sensor and the controller can be increased, and the latency of the overall system can be reduced. Integrate multiple cameras on the manipulator to form a stereoscopic vision system to improve the spatial recognition ability of the target object. For example, use two or more cameras to obtain the three-dimensional point cloud data of the target object, and then determine the position and posture of the object through point cloud registration. Fuse the data of the camera, IMU, and force sensor to establish a multi-modal sensor information fusion system to improve the intelligent perception ability of the manipulator. For example, when the visual system fails due to changes in lighting, the IMU can provide posture reference data to ensure the stability of the system.

[0114] At the same time, the manipulator design is optimized to ensure balance and earthquake resistance, and the overall accuracy and response speed are improved through high-speed communication and multi-camera collaboration, enhancing the intelligence and safety of the system. Specifically, the structure of the manipulator is optimized to have good symmetry, lower the center of gravity, increase stability, and thus reduce the jitter caused by inertia. High-strength lightweight materials such as carbon fiber or aluminum alloy are selected to reduce the mass of the manipulator, thereby improving dynamic performance and earthquake resistance.

[0115] S4. Grasping and packing the target stamping parts according to the planned trajectory of the flexible manipulator.

[0116] During the process of grasping the target stamping part according to the planned trajectory of the flexible manipulator, the position or posture of the target stamping part is monitored in real time to see if it is offset. If so, in addition to re-predicting the contact points of the flexible manipulator through the pre-trained contact point prediction model, it is also necessary to re-plan the trajectory and update the planned trajectory.

[0117] The present invention monitors in real time whether the position or posture of the target stamping part is offset during the movement of the flexible manipulator. If so, the contact point of the flexible manipulator is predicted again through the pre-trained contact point prediction model, and the trajectory is re-planned, realizing real-time detection and dynamic adjustment of the grasping action, which can meet the needs of high-speed production lines, thereby significantly improving the automation level of the production line and the overall production efficiency. If an error occurs during the grasping or packing process, such as grasping failure or body posture deviation, automatic detection is performed and the recovery mechanism is started to reposition the contact point and resume operation.

[0118] The present invention integrates flexible manipulators and real-time sensor data processing technology and machine learning models, and uses a pre-trained contact point prediction model to predict the contact points of the flexible manipulator in real time, thereby performing more accurate trajectory planning, and can adaptively process stampings of different shapes and sizes, enhancing the adaptability and flexibility of the system, significantly improving grasping accuracy, reducing grasping failures, and reducing the risk of damage to the surface of stampings. The present invention further shortens the packing time and improves the packing efficiency by using a strategy of dynamic trajectory planning for the end effector of the flexible manipulator based on real-time sensor data processing technology.

[0119] The high-performance control and real-time response capabilities of the present invention enable the grasping and packing operations of automobile stamping parts to meet the needs of high-speed production lines, realize real-time detection of target status and dynamic adjustment of grasping actions, and significantly improve the automation level and overall production efficiency of the production line. The present invention has good adaptability to special-shaped stamping parts with complex shapes, and has taken the lead in completing the multi-robot rapid positioning and efficient and accurate packing technology for special-shaped stamping parts, achieving a packing accuracy of 5000±2.5mm and a packing speed of 15 pieces / min, improving the grasping accuracy and packing efficiency.

[0120] On the basis of the above method embodiment, the present invention also discloses a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator. Figure 2 The figure shows the operation process of a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator of the present invention. The system includes the following functional modules:

[0121] Data acquisition module: used to obtain the three-dimensional point cloud data of the target stamping parts on the conveyor belt in real time through sensors, extract key geometric features of the target stamping parts, and determine the type, position and posture of the target stamping parts;

[0122] Contact point prediction module: used to input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact point of the flexible manipulator;

[0123] Trajectory planning module: used for performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact point of the flexible manipulator to obtain a planned trajectory;

[0124] Manipulator control module: used to grasp and pack target stamping parts according to the planned trajectory of the flexible manipulator.

[0125] The above system embodiments and method embodiments correspond one to one, and for a brief description of the system embodiments, please refer to the method embodiments.

[0126] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory, and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0127] The present invention also discloses a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to implement all or part of the steps of the method described in the embodiment of the present invention. The storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk, and other media that can store program codes.

[0128] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be distributed to multiple network units. A person skilled in the art may select some or all of the modules according to actual needs to achieve the purpose of the solution of this embodiment without creative effort.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator, characterized in that: The method comprises: Acquire the three-dimensional point cloud data of the target stamping part on the conveyor belt in real time, extract the key geometric features of the target stamping part, and determine the type, position and posture of the target stamping part; The type, key geometric features and posture of the target stamping part are input into the pre-trained contact point prediction model to predict the contact point of the flexible manipulator; Perform trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory; The target stamping parts are grasped and packed according to the planned trajectory of the flexible manipulator.

2. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: Determining the type of the target stamping part specifically includes: The three-dimensional point cloud data of the body-in-white is acquired, the three-dimensional point cloud data of the target stamping part is matched with the three-dimensional point cloud data of the body-in-white, and the type of the target stamping part is determined according to the point cloud matching result.

3. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: The pre-trained contact point prediction model is trained by historical captured data of stamping parts, and the training method is as follows: Acquire historical grasping data of stamping parts, wherein the historical grasping data includes the type of stamping parts, key geometric features, grasping posture and contact points of the flexible manipulator; the key geometric features are corner points, edge features and normal vectors extracted from the three-dimensional point cloud data of the corresponding stamping parts; The type, key geometric features and posture of the stamped parts in the historical grasping data are used as sample attributes, and the contact points of the flexible manipulator are used as sample labels to construct a data set; The machine learning model is trained using the data set to obtain a pre-trained contact point prediction model.

4. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 3 is characterized in that: The complete formula of the contact point prediction model can be expressed as: y=W (2) σ(W (1) X+b (1) )+b (2) Where X is the input vector, the input vector is the vector corresponding to the key geometric feature, W (2) is the output layer weight matrix, W (1) is the weight matrix, b (1) and b (2) is the bias vector and σ(x) is the activation function.

5. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: The optimization goal of trajectory planning of the flexible manipulator is: the function value of the cost function is minimized and the grasping error is minimized; The expression of the cost function is: Where: J is the cost function, t represents time, T is the time period of grasping and packing operations, u(t) is the control input, that is, force or torque; q(t) is the acceleration of the flexible manipulator joint, v(t) is the speed of the flexible manipulator end effector, w1, w2, and w3 are weight coefficients respectively.

6. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: After inputting the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model and predicting the contact point of the flexible manipulator, the trajectory planning of the flexible manipulator is performed according to the position, posture and predicted contact point of the target stamping part, and before obtaining the planned trajectory, the method further includes: If the predicted contact points of the flexible manipulator involve weak parts of the vehicle body, the contact points are adjusted so that each contact point is located in a relatively strong part of the structure.

7. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: During the process of grasping the target stamping part according to the planned trajectory of the flexible manipulator, the position or posture of the target stamping part is monitored in real time to see if it is offset. If so, the contact point of the flexible manipulator is re-predicted through the pre-trained contact point prediction model, and the trajectory is re-planned to update the planned trajectory.

8. A dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator, characterized in that: The system comprises: Data acquisition module: used to obtain the three-dimensional point cloud data of the target stamping parts on the conveyor belt in real time, extract the key geometric features of the target stamping parts, and determine the type, position and posture of the target stamping parts; Contact point prediction module: used to input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact point of the flexible manipulator; Trajectory planning module: used to plan the trajectory of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact point of the flexible manipulator to obtain the planned trajectory; Manipulator control module: used to grasp and pack target stamping parts according to the planned trajectory of the flexible manipulator.

9. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; Wherein, the processor, memory, and communication interface communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 7.

Images