Dynamic grasping and packing method and system for automobile stamping parts based on flexible manipulator

Abstract

The present invention discloses a method and system for dynamic grasping and packing of automotive stamping parts based on a flexible manipulator. The method comprises: acquiring three-dimensional point cloud data of target stamping parts on a conveyor belt in real time, extracting key geometric features of the target stamping parts, and determining the type, position, and posture of the target stamping parts; inputting the type, key geometric features, and posture of the target stamping parts into a pre-trained contact point prediction model to predict the contact points of the flexible manipulator; planning the trajectory of the flexible manipulator based on the position and posture of the target stamping parts and the predicted contact points of the flexible manipulator to obtain a planned trajectory; and grasping and packing the target stamping parts according to the planned trajectory of the flexible manipulator. The present invention can flexibly respond to the grasping needs of stamping parts of different shapes, sizes, and positions, and also has good adaptability to complex and special-shaped stamping parts, thereby improving grasping accuracy and packing efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of robot control, and in particular relates to a method and system for dynamically grasping and packing automobile stamping parts based on a flexible manipulator. Background Art

[0002] In the automotive manufacturing industry, stamped parts are essential components for vehicle bodies and other key components, often requiring multiple processing steps. On the production line, grabbing and boxing large numbers of stamped parts from conveyor belts is a critical step, directly impacting overall production line efficiency. However, traditional grabbing and boxing methods often rely on rigid robotic arms, which are susceptible to limitations due to part shape complexity and positional variations. This results in insufficient grasping accuracy, slow speeds, and even potential surface damage.

[0003] Existing automated packaging systems typically use fixed-path manipulators or template-matching-based grasping methods, which often have difficulty handling complex or diverse stamping parts. Furthermore, the shapes and sizes of stamping parts on the production line vary, and their position and posture on the conveyor belt are difficult to accurately predict, severely limiting the grasping efficiency of traditional manipulators. At the same time, with the continuous advancement of automobile manufacturing processes, higher requirements are being placed on the automation level and packaging efficiency of production lines. To meet these challenges, flexible manipulators have gradually become a hot topic in research and application. Flexible manipulators, through their multi-joint structure and flexible end effectors, can, to a certain extent, adapt to the grasping needs of stamping parts of different shapes, sizes, and positions. However, the flexible manipulator systems currently on the market still have shortcomings in dynamic grasping and real-time response, and further optimization is needed to improve packaging efficiency and adaptability.

[0004] Therefore, there is an urgent need for a method that can efficiently and accurately grab various types of automotive stamping parts from the conveyor belt and quickly pack them into boxes to meet the modern automotive manufacturing industry's demand for automation and efficient production. Summary of the Invention

[0005] In view of this, the present invention proposes a method and system for dynamically grasping and packing automobile stamping parts based on a flexible manipulator, which is used to solve the problem of poor packing efficiency of existing manipulators.

[0006] In a first aspect, the present invention discloses a method for dynamically grasping and packing automobile stamping parts based on a flexible manipulator, the method comprising:

[0007] Acquire three-dimensional point cloud data of a target stamping part on a conveyor belt in real time, extract key geometric features of the target stamping part, and determine the type, position, and posture of the target stamping part;

[0008] The type, key geometric features, and posture of the target stamping part are input into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator.

[0009] Performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory;

[0010] The target stamping parts are grasped and packed according to the planned trajectory of the flexible manipulator.

[0011] On the basis of the above technical solution, preferably, determining the type of the target stamping part specifically includes:

[0012] The three-dimensional point cloud data of the body-in-white is acquired, the three-dimensional point cloud data of the target stamping part is matched with the three-dimensional point cloud data of the body-in-white, and the type of the target stamping part is determined according to the point cloud matching result.

[0013] On the basis of the above technical solution, preferably, the pre-trained contact point prediction model is obtained by training with historical captured data of stamping parts, and the training method is:

[0014] Acquire historical grasping data of the stamped part, the historical grasping data including the type of the stamped part, key geometric features, grasping posture, and contact points of the flexible manipulator; the key geometric features are corner points, edge features, and normal vectors extracted from the three-dimensional point cloud data of the corresponding stamped part;

[0015] The data set is constructed by using the type, key geometric features, and grasping posture of the stamped parts in the historical grasping data as sample attributes and the contact points of the flexible manipulator as sample labels;

[0016] The machine learning model is trained using the data set to obtain a pre-trained touch point prediction model.

[0017] Based on the above technical solution, preferably, the complete formula of the contact point prediction model can be expressed as:

[0018] y=W (2) σ(W (1) X+b (1) )+b (2)

[0019] Where X is the input vector, the input vector is the vector corresponding to the key geometric feature, W (2) is the output layer weight matrix, W (1) is the weight matrix, b (1) and b (2) is the bias vector, and σ(x) is the activation function.

[0020] On the basis of the above technical solution, preferably, the optimization goal of trajectory planning of the flexible manipulator is: the function value of the cost function is minimized and the grasping error is minimized;

[0021] The expression of the cost function is:

[0022]

[0023] Where: J is the cost function, t represents time, T is the time period of grasping and packing operations, u(t) is the control input, that is, force or torque; q(t) is the acceleration of the flexible manipulator joint, v(t) is the velocity of the flexible manipulator end effector, and w1, w2, and w3 are weight coefficients respectively.

[0024] On the basis of the above technical solution, preferably, the method further comprises the following steps: inputting the type, key geometric features and posture of the target stamping part into a pre-trained contact point prediction model, predicting the contact points of the flexible manipulator, and performing trajectory planning of the flexible manipulator according to the position, posture and predicted contact points of the target stamping part, and obtaining the planned trajectory:

[0025] If the predicted contact points of the flexible manipulator involve weak parts of the vehicle body, the contact points are adjusted so that each contact point is located in a relatively strong part of the structure.

[0026] On the basis of the above technical solution, preferably, during the process of grasping the target stamping part according to the planned trajectory of the flexible manipulator, the position or posture of the target stamping part is monitored in real time to see if it is offset. If so, the contact point of the flexible manipulator is re-predicted through the pre-trained contact point prediction model, and the trajectory is re-planned to update the planned trajectory.

[0027] The second aspect of the present invention discloses a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator, the system comprising:

[0028] Data acquisition module: used to acquire the three-dimensional point cloud data of the target stamping part on the conveyor belt in real time, extract the key geometric features of the target stamping part, and determine the type, position and posture of the target stamping part;

[0029] Contact point prediction module: used to input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator;

[0030] Trajectory planning module: used for performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory;

[0031] Manipulator control module: used to grab and pack target stamping parts according to the planned trajectory of the flexible manipulator.

[0032] A third aspect of the present invention discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus;

[0033] The processor, memory, and communication interface communicate with each other via the bus.

[0034] The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to the first aspect of the present invention.

[0035] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to the first aspect of the present invention.

[0036] The present invention has the following beneficial effects compared to the prior art:

[0037] 1) The present invention acquires the three-dimensional point cloud data of the target stamping parts on the conveyor belt in real time, extracts the key geometric features of the target stamping parts, determines the type, position and posture of the target stamping parts, and predicts the contact points of the flexible manipulator through a pre-trained contact point prediction model, thereby performing more accurate trajectory planning. It can flexibly respond to the grasping needs of stamping parts of different shapes, sizes and positions, and also has good adaptability to complex-shaped special-shaped stamping parts, thereby improving the grasping accuracy and packing efficiency.

[0038] 2) The present invention constructs a data set based on the historical grasping data of stamping parts, and trains a machine learning model through the data set to obtain a pre-trained contact point prediction model. The contact point prediction model can predict the optimal contact point of the flexible manipulator based on the type of stamping parts, key geometric features and posture during grasping, thereby avoiding damage to the stamping parts due to improper contact points between the manipulator and the stamping parts, and improving the grasping success rate.

[0039] 3) The present invention defines the optimization goal through the cost function of trajectory planning, which can ensure the grasping accuracy while making the trajectory smooth, and can improve the stability of the flexible manipulator's high-speed grasping and packing operations.

[0040] 4) The present invention plans the trajectory of the flexible manipulator based on the position, posture and predicted contact points of the target stamping part, and monitors in real time whether the position or posture of the target stamping part is offset during the movement of the flexible manipulator. If so, the contact points of the flexible manipulator are re-predicted through the pre-trained contact point prediction model, and the trajectory is re-planned. It can adapt to the needs of high-speed production lines, realize real-time detection and dynamic adjustment of grasping actions, thereby significantly improving the automation level of the production line and the overall production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 This is a flow chart of a method for dynamically grasping and packing automobile stamping parts based on a flexible manipulator of the present invention;

[0043] Figure 2 Shown is a schematic diagram of the operation process of a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator according to the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] See also Figure 1 The present invention discloses a method for dynamically grasping and packing automobile stamping parts based on a flexible manipulator, the method comprising:

[0046] S1. Acquire the three-dimensional point cloud data of the target stamping part on the conveyor belt in real time, extract the key geometric features of the target stamping part, and determine the type, position and posture of the target stamping part.

[0047] In an embodiment of the present invention, multiple sensors are mounted on the robotic arm of an industrial robot or on both sides of a conveyor belt to collect data from the high-speed conveyor belt. For example, a laser scanner can collect real-time 3D point cloud data of a target stamping part on the conveyor belt, while a high-resolution camera can collect real-time image data of the target stamping part.

[0048] The key geometric features of the three-dimensional point cloud data of the target stamping part, such as corner points, edge features, normal vectors, etc., are extracted. Through these key geometric features, the type of the target stamping part can be determined.

[0049] Specifically, the three-dimensional point cloud data of the body in white is obtained, and based on the key geometric features of the three-dimensional point cloud data of the target stamping part, the three-dimensional point cloud data of the target stamping part is matched with the three-dimensional point cloud data of the body in white, and the type of the target stamping part is determined according to the point cloud matching result.

[0050] Assume that the 3D point cloud of the body in white is represented as a set, where p i (x i ,y i ,z i ) is the i-th point of the 3D point cloud data of the body in white in 3D space, N is the total number of points, Represents the three-dimensional real number field. The geometric shape of each stamping part in three-dimensional space can also be represented by a point cloud, recorded as a set where q j (x j ,y j ,z j ) is the j-th point of the three-dimensional point cloud data of the stamping part in the three-dimensional space, and M is the total number of points.

[0051] The rigid transformation matrix can be used to match the body in white with the stamping part. The rigid transformation matrix includes the rotation matrix R and the translation vector t, so that the distance between the point sets P and Q is minimized. The matching goal is to solve:

[0052]

[0053] where R∈SO(3) is the rotation matrix, is the translation vector, q j ∈Q. This objective function can be solved by the iterative closest point algorithm.

[0054] The position and posture of the target stamping part on the conveyor belt can be further determined by the three-dimensional point cloud data of the target stamping part collected in real time, or the position and posture of the target stamping part on the conveyor belt can be determined by the image data collected by the high-resolution camera.

[0055] S2. Input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator.

[0056] The pre-trained contact point prediction model is trained by historical captured data of stamping parts, and its training method is as follows:

[0057] A. Acquire historical grasping data of stamped parts, wherein the historical grasping data includes the type of stamped parts, key geometric features, grasping posture, and contact points of the flexible manipulator; wherein the contact points of the flexible manipulator are the contact points between the flexible manipulator and the target stamped parts;

[0058] B. Constructing a dataset using the type, key geometric features, and grasping posture of the stamped parts in the historical grasping data as sample attributes and the contact points of the flexible manipulator as sample labels;

[0059] When constructing a dataset, in order to prevent insufficient historical data or low sample quality, the dataset can be expanded or label optimization can be performed by expanding sample labels.

[0060] Therefore, when making sample labels, the present invention determines the type of the target stamping part, identifies the key parts of the target stamping part, determines the key structural features of the target stamping part, and selects appropriate contact points through a contact point optimization strategy.

[0061] Specifically, taking automobile stamping parts (such as door inner panels) as an example, the door inner panels have reinforcing ribs, mounting holes and support areas. When grasping such structures, first identify the key parts of the target stamping parts and determine the key structural features of the target stamping parts. For example, the reinforcing ribs and support areas of the door inner panels are usually relatively strong parts of the structure and not easy to deform. These areas can be selected when selecting contact points to make labels to avoid damage and deformation of the stamping parts during the grasping process due to improper selection of contact points. Hole areas such as bolt mounting holes or connection holes are also suitable points for grasping. Through specially designed flexible robotic grippers or suction cups, these positions can be selected as contact points to avoid grasping weak edge areas. In this way, the contact points can be optimized to ensure that the target stamping parts are not deformed or damaged.

[0062] In addition, the center of gravity of automotive stamping parts such as door inner panels is usually unevenly distributed. Therefore, it is necessary to calculate the geometric shape and weight distribution of the stamping parts to find the most suitable combination of contact points to ensure that the robot grasps them in a balanced manner and avoid tilting or instability of the stamping parts due to improper contact point positions.

[0063] For larger stamping parts, a multi-point gripping solution can be used. This involves selecting multiple contact points and using multiple manipulators to evenly distribute the tensile force, further reducing the risk of tilting during gripping. For example, the ribs on both sides and the mounting holes in the middle can be gripped simultaneously to achieve optimal pulling balance.

[0064] C. Train a machine learning model using the dataset to obtain a pre-trained touch point prediction model.

[0065] The present invention trains a machine learning model through a large amount of historical grasping operation data to automatically predict the optimal contact points of different stamping parts. According to the scheme of constructing a data set in step B, the contact point prediction model of the present invention can identify which parts are suitable for grasping (such as areas with strong structures) through training, and adjust the contact points in real time according to the posture of the stamping part.

[0066] A neural network-based contact point prediction model for a flexible manipulator is constructed. The complete neural network calculation formula includes multiple layers of linear transformations, nonlinear activation functions, and loss calculations. The input data is set as the type of stamping part, key geometric features (such as corner points, edge features, and normal vectors), and the posture during grasping. The dimension of the input vector X is n (i.e., the total number of all input features):

[0067]

[0068] Here, X is a column vector containing the feature data for all samples. The neural network's input layer receives data describing the part type, key geometric features (such as corners, edges, and normal vectors), and the gripping pose. The input features are represented as vectors as X. Each input feature plays a different role in the overall model, providing important geometric and pose information to help predict the contact points of the flexible manipulator.

[0069] Assume we use a hidden layer with k neurons, whose calculation is a combination of the input layer through linear transformation, bias addition and nonlinear activation function. The hidden layer output H is:

[0070] H=σ(W (1) X+b (1) )

[0071] Where W(1) is a weight matrix of size k×n (the weight of the first layer), b(1) is a bias vector of size k×1, and σ(X) is an activation function. Commonly used nonlinear functions include ReLU (Rectified Linear Unit), Sigmoid, and Tanh. Specifically, the definition of the ReLU nonlinear function is as follows:

[0072] σ(z)=max(0,z)

[0073] The definition of the Sigmoid nonlinear function is as follows:

[0074]

[0075] The Tanh nonlinear function is defined as follows:

[0076]

[0077] Therefore, the output formula of each neuron in the hidden layer is:

[0078]

[0079] This means that the output h of each neuron in the hidden layer j It is the value obtained by weighted summation of the input vector X, adding the bias term, and then calculating it through the activation function.

[0080] The output layer converts the output H of the hidden layer into the predicted value y of the contact point. Assuming that the output layer has m outputs (for example, if you want to predict the contact point position in 3D space, you may need three outputs corresponding to the x, y, and z axes), the calculation formula of the output layer is:

[0081] y=W (2) H+b (2)

[0082] Among them, W (2) is a weight matrix of size m×k, b (2) is a bias vector of size m×1.

[0083] The output formula for each neuron in the output layer is:

[0084]

[0085] Combining the calculation processes of the above layers, the complete formula of the flexible manipulator contact point prediction model can be expressed as:

[0086] y=W (2) σ(W (1) X+b (1) )+b (2)

[0087] Among them, X is the input vector, the input vector is the vector corresponding to the key geometric feature, W (2) is the output layer weight matrix, W (1) is the weight matrix, b (1) and b (2) is the bias vector, σ(x) is the activation function;

[0088] The input layer receives information about the stamped part's geometric features, posture, and other aspects as input to the neural network. Information is passed layer by layer through a linear weighted summation of nonlinear activation functions, extracting the complex patterns of the input features. The final layer generates predictions of the flexible manipulator's contact points, which may be continuous coordinates or probabilities of contact point categories. The accuracy of the predictions is measured using a loss function, and the backpropagation algorithm gradually optimizes the network's weights and biases, enabling the model to better predict contact points. This layer-by-layer calculation and weight adjustment mechanism enables the neural network to learn from the complex features of the input and generate accurate predictions of contact points.

[0089] In actual operation, the position and posture information of the target stamping part is captured in real time through sensors. If the target stamping part undergoes slight displacement or posture changes during the transmission process, the contact point position of the robot is adaptively adjusted through the pre-trained contact point prediction model to ensure accurate grasping.

[0090] In addition, if the final predicted contact points of the flexible manipulator still involve weak parts of the car body, the contact points will be re-optimized so that each contact point is located in a relatively strong part of the structure, such as a pillar or frame, to ensure that the stamping parts are not deformed or damaged.

[0091] The present invention constructs a data set based on the historical grasping data of stamping parts, and trains a machine learning model through the data set to obtain a pre-trained contact point prediction model. The contact point prediction model can predict the optimal contact point of the flexible manipulator based on the type of stamping parts, key geometric features and posture during grasping, and perform label optimization through the contact point optimization strategy, which can avoid damage to the stamping parts due to improper contact points between the manipulator and the stamping parts and improve the grasping success rate.

[0092] S3. Perform trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory.

[0093] Assuming that the robot used in the present invention has n joints, the forward kinematic equation representing the position of the flexible manipulator is:

[0094] T(θ)=T1(θ1)·T2(θ2)·...·T n (θ n )

[0095] where θ k (k=1,2,…,n) is the angle of the kth joint, T k (θ k ) is the transformation matrix representing the joint pose.

[0096] The position of the flexible manipulator's end effector can be determined based on the position of the flexible manipulator, thereby performing trajectory planning.

[0097] In an embodiment of the present invention, an A* or Dijkstra algorithm is used for trajectory planning. Specifically, after the real-time position information of the target stamping part is identified, a shortest collision-free path is generated by the A* or Dijkstra algorithm.

[0098] In an embodiment of the present invention, the optimization goal of trajectory planning for the flexible manipulator is to minimize the function value of the cost function and minimize the grasping error.

[0099] The expression of the cost function is:

[0100]

[0101] Where: J is the cost function, t represents time, T is the time period of grasping and packing operations, u(t) is the control input, i.e. force or torque; q(t) is the acceleration of the flexible manipulator joint, It can represent the smoothness of the trajectory; v(t) is the velocity of the end effector of the flexible manipulator, and w1, w2, and w3 are weight coefficients respectively.

[0102] The present invention defines the optimization target through the cost function of trajectory planning, which can make the trajectory smooth while ensuring the grasping accuracy, and can improve the stability of the flexible manipulator's high-speed grasping and packing operations.

[0103] Taking the interior door panel as an example, after the real-time position information of the door panel on the conveyor belt is identified, the A* or Dijkstra algorithm is used to generate a shortest and collision-free path. When the door panel moves on the conveyor belt, the flexible robot arm needs to grab the reinforcement area from above, and the path needs to avoid collision with other mechanical parts.

[0104] If the door panel changes position during transport, the flexible robot's planned trajectory also needs to be dynamically adjusted. For example, if the door panel tilts slightly, the trajectory planning algorithm will replan the flexible robot's motion trajectory based on sensor feedback to ensure that the robot can successfully grasp the door panel's contact point.

[0105] Stamped parts like door inner panels often have a certain surface curvature. To ensure smooth grasping, the flexible robot's trajectory can be planned based on the curvature of the stamped part, allowing the flexible robot to maintain close contact with the door inner panel during movement. For example, before grasping, the flexible robot can gradually approach the door from the side rather than making direct vertical contact to avoid uneven force caused by sudden contact.

[0106] The flexible manipulator of this invention possesses multiple degrees of freedom, making it possible to utilize these advantages in path planning for complex motions. For example, a car door inner panel might be moving rapidly on a conveyor belt, and the flexible manipulator could rotate and bend to match the panel's movement and position changes, achieving precise traction.

[0107] Furthermore, door inner panels may coexist on the same conveyor belt as other stamped parts or production equipment. During the grasping process, the robot's grasping trajectory must not interfere with other parts of the stamped part. To avoid collisions during the grasping process, the flexible robot relies on real-time feedback from sensors to perform real-time obstacle avoidance detection and plan an obstacle avoidance path. For example, sensors can detect the position and shape of other workpieces on the conveyor belt in real time. Based on this information, trajectory planning is performed, taking into account the robot's contact angle and movement trajectory when approaching the door inner panel, and instructing the robot to bypass these workpieces to avoid collisions and ensure smooth and accurate grasping.

[0108] After the grasping is completed, the flexible robot also needs to place the target stamping part in the designated packing position. The trajectory planning in this process also needs to be precisely designed to improve the packing efficiency and the stability of the placement of the target stamping part.

[0109] For example, door inner panels must be placed in a specific order and orientation to maximize space within the packaging space. Therefore, determining the optimal packing sequence is crucial. For example, door inner panels can be placed vertically or horizontally, depending on their shape, to minimize gaps during packing. During multi-layer packing, the robot must dynamically adjust the placement path of the next door inner panel based on the position of already packed doors. For example, when several layers of door inner panels have already been placed in the box, the robot must raise its gripping point and adjust the gripping path to ensure the next door is securely placed.

[0110] Furthermore, there is a risk of stamped parts colliding during the packing process, especially when packing space is limited. Sensors monitor the distance between the door inner panel and other workpieces in real time. If an impending collision is detected, the packing path should be adjusted immediately or the packing sequence should be replanned to ensure that the stamped parts are not damaged during packing.

[0111] In one possible implementation, a camera is added to the manipulator for visual monitoring and to achieve anti-shake and collision avoidance. By selecting a high-definition, low-latency camera and a depth sensor, combined with real-time image processing and anti-shake algorithms, stability during movement can be ensured. Image processing algorithms, such as OpenCV, perform edge detection and target recognition on images to ensure that the manipulator can correctly locate the type, position and posture of the target object. Combining IMU (inertial measurement unit) and visual information, the motion trajectory of the manipulator is smoothed through Kalman filter or PID control to reduce the accuracy deviation of the manipulator caused by vibration. A machine learning model is used to predict the possible jitter that may occur during movement, and the trajectory is fine-tuned and compensated in advance. Through deep learning models, such as convolutional neural networks (CNN), the dynamic motion trend of the target is predicted, thereby improving the stability of grasping and packing.

[0112] At the same time, vision is integrated with data from other sensors (such as IMUs and force sensors) to provide real-time feedback and adjustments through collision detection and path planning to avoid obstacles. This system combines visual information with data from other sensors, such as IMUs and force sensors, to establish a comprehensive environmental perception system. During operation, if the robot detects displacement of the object it is about to contact or an area unsuitable for grasping (such as a weak part of a car body), it automatically adjusts the grasping point. Path planning algorithms such as A* and Dijkstra are used to generate the robot's optimal motion trajectory to avoid collisions. The optimization goal is to minimize the cost function and grasping error. The cost function can include a comprehensive consideration of the robot's end-point velocity, acceleration, and torque input to ensure a smooth and stable trajectory. A laser scanner or depth camera monitors the robot's surroundings in real time, dynamically updating the path to avoid sudden obstacles. Force sensors also detect the contact force between the robot and the environment to prevent damage to the workpiece due to excessive force.

[0113] By using high-speed communication methods such as industrial Ethernet (such as EtherCAT), the data transmission speed between sensors and controllers can be increased, reducing overall system latency. Integrating multiple cameras into the robot arm to form a stereo vision system enhances the spatial recognition capability of the target object. For example, two or more cameras can be used to acquire 3D point cloud data of the target object, and then the object's position and posture can be determined through point cloud registration. By fusing data from cameras, IMUs, and force sensors to establish a multimodal sensor information fusion system, the robot arm's intelligent perception capabilities can be enhanced. For example, when the vision system fails due to changes in lighting, the IMU can provide posture reference data to ensure system stability.

[0114] At the same time, the manipulator design was optimized to ensure balance and vibration resistance. High-speed communication and multi-camera collaboration were used to improve overall accuracy and response speed, enhancing the system's intelligence and safety. Specifically, the manipulator's structure was optimized to achieve good symmetry, lower its center of gravity, and increase stability, thereby reducing vibration caused by inertia. High-strength, lightweight materials such as carbon fiber or aluminum alloy were selected to reduce the manipulator's mass, thereby improving dynamic performance and vibration resistance.

[0115] S4. Grab and pack the target stamping parts according to the planned trajectory of the flexible manipulator.

[0116] During the process of grasping the target stamping part according to the planned trajectory of the flexible manipulator, the position or posture of the target stamping part is monitored in real time to see if it is offset. If so, in addition to re-predicting the contact point of the flexible manipulator through the pre-trained contact point prediction model, the trajectory planning needs to be re-performed to update the planned trajectory.

[0117] The present invention monitors the position or posture of the target stamping part in real time during the movement of the flexible manipulator. If so, the flexible manipulator's contact points are re-predicted using a pre-trained contact point prediction model and trajectory planning is performed again. This enables real-time detection and dynamic adjustment of the grasping action, adapting to the needs of high-speed production lines, thereby significantly improving the production line's automation level and overall production efficiency. If an error occurs during grasping or packing, such as a grasping failure or a deviation in the body's posture, automatic detection is performed and a recovery mechanism is activated to reposition the contact points and resume operation.

[0118] This invention integrates a flexible manipulator with real-time sensor data processing technology and a machine learning model. Using a pre-trained contact point prediction model, the flexible manipulator's contact points are predicted in real time, enabling more accurate trajectory planning. This system can adaptively handle stamping parts of varying shapes and sizes, enhancing the system's adaptability and flexibility, significantly improving grasping accuracy, reducing grasping failures, and lowering the risk of surface damage to stamping parts. The invention's strategy of dynamic trajectory planning for the flexible manipulator's end effector based on real-time sensor data processing further shortens packaging time and improves efficiency.

[0119] The high-performance control and real-time response capabilities of this invention enable the grasping and packing of automotive stamping parts to meet the demands of high-speed production lines. It enables real-time detection of target states and dynamic adjustment of grasping actions, significantly improving the automation level and overall production efficiency of the production line. This invention is highly adaptable to complex, irregularly shaped stamping parts and has pioneered the rapid positioning and efficient, precise packing of irregularly shaped stamping parts by multiple robots, achieving a packing accuracy of 5000±2.5mm and a packing speed of 15 pieces / min, improving both grasping accuracy and packing efficiency.

[0120] On the basis of the above method embodiment, the present invention also discloses a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator. Figure 2 The figure shows the operation process of a dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator of the present invention. The system includes the following functional modules:

[0121] Data acquisition module: used to obtain three-dimensional point cloud data of the target stamping parts on the conveyor belt in real time through sensors, extract key geometric features of the target stamping parts, and determine the type, position and posture of the target stamping parts;

[0122] Contact point prediction module: used to input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator;

[0123] Trajectory planning module: used for performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory;

[0124] Manipulator control module: used to grab and pack target stamping parts according to the planned trajectory of the flexible manipulator.

[0125] The above system embodiments and method embodiments correspond one to one. For a brief description of the system embodiments, please refer to the method embodiments.

[0126] The present invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface and a bus; wherein the processor, memory and communication interface communicate with each other through the bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the aforementioned method of the present invention.

[0127] The present invention also discloses a computer-readable storage medium storing computer instructions that cause the computer to implement all or part of the steps of the method described in the embodiments of the present invention. The storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0128] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be distributed across multiple network units. A person skilled in the art may, without inventive effort, select some or all of the modules as needed to achieve the objectives of this embodiment.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator, characterized in that: The method comprises: Acquire three-dimensional point cloud data of a target stamping part on a conveyor belt in real time, extract key geometric features of the target stamping part, and determine the type, position, and posture of the target stamping part; The type, key geometric features, and posture of the target stamping part are input into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator. Performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory; Grasping and packing target stamping parts according to the planned trajectory of the flexible manipulator; The pre-trained contact point prediction model is trained by historical captured data of stamping parts, and its training method is as follows: Acquire historical grasping data of the stamped part, the historical grasping data including the type of the stamped part, key geometric features, grasping posture, and contact points of the flexible manipulator; the key geometric features are corner points, edge features, and normal vectors extracted from the three-dimensional point cloud data of the corresponding stamped part; The data set is constructed by using the type, key geometric features, and grasping posture of the stamped parts in the historical grasping data as sample attributes and the contact points of the flexible manipulator as sample labels; Training a machine learning model using the data set to obtain a pre-trained touch point prediction model; The complete formula of the touch point prediction model can be expressed as: ； Where X is the input vector, the input vector is the vector corresponding to the key geometric feature, W (2) is the output layer weight matrix, W (1) is the weight matrix, b (1) and b (2) is the bias vector, and σ(x) is the activation function.

2. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: Determining the type of the target stamping part specifically includes: The three-dimensional point cloud data of the body-in-white is acquired, the three-dimensional point cloud data of the target stamping part is matched with the three-dimensional point cloud data of the body-in-white, and the type of the target stamping part is determined according to the point cloud matching result.

3. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: The optimization goal of trajectory planning for the flexible manipulator is to minimize the function value of the cost function and minimize the grasping error; The expression of the cost function is: ； Where: J is the cost function, t represents time, T is the time period of the grabbing and packing operation, u ( t ) is the control input, i.e., force or torque; q ( t ) is the acceleration of the flexible manipulator joint, v ( t ) is the speed of the flexible manipulator end effector, w 1. w 2. w 3 are weight coefficients respectively.

4. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: After inputting the type, key geometric features, and posture of the target stamping part into the pre-trained contact point prediction model and predicting the contact points of the flexible manipulator, the trajectory of the flexible manipulator is planned according to the position, posture, and predicted contact points of the target stamping part, and before obtaining the planned trajectory, the method further includes: If the predicted contact points of the flexible manipulator involve weak parts of the vehicle body, the contact points are adjusted so that each contact point is located in a relatively strong part of the structure.

5. The method for dynamic grasping and packing of automobile stamping parts based on a flexible manipulator according to claim 1 is characterized in that: During the process of grasping the target stamping part according to the planned trajectory of the flexible manipulator, the position or posture of the target stamping part is monitored in real time to see if it is offset. If so, the contact point of the flexible manipulator is re-predicted through the pre-trained contact point prediction model, and the trajectory is re-planned to update the planned trajectory.

6. A dynamic grasping and packing system for automobile stamping parts based on a flexible manipulator, characterized in that: The system is applied to the method according to any one of claims 1 to 5, and the system includes: Data acquisition module: used to acquire the three-dimensional point cloud data of the target stamping part on the conveyor belt in real time, extract the key geometric features of the target stamping part, and determine the type, position and posture of the target stamping part; Contact point prediction module: used to input the type, key geometric features and posture of the target stamping part into the pre-trained contact point prediction model to predict the contact points of the flexible manipulator; Trajectory planning module: used for performing trajectory planning of the flexible manipulator according to the position and posture of the target stamping part and the predicted contact points of the flexible manipulator to obtain a planned trajectory; Manipulator control module: used to grab and pack target stamping parts according to the planned trajectory of the flexible manipulator.

7. An electronic device, characterized in that: include: at least one processor, at least one memory, a communication interface, and a bus; The processor, memory, and communication interface communicate with each other via the bus. The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to implement the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions enable a computer to implement the method according to any one of claims 1 to 5.

Images