Self-adaptive taking and placing system based on placement state of special-shaped part structure

Abstract

The invention relates to a self-adaptive taking and placing system based on the placement state of a special-shaped part structure, and relates to the technical field of intelligent workpiece carrying. Comprising an image acquisition and simulation module for performing three-dimensional reconstruction based on a binocular vision image and identifying and matching a 3D part model, and a grabbing point identification and screening module determines a grabbing mode and automatically acquires grabbing points according to characteristics and placement requirements of the 3D part model; the mechanical arm action module automatically optimizes a minimum posture adjustment path after grabbing is achieved, and rapid posture adjustment and transfer are achieved. According to the method, the grabbing mode is autonomously determined by analyzing the shape and material characteristics of the special-shaped part, an optimization model is constructed in the corresponding grabbing mode, multiple sets of grabbing points corresponding to the special-shaped part are found, screening is conducted according to the actual placement situation, after grabbing is conducted, the special-shaped part is automatically rotated to the target posture according to the minimum rotation path, the placement requirement is matched, and the grabbing efficiency is improved. The whole process is combined with an optimization algorithm through intelligent recognition, and high automation is achieved.

Description

Technical Field

[0001] This invention relates to the field of intelligent workpiece handling technology, specifically an adaptive pick-and-place system based on the placement state of irregularly shaped parts. Background Technology

[0002] With the development of intelligent technology, traditional manufacturing industries are also rapidly upgrading to intelligent manufacturing. In traditional manufacturing industries, workpieces at the process transition stage rely on manual sorting and transfer to the designated location of the next process station, which is labor-intensive and time-consuming.

[0003] In existing technologies, end effectors on robotic arms are used to pick up and transfer workpieces. These end effectors generally employ mechanical gripping or suction cup gripping (including vacuum suction cups and electromagnetic suction cups). For workpieces with regular shapes, such as flat or round shafts, the picking and placing workpieces can be completed simply by using machine vision to bring the robotic arm to a braking position. However, for irregularly shaped parts, especially those in a free-placed state, their varied postures and complex structures make it difficult for the transmission-based robotic arm picking and placing system to meet real-world needs.

[0004] While existing technologies utilize binocular vision to ensure the gripping accuracy of robotic arms, the gripping point position still requires manual setting. When the gripping point is obstructed by irregularly shaped parts, posture adjustments are necessary, reducing the efficiency and automation level of part handling. Furthermore, current technologies lack the ability to adjust the posture within the shortest possible motion steps from gripping to placement, thus failing to meet the posture requirements of subsequent processes while minimizing wear on the transmission structure. Therefore, we propose an adaptive handling system based on the placement state of irregularly shaped parts. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive pick-and-place system based on the placement state of irregularly shaped parts.

[0006] The technical problem solved by this invention is:

[0007] (1) When the existing robotic arm grasps irregularly shaped parts, the grasping point needs to be set manually in advance before the robotic arm can be driven to grasp them.

[0008] (2) Existing robotic arms cannot automatically change their gripping methods based on the characteristics of irregularly shaped parts;

[0009] (3) When picking up parts that require a specific orientation after transfer, it is impossible to select the smallest orientation adjustment path, resulting in ineffective wear of transmission components and ineffective energy consumption.

[0010] This invention can be achieved through the following technical solution: an adaptive pick-and-place system based on the placement state of irregularly shaped parts, comprising:

[0011] The image acquisition and simulation module uses binocular vision technology to acquire images and reconstruct 3D models of parts in the placement area, and matches them with 3D part models in the data storage module to identify the corresponding 3D part models.

[0012] The gripping point recognition and filtering module determines the appropriate gripping method based on the shape characteristics, material, and whether there are any placement posture requirements of the irregular part. Then, it automatically obtains the gripping points of the corresponding irregular part according to different gripping methods.

[0013] The gripping strategy selection module identifies and marks the corresponding positions or areas of each gripping point in the image based on the coordinates of the gripping points in the relative coordinate system constructed based on the center of gravity of the part, and prioritizes the selection of irregular parts whose gripping points are not covered from the placement area for gripping; during gripping, the absolute coordinates of the gripping points are determined so that the end effector of the robotic arm should reach the gripping position.

[0014] The robotic arm motion module constructs an attitude adjustment optimization model with the goal of minimizing the rotation amount of the robotic arm driving the irregular part to rotate from the current attitude to the target attitude. It uses a swarm intelligence algorithm to solve for the minimum rotation path and adjusts the attitude according to the path, while performing transfer actions during the adjustment process.

[0015] A further technical improvement of the present invention lies in the process by which the image acquisition and simulation module combines the 3D reconstruction model with the 3D part model, including:

[0016] Calculate the volume of the 3D reconstructed model and filter out a set of 3D part models whose volume deviation is within a set range from the data storage module;

[0017] The overlap of each element in the 3D reconstructed model and the 3D part model set is calculated one by one. The formula is: overlap = overlap volume / (sum of volumes of the two models - overlap volume).

[0018] When the overlap is greater than the overlap threshold, select the corresponding 3D part model for subsequent operations.

[0019] A further technical improvement of the present invention is that: the gripping method for irregularly shaped parts includes mechanical gripping and adsorption gripping. The gripping point identification and screening module prioritizes adsorption gripping when the parts are made of easily breakable materials, magnetic metal materials, or have a wide flat surface. In other cases, mechanical gripping is used. Mechanical gripping is also used when there are requirements for the placement posture of the gripped parts.

[0020] A further technical improvement of the present invention is that, in the adsorption-type grasping method, the process of the grasping point identification and screening module acquiring the grasping point includes:

[0021] Based on the 3D part model, at least one adsorption surface is selected, and a perpendicular line is drawn from the center of the 3D part model to the selected adsorption surface, with the foot of the perpendicular as the adsorption center point.

[0022] On the premise that all adsorption points are in complete contact with the adsorption surface, calculate the range of the center deviation rate between the adsorption center point and the geometric center of the plane formed by the adsorption point matrix.

[0023] When the center deviation rate can be 0, that is, when the adsorption center can coincide with the above geometric center, the corresponding adsorption surface is marked as the preferred adsorption surface;

[0024] When the center deviation rate does not exceed the set threshold, the corresponding adsorption surface is marked as an additional adsorption surface; otherwise, it is marked as a dangerous adsorption surface.

[0025] For the additional adsorption surface and the preferred adsorption surface, the optimization objective is to minimize the deviation rate of the central sheet. The constraint is that the projection of any adsorption point along the vertical direction of the centroid falls on the adsorption surface, thereby obtaining the coordinates of each adsorption point in the relative coordinate system with the centroid as the origin, which are the coordinates of the grab point.

[0026] A further technical improvement of the present invention is that the formula for calculating the center deviation rate is: Where Δl is the deviation distance between the geometric center of the plane formed by the adsorption point matrix and the adsorption center, and d max This represents the distance between the two adsorption points that are furthest apart in the adsorption point matrix.

[0027] A further technical improvement of the present invention is that, in the mechanical grasping method, the process by which the grasping point identification and filtering module acquires the grasping point includes:

[0028] Determine the stable placement of irregularly shaped parts;

[0029] Under the above conditions, the 3D part model is sliced ​​in the vertical direction to obtain multiple slices;

[0030] Ignore slices with discontinuous areas, and calculate the outer contour areas of the upper and lower cut surfaces of the remaining slices in turn. Then, calculate the outer contour areas S of the upper and lower cut surfaces. 上 、S 下 Calculate the section deviation rate of the corresponding slice.

[0031] The optimization objective is to minimize the variance of the distances between multiple gripping points and the center of gravity, and constraints are set as follows: The coordinates of one or more sets of points in a relative coordinate system with the centroid as the origin are obtained by using a swarm intelligence algorithm, which are the coordinates of the grab points.

[0032] Where G represents the center of gravity, Ai Indicates the crawl point, This represents the spatial threshold obtained by projecting the plane defined by connecting the grab points sequentially into the vertical direction. This represents the feature threshold formed by the feature surfaces marked on the 3D part model.

[0033] A further technical improvement of the present invention is that, for the selection of the parts to be grasped, the grasping strategy selection module marks the grasping point location area in the placement area image according to the grasping point coordinates, and prioritizes grasping irregularly shaped parts whose grasping points are completely covered in the placement area:

[0034] Adsorption-type gripping selects and grips irregularly shaped parts in the order of having a preferred adsorption surface and then an additional adsorption surface.

[0035] Mechanical gripping selects the part with the highest overlap between its posture and the model posture during the layered slicing process in the gripping point recognition and filtering module from among the irregularly shaped parts located on the surface of the placement area and whose gripping position or area is not covered.

[0036] A further technical improvement of the present invention is as follows: for the selection of the gripping point, the gripping strategy selection module uses a 3D part model to simulate the current placement state of the irregular part in the placement area, constructs an absolute spatial coordinate system with the current working area, determines the absolute coordinates of the center of gravity of the irregular part in the absolute coordinate system, obtains the absolute coordinates of the gripping point based on the gripping point coordinates and the absolute coordinates of the center of gravity of the irregular part, and thus determines the gripping position that the end effector of the robotic arm should reach.

[0037] In adsorption-based grasping, once the adsorption surface is determined, the absolute coordinates of the grasp can be determined based on the absolute coordinates of the adsorption center.

[0038] In mechanical gripping, the point with the shortest sum of distances to the center of gravity is selected from a set of multiple points that can be gripped as the gripping point.

[0039] A further technical improvement of this invention is that: the mechanical motion module constructs an attitude adjustment optimization model, with the optimization objective being to minimize the rotation amount by which the robotic arm drives the irregularly shaped part to rotate from the current attitude to the target attitude. The constraints include: the joint motion angle of the robotic arm is within the allowable range, the torque of the robotic arm joint at each angle is within the set range, and a rotational space domain is set to avoid collisions; the joint angle of the robotic arm is used as a decision variable; the ant colony algorithm is used to solve the minimum rotation path; the robotic arm motion module adjusts the attitude according to the minimum rotation path and performs transfer actions during the adjustment process.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. This invention uses an image acquisition and simulation module to match images acquired by binocular vision with 3D part models in a data storage module to identify the corresponding 3D part models. Then, a gripping point recognition and filtering module automatically determines the preferred gripping method based on the shape features, material properties, and whether there are any placement posture requirements of the irregular part. The robotic arm then changes the end effector under the corresponding gripping method to perform subsequent gripping work.

[0042] 2. After determining a suitable gripping method for irregularly shaped parts, this invention constructs different optimization models to obtain the optimal gripping position. Specifically, in adsorption-type gripping, the preferred adsorption surface and the additional adsorption surface are marked according to the center deviation rate between the geometric center of the plane formed by the adsorption point matrix and the adsorption center. The coordinates of the adsorption point (grip point coordinates) with the smallest center deviation rate are found on the above adsorption surfaces to ensure the feasibility and stability of gripping. In mechanical gripping, the irregularly shaped parts are sliced ​​in a stable placement state, and the deviation rate of the outer contour area of ​​the upper and lower surfaces of each slice is calculated to determine the structural form of the corresponding slice. The coordinates of the gripping point with the smallest variance of the distance between the gripping point and the center of gravity are found on the outer contour of each slice to ensure the stability of gripping. This greatly saves the tedious work of manually finding gripping points and positioning before gripping, improves efficiency, and provides selectivity for the setting of multiple adsorption surfaces and gripping points, based on the possibility of gripping point occlusion or coverage due to different placement states of the parts.

[0043] 3. This invention uses a posture adjustment and optimization model built in the robotic arm motion module to optimize the rotation path between the current posture and the target posture of the part when there are requirements for the placement state of the part after it is grasped. This minimizes the amount of rotation, reduces wear on the robotic arm parts and the falling of parts due to excessive movements, and also reduces the movement time. Attached Figure Description

[0044] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0045] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0046] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0047] See also Figure 1 As shown, an adaptive picking and placing system based on the placement state of irregularly shaped parts includes an image acquisition and simulation module, a gripping strategy selection module, a robotic arm motion module, a gripping point recognition and filtering module, and a data storage module.

[0048] The image acquisition and simulation module includes a binocular stereo vision camera fixed at a certain height above the parts placement area to capture images of irregularly shaped parts. Based on binocular vision technology, it acquires two-dimensional image coordinates and depth information, converts the depth information and two-dimensional image coordinates of each pixel into points in three-dimensional space, and traverses the entire two-dimensional image to generate a point cloud image containing a large number of three-dimensional coordinate points. Based on the point cloud image, it completes the three-dimensional reconstruction of irregularly shaped parts in a certain pose within the parts placement area. Specifically, the three-dimensional reconstruction process includes point cloud data preprocessing (filtering and denoising, local data downsampling, and data interpolation), point cloud data calculation and registration, data fusion, and surface reconstruction. These are existing technologies and will not be elaborated here.

[0049] The data storage module stores a large number of 3D part models drawn by 3D drawing software. It obtains 3D part models with volume deviation within a controllable range and compares them with the 3D reconstruction model one by one. The overlap is calculated as: overlap = overlap volume / (sum of volumes of the two models - overlap volume). If the overlap is greater than the overlap threshold, the corresponding 3D part model is directly used to replace the 3D reconstruction model for subsequent operations.

[0050] The gripping point recognition and filtering module filters out gripping points suitable for the corresponding gripping methods based on the above 3D part model. The gripping methods include mechanical gripping and suction gripping. The specific process includes:

[0051] Step 1: Determine the gripping method for irregularly shaped parts

[0052] Common gripping methods include mechanical gripping with flexible claws and adsorption gripping using vacuum adsorption or electromagnetic adsorption.

[0053] For irregularly shaped parts made of easily breakable materials (such as glass), vacuum adsorption should be used as the preferred method for gripping.

[0054] For irregularly shaped parts with large flat surfaces, mechanical gripping is limited by the size of the clamping fixture, so adsorption gripping is preferred; furthermore, when the surface finish is high and the mass is small, vacuum adsorption is used; when the material of the irregularly shaped part is mainly magnetic metal, electromagnetic adsorption can be used.

[0055] In other cases, mechanical gripping methods are generally used to transfer irregularly shaped parts.

[0056] Based on the above principles, when there are requirements for the placement posture of the transferred parts, mechanical gripping should be used because vacuum adsorption gripping is prone to falling when the posture changes, while electromagnetic adsorption gripping is difficult to control the posture of irregularly shaped parts during gripping.

[0057] Based on the shape characteristics, material, and whether there are any placement requirements of the 3D part model, and in accordance with the above principles, select a gripping method suitable for the corresponding irregular part.

[0058] Step 2: Determine the gripping points for irregularly shaped parts

[0059] The density of the solid part of the irregular part described in this application is uniform. Therefore, the coordinates of the center of gravity of the irregular part can be quickly determined by computer software based on the specific features of the 3D part model. In the 3D part model, the center of gravity is taken as the origin of the coordinate system, and three perpendicular and non-coplanar straight lines are drawn from the origin to construct a relative spatial coordinate system. Then, any point on the irregular part has a definite coordinate position relative to the center of gravity.

[0060] For adsorption-based grasping:

[0061] First, select a wide plane (not limited to one) that can be used as an adsorption surface based on the 3D part model. Draw perpendicular lines from the centroid of the 3D part model to the selected adsorption surfaces to obtain the foot of the perpendicular, and mark the foot of the perpendicular as the adsorption center point.

[0062] When performing adsorption gripping, the geometric center of the plane formed by the adsorption point matrix at the end of the robotic arm coincides with the adsorption center, and at this time each adsorption point can fully contact the wide plane of the part. The corresponding adsorption surface is marked as the preferred adsorption surface, and the coordinates of the corresponding adsorption center point are recorded.

[0063] When the geometric center of the plane formed by the adsorption point matrix at the end of the robotic arm coincides with the adsorption center, and at least one adsorption point cannot make complete contact with the wide plane of the part, it is necessary to make a certain deviation distance between the adsorption center and the geometric center of the plane formed by the adsorption point matrix in order to make all adsorption points contact the wide plane of the part.

[0064] The limit deviation distance is determined by ensuring that all adsorption points in the adsorption point matrix are in complete contact with the wide plane of the part. The corresponding center deviation rate is then calculated and compared with a set threshold.

[0065] When the center deviation rate is greater than the set threshold, the adsorption surface cannot meet the safe adsorption gripping requirement, and the adsorption surface is marked as a dangerous adsorption surface.

[0066] If the center deviation rate does not exceed the set threshold, the adsorption surface meets the requirements for safe adsorption gripping, and the adsorption surface is marked as an additional adsorption surface.

[0067] It should be noted that the formula for calculating the center deviation rate ε is: Where Δl is the deviation distance between the geometric center of the plane formed by the adsorption point matrix and the adsorption center, and d max This represents the distance between the two farthest adsorption points in the adsorption point matrix. When the adsorption point matrix is ​​rectangular... L1 and L2 are the length and width of the rectangle, respectively.

[0068] In summary, for the adsorption-type gripping method, the optimization objective is to minimize the center deviation rate, i.e., minε; the constraint condition is that the projection of any adsorption point along the vertical direction of the center of gravity falls on the adsorption surface; thus, the coordinates of the adsorption point on the corresponding adsorption surface are obtained. These adsorption point coordinates are relative to the center of gravity of the part, i.e., the gripping point coordinates.

[0069] For mechanical gripping, let's take a three-finger gripper as an example:

[0070] Define the rule constraints for the gripping points in mechanical gripping:

[0071] (1) The line connecting the three gripping points must include the area where the center of gravity is located;

[0072] (2) The distance from the center of gravity to the three gripping points should be as equal as possible to ensure balanced force distribution;

[0073] (3) The gripping point should avoid the feature surface of the hole and groove, the positioning reference surface and the high-precision working surface as much as possible;

[0074] (4) The projected area of ​​the three grab points in the horizontal plane should be as large as possible.

[0075] The placement state in which the support point covers the largest area and the center of gravity is projected onto the horizontal plane within the support point coverage area is selected as the stable placement state.

[0076] In this placement state, the 3D part model is sliced ​​vertically, that is, sliced ​​horizontally according to the set thickness. The slice thickness is positively correlated with the height of the part in the stable placement state, and the slice thickness does not exceed twice the height of the gripper surface.

[0077] After obtaining multiple slices through layered slicing, the outer contour areas of the upper and lower cut surfaces of each slice are calculated sequentially, ignoring slices with discontinuous areas. Then, the cut surface deviation rate is calculated. Among them, S 上 、S 下 The two values ​​represent the outer contour areas of the upper and lower cut surfaces of the corresponding slice, respectively. The cut surface deviation rate is marked with a sign. When the slice deviation rate is positive, it means that the upper cut surface is larger than the lower cut surface, and the slice has an inverted cone structure. Otherwise, the slice has a columnar or upright cone structure.

[0078] Construct an optimization model, assuming the coordinates of the three grab points are A1(x1,y1,z1), A2(x2,y2,z2), and A3(x3,y3,z3), and the centroid coordinates are G(x G ,y G ,zG );

[0079] The optimization objective is to minimize the variance of the distances between the three gripping points and the center of gravity, i.e., to make the distances as equal as possible. 2 Among them, variance |A1G|、|A2G|、|A3G|、 These represent the distances from the three gripping points to the center of gravity, and the average distance from each point to the center of gravity, respectively.

[0080] The constraints are:

[0081] (1) The spatial threshold is obtained by projecting the planar threshold defined by the line connecting the three grasping points into the vertical direction. The center of gravity is located within the spatial threshold, that is

[0082] (2) Slice deviation rate

[0083] (3) Mark the feature surfaces of holes and slots, the positioning reference surfaces, and the high-precision working surfaces according to the 3D part model, and obtain the feature surface thresholds of the above surfaces. Therefore, the three grab points do not fall within these feature surfaces, that is: and

[0084] Genetic algorithms, ant colony algorithms, and other swarm intelligence algorithms are used to solve the optimization model to obtain one or more sets of points that meet the conditions.

[0085] Under the above stable placement state, there is no In this case, the placement state is reversed and the slicing and layering process is repeated. After updating parameters such as the centroid coordinates, the optimization model is solved to obtain one or more sets of points that meet the conditions.

[0086] The coordinates of the above point set are relative point set coordinates based on the centroid, that is, the coordinates of the grab point.

[0087] After determining the gripping method and gripping points, the gripping strategy selection module determines the position or area of ​​the gripping point on the 3D part model based on the coordinates of the gripping point. After acquiring the image of the placement area using binocular vision technology, the module identifies and marks the position or area corresponding to each gripping point in the image and displays it using a highlighted rectangle.

[0088] Prioritize picking up irregularly shaped parts whose gripping points are not completely covered from the placement area:

[0089] For adsorption-type gripping, priority is given to gripping irregularly shaped parts with preferred adsorption surfaces. When the preferred adsorption surface is covered or blocked, the irregularly shaped parts with additional adsorption surfaces are marked for gripping. The identification and marking are re-performed after each gripping.

[0090] For mechanical gripping, among the irregularly shaped parts located on the surface of the placement area and whose gripping position or area is not covered, the part with the highest overlap between its posture and the model posture during the layered slicing process in the gripping point recognition and filtering module is selected for gripping.

[0091] The grasping strategy selection module uses a 3D part model to simulate the current placement state of the irregular part in the placement area, and constructs an absolute spatial coordinate system based on the current working area. During the grasping process, the absolute coordinates of the center of gravity of the irregular part in the absolute coordinate system are first determined. Based on the coordinates of the grasping point and the absolute coordinates of the center of gravity of the irregular part, the absolute coordinates of the grasping point are obtained, thereby determining the grasping position that the end effector of the robotic arm should reach.

[0092] After the robotic arm motion module drives the end effector to the gripping position, it performs gripping actions according to different gripping methods and locates the gripping point. It should be noted that for mechanical gripping, the point with the shortest sum of distances to the center of gravity is selected from multiple sets of points that can be gripped as the gripping point.

[0093] When there are requirements for the placement of parts after they are grasped, in order to reduce wear on the robotic arm parts and the parts falling due to excessive movements, as well as to reduce the movement time, the robotic arm motion module also has a posture adjustment optimization model. The optimization objective is to minimize the amount of rotation of the robotic arm that drives the irregular part from the current posture to the target posture. The constraints include: the robotic arm joint motion angle is within the allowable range, the torque of the robotic arm joint at each angle is within the set range, and the rotation space domain is set to avoid collisions. The robotic arm joint angle is used as a decision variable, and the ant colony algorithm is used to solve the minimum rotation path.

[0094] Specifically, quaternions are used to represent the current attitude and the target attitude, respectively, and the angular difference between the two attitudes, i.e., the rotation, is calculated. A quaternion is a rotation represented by one real number and three imaginary numbers, which constitute a four-dimensional hypercomplex space. A quaternion can be viewed as consisting of a scalar part and a vector part, if an object revolves around a unit vector... Rotate by an angle θ, then the corresponding quaternion is:

[0095] The robotic arm motion module adjusts its posture according to the minimum rotation path and performs transfer actions during the adjustment process.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An adaptive pick-and-place system based on the placement status of special-shaped parts, characterized in that: include: The image acquisition and simulation module acquires images and performs three-dimensional reconstruction of the parts in the display area based on binocular vision technology, matches them with the 3D part models in the data storage module, and identifies the corresponding 3D part models; The grasping point identification and screening module determines the corresponding grasping method according to the shape characteristics, material and whether there are any placement requirements of the special-shaped parts, and then automatically obtains the grasping points of the corresponding special-shaped parts according to different grasping methods; The grasping strategy selection module identifies and marks the corresponding position or area of ​​each grasping point in the image according to the grasping point coordinates in the relative coordinate system constructed based on the center of gravity of the part, and preferentially selects the special-shaped parts whose grasping points are not covered from the placement area for grasping; when grasping, the absolute coordinates of the grasping point are determined so that the end picker of the robot arm should reach the grasping position; The robot arm motion module constructs a posture adjustment optimization model with the optimization goal of minimizing the rotation amount of the special-shaped parts driven by the robot arm from the current posture to the target posture, and solves the minimum rotation path through the swarm intelligence algorithm, and adjusts the posture according to the path, while performing transfer actions during the adjustment process.

2. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 1, characterized in that: The process of the image acquisition and simulation module combining the three-dimensional reconstruction model with the 3D part model includes: Calculate the volume of the 3D reconstructed model, and select a set of 3D part models whose volume deviation is within a set range from the data storage module; Calculate the overlap degree of each element in the 3D reconstructed model and the 3D part model set one by one, and the calculation formula is: overlap degree = overlap volume / (the sum of the volumes of the two models - overlap volume); When the overlap degree is greater than the overlap degree threshold, the corresponding 3D part model is selected for subsequent operations.

3. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 1, characterized in that: The grabbing methods for special-shaped parts include mechanical clamping and adsorption grabbing. The grabbing point identification and screening module gives priority to adsorption grabbing when using fragile materials, magnetic metal materials or materials with a wide surface. In other cases, mechanical grabbing is used; and when there are requirements for the placement posture of the grasped parts, mechanical grabbing is also used.

4. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 3, characterized in that: In the adsorption-type grabbing method, the process of the grabbing point identification and screening module acquiring the grabbing point includes: At least one adsorption surface is selected according to the 3D part model, and a perpendicular line is drawn from the center of the 3D part model to the selected adsorption surface, and the foot of the perpendicular line is taken as the adsorption center point; On the basis of ensuring that all adsorption points are in complete contact with the adsorption surface, the center deviation rate range value between the adsorption center point and the geometric center of the plane formed by the adsorption point matrix is ​​calculated; When the center deviation rate can be 0, that is, when the adsorption center and the above-mentioned geometric center can coincide, the corresponding adsorption surface is marked as the preferred adsorption surface; When the center deviation rate does not exceed the set threshold, the corresponding adsorption surface is marked as an additional adsorption surface, otherwise it is marked as a dangerous adsorption surface; For the additional adsorption surface and the priority adsorption surface, the optimization goal is to minimize the deviation rate of the center piece. The constraint condition is that the projection of any adsorption point along the vertical line of the center of gravity falls on the adsorption surface, so as to obtain the coordinates of each adsorption point in the relative coordinate system with the center of gravity as the origin, which is the coordinates of the grabbing point.

5. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 4, characterized in that: The calculation formula of the center deviation rate is: Where Δl is the deviation distance between the geometric center of the plane formed by the adsorption point matrix and the adsorption center, d max Indicates the distance between the two farthest adsorption points in the adsorption point matrix.

6. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 4, characterized in that: In the mechanical grasping method, the process of the grasping point identification and screening module acquiring the grasping point includes: Determine the stable placement of special-shaped parts; In the above state, the 3D part model is subjected to layered slicing processing in the vertical direction to obtain a plurality of slices; Ignore the slices with discontinuous areas, and calculate the outer contour areas of the upper and lower sections of the remaining slices in turn. 上 , S 下 Calculate the section deviation rate of the corresponding slice The optimization goal is to minimize the variance of the distances between multiple grasping points and the center of gravity, and set the constraints: ① ② ③ A swarm intelligence algorithm is used to solve and obtain one or more sets of point set coordinates in a relative coordinate system with the center of gravity as the origin that meet the conditions, which are the grabbing point coordinates; Among them, G represents the center of gravity, A i Represents the grab point, It represents the spatial threshold obtained by projecting the plane determined by connecting the grasping points in sequence in the vertical direction. Indicates the feature face threshold consisting of the feature faces marked by the 3D part model.

7. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 6, characterized in that: For the selection of grasped parts, the grasping strategy selection module marks the grasping point location area in the placement area image according to the grasping point coordinates, and preferentially selects the grasping points from the placement area and the special-shaped parts that are completely covered by the grasping points for grasping: Adsorption-type grasping selects and grasps special-shaped parts in the order of the existing priority adsorption surface and the additional adsorption surface; Mechanical grasping selects the part with the highest degree of coincidence between the part posture and the model posture during the layered slicing processing in the grasping point identification and filtering module from among the special-shaped parts located on the surface of the placement area and whose grasping position or area is not covered.

8. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 7, characterized in that: For the selection of the grasping point, the grasping strategy selection module uses the 3D part model to simulate the current placement state of the special-shaped part in the placement area, constructs an absolute space coordinate system based on the current working area, determines the absolute coordinates of the center of gravity of the special-shaped part in the absolute coordinate system, and obtains the absolute coordinates of the grasping point based on the coordinates of the grasping point and the absolute coordinates of the center of gravity of the special-shaped part, thereby determining the grasping position that the end picker of the robot arm should reach; In adsorption-type grasping, after determining the adsorption surface, the absolute coordinates of the grasping can be determined based on the absolute coordinates of the adsorption center; In mechanical grasping, the point with the shortest sum of distances to the center of gravity is selected from multiple groups of points that can be grasped as the grasping point for grasping.

9. The adaptive pick-and-place system based on the placement state of special-shaped parts according to claim 1, characterized in that: The mechanical action module is constructed with a posture adjustment optimization model, with the optimization goal of minimizing the rotation amount of the special-shaped parts driven by the robot arm from the current posture to the target posture. The constraints include: the robot arm joint motion angle is within the allowable range, the torque of the robot arm joint at each angle is within the set range, and the rotation space domain is set to avoid collision; the robot arm joint angle is used as a decision variable; the ant colony algorithm is used to solve the minimum rotation path; the robot arm action module adjusts the posture according to the minimum rotation path, and performs transfer actions during the adjustment process.

Images