An adaptive pick-and-place system based on the placement state of irregularly shaped parts

Abstract

This invention relates to an adaptive pick-and-place system based on the placement state of irregularly shaped parts, belonging to the field of intelligent workpiece handling technology. It includes an image acquisition and simulation module that performs 3D reconstruction based on binocular vision images and identifies and matches a 3D part model; a gripping point identification and filtering module that determines the gripping method and automatically acquires gripping points based on the characteristics of the 3D part model itself and placement requirements; and a robotic arm motion module that automatically optimizes the minimum posture adjustment path after gripping, achieving rapid posture adjustment and transfer. This invention autonomously determines the gripping method by analyzing the shape and material characteristics of the irregularly shaped parts, constructs an optimization model within the corresponding gripping method, finds multiple sets of gripping points corresponding to the irregularly shaped parts, filters them according to the actual placement situation, and automatically rotates to the target posture according to the minimum rotation path after gripping, matching the placement requirements. The entire process achieves a high degree of automation through the combination of intelligent recognition and optimization algorithms.

Description

Technical Field

[0001] This invention relates to the field of intelligent workpiece handling technology, specifically an adaptive pick-and-place system based on the placement state of irregularly shaped parts. Background Technology

[0002] With the development of intelligent technology, traditional manufacturing industries are also rapidly upgrading to intelligent manufacturing. In traditional manufacturing industries, workpieces at the process transition stage rely on manual sorting and transfer to the designated location of the next process station, which is labor-intensive and time-consuming.

[0003] In existing technologies, end effectors on robotic arms are used to pick up and transfer workpieces. These end effectors generally employ mechanical gripping or suction cup gripping (including vacuum suction cups and electromagnetic suction cups). For workpieces with regular shapes, such as flat or round shafts, the picking and placing workpieces can be completed simply by using machine vision to bring the robotic arm to a braking position. However, for irregularly shaped parts, especially those in a free-placed state, their varied postures and complex structures make it difficult for the transmission-based robotic arm picking and placing system to meet real-world needs.

[0004] While existing technologies utilize binocular vision to ensure the gripping accuracy of robotic arms, the gripping point position still requires manual setting. When the gripping point is obstructed by irregularly shaped parts, posture adjustments are necessary, reducing the efficiency and automation level of part handling. Furthermore, current technologies lack the ability to adjust the posture within the shortest possible motion steps from gripping to placement, thus failing to meet the posture requirements of subsequent processes while minimizing wear on the transmission structure. Therefore, we propose an adaptive handling system based on the placement state of irregularly shaped parts. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive pick-and-place system based on the placement state of irregularly shaped parts.

[0006] The technical problem solved by this invention is:

[0007] When a robotic arm grasps irregularly shaped parts, the grasping points need to be pre-set manually before the robotic arm can be driven to grasp them.

[0008] Existing robotic arms cannot automatically change their gripping methods based on the characteristics of irregularly shaped parts;

[0009] When picking up parts that require a specific orientation after transfer, it is impossible to select the smallest orientation adjustment path, resulting in ineffective wear of transmission components and ineffective energy consumption.

[0010] This invention can be achieved through the following technical solution: an adaptive pick-and-place system based on the placement state of irregularly shaped parts, comprising:

[0011] The image acquisition and simulation module uses binocular vision technology to acquire images and reconstruct 3D models of parts in the placement area, and matches them with 3D part models in the data storage module to identify the corresponding 3D part models.

[0012] The gripping point recognition and filtering module determines the appropriate gripping method based on the shape characteristics, material, and whether there are any placement posture requirements of the irregular part. Then, it automatically obtains the gripping points of the corresponding irregular part according to different gripping methods.

[0013] The gripping strategy selection module identifies and marks the corresponding positions or areas of each gripping point in the image based on the coordinates of the gripping points in the relative coordinate system constructed based on the center of gravity of the part, and prioritizes the selection of irregular parts whose gripping points are not covered from the placement area for gripping; during gripping, the absolute coordinates of the gripping points are determined so that the end effector of the robotic arm can reach the gripping position.

[0014] The robotic arm motion module constructs an attitude adjustment optimization model with the goal of minimizing the rotation amount of the robotic arm driving the irregular part to rotate from the current attitude to the target attitude. It uses a swarm intelligence algorithm to solve for the minimum rotation path and adjusts the attitude according to the path, while performing transfer actions during the adjustment process.

[0015] A further technical improvement of the present invention lies in the following: the image acquisition and simulation module performs image acquisition and 3D reconstruction of the parts in the placement area based on binocular vision technology, and matches them with the 3D part models in the data storage module to identify the corresponding 3D part models, including:

[0016] Calculate the volume of the 3D reconstructed model and filter out a set of 3D part models whose volume deviation is within a set range from the data storage module;

[0017] The overlap of each element in the 3D reconstructed model and the 3D part model set is calculated one by one. The formula is: overlap = overlap volume / (sum of the volumes of the two models - overlap volume).

[0018] When the overlap is greater than the overlap threshold, select the corresponding 3D part model for subsequent operations.

[0019] A further technical improvement of the present invention is that: the gripping method for irregularly shaped parts includes mechanical gripping and adsorption gripping. The gripping point identification and screening module prioritizes adsorption gripping when the parts are made of easily breakable materials, magnetic metal materials, or have a wide flat surface. In other cases, mechanical gripping is used. Mechanical gripping is also used when there are requirements for the placement posture of the gripped parts.

[0020] A further technical improvement of the present invention is that, in the adsorption-type grasping method, the process of the grasping point identification and screening module acquiring the grasping point includes:

[0021] Based on the 3D part model, at least one adsorption surface is selected, and a perpendicular line is drawn from the center of the 3D part model to the selected adsorption surface, with the foot of the perpendicular as the adsorption center point.

[0022] On the premise that all adsorption points are in complete contact with the adsorption surface, calculate the range of the center deviation rate between the adsorption center point and the geometric center of the plane formed by the adsorption point matrix.

[0023] When the center deviation rate can be 0, that is, when the adsorption center can coincide with the above geometric center, the corresponding adsorption surface is marked as the preferred adsorption surface;

[0024] When the center deviation rate does not exceed the set threshold, the corresponding adsorption surface is marked as an additional adsorption surface; otherwise, it is marked as a dangerous adsorption surface.

[0025] For the additional adsorption surface and the preferred adsorption surface, the optimization objective is to minimize the deviation rate of the central sheet. The constraint is that the projection of any adsorption point along the vertical direction of the centroid falls on the adsorption surface, thereby obtaining the coordinates of each adsorption point in the relative coordinate system with the centroid as the origin, which are the coordinates of the grab point.

[0026] A further technical improvement of the present invention is that the formula for calculating the center deviation rate is: ,in, The distance between the geometric center of the plane formed by the adsorption point matrix and the adsorption center is denoted by . This represents the distance between the two adsorption points that are furthest apart in the adsorption point matrix.

[0027] A further technical improvement of the present invention is that, in the mechanical grasping method, the process by which the grasping point identification and filtering module acquires the grasping point includes:

[0028] Determine the stable placement of irregularly shaped parts;

[0029] Under the above conditions, the 3D part model is sliced ​​in the vertical direction to obtain multiple slices;

[0030] Ignore slices with discontinuous areas, and calculate the outer contour areas of the upper and lower cut surfaces of the remaining slices in turn. Calculate the section deviation rate of the corresponding slice. ;

[0031] The optimization objective is to minimize the variance of the distances between multiple grab points and the center of gravity, and the following constraints are set: ① ;② ;③ The coordinates of one or more sets of points in a relative coordinate system with the centroid as the origin are obtained by using a swarm intelligence algorithm; these are the coordinates of the grab points.

[0032] in, Indicates the center of gravity. Indicates the grab point, This represents the spatial threshold obtained by projecting the plane defined by connecting the grab points sequentially into the vertical direction. This represents the feature threshold formed by the feature surfaces marked on the 3D part model.

[0033] A further technical improvement of the present invention is that, for the selection of the parts to be grasped, the grasping strategy selection module marks the grasping point location area in the placement area image according to the grasping point coordinates, and prioritizes the grasping of irregularly shaped parts whose grasping points are not completely covered from the placement area:

[0034] Adsorption-type gripping selects and grips irregularly shaped parts in the order of having a preferred adsorption surface and then an additional adsorption surface.

[0035] Mechanical gripping selects the part with the highest overlap between its posture and the model posture during the layered slicing process in the gripping point recognition and filtering module from among the irregularly shaped parts located on the surface of the placement area and whose gripping position or area is not covered.

[0036] A further technical improvement of the present invention is as follows: for the selection of the gripping point, the gripping strategy selection module uses a 3D part model to simulate the current placement state of the irregular part in the placement area, constructs an absolute spatial coordinate system with the current working area, determines the absolute coordinates of the center of gravity of the irregular part in the absolute coordinate system, obtains the absolute coordinates of the gripping point based on the gripping point coordinates and the absolute coordinates of the center of gravity of the irregular part, and thus determines the gripping position that the end effector of the robotic arm should reach.

[0037] In adsorption-based grasping, once the adsorption surface is determined, the absolute coordinates of the grasp can be determined based on the absolute coordinates of the adsorption center.

[0038] In mechanical gripping, the point with the shortest sum of distances to the center of gravity is selected from a set of multiple points that can be gripped as the gripping point.

[0039] A further technical improvement of this invention is that: the robotic arm motion module constructs an attitude adjustment optimization model, with the optimization objective being to minimize the rotation amount by which the robotic arm drives the irregularly shaped part to rotate from the current attitude to the target attitude. The constraints include: the robotic arm joint motion angle is within the allowable range, the torque of the robotic arm joint at each angle is within the set range, and a rotational space domain is set to avoid collisions; the robotic arm joint angle is used as a decision variable; the ant colony algorithm is used to solve the minimum rotation path; the robotic arm motion module adjusts the attitude according to the minimum rotation path and performs transfer actions during the adjustment process.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. This invention uses an image acquisition and simulation module to match images acquired by binocular vision with 3D part models in a data storage module to identify the corresponding 3D part models. Then, a gripping point recognition and filtering module automatically determines the preferred gripping method based on the shape features, material properties, and whether there are any placement posture requirements of the irregular part. The robotic arm then changes the end effector under the corresponding gripping method to perform subsequent gripping work.

[0042] 2. After determining a suitable gripping method for irregularly shaped parts, this invention constructs different optimization models to obtain the optimal gripping position. Specifically, in adsorption-type gripping, the preferred adsorption surface and the additional adsorption surface are marked according to the center deviation rate between the geometric center of the plane formed by the adsorption point matrix and the adsorption center. The coordinates of the adsorption point (grip point coordinates) with the smallest center deviation rate are found on the above adsorption surfaces to ensure the feasibility and stability of gripping. In mechanical gripping, the irregularly shaped parts are sliced ​​in a stable placement state, and the deviation rate of the outer contour area of ​​the upper and lower surfaces of each slice is calculated to determine the structural form of the corresponding slice. The coordinates of the gripping point with the smallest variance of the distance between the gripping point and the center of gravity are found on the outer contour of each slice to ensure the stability of gripping. This greatly saves the tedious work of manually finding gripping points and positioning before gripping, improving efficiency. Moreover, based on the different placement states of the parts, there may be situations where gripping points are obstructed or covered. The setting of multiple adsorption surfaces and gripping points also provides selectivity.

[0043] 3. This invention uses a posture adjustment and optimization model built in the robotic arm motion module to optimize the rotation path between the current posture and the target posture of the part when there are requirements for the placement state of the part after it is grasped. This minimizes the amount of rotation, reduces wear on the robotic arm parts and the falling of parts due to excessive movements, and also reduces the movement time. Attached Figure Description

[0044] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0045] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0046] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0047] Please see Figure 1 As shown, an adaptive picking and placing system based on the placement state of irregularly shaped parts includes an image acquisition and simulation module, a gripping strategy selection module, a robotic arm motion module, a gripping point recognition and filtering module, and a data storage module.

[0048] The image acquisition and simulation module includes a binocular stereo vision camera fixed at a certain height above the parts placement area to capture images of irregularly shaped parts. Based on binocular vision technology, it acquires two-dimensional image coordinates and depth information, converts the depth information and two-dimensional image coordinates of each pixel into points in three-dimensional space, and traverses the entire two-dimensional image to generate a point cloud image containing a large number of three-dimensional coordinate points. Based on the point cloud image, it completes the three-dimensional reconstruction of irregularly shaped parts in a certain pose within the parts placement area. Specifically, the three-dimensional reconstruction process includes point cloud data preprocessing (filtering and denoising, local data downsampling, and data interpolation), point cloud data calculation and registration, data fusion, and surface reconstruction. These are existing technologies and will not be elaborated here.

[0049] The data storage module stores a large number of 3D part models drawn by 3D drawing software. It obtains 3D part models with volume deviation within a controllable range and compares them one by one with the 3D reconstruction model. The overlap is calculated as: overlap = overlap volume / (sum of volumes of the two models - overlap volume). If the overlap is greater than the overlap threshold, the corresponding 3D part model is directly used to replace the 3D reconstruction model for subsequent operations.

[0050] The gripping point recognition and filtering module filters out gripping points suitable for the corresponding gripping methods based on the above 3D part model. The gripping methods include mechanical gripping and suction gripping. The specific process includes:

[0051] Step 1: Determine the gripping method for irregularly shaped parts

[0052] Common gripping methods include mechanical gripping with flexible claws and adsorption gripping using vacuum adsorption or electromagnetic adsorption.

[0053] For irregularly shaped parts made of easily breakable materials (such as glass), vacuum adsorption should be used as the preferred method for gripping.

[0054] For irregularly shaped parts with large flat surfaces, mechanical gripping is limited by the size of the clamping fixture, so adsorption gripping is preferred; furthermore, when the surface finish is high and the mass is small, vacuum adsorption is used; when the material of the irregularly shaped part is mainly magnetic metal, electromagnetic adsorption can be used.

[0055] In other cases, mechanical gripping methods are generally used to transfer irregularly shaped parts.

[0056] Based on the above principles, when there are requirements for the placement posture of the transferred parts, mechanical gripping should be used because vacuum adsorption gripping is prone to falling when the posture changes, while electromagnetic adsorption gripping is difficult to control the posture of irregularly shaped parts during gripping.

[0057] Based on the shape characteristics, material, and whether there are any placement requirements of the 3D part model, and in accordance with the above principles, select a gripping method suitable for the corresponding irregular part.

[0058] Step 2: Determine the gripping points for irregularly shaped parts

[0059] The density of the solid part of the irregular part described in this application is uniform. Therefore, the coordinates of the center of gravity of the irregular part can be quickly determined by computer software based on the specific features of the 3D part model. In the 3D part model, the center of gravity is taken as the origin of the coordinate system, and three perpendicular and non-coplanar straight lines are drawn from the origin to construct a relative spatial coordinate system. Then, any point on the irregular part has a definite coordinate position relative to the center of gravity.

[0060] For adsorption-based grasping:

[0061] First, select a wide plane (not limited to one) that can be used as an adsorption surface based on the 3D part model. Draw perpendicular lines from the centroid of the 3D part model to the selected adsorption surfaces to obtain the foot of the perpendicular, and mark the foot of the perpendicular as the adsorption center point.

[0062] When performing adsorption gripping, the geometric center of the plane formed by the adsorption point matrix at the end of the robotic arm coincides with the adsorption center, and at this time each adsorption point can fully contact the wide plane of the part. The corresponding adsorption surface is marked as the preferred adsorption surface, and the coordinates of the corresponding adsorption center point are recorded.

[0063] When the geometric center of the plane formed by the adsorption point matrix at the end of the robotic arm coincides with the adsorption center, and at least one adsorption point cannot make complete contact with the wide plane of the part, it is necessary to make a certain deviation distance between the adsorption center and the geometric center of the plane formed by the adsorption point matrix in order to make all adsorption points contact the wide plane of the part.

[0064] The limit deviation distance is determined by ensuring that all adsorption points in the adsorption point matrix are in complete contact with the wide plane of the part. The corresponding center deviation rate is then calculated and compared with a set threshold.

[0065] When the center deviation rate is greater than the set threshold, the adsorption surface cannot meet the safe adsorption gripping requirement, and the adsorption surface is marked as a dangerous adsorption surface.

[0066] If the center deviation rate does not exceed the set threshold, the adsorption surface meets the requirements for safe adsorption gripping, and the adsorption surface is marked as an additional adsorption surface.

[0067] It should be noted that the center deviation rate The calculation formula is ,in, The distance between the geometric center of the plane formed by the adsorption point matrix and the adsorption center is denoted by . This represents the distance between the two farthest adsorption points in the adsorption point matrix. When the adsorption point matrix is ​​rectangular... , , These are the length and width of the rectangle, respectively.

[0068] In summary, for adsorption-based grasping methods, the optimization objective is to minimize the center deviation rate. The constraint condition is that the projection of any adsorption point along the vertical direction of the center of gravity falls on the adsorption surface; thus, the coordinates of the adsorption point on the corresponding adsorption surface are obtained. These coordinates are relative to the center of gravity of the part, that is, the coordinates of the gripping point.

[0069] For mechanical gripping, let's take a three-finger gripper as an example:

[0070] Define the rule constraints for the gripping points in mechanical gripping:

[0071] The line connecting the three gripping points must include the area where the center of gravity is located;

[0072] The distance from the center of gravity to the three gripping points should be as equal as possible to ensure balanced force distribution;

[0073] (3) The gripping point should avoid the feature surface of the hole and groove, the positioning reference surface and the high-precision working surface as much as possible;

[0074] (4) The projected area of ​​the three grab points in the horizontal plane should be as large as possible.

[0075] The placement state in which the support point covers the largest area and the center of gravity is projected onto the horizontal plane within the support point coverage area is selected as the stable placement state.

[0076] In this placement state, the 3D part model is sliced ​​vertically, that is, sliced ​​horizontally according to the set thickness. The slice thickness is positively correlated with the height of the part in the stable placement state, and the slice thickness does not exceed twice the height of the gripper surface.

[0077] After obtaining multiple slices through layered slicing, the outer contour areas of the upper and lower cut surfaces of each slice are calculated sequentially, ignoring slices with discontinuous areas. Then, the cut surface deviation rate is calculated. ,in, The two values ​​represent the outer contour areas of the upper and lower cut surfaces of the corresponding slice, respectively. The cut surface deviation rate is marked with a sign. When the slice deviation rate is positive, it means that the upper cut surface is larger than the lower cut surface, and the slice has an inverted cone structure. Otherwise, the slice has a columnar or upright cone structure.

[0078] Construct an optimization model, and let the coordinates of the three grab points be as follows: , , The centroid coordinates are ;

[0079] The optimization objective is to minimize the variance of the distances between the three gripping points and the center of gravity, making them as equal as possible. Among them, variance , These represent the distances from the three gripping points to the center of gravity, and the average distance from each point to the center of gravity, respectively.

[0080] The constraints are:

[0081] (1) The spatial threshold is obtained by projecting the planar threshold defined by the line connecting the three grasping points into the vertical direction. Then the center of gravity is located within the spatial threshold, that is ;

[0082] (2) Slice deviation rate ;

[0083] (3) Mark the hole and groove feature surfaces, positioning reference surfaces and high-precision working surfaces according to the 3D part model, and obtain the feature surface thresholds of the above surfaces. If the three grab points do not fall within these feature surfaces, then: , and ;

[0084] Genetic algorithms, ant colony algorithms, and other swarm intelligence algorithms are used to solve the optimization model to obtain one or more sets of points that meet the conditions.

[0085] Under the above stable placement state, there is no In this case, the placement state is reversed and the slicing and layering process is repeated. After updating parameters such as the centroid coordinates, the optimization model is solved to obtain one or more sets of points that meet the conditions.

[0086] The coordinates of the above point set are relative point set coordinates based on the centroid, that is, the coordinates of the grab point.

[0087] After determining the gripping method and gripping points, the gripping strategy selection module determines the position or area of ​​the gripping point on the 3D part model based on the coordinates of the gripping point. After acquiring the image of the placement area using binocular vision technology, the module identifies and marks the position or area corresponding to each gripping point in the image and displays it using a highlighted rectangle.

[0088] Prioritize picking up irregularly shaped parts whose gripping points are not completely covered from the placement area:

[0089] For adsorption-type gripping, priority is given to gripping irregularly shaped parts with preferred adsorption surfaces. When the preferred adsorption surface is covered or blocked, the irregularly shaped parts with additional adsorption surfaces are marked for gripping. The identification and marking are re-performed after each gripping.

[0090] For mechanical gripping, among the irregularly shaped parts located on the surface of the placement area and whose gripping position or area is not covered, the part with the highest overlap between its posture and the model posture during the layered slicing process in the gripping point recognition and filtering module is selected for gripping.

[0091] The grasping strategy selection module uses a 3D part model to simulate the current placement state of the irregular part in the placement area, and constructs an absolute spatial coordinate system based on the current working area. During the grasping process, the absolute coordinates of the center of gravity of the irregular part in the absolute coordinate system are first determined. Based on the coordinates of the grasping point and the absolute coordinates of the center of gravity of the irregular part, the absolute coordinates of the grasping point are obtained, thereby determining the grasping position that the end effector of the robotic arm should reach.

[0092] After the robotic arm motion module drives the end effector to the gripping position, it performs gripping actions according to different gripping methods and locates the gripping point. It should be noted that for mechanical gripping, the point with the shortest sum of distances to the center of gravity is selected from multiple sets of points that can be gripped as the gripping point.

[0093] When there are requirements for the placement of parts after they are grasped, in order to reduce wear on the robotic arm parts and the parts falling due to excessive movements, as well as to reduce the movement time, the robotic arm motion module also has a posture adjustment optimization model. The optimization objective is to minimize the amount of rotation of the robotic arm that drives the irregular part from the current posture to the target posture. The constraints include: the robotic arm joint motion angle is within the allowable range, the torque of the robotic arm joint at each angle is within the set range, and the rotation space domain is set to avoid collisions. The robotic arm joint angle is used as a decision variable, and the ant colony algorithm is used to solve the minimum rotation path.

[0094] Specifically, quaternions are used to represent the current attitude and the target attitude, respectively, and the angular difference between the two attitudes, i.e., the rotation, is calculated. A quaternion is a rotation represented by one real number and three imaginary numbers, which constitute a four-dimensional hypercomplex space. A quaternion can be viewed as consisting of a scalar part and a vector part, if an object revolves around a unit vector... Rotation Angle, then the corresponding quaternion is .

[0095] The robotic arm motion module adjusts its posture according to the minimum rotation path and performs transfer actions during the adjustment process.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An adaptive pick-and-place system based on the placement state of irregularly shaped parts, characterized in that, include: The image acquisition and simulation module uses binocular vision technology to acquire images and reconstruct 3D models of parts in the placement area, and matches them with 3D part models in the data storage module to identify the corresponding 3D part models. The gripping point recognition and filtering module determines the appropriate gripping method based on the shape characteristics, material, and whether there are any placement posture requirements of the irregular part. Then, it automatically obtains the gripping points of the corresponding irregular part according to different gripping methods. The gripping strategy selection module identifies and marks the corresponding positions or areas of each gripping point in the image based on the coordinates of the gripping points in the relative coordinate system constructed based on the center of gravity of the part, and prioritizes the selection of irregular parts whose gripping points are not covered from the placement area for gripping; during gripping, the absolute coordinates of the gripping points are determined so that the end effector of the robotic arm can reach the gripping position. The robotic arm motion module constructs an attitude adjustment optimization model with the goal of minimizing the rotation amount of the robotic arm driving the irregular part to rotate from the current attitude to the target attitude. It uses a swarm intelligence algorithm to solve for the minimum rotation path and adjusts the attitude according to the path, while performing transfer actions during the adjustment process.

2. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 1, characterized in that, The image acquisition and simulation module uses binocular vision technology to acquire images and perform 3D reconstruction of the parts in the placement area, and matches them with the 3D part models in the data storage module to identify the corresponding 3D part models. The process includes: Calculate the volume of the 3D reconstructed model and filter out a set of 3D part models whose volume deviation is within a set range from the data storage module; The overlap of each element in the 3D reconstructed model and the 3D part model set is calculated one by one. The formula is: overlap = overlap volume / (sum of the volumes of the two models - overlap volume). When the overlap is greater than the overlap threshold, select the corresponding 3D part model for subsequent operations.

3. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 1, characterized in that, The gripping methods for irregularly shaped parts include mechanical gripping and adsorption gripping. The gripping point identification and screening module prioritizes adsorption gripping when the parts are made of easily breakable materials, magnetic metal materials, or have a wide flat surface. In other cases, mechanical gripping is used. Mechanical gripping is also used when there are requirements for the placement posture of the gripped parts.

4. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 3, characterized in that, In the adsorption-based grasping method, the process by which the grasping point identification and filtering module acquires grasping points includes: Based on the 3D part model, at least one adsorption surface is selected, and a perpendicular line is drawn from the center of the 3D part model to the selected adsorption surface, with the foot of the perpendicular as the adsorption center point. On the premise that all adsorption points are in complete contact with the adsorption surface, calculate the range of the center deviation rate between the adsorption center point and the geometric center of the plane formed by the adsorption point matrix. When the center deviation rate can be 0, that is, when the adsorption center can coincide with the above geometric center, the corresponding adsorption surface is marked as the preferred adsorption surface; When the center deviation rate does not exceed the set threshold, the corresponding adsorption surface is marked as an additional adsorption surface; otherwise, it is marked as a dangerous adsorption surface. For the additional adsorption surface and the preferred adsorption surface, the optimization objective is to minimize the deviation rate of the central sheet. The constraint is that the projection of any adsorption point along the vertical direction of the centroid falls on the adsorption surface, thereby obtaining the coordinates of each adsorption point in the relative coordinate system with the centroid as the origin, which are the coordinates of the grab point.

5. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 4, characterized in that, The formula for calculating the center deviation rate is: ,in, The distance between the geometric center of the plane formed by the adsorption point matrix and the adsorption center is denoted by . This represents the distance between the two adsorption points that are furthest apart in the adsorption point matrix.

6. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 4, characterized in that, In the mechanical grasping method, the process by which the grasping point identification and filtering module acquires grasping points includes: Determine the stable placement of irregularly shaped parts; Under the above conditions, the 3D part model is sliced ​​in the vertical direction to obtain multiple slices; Ignore slices with discontinuous areas, and calculate the outer contour areas of the upper and lower cut surfaces of the remaining slices in turn. Calculate the section deviation rate of the corresponding slice. ; The optimization objective is to minimize the variance of the distances between multiple grab points and the center of gravity, and the following constraints are set: ① ;② ;③ The coordinates of one or more sets of points in a relative coordinate system with the centroid as the origin are obtained by using a swarm intelligence algorithm; these are the coordinates of the grab points. in, Indicates the center of gravity. Indicates the grab point, This represents the spatial threshold obtained by projecting the plane defined by connecting the grab points sequentially into the vertical direction. This represents the feature threshold formed by the feature surfaces marked on the 3D part model.

7. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 6, characterized in that, Regarding the selection of parts to be gripped, the gripping strategy selection module marks the gripping point location area in the placement area image based on the gripping point coordinates, and prioritizes gripping irregularly shaped parts whose gripping points are not completely covered from the placement area: Adsorption-type gripping selects and grips irregularly shaped parts in the order of having a preferred adsorption surface and then an additional adsorption surface. Mechanical gripping selects the part with the highest overlap between its posture and the model posture during the layered slicing process in the gripping point recognition and filtering module from among the irregularly shaped parts located on the surface of the placement area and whose gripping position or area is not covered.

8. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 7, characterized in that, Regarding the selection of gripping points, the gripping strategy selection module uses a 3D part model to simulate the current placement state of the irregular part in the placement area, constructs an absolute spatial coordinate system with the current working area, determines the absolute coordinates of the center of gravity of the irregular part in the absolute coordinate system, obtains the absolute coordinates of the gripping point based on the coordinates of the gripping point and the absolute coordinates of the center of gravity of the irregular part, and thus determines the gripping position that the end effector of the robotic arm should reach. In adsorption-based grasping, once the adsorption surface is determined, the absolute coordinates of the grasp can be determined based on the absolute coordinates of the adsorption center. In mechanical gripping, the point with the shortest sum of distances to the center of gravity is selected from a set of multiple points that can be gripped as the gripping point.

9. The adaptive pick-and-place system based on the placement state of irregularly shaped parts according to claim 1, characterized in that, The robotic arm motion module constructs an attitude adjustment optimization model, with the optimization objective being to minimize the rotation amount by which the robotic arm drives the irregular part to rotate from the current attitude to the target attitude. The constraints include: the robotic arm joint motion angles are within the allowable range, the torque of the robotic arm joints at each angle is within the set range, and a rotation space domain is set to avoid collisions. The robotic arm joint angles are used as decision variables. The minimum rotation path is solved using an ant colony algorithm. The robotic arm motion module adjusts the attitude according to the minimum rotation path and performs transfer actions during the adjustment process.

Images