Robot grasping object selection method based on image recognition

Abstract

The present invention discloses a method for selecting a robot grasping object based on image recognition, and relates to the field of robot visual recognition and grasping technology. First, the depth camera is adjusted to shoot and obtain the image to be processed according to the working scene and the characteristics of the workpiece, and then the image is preprocessed and the connected domain is extracted. The area is calculated by pixel point counting and shape fitting, and the threshold screening is determined according to the size of the gripper and the pre-stored surface area. Subsequently, the extreme depth points of the connected domain contour are determined, the depth ratio is calculated, the optimal clamping point is determined according to the characteristics of the workpiece, the clamping plane is constructed, and the grasping distance is calculated. Finally, the clamping priority is weightedly calculated by comprehensively considering the connected domain area, the depth ratio and the grasping distance to determine the optimal grasping object. This method can effectively deal with the problems of workpiece accumulation, occlusion, uneven lighting and interference, reduce the amount of calculation, and improve the robot's grasping accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of robot visual recognition, and in particular to a method for selecting a robot grasping object based on image recognition. Background Art

[0002] In the scenario where a large number of workpieces are piled up in disorder, occlusion is a very common situation. Some workpieces may be completely covered by other workpieces, or only a part of them is exposed. This makes it difficult for the robot to obtain complete workpiece feature information when identifying through the visual system. For example, for some workpieces with specific shape contours, textures or logos to assist in identification, if the key parts are blocked, the traditional image recognition algorithm may fail. In addition, the lighting conditions in the actual working environment are variable and uneven, which will affect the appearance of the workpiece. Reflective areas may be produced under strong light, and the shadow part will cause some details of the workpiece to be lost, which will interfere with the robot's visual recognition system to accurately extract the real features of the workpiece. Even if the workpiece is identified, the feasibility of grasping must be considered when determining it as the object to be grasped. Due to interference between workpieces, that is, the complex positional relationship between each other, it may happen that although a workpiece is identified, it is tightly stuck by other workpieces around it, and it is difficult for the robot arm to perform grasping without touching the surrounding workpieces. For workpieces with complex shapes, such as castings with irregular surfaces, the grasping angle and path of the robot arm need to be considered to avoid collision with surrounding workpieces during grasping.

[0003] Due to the above reasons, the grasping of disorderly stacked workpieces requires the use of high-precision visual inspection algorithms, or the combination of other expensive sensors to comprehensively judge whether the workpieces meet the requirements of grasping and screening. It is also necessary to accurately model the spatial position and shape of the workpiece, predict the grasping path, calculate the grasping angle, etc., to ensure that there will be no collision and interference with other workpieces when grasping the target workpiece, and to ensure that the entire grasping operation proceeds smoothly. This spatial layout analysis and grasping path planning involve complex geometric and kinematic calculations. Summary of the invention

[0004] The present invention provides a method for selecting a robot grasping object based on image recognition, which is characterized by comprising the following steps:

[0005] Step 1, obtaining an image to be processed of a workpiece to be grasped;

[0006] Step 2, connected domain extraction and screening: pre-process the image to be processed, extract the connected domain of the image, calculate the area of ​​each connected domain, and compare it with the first threshold, and screen out multiple connected domains whose area is greater than the first threshold;

[0007] Step 3, depth ratio screening: determine the maximum depth point and the minimum depth point of the contour of the connected domain, and calculate the depth ratio of the maximum depth point to the minimum depth point;

[0008] Step 4, grasping distance screening: determine the optimal clamping point in the contour of the connected domain that is suitable for contact with the gripper, construct the clamping plane based on the spatial geometric constraint equation, and calculate the distance between the midpoint of the connected domain and the clamping plane;

[0009] Step 5, calculate the gripping priority: calculate the gripping priority of each connected domain according to the recorded connected domain area, depth ratio, and grasping distance, comprehensively evaluate the grasping feasibility and efficiency of each connected domain, and determine the optimal grasping object.

[0010] Specifically, the maximum depth point and the minimum depth point of the contour are determined by calculating the depth of each point on the contour of the connected domain from the reference point, and sorting to obtain the maximum depth point and the minimum depth point on the contour of each connected domain.

[0011] Specifically, the maximum depth point and the minimum depth point of the contour are determined as follows: the depth of each point in the connected domain from the reference point is calculated, the maximum depth point and the minimum depth point in the connected domain are connected, and the line connecting the two points and the contour of the connected domain produce two intersection points. The intersection point with a larger depth from the reference point is recorded as the maximum depth point of the contour, and the intersection point with a smaller depth from the reference point is recorded as the minimum depth point of the contour. The ratio of the depth of the maximum depth point of the contour to the depth of the minimum depth point of the contour is recorded as the depth ratio.

[0012] Specifically, the calculation formula of the clamping priority P is: P=A*S / lnD+B*L, where S is the area of ​​the connected domain, D is the depth ratio, L is the grasping distance, and A and B are proportional coefficients.

[0013] Specifically, the calculation formula for the clamping priority P is: P=(A*S / lnD+B*L)*C*(1-η), where S is the area of ​​the connected domain, D is the depth ratio, L is the grasping distance, A, B, and C are proportional coefficients, and η is the correction factor.

[0014] Specifically, the connected domain is extracted by using a threshold segmentation algorithm to segment the image into a foreground and a background according to the grayscale histogram of the image, and performing a connected domain labeling algorithm on the segmented binary image to complete the extraction of the connected domain.

[0015] Specifically, the area of ​​the connected domain is calculated by combining pixel counting with shape fitting. A geometric formula is used for the connected domain with regular shape, and a polygonal approximation algorithm is used for the irregular connected domain.

[0016] Specifically, the contour of the connected domain is determined by performing Gaussian filtering on the image to remove noise, then calculating the gradient amplitude and direction of the image, and extracting the edge contour of the connected domain through non-maximum suppression and double threshold detection.

[0017] Specifically, the preprocessing of the image to be processed includes grayscale conversion, and an adaptive grayscale stretching algorithm is used to dynamically adjust the grayscale range of the image according to the reflective characteristics of the workpiece surface.

[0018] Specifically, the midpoint of the connected domain is the center point of the connected domain, or the midpoint of the line connecting the maximum depth point of the contour and the minimum depth point of the contour, or the midpoint of the line connecting the maximum depth point and the minimum depth point in the connected domain.

[0019] Beneficial technical effects: The robot grasping object selection method based on image recognition of the present invention has significant beneficial technical effects in terms of accuracy, stability, efficiency and adaptability. It can obtain accurate depth information under complex lighting and surface conditions of workpieces of different materials, providing a comprehensive and accurate data basis for subsequent connected domain extraction and workpiece identification. Compared with traditional methods, the semantic segmentation network based on deep learning is used to extract connected domains, which can more accurately identify the workpiece area, effectively reduce misjudgments and missed judgments, and greatly improve the accuracy of workpiece identification. The connected domain, depth ratio, and clamping distance are used to judge the clamping priority, which reduces the number of grasping failures and re-grasping, further improves the grasping efficiency, and enables the robot to quickly and efficiently complete the grasping task in complex and disordered workpiece stacking scenes, and can adapt to various types, shapes, sizes and materials of workpiece grasping tasks, and realize accurate identification, screening and efficient grasping of different workpieces, with a wide range of applications and strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Attached Figure 1 A flowchart of a method for selecting an object to be grasped by a robot based on image recognition according to the present invention;

[0021] Attached Figure 2 It is a schematic diagram of the maximum depth point and the minimum depth point of the profile of the present invention. DETAILED DESCRIPTION

[0022] like Figure 1 As shown, the present invention provides a method for selecting a robot grasping object based on image recognition, and the specific steps include:

[0023] 1. Obtain the image to be processed of the workpiece to be grasped: According to the lighting conditions of the working scene, the material and color characteristics of the workpiece, the emission frequency and intensity of the depth camera are automatically adjusted, and the disorderly stacked workpieces are photographed in all directions. The captured image data is transmitted to the image fusion and preprocessing unit to obtain the object to be processed.

[0024] The image to be processed can be obtained by shooting with a depth camera. The depth camera can simultaneously obtain the two-dimensional image information and depth information of the workpiece. The depth camera is based on technologies such as structured light, time of flight or binocular vision, and calculates the depth value of each point in the scene by emitting specific light or using binocular parallax. For example, a structured light depth camera emits a coded light pattern to the surface of the workpiece and calculates the depth based on the deformation of the reflected light pattern; a time of flight depth camera determines the distance by measuring the time difference between the emission and reflection of the light pulse; a binocular vision depth camera uses two cameras to shoot the same object from different angles and calculates the depth through parallax. The depth camera can accurately calculate the depth value of each point in the scene by emitting coded light patterns of multiple frequencies to the surface of the workpiece, using the complex deformation of the reflected light pattern and the phase difference information of light of different frequencies. This depth measurement method can obtain more accurate and stable depth information under complex lighting conditions and on the surfaces of workpieces of different materials.

[0025] The image to be processed can be taken at a specific position on the production line under specific lighting conditions, or it can be a composite image obtained by electronic equipment by fusing images taken at multiple angles or at different times through image synthesis technology. The purpose is to more comprehensively and accurately reflect the actual state of the workpiece. The image to be processed can be the image data directly obtained by the depth camera taking pictures of multiple disorderly stacked workpieces, which contains rich information such as the color, texture, shape and depth of the workpiece. The image to be processed can not only include the color and texture information of the workpiece, but also effectively integrate the depth information to construct a three-dimensional image model with rich spatial and visual features, which provides a comprehensive and accurate data basis for the subsequent connected domain extraction and determination of the workpiece to be grasped.

[0026] 2. Connected domain extraction and screening: Preprocess the image to be processed, then extract the connected domain of the image, calculate the area of ​​each connected domain, and compare it with the first threshold, screen out multiple connected domains with an area greater than the first threshold, and record their relevant information.

[0027] The specific steps are as follows: inputting the image to be processed into the connected domain extraction module, preprocessing the image, including corrosion operation to remove small area noise, dilation operation to connect adjacent workpiece areas, and extracting preliminary connected domains; for each connected domain, the area is calculated by combining pixel counting with shape fitting, and according to the pre-stored workpiece type, a suitable first threshold is selected from the threshold range determined by cluster analysis for screening, and the connected domains with an area greater than the first threshold and their related information, such as position, shape features, etc., are retained.

[0028] Preprocessing operation, firstly, grayscale processing is performed on the acquired image to be processed, and the color image is converted into a grayscale image to simplify the subsequent calculation. The grayscale processing can adopt an adaptive grayscale stretching algorithm to dynamically adjust the grayscale range of the image according to the reflective characteristics of the workpiece surface, enhance the image details, and prevent the high contrast between the highlight area and the shadow area in the image due to the strong reflectivity of the workpiece surface, which affects the accuracy of the connected domain extraction. Then, the threshold segmentation algorithm is used to determine a suitable threshold according to the grayscale histogram of the image, and the image is divided into two parts: foreground and background. The foreground part is the possible workpiece area. Then, the connected domain marking algorithm is performed on the segmented binary image, such as the connected domain marking algorithm based on the scan line or the connected domain marking algorithm based on seed filling. Taking the connected domain marking algorithm based on the scan line as an example, the image is scanned line by line from the upper left corner. When encountering a foreground pixel, its connectivity with the adjacent pixels is judged. If it is a new connected area, a new label is assigned; if it is connected to the marked connected area, it is marked with the same connected area number, thereby completing the extraction of the connected domain. In addition to grayscale processing, image filtering operations can also be performed as needed to remove noise interference in the image. For example, using the Gaussian filtering algorithm, by setting the appropriate Gaussian kernel size and standard deviation, the image is convolved to smooth the image and reduce the impact of noise on the connected domain extraction and subsequent calculations. At the same time, image enhancement operations such as histogram equalization can be performed as needed to improve the contrast of the image, make the features of the artifact more obvious, and facilitate the accurate extraction of the connected domain. The extraction of the connected domain can also be performed using a semantic segmentation network based on deep learning for preliminary segmentation, so as to accurately identify the artifact area and background area in the image, thereby improving the accuracy and efficiency of recognition.

[0029] The calculation of the area of ​​the connected domain is based on a method that combines pixel counting with shape fitting. For connected domains with regular shapes, such as circles or rectangles, the area is accurately calculated using geometric formulas by fitting their geometric shapes; for connected domains with irregular shapes, a polygon approximation algorithm is used on the basis of pixel counting to approximate the edge of the connected domain with a polygon, and then the polygon area is calculated to improve the accuracy of area calculation. Pixel counting is to calculate the area by traversing the number of pixels in the connected domain. For binary images, the foreground pixel value in the connected domain is 1, and the background pixel value is 0. The number of pixels with a pixel value of 1 in each connected domain is counted to obtain the area of ​​the connected domain.

[0030] The setting of the first threshold is related to the size of the gripper and the size of the surface area of ​​the pre-stored multiple grasping objects. Its purpose is to screen out the connected domains with a large area, which may be complete workpieces and can be grasped by the gripper. For example, if the opening range of the gripper is small, it may not be possible to grasp the workpiece with a large area, so the first threshold cannot be set too high; at the same time, if the average surface area of ​​the pre-stored grasping object is small, the first threshold should also be adjusted accordingly to avoid mistaking the partial area of ​​the small workpiece or noise as a graspable object. The specific setting can be determined by experiments and empirical data, such as calculating the minimum and maximum area range that can be grasped according to the maximum opening diameter of the gripper, and then combining the surface area distribution of the grasping object to determine a suitable first threshold range. The setting of the first threshold can introduce the clustering analysis algorithm in machine learning, cluster the connected domain area data of a large number of workpieces of different sizes, and determine multiple threshold intervals according to the size range of the gripper and the average surface area distribution of workpieces of different categories. In actual screening, according to the preliminary prediction of the workpiece, the appropriate threshold interval is dynamically selected, so as to more accurately screen out the connected domains that may be complete workpieces and suitable for gripping by the gripper.

[0031] The pre-stored surface area of ​​various grasping objects refers to the standard surface area data of various possible grasping workpieces pre-stored in the robot control system. These data can be obtained by measuring and statistically analyzing a large number of similar workpieces. For example, for a certain type of bolt, the surface area of ​​multiple samples is measured, and the average value is calculated and stored in the system. The pre-stored surface area data helps to make preliminary judgments and screening of connected domains during the image recognition process, and improves the accuracy and efficiency of grasping object selection.

[0032] The relative size of the connected domain area and the first threshold of the surface area of ​​the associated pre-stored multiple grasping objects reflects the relative size of the connected domain in the image. The ratio of the connected domain to the first threshold reflects the ratio of the connected domain area to the pre-stored average surface area of ​​this type of workpiece, which can help determine whether the connected domain is likely to be a complete workpiece. For example, if the ratio of the connected domain area to the surface area of ​​the pre-stored grasping object is close to 1, and the connected domain accounts for a relatively large proportion of the image, then the connected domain is likely to be a complete and large workpiece that can be directly grasped; if the ratio is in the middle, further screening is required; if the ratio is very small, it may be part of the workpiece or noise, and is not suitable for grasping.

[0033] 3. Depth ratio screening: determine the maximum depth point and the minimum depth point of the contour of the connected domain, calculate and record the depth ratio of the maximum depth point and the minimum depth point of the contour of each connected domain.

[0034] Alternatively, after determining the maximum depth point and the minimum depth point of the contour of the connected domain, calculating and recording the depth ratio of the maximum depth point and the minimum depth point of the contour of each connected domain, the depth ratio can be compared with the second threshold, and multiple connected domains close to the second threshold value can be screened out, and their related information can be recorded. Through this alternative method, objects suitable for grasping can be further screened based on the connected domain area screening by depth ratio screening. The second threshold can be specifically set according to the workpiece type and workpiece distribution, for example, it can be set to 1 or other.

[0035] Taking the disorderly stacked bolts and nuts as an example, combined with the attached Figure 2 This paper introduces the determination of the maximum depth point and the minimum depth point of the contour. Figure 2 Taking the nut workpiece (1) in the figure as an example, the depth of each point on the contour of the connected domain from the reference point (2) is calculated, and the maximum depth point (3) and the minimum depth point (4) on the contour of each connected domain are obtained by sorting.

[0036] The maximum depth point and the minimum depth point of the contour can be determined by other methods. Figure 2 Taking the bolt workpiece (5) in the figure as an example, the depth of each point in the connected domain from the reference point is calculated, the maximum depth point (6) and the minimum depth point (7) in the connected domain are connected, and the line connecting the two points and the contour of the connected domain generate two intersection points (6, 8). In this embodiment, the intersection point (6) with a larger depth from the reference point is recorded as the maximum depth point of the contour, and the intersection point (8) with a smaller depth from the reference point is recorded as the minimum depth point of the contour. The ratio of the depth of the maximum depth point of the contour to the depth of the minimum depth point of the contour is recorded as the depth ratio.

[0037] The reference point can be selected as a depth camera or a point directly above multiple artifacts that are piled up in a disorderly manner.

[0038] Depth measurement method: Use the depth information obtained by the depth camera to measure the depth of each point in the connected domain from the reference point. Each pixel of the depth camera corresponds to a depth value. Through coordinate conversion and indexing, the depth data of each point in the connected domain can be obtained. For example, for a pixel with coordinates (x, y) in the image, its corresponding depth value is z. The image coordinates can be converted into three-dimensional space coordinates (x, y, z) through the calibration parameters of the depth camera to obtain the depth of the point from the reference point.

[0039] The contour determination adopts a method based on sub-pixel edge detection, combined with image gradient information and directional derivatives, to accurately locate the edge contour of the connected domain. For edge detection, the Canny edge detection algorithm can be used. First, the image is Gaussian filtered to remove noise, and then the gradient amplitude and direction of the image are calculated. The edge contour of the connected domain is extracted through non-maximum suppression and double threshold detection. The extracted edge contour can be simplified and described by chain code representation or polygon approximation for subsequent calculations.

[0040] When connecting the maximum depth point and the minimum depth point in a connected domain, if the surface of the connected domain is a curved surface, it is necessary to consider the extension of the connecting line along the surface. One method is to use a surface fitting algorithm, such as least squares surface fitting, to fit a surface model based on the point cloud data in the connected domain, and then project the connecting line of the maximum depth point and the minimum depth point onto the surface to obtain the connecting line along the surface. Another method is to use an interpolation algorithm, such as cubic spline interpolation, to perform interpolation calculations along the surface between the maximum depth point and the minimum depth point to obtain an approximate curve of the connecting line. For the curved part on the contour, Bezier curve fitting can be used to accurately represent the contour curve with a small number of control points, thereby reducing the amount of data while improving the accuracy of the contour description.

[0041] 4. Grasping distance screening: Determine the optimal gripping point in the contour of the connected domain that is suitable for contact with the gripper, construct the gripping plane based on the spatial geometric constraint equation, and calculate the distance between the midpoint of the connected domain and the gripping plane. If the midpoint is within the gripping plane, the distance is 0. The midpoint of the connected domain can be the center point of the connected domain, the midpoint of the connecting line between the maximum depth point and the minimum depth point of the contour, or the midpoint of the connecting line between the maximum depth point and the minimum depth point in the connected domain.

[0042] Alternatively, after calculating the distance between the midpoint of the connected domain and the clamping plane, the distance between the midpoint of the connected domain and the clamping plane, i.e., the grasping distance, can be compared with the third threshold, and multiple connected domains less than the third threshold can be screened out, and their related information can be recorded. Through this alternative method, objects suitable for grasping can be further screened based on the depth ratio screening by grasping distance screening. The third threshold can be specifically set according to the robot configuration, terminal clamp configuration, workpiece type, and workpiece distribution.

[0043] The construction of the clamping plane can adopt a fitting method based on spatial geometric constraints. Considering the coordinated motion and spatial constraints of multiple grippers during the grasping process, after determining the optimal clamping point, a clamping plane that meets the robot kinematic requirements and can ensure stable grasping of the workpiece can be constructed by solving the spatial geometric constraint equation.

[0044] The clamping point is the point where the jaws contact the workpiece. For workpieces of different shapes, the choice of clamping point is different. For example, for cylindrical workpieces, the clamping point is usually selected on the side of the cylinder; for rectangular workpieces, the clamping point can be selected on the edge or plane of the side. The optimal clamping point is the point in the outer contour of the connected domain that is most suitable for the jaws to contact and stably grasp the workpiece based on the shape of the jaws and the characteristics of the workpiece. To determine the optimal clamping point, it is necessary to consider the center of gravity distribution of the workpiece, the symmetry of the shape, and the grasping method of the jaws. For example, for workpieces with symmetrical shapes, the optimal clamping point is usually selected on the outer contour near the center of symmetry; for workpieces with offset center of gravity, the clamping point should be close to the center of gravity to ensure stability during grasping.

[0045] The multiple optimal gripping points of multiple grippers constitute the gripping plane. The determination of the gripping plane is very important for judging the feasibility and stability of grasping. For example, for a dual-gripper robot, the two optimal gripping points determine a straight line, and the plane where this straight line lies is the gripping plane; for a multi-gripper robot, a plane is fitted through multiple optimal gripping points as the gripping plane.

[0046] 5. Calculate the clamping priority: According to the recorded connected domain area, depth ratio, and grasping distance, calculate the clamping priority P of each connected domain, comprehensively evaluate the grasping feasibility and efficiency of each connected domain, and determine the optimal grasping object. Alternatively, the clamping priority calculation can be performed only for the screened connected domains, thereby reducing the amount of calculation. The specific calculation formula is: P=A*S / lnD+B*L, where S is the area of ​​the connected domain, D is the depth ratio, L is the grasping distance, and A and B are proportional coefficients. The clamping priority combines the size of the connected domain, the depth ratio, and the grasping distance. The larger the area of ​​the connected domain, the closer the depth ratio is to 1, the smaller the grasping distance, the greater the clamping priority, and the more suitable it is for priority grasping.

[0047] The larger the area of ​​the connected domain, the more complete the workpiece may be and the easier it is to grasp; the closer the depth ratio is to 1, the less likely the workpiece is to be interfered with, and the less likely it is to tilt or collide during grasping; the smaller the grasping distance, the more stable and efficient the robot is during the grasping process. The gripping priority is calculated using the formula, and the proportional coefficients A and B can be adjusted according to the actual application scenario and the performance requirements of the robot. For example, in a logistics warehouse scenario with high requirements for grasping efficiency, the value of B can be appropriately increased so that the grasping distance has a greater impact on the gripping priority; in a precision assembly scenario with high requirements for grasping interference, the value of A can be increased to highlight the importance of the connected domain area and the depth ratio.

[0048] The calculation of the clamping priority can also introduce a correction factor η. The correction factor η can take into account the stability risk of the workpiece (such as surface curvature, center of gravity offset, etc.), the collision risk during the grasping process (based on the distribution of obstacles in the surrounding environment and the prediction of the robot's motion path), and the cost of re-grasping after grasping failure (including time cost and possible damage cost to the workpiece). Taking the surface curvature as an example, for areas with large curvature changes, corresponding weights are given so that the priority calculation can more accurately reflect the flatness of the workpiece surface and the difficulty of grasping. For example, in a curved surface area with a large curvature, even if the depth difference is small, instability may still occur during grasping due to the influence of curvature. Curvature weighting can effectively identify this potential risk. The calculation formula becomes: P=(A*S / lnD+B*L)*C*(1-η), where S is the area of ​​the connected domain, D is the depth ratio, L is the grasping distance, and A, B, and C are proportional coefficients that are dynamically adjusted according to the actual application scenario and robot performance requirements.

[0049] The screening method of the present invention can be used in conjunction with other robot grasping methods, such as combining factors such as whether the grasping process hits the wall, whether there is interference, and the highest route efficiency, through comprehensive analysis and optimization algorithms, the optimal grasping object is determined, and the appropriate grasping path is selected, so as to achieve efficient and accurate grasping operations of the robot in complex and disordered workpiece accumulation scenes, and improve the overall efficiency and quality of industrial production and logistics automation. The screening method of the present invention can also be combined with a robot decision-making system based on reinforcement learning. The reinforcement learning system optimizes the grasping strategy through continuous trial and error and feedback learning of the robot in the actual grasping process. For example, when the robot encounters an unexpected situation (such as collision, grasping failure, etc.) when grasping a workpiece, the reinforcement learning system will adjust the subsequent grasping decision according to the current environmental state and grasping results, including re-evaluating the clamping priority of other workpieces, optimizing the grasping path planning, etc., so as to achieve the robot's adaptive and efficient grasping operation in complex and disordered workpiece accumulation scenes, and continuously improve the intelligence level and reliability of the entire grasping system.

[0050] Specifically, the specific identification of the grasped object can be introduced in the grasped object selection method of the present application, so that after the grasped object of the present application is selected, further screening can be performed through grasped object identification. The specific screening can be performed by extracting the local features and overall features of the grasped object, and combining the local features and overall features for screening. The local features include contour features, connected domain features and connecting line features, and the overall features include line density features. The extracted feature parameters are normalized, and the weights of different features are determined by the hierarchical analysis method to obtain the weighted feature vector. The feature vector of the grasped image to be further screened is compared with the feature vectors of various pre-stored workpiece types. The dynamic time warping distance is combined with the feature vector to calculate the similarity. The weighting coefficient is introduced to adjust the dynamic time warping distance. When the adjusted distance is less than the set threshold, the workpiece type is determined, thereby completing further screening, and then subsequent grasping is performed. The contour feature can adopt the Hu invariant moment of the contour, the connected domain feature can adopt the area and / or circularity and / or rectangularity of the connected domain, and the connecting line feature can adopt the straight line segment in the Hough transform image, mapping the straight line in the image space to the parameter space, and determining the parameters of the straight line, such as the slope, intercept, etc., by finding the peak in the parameter space.

[0051] In the several embodiments provided by the present invention, it should be understood that the disclosed method can be implemented in other ways. For example, the above-described embodiments of the invention are only illustrative, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0052] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0053] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0054] It is obvious to a person skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the essential characteristics of the present invention.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for selecting objects to be grasped by a robot based on image recognition, characterized in that: The following steps are involved: Step 1, obtaining the image to be processed of the workpiece to be grasped; this step includes: automatically adjusting the emission frequency and intensity of the depth camera according to the lighting conditions of the working scene, the material and color characteristics of the workpiece, and taking a full-scale photo of the workpiece; Step 2, connected domain extraction and screening: pre-processing the image to be processed, extracting the connected domain of the image, calculating the area of ​​each connected domain, and comparing it with the first threshold, screening out multiple connected domains whose area is greater than the first threshold; wherein the pre-processing includes: graying the image to be processed, the graying adopts an adaptive grayscale stretching algorithm, and dynamically adjusts the grayscale range of the image according to the reflection characteristics of the workpiece surface; Step 3, depth ratio screening: determine the maximum depth point and the minimum depth point of the contour of the connected domain, and calculate the depth ratio of the maximum depth point to the minimum depth point; Step 4, grasping distance screening: determine the optimal clamping point in the contour of the connected domain that is suitable for contact with the gripper, construct the clamping plane based on the spatial geometric constraint equation, and calculate the distance between the midpoint of the connected domain and the clamping plane; Step 5, calculate the gripping priority: calculate the gripping priority of each connected domain according to the recorded connected domain area, depth ratio, and grasping distance, comprehensively evaluate the grasping feasibility and efficiency of each connected domain, and determine the optimal grasping object.

2. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The maximum depth point and the minimum depth point of the contour are determined by calculating the depth of each point on the contour of the connected domain from the reference point, and sorting to obtain the maximum depth point and the minimum depth point on the contour of each connected domain.

3. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The maximum depth point and the minimum depth point of the contour are determined by calculating the depth of each point in the connected domain from the reference point, connecting the maximum depth point and the minimum depth point in the connected domain, and the line connecting the two points and the contour of the connected domain generate two intersection points. The intersection point with a larger depth from the reference point is recorded as the maximum depth point of the contour, and the intersection point with a smaller depth from the reference point is recorded as the minimum depth point of the contour. The ratio of the depth of the maximum depth point of the contour to the depth of the minimum depth point of the contour is recorded as the depth ratio.

4. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The calculation formula of gripping priority P is: P=A*S / lnD+B*L, where S is the area of ​​the connected domain, D is the depth ratio, L is the gripping distance, and A and B are proportional coefficients.

5. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The calculation formula of gripping priority P is: P=(A*S / lnD+B*L)*C*(1-η), where S is the area of ​​the connected domain, D is the depth ratio, L is the gripping distance, A, B, and C are proportional coefficients, and η is the correction factor.

6. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The extraction of connected domains is as follows: using a threshold segmentation algorithm, according to the grayscale histogram of the image, the image is segmented into foreground and background, and the connected domain labeling algorithm is performed on the segmented binary image to complete the extraction of the connected domain.

7. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The area of ​​the connected domain is calculated by combining pixel counting and shape fitting. The geometric formula is used for the connected domain with regular shape, and the polygonal approximation algorithm is used for the irregular connected domain.

8. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The contour of the connected domain is determined by performing Gaussian filtering on the image to remove noise, then calculating the gradient amplitude and direction of the image, and extracting the edge contour of the connected domain through non-maximum suppression and double threshold detection.

9. The method for selecting objects to be grasped by a robot based on image recognition according to claim 1, characterized in that: The midpoint of the connected domain is the center point of the connected domain, or the midpoint of the line connecting the maximum depth point of the contour and the minimum depth point of the contour, or the midpoint of the line connecting the maximum depth point and the minimum depth point in the connected domain.

Images