Robot grasping precision adaptive identification method based on multi-source data acquisition

Abstract

The application discloses a robot grabbing precision self-adaptive identification method based on multi-source data collection, relates to the technical field of grabbing identification, and comprises the following steps: obtaining a target grabbing article image based on a camera and marking the target grabbing article image as a real-time two-dimensional article image; obtaining a first gray threshold based on the real-time two-dimensional article image; obtaining a real-time segmentation contour based on the real-time two-dimensional article image and the first gray threshold; obtaining a real-time contour included angle based on the real-time segmentation contour; obtaining a historical contour included angle based on articles without mutual occlusion, obtaining a first included angle threshold and a second included angle threshold based on the historical contour included angle; obtaining a real-time single-object contour point, an occluded contour and a non-occluded contour based on the real-time contour included angle, the first included angle threshold and the second included angle threshold; and performing different grabbing identifications based on the occluded contour and the non-occluded contour; and the application is used for solving the problem that in the prior art, it is difficult to accurately identify articles with light intensity fluctuation and object stacking, thereby leading to inaccurate grabbing of each single article.

Description

Technical Field

[0001] This invention relates to the field of grasping and recognition technology, specifically to an adaptive recognition method for robot grasping accuracy based on multi-source data acquisition. Background Technology

[0002] Industrial robots have been widely used in scenarios such as parts sorting, precision assembly, and flexible material handling. High-precision and high-stability autonomous grasping capabilities are the core foundation for robots to achieve intelligent operations; therefore, it is necessary to identify the grasping accuracy of robots.

[0003] Current robot grasping and recognition solutions mostly rely on a single vision sensor to complete target localization and grasping, or to grasp objects placed in fixed positions. They cannot handle entire rows. However, industrial sites generally have fluctuations in lighting intensity, which leads to inaccurate positioning of objects in 2D recognition. At the same time, it is difficult to accurately obtain a 3D coordinate system from 2D images. Furthermore, 3D imaging easily identifies stacked objects as a single item, resulting in inaccurate grasping. Existing technologies cannot accurately identify objects with fluctuating lighting intensity and stacked objects, resulting in inaccurate grasping of each individual item. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art. It involves acquiring a target object image based on a camera and labeling it as a real-time two-dimensional object image; obtaining a first grayscale threshold based on the real-time two-dimensional object image; obtaining a real-time segmented contour based on the real-time two-dimensional object image and the first grayscale threshold; obtaining a real-time contour angle based on the real-time segmented contour; obtaining historical contour angles based on objects without mutual occlusion; obtaining a first angle threshold and a second angle threshold based on the historical contour angles; obtaining real-time single object contour points, occluded contours, and unoccluded contours based on the real-time contour angles, the first angle threshold, and the second angle threshold; and performing different grasping recognition based on the occluded and unoccluded contours. This addresses the problem in the prior art where it is difficult to accurately identify objects with fluctuating lighting intensity and object stacking, leading to inaccurate grasping of each individual object.

[0005] To achieve the above objectives, this application provides an adaptive recognition method for robot grasping accuracy based on multi-source data acquisition, comprising the following steps:

[0006] S100: Acquire an image of the target object based on the camera and label it as a real-time two-dimensional object image; obtain a first grayscale threshold based on the real-time two-dimensional object image; obtain a real-time segmented contour based on the real-time two-dimensional object image and the first grayscale threshold; obtain the real-time contour angle based on the real-time segmented contour.

[0007] S200, obtain the historical contour angle based on items that do not obstruct each other, and obtain the first angle threshold and the second angle threshold based on the historical contour angle;

[0008] S300, based on the real-time contour angle, the first angle threshold and the second angle threshold, obtains the real-time single object contour points, occluded contours and unoccluded contours.

[0009] S400 performs different grasping and recognition based on occluded and unoccluded contours.

[0010] Furthermore, S100 includes the following sub-steps:

[0011] A camera is installed above the center of the item placement area, facing the item placement area with a black background. The camera then captures images of the target items and marks them as real-time two-dimensional item images.

[0012] The RGB values ​​of each pixel in the real-time 2D object image are converted into grayscale values ​​to obtain a grayscale image, which is then marked as a real-time 2D object grayscale image.

[0013] Furthermore, S100 also includes the following sub-steps:

[0014] Mark the grayscale value of each pixel in the real-time 2D grayscale image of the object as the real-time object grayscale value;

[0015] Divide the grayscale value range from 0 to 255 into N grayscale value ranges of equal length, and mark them as the first grayscale range.

[0016] Obtain the number of real-time grayscale values ​​of items within each first grayscale range, and mark them as the number within the first grayscale range;

[0017] Mark the median value of the first grayscale range as the median value of the first range;

[0018] Sort the quantities in the first range from left to right according to the median value in the first range, from smallest to largest.

[0019] The quantity within the first range that is greater than the quantity on the left and right sides is marked as the first target quantity;

[0020] Obtain the first target quantity from left to right and mark it as the second target quantity;

[0021] Get the first quantity of the first target from right to left, and mark it as the quantity of the third target;

[0022] The quantity within the first range that is the smallest between the second and third target quantities is marked as the fourth target quantity;

[0023] Obtain the median value of the first range corresponding to the number of fourth targets, and mark it as the first grayscale threshold.

[0024] Furthermore, S100 also includes the following sub-steps:

[0025] Obtain the real-time grayscale value of the item that is greater than the first grayscale threshold, and mark it as the real-time segmented grayscale value;

[0026] Obtain a continuous region composed of pixels with real-time grayscale values ​​and mark it as the real-time segmentation region;

[0027] Obtain the contour of the real-time segmented region and mark it as the real-time segmented contour.

[0028] Furthermore, S100 also includes the following sub-steps:

[0029] Establish a Cartesian coordinate system and label it as the real-time segmentation coordinate system;

[0030] Place the real-time segmented contour in the first quadrant of the real-time segmentation coordinate system;

[0031] Draw a point at intervals M on the real-time segmented contour and mark it as a real-time segmented contour point;

[0032] Connect adjacent real-time segmented contour points to obtain line segments, which are then marked as real-time contour line segments;

[0033] Obtain the included angle between adjacent real-time contour segments and mark it as the real-time contour angle.

[0034] Furthermore, S200 includes the following sub-steps:

[0035] Obtain the real-time outline angle of items that do not obstruct each other, and mark it as the historical outline angle;

[0036] Obtain the included angles of F1 historical contours;

[0037] A Cartesian coordinate system is established with the included angle of the historical contours as the horizontal axis value, and it is marked as the first historical coordinate system;

[0038] Obtain the mean of the historical contour angles and mark it as the historical mean angle;

[0039] Obtain the point on the horizontal axis of the first historical coordinate system where the historical mean angle is located, and mark it as the historical mean point;

[0040] Obtain the straight line that passes through the historical mean point and uses it as the vertical axis of the first historical coordinate system; mark it as the straight line in history.

[0041] In the first quadrant of the first historical coordinate system, the coordinates of points where the horizontal axis of the first historical coordinate system is the historical profile angle and the vertical axis of the first historical coordinate system is any value are marked as historical angle coordinate points.

[0042] Furthermore, S200 also includes the following sub-steps:

[0043] Get half of F1 and label it F2;

[0044] The first historical line begins to move in the negative direction of the horizontal axis of the first historical coordinate system by a line parallel to the vertical axis of the first historical coordinate system, and is marked as the first translation line; the first historical line begins to move in the positive direction of the horizontal axis of the first historical coordinate system by a line parallel to the vertical axis of the first historical coordinate system, and is marked as the second translation line; the first translation line and the second translation line move the same distance; the historical angle coordinates between the first translation line and the second translation line are obtained and marked as the first intermediate distribution coordinate point; when the first intermediate distribution coordinate point is equal to or greater than F2, the movement of the first translation line and the second translation line is stopped, and the movement of the first translation line and the second translation line at this stopped position is obtained and marked as the first reference line and the second reference line, respectively;

[0045] Set a ratio, labeled B1; obtain the distance between the first reference line and the second reference line, labeled D1;

[0046] Obtain the product of B1 and D1, and label it as D2;

[0047] The first reference line is moved by D2 in the negative direction of the horizontal axis of the first historical coordinate system to obtain a straight line, which is marked as the first threshold line.

[0048] The second reference line is moved by D2 in the positive direction of the horizontal axis of the first historical coordinate system to obtain a straight line, which is marked as the second threshold line.

[0049] Obtain the historical angle coordinates between the first threshold line and the second threshold line, and mark them as historical filter coordinates.

[0050] Obtain the minimum and maximum x-coordinates of all historical filtered coordinate points, and mark them as the first included angle threshold and the second included angle threshold, respectively.

[0051] Furthermore, S300 includes the following sub-steps:

[0052] Searching towards both sides of the real-time segmented contour from any real-time segmented contour point, if the angle of the real-time contour is outside the range of the first angle threshold to the second angle threshold, the search stops and the real-time segmented contour corresponding to the searched real-time segmented contour point is identified as an occluded contour; if the angle of the real-time contour is not outside the range of the first angle threshold to the second angle threshold, and the search returns to the real-time segmented contour point, the real-time segmented contour corresponding to the searched real-time segmented contour point is identified as an unoccluded contour.

[0053] Furthermore, S400 includes the following sub-steps:

[0054] If the outline is unobstructed, then the 3D outline points of a target object within the real-time segmented outline are obtained through 3D scanning and marked as real-time single-object 3D points.

[0055] The robot grasps each target item based on real-time single-object 3D point data.

[0056] Furthermore, S400 also includes the following sub-steps:

[0057] If the outline is occluded, each real-time segmented outline point found in the search will be marked as a real-time single object outline point.

[0058] The function is obtained by fitting the real-time single object contour points with the equation of a circle, and it is marked as the real-time fitted contour.

[0059] Each real-time fitted contour region corresponds to a target item that needs to be captured.

[0060] Then, the 3D contour points of a target object within the real-time fitted contour are obtained through 3D scanning and marked as real-time single-object 3D points.

[0061] The robot grasps each target item based on real-time single-object 3D point data.

[0062] The beneficial effects of this invention are as follows: This invention acquires images of target objects for grasping based on a camera and marks them as real-time two-dimensional object images; obtains a first grayscale threshold based on the real-time two-dimensional object images; obtains real-time segmented contours based on the real-time two-dimensional object images and the first grayscale threshold; obtains real-time contour angles based on the real-time segmented contours; obtains historical contour angles based on objects without mutual occlusion, and obtains a first angle threshold and a second angle threshold based on the historical contour angles; obtains real-time single object contour points, occluded contours, and unoccluded contours based on the real-time contour angles, the first angle threshold, and the second angle threshold; and performs different grasping recognition based on occluded and unoccluded contours. Its advantages include adaptability to different lighting changes and accurate recognition of stacked objects, thus improving the accuracy of object grasping.

[0063] This invention obtains real-time single-object contour points, occluded contours, and unoccluded contours based on real-time contour angles, a first angle threshold, and a second angle threshold. Its advantage lies in its ability to identify occlusions of stacked items, thereby improving the accuracy of object grasping. Attached Figure Description

[0064] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0065] Figure 2 This is a schematic diagram of the real-time segmented contour points, historical straight lines, first reference straight lines, second reference straight lines, first threshold straight lines, and second threshold straight lines of the present invention.

[0066] Figure 3 This is a schematic diagram of the real-time segmented contour and real-time segmented contour points of the present invention.

[0067] Figure 4 This is a schematic diagram of the real-time contour line segment and the real-time contour angle of the present invention;

[0068] Figure 5 This is a schematic diagram of the real-time single-object contour points and real-time fitted contours of the present invention. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0070] Example 1, please refer to Figure 1 As shown, this application provides an adaptive recognition method for robot grasping accuracy based on multi-source data acquisition, including the following steps:

[0071] Step S100: Acquire an image of the target object using a camera and label it as a real-time 2D object image; obtain a first grayscale threshold based on the real-time 2D object image; obtain a real-time segmentation contour based on the real-time 2D object image and the first grayscale threshold; obtain the real-time contour angle based on the real-time segmentation contour; Step S100 includes the following sub-steps:

[0072] Step S101: Install a camera above the center of the item placement area, with the camera facing the item placement area. The background of the item placement area is black. Then, use the camera to acquire an image of the target grabbing item and mark it as a real-time two-dimensional item image. The target grabbing item is mainly for parts that are circular pieces. The background is set to black, which is different from the color of the target grabbing item. For example, the target grabbing item is silver-white.

[0073] Step S102: Convert the RGB values ​​of each pixel in the real-time two-dimensional object image into grayscale values ​​to obtain a grayscale image, which is then marked as a real-time two-dimensional object grayscale image.

[0074] Step S103: Mark the gray value of each pixel in the real-time two-dimensional object grayscale image as the real-time object grayscale value;

[0075] Step S104: Divide the grayscale value range from 0 to 255 into N grayscale value ranges of equal length, and mark them as the first grayscale range; for example, N is 8.

[0076] Step S105: Obtain the number of real-time item grayscale values ​​within each first grayscale range and mark them as the number within the first grayscale range;

[0077] Step S106: Mark the median value of the first grayscale range as the median value of the first range;

[0078] Step S107: Sort the quantities in the first range from left to right according to the median value in the first range from smallest to largest;

[0079] Step S108: Mark the quantity within the first range that is greater than the quantity on the left and right sides as the first target quantity;

[0080] Step S109: Obtain the first target quantity from left to right and mark it as the second target quantity; obtain the first target quantity from right to left and mark it as the third target quantity;

[0081] Step S110: Obtain the quantity within the first range that is the smallest between the second target quantity and the third target quantity, and mark it as the fourth target quantity;

[0082] Step S111: Obtain the median value of the first range corresponding to the number of fourth targets, and mark it as the first grayscale threshold;

[0083] Step S112: Obtain the real-time grayscale value of the item that is greater than the first grayscale threshold and mark it as the real-time segmentation grayscale value; because the background is black, that is, the grayscale value of the background is low and the grayscale value of the target grasped item is high, the real-time segmentation grayscale value is the segmentation grayscale value between the background and the target grasped item; it can adapt to changes in lighting.

[0084] In practical applications, the first grayscale threshold is set to 111. Real-time grayscale values ​​of items with a value greater than 111 are then identified and marked as real-time segmented grayscale values.

[0085] Step S113: Obtain the continuous region composed of pixels with real-time grayscale values ​​and mark it as the real-time segmentation region;

[0086] Step S114: Obtain the contour of the real-time segmented region and mark it as the real-time segmented contour;

[0087] For practical applications, please refer to Figure 4 As shown, this is a schematic diagram of obtaining the real-time segmented contour, that is, the contour of the target object to be grabbed.

[0088] Step S115: Establish a Cartesian coordinate system and mark it as the real-time segmentation coordinate system;

[0089] Step S116: Place the real-time segmented contour in the first quadrant of the real-time segmentation coordinate system;

[0090] Step S117: Draw a point at intervals M on the real-time segmented contour and mark it as a real-time segmented contour point; for example, M is 0.4cm; the interval distance is the distance along the real-time segmented contour, not a straight line distance;

[0091] For practical applications, please refer to Figure 3As shown, a point is drawn at each interval of 0.4cm on the real-time segmented contour and marked as a real-time segmented contour point; a schematic diagram of the real-time segmented contour points is obtained.

[0092] Step S118: Connect adjacent real-time segmented contour points to obtain line segments, and mark them as real-time contour line segments;

[0093] Step S119: Obtain the angle between adjacent real-time contour segments and mark it as the real-time contour angle. Since the target grabbing item is a circular piece, the real-time contour angle of non-overlapping target grabbing items remains basically unchanged. Therefore, the real-time contour angle can be used to determine whether the target grabbing items overlap.

[0094] For practical applications, please refer to Figure 4 As shown, this is a schematic diagram of the angle between a real-time contour line segment and the real-time contour. For example, the angle between a real-time contour segment and the real-time contour is 168.0°.

[0095] Step S200: Obtain the historical contour angle based on items that do not obstruct each other, and obtain a first angle threshold and a second angle threshold based on the historical contour angle; Step S200 includes the following sub-steps:

[0096] Step S201: Obtain the real-time contour angle of items without mutual occlusion and mark it as the historical contour angle; in order to obtain the distribution range of the real-time contour angle under a single circular contour; because the boundary of the contour acquisition may fluctuate, the historical contour angle will fluctuate slightly.

[0097] Step S202: Obtain F1 historical contour angles; for example, F1 is 200.

[0098] Step S203: Establish a Cartesian coordinate system with the historical contour angle as the horizontal axis value, and mark it as the first historical coordinate system; visualize the data to make it easier to analyze the historical contour angle.

[0099] Step S204: Obtain the average value of the historical contour angles and mark it as the historical angle average.

[0100] Step S205: Obtain the point on the horizontal axis of the first historical coordinate system where the historical mean angle is located, and mark it as the historical mean point;

[0101] Step S206: Obtain the straight line passing through the historical mean point and using it as the vertical axis of the first historical coordinate system, and mark it as the straight line in history;

[0102] Step S207: In the first quadrant of the first historical coordinate system, coordinate points whose horizontal axis value is the historical contour angle and whose vertical axis value is any value are marked as historical angle coordinate points; historical angle coordinate points represent the distribution of historical contour angles.

[0103] For practical applications, please refer to Figure 2 As shown, the average historical included angle is 168.0°, so the straight line in the history is X=168.0°. At the same time, a schematic diagram of the coordinate points of the historical included angle is obtained.

[0104] Step S208: Obtain half of F1 and mark it as F2; ​​if F1 is 200, then F2 is 100.

[0105] Step S209: The first historical straight line begins to move in the negative direction of the horizontal axis of the first historical coordinate system by a straight line parallel to the vertical axis of the first historical coordinate system, and is marked as the first translation straight line; the first historical straight line begins to move in the positive direction of the horizontal axis of the first historical coordinate system by a straight line parallel to the vertical axis of the first historical coordinate system, and is marked as the second translation straight line; the first translation straight line and the second translation straight line move the same distance; the historical angle coordinate point between the first translation straight line and the second translation straight line is obtained and marked as the first intermediate distribution coordinate point; when the first intermediate distribution coordinate point is equal to or greater than F2, the movement of the first translation straight line and the second translation straight line is stopped, and the movement of the first translation straight line and the second translation straight line at this stopped position is obtained and marked as the first reference straight line and the second reference straight line, respectively; in order to obtain the distribution length of the intermediate historical contour angle;

[0106] For practical applications, please refer to Figure 2 As shown, the first reference line is X=167.8° and the second reference line is X=168.2°.

[0107] Step S210: Set a ratio, labeled B1; obtain the distance between the first reference line and the second reference line, labeled D1; obtain the product of B1 and D1, labeled D2; B1 is 3; because the historical contour angles are concentrated, the dispersion of the historical contour angles can be set by D1 to filter out the minimum and maximum values; referring to the quartile filtering method, B1 is 1.5;

[0108] For practical applications, please refer to Figure 2 As shown, the distance between the first and second reference lines is 0.4°, therefore D2 is 0.4 × 1.5 = 0.6°; historical profile angle.

[0109] Step S211: Move the first reference line D2 in the negative direction of the horizontal axis of the first historical coordinate system to obtain a straight line, and mark it as the first threshold line;

[0110] Step S212: Move the second reference line D2 in the positive direction of the horizontal axis of the first historical coordinate system to obtain a straight line, and mark it as the second threshold line;

[0111] Step S213: Obtain the historical angle coordinates between the first threshold line and the second threshold line, and mark them as historical filtering coordinates.

[0112] Step S214: Obtain the minimum and maximum values ​​of the abscissas of all historical filtered coordinate points, and mark them as the first included angle threshold and the second included angle threshold, respectively; the minimum value of the real-time included angle of a normal single circle contour is the first included angle threshold, and the maximum value is the second included angle threshold.

[0113] For practical applications, please refer to Figure 2 As shown, the first threshold line obtained is X=167.2°, the first threshold line obtained is X=168.8°, the first included angle threshold is 167.2°, and the second included angle threshold is 168.8°.

[0114] Step S300: Obtain real-time single object contour points, occluded contours, and unoccluded contours based on the real-time contour angle, the first angle threshold, and the second angle threshold; Step S300 includes the following sub-steps:

[0115] Step S301: Using any real-time segmented contour point, search to both sides of the real-time segmented contour. If the angle of the real-time contour is not within the range of the first angle threshold to the second angle threshold, stop the search and identify the real-time segmented contour corresponding to the searched real-time segmented contour point as an occluded contour. If the angle of the real-time contour is not within the range of the first angle threshold to the second angle threshold, and the search returns to the real-time segmented contour point, the real-time segmented contour corresponding to the searched real-time segmented contour point is identified as an unoccluded contour.

[0116] Step S400 involves performing different grasping and recognition operations based on occluded and unoccluded contours; Step S400 includes the following sub-steps:

[0117] Step S401: If the outline is unoccluded, then obtain the 3D outline point of a target object within the real-time segmented outline through 3D scanning, and mark it as a real-time single-object 3D point.

[0118] Step S402: Based on real-time single-object 3D points, the robot grasps each target item; for unoccluded items, the 3D coordinates of the target item are directly obtained through 3D; when executing the grasping program, the unoccluded contours are grasped first.

[0119] Step S403: If it is an occluded contour, mark the real-time segmented contour points found each time as real-time single object contour points.

[0120] Step S404: Fit the real-time single object contour points to the equation of a circle to obtain a function, and mark it as the real-time fitted contour; the real-time fitted contour occludes and restores the contour.

[0121] Step S405: Each real-time fitted contour region corresponds to a target object to be captured; then, the 3D contour points of the target object within the real-time fitted contour are obtained through 3D scanning and marked as real-time single-object 3D points; contour restoration of the occlusion contour more accurately obtains the 3D coordinate points of the target object.

[0122] Step S406: Based on real-time single-object 3D points, the robot grasps each target item.

[0123] Example 2: This application also provides an electronic device, which may include: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps of the robot grasping accuracy adaptive recognition method based on multi-source data acquisition are performed to achieve the following functions: acquiring an image of the target grasped object based on a camera and marking it as a real-time two-dimensional object image; obtaining a first grayscale threshold based on the real-time two-dimensional object image; obtaining a real-time segmented contour based on the real-time two-dimensional object image and the first grayscale threshold; obtaining a real-time contour angle based on the real-time segmented contour; obtaining historical contour angles based on objects without mutual occlusion, and obtaining a first angle threshold and a second angle threshold based on the historical contour angles; obtaining real-time single object contour points, occluded contours, and unoccluded contours based on the real-time contour angles, the first angle threshold, and the second angle threshold; and performing different grasping recognitions based on the occluded contours and unoccluded contours.

[0124] Furthermore, when the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0125] Example 3: This application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the robot grasping accuracy adaptive recognition method based on multi-source data acquisition provided by the above methods. The method includes: acquiring a target grasping object image based on a camera and marking it as a real-time two-dimensional object image; acquiring a first grayscale threshold based on the real-time two-dimensional object image; acquiring a real-time segmented contour based on the real-time two-dimensional object image and the first grayscale threshold; acquiring a real-time contour angle based on the real-time segmented contour; acquiring historical contour angles based on objects without mutual occlusion; acquiring a first angle threshold and a second angle threshold based on the historical contour angles; acquiring real-time single object contour points, occluded contours, and unoccluded contours based on the real-time contour angles, the first angle threshold, and the second angle threshold; and performing different grasping recognitions based on the occluded contours and unoccluded contours.

[0126] Example 4: This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the steps of the above-mentioned adaptive recognition method for robot grasping accuracy based on multi-source data acquisition to achieve the following functions: acquiring an image of the target object to be grasped based on a camera and marking it as a real-time two-dimensional object image; obtaining a first grayscale threshold based on the real-time two-dimensional object image; obtaining a real-time segmented contour based on the real-time two-dimensional object image and the first grayscale threshold; obtaining a real-time contour angle based on the real-time segmented contour; obtaining historical contour angles based on objects without mutual occlusion, and obtaining a first angle threshold and a second angle threshold based on the historical contour angles; obtaining real-time single object contour points, occluded contours, and unoccluded contours based on the real-time contour angles, the first angle threshold, and the second angle threshold; and performing different grasping recognitions based on the occluded and unoccluded contours.

[0127] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the above technical solutions, in essence or in terms of their contribution to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.

[0128] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An adaptive recognition method for robot grasping accuracy based on multi-source data acquisition, characterized in that, Includes the following steps: S100: Acquire an image of the target object based on the camera and label it as a real-time two-dimensional object image; obtain a first grayscale threshold based on the real-time two-dimensional object image; obtain a real-time segmentation contour based on the real-time two-dimensional object image and the first grayscale threshold; Real-time contour angle is obtained based on real-time contour segmentation; S200, obtain the historical contour angle based on items that do not obstruct each other, and obtain the first angle threshold and the second angle threshold based on the historical contour angle; S300, based on the real-time contour angle, the first angle threshold and the second angle threshold, obtains the real-time single object contour points, occluded contours and unoccluded contours. S400 performs different grasping and recognition based on occluded and unoccluded contours.

2. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 1, characterized in that, S100 includes the following sub-steps: A camera is installed above the center of the item placement area, facing the item placement area with a black background. The camera then captures images of the target items and marks them as real-time two-dimensional item images. The RGB values ​​of each pixel in the real-time 2D object image are converted into grayscale values ​​to obtain a grayscale image, which is then marked as a real-time 2D object grayscale image.

3. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 2, characterized in that, S100 also includes the following sub-steps: Mark the grayscale value of each pixel in the real-time 2D grayscale image of the object as the real-time object grayscale value; Divide the grayscale value range from 0 to 255 into N grayscale value ranges of equal length, and mark them as the first grayscale range. Obtain the number of real-time grayscale values ​​of items within each first grayscale range, and mark them as the number within the first grayscale range; Mark the median value of the first grayscale range as the median value of the first range; Sort the quantities in the first range from left to right according to the median value in the first range, from smallest to largest. The quantity within the first range that is greater than the quantity on the left and right sides is marked as the first target quantity; Obtain the first target quantity from left to right and mark it as the second target quantity; Get the first quantity of the first target from right to left, and mark it as the quantity of the third target; The quantity within the first range that is the smallest between the second and third target quantities is marked as the fourth target quantity; Obtain the median value of the first range corresponding to the number of fourth targets, and mark it as the first grayscale threshold.

4. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 3, characterized in that, S100 also includes the following sub-steps: Obtain the real-time grayscale value of the item that is greater than the first grayscale threshold, and mark it as the real-time segmented grayscale value; Obtain a continuous region composed of pixels with real-time grayscale values ​​and mark it as the real-time segmentation region; Obtain the contour of the real-time segmented region and mark it as the real-time segmented contour.

5. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 4, characterized in that, S100 also includes the following sub-steps: Establish a Cartesian coordinate system and label it as the real-time segmentation coordinate system; Place the real-time segmented contour in the first quadrant of the real-time segmentation coordinate system; Draw a point at intervals M on the real-time segmented contour and mark it as a real-time segmented contour point; Connect adjacent real-time segmented contour points to obtain line segments, which are then marked as real-time contour line segments; Obtain the included angle between adjacent real-time contour segments and mark it as the real-time contour angle.

6. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 5, characterized in that, S200 includes the following sub-steps: Obtain the real-time outline angle of items that do not obstruct each other, and mark it as the historical outline angle; Obtain the included angles of F1 historical contours; A Cartesian coordinate system is established with the included angle of the historical contours as the horizontal axis value, and it is marked as the first historical coordinate system; Obtain the mean of the historical contour angles and mark it as the historical mean angle; Obtain the point on the horizontal axis of the first historical coordinate system where the historical mean angle is located, and mark it as the historical mean point; Obtain the straight line that passes through the historical mean point and uses it as the vertical axis of the first historical coordinate system; mark it as the straight line in history. In the first quadrant of the first historical coordinate system, the coordinates of points where the horizontal axis of the first historical coordinate system is the historical profile angle and the vertical axis of the first historical coordinate system is any value are marked as historical angle coordinate points.

7. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 6, characterized in that, S200 also includes the following sub-steps: Get half of F1 and label it F2; The first historical line begins to move in the negative direction of the horizontal axis of the first historical coordinate system by a line parallel to the vertical axis of the first historical coordinate system, and is marked as the first translation line; the first historical line begins to move in the positive direction of the horizontal axis of the first historical coordinate system by a line parallel to the vertical axis of the first historical coordinate system, and is marked as the second translation line; the first translation line and the second translation line move the same distance; the historical angle coordinates between the first translation line and the second translation line are obtained and marked as the first intermediate distribution coordinate point; when the first intermediate distribution coordinate point is equal to or greater than F2, the movement of the first translation line and the second translation line is stopped, and the movement of the first translation line and the second translation line at this stopped position is obtained and marked as the first reference line and the second reference line, respectively; Set a ratio, labeled B1; obtain the distance between the first reference line and the second reference line, labeled D1; Obtain the product of B1 and D1, and label it as D2; The first reference line is moved by D2 in the negative direction of the horizontal axis of the first historical coordinate system to obtain a straight line, which is marked as the first threshold line. The second reference line is moved by D2 in the positive direction of the horizontal axis of the first historical coordinate system to obtain a straight line, which is marked as the second threshold line. Obtain the historical angle coordinates between the first threshold line and the second threshold line, and mark them as historical filter coordinate points; Obtain the minimum and maximum x-coordinates of all historical filtered coordinate points, and label them as the first included angle threshold and the second included angle threshold, respectively.

8. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 7, characterized in that, S300 includes the following sub-steps: Searching towards both sides of the real-time segmented contour from any real-time segmented contour point, if the angle of the real-time contour is outside the range of the first angle threshold to the second angle threshold, the search stops and the real-time segmented contour corresponding to the searched real-time segmented contour point is identified as an occluded contour; if the angle of the real-time contour is not outside the range of the first angle threshold to the second angle threshold, and the search returns to the real-time segmented contour point, the real-time segmented contour corresponding to the searched real-time segmented contour point is identified as an unoccluded contour.

9. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 8, characterized in that, S400 includes the following sub-steps: If the outline is unobstructed, then the 3D outline points of a target object within the real-time segmented outline are obtained through 3D scanning and marked as real-time single-object 3D points. The robot grasps each target item based on real-time single-object 3D point data.

10. The adaptive recognition method for robot grasping accuracy based on multi-source data acquisition according to claim 9, characterized in that, S400 also includes the following sub-steps: If the outline is occluded, each real-time segmented outline point found in the search will be marked as a real-time single object outline point. The function is obtained by fitting the real-time single object contour points with the equation of a circle, and it is marked as the real-time fitted contour. Each real-time fitted contour region corresponds to a target item that needs to be captured. Then, the 3D contour points of a target object within the real-time fitted contour are obtained through 3D scanning and marked as real-time single-object 3D points. The robot grasps each target item based on real-time single-object 3D point data.

Images