Robot adaptive material grabbing method and device based on machine vision

Abstract

The application relates to a robot adaptive material grabbing method and device based on machine vision, which comprises the following steps: acquiring a first image and a second image of target material based on a binocular camera; performing fusion processing on the first image and the second image to obtain a to-be-analyzed image; performing edge detection on the to-be-analyzed image, excluding interference edges, obtaining a material edge, marking the material edge in the first image and the second image to obtain a marked first image and a marked second image; extracting 3D information based on the marked first image and the marked second image; reading robot mechanical arm coordinates, and performing path planning based on the robot mechanical arm coordinates, material position coordinates and all obstacle position coordinates to obtain a planned path; and controlling the mechanical arm to grab the material based on the planned path. The application can not only satisfy robot adaptive material grabbing, but also improve the material grabbing success rate and efficiency of the mechanical arm.

Description

Technical Field

[0001] The present invention relates to the technical field related to intelligent robots, and in particular to a robot adaptive material grasping method and device based on machine vision. Background Art

[0002] With the advancement of industrial automation and intelligence, traditional automation systems often rely on fixed-shaped materials or predefined grasping tasks. However, in practice, the complex shapes, sizes, and postures of objects increase the difficulty of grasping tasks. One area that needs to be addressed is how to adaptively grasp complex and irregularly shaped materials.

[0003] In addition, after determining the edge shape of the material, another problem that needs to be solved is path planning for the robot's robotic arm to grasp the material. This is because there may be some obstacles in the three-dimensional space where the material exists. If the path is simply planned based on the shortest distance, it often cannot meet actual needs. Summary of the Invention

[0004] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a robot adaptive material grasping method and device based on machine vision.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] Specifically, a robot adaptive material grasping method based on machine vision is proposed, including the following:

[0007] Acquire a first image and a second image of a target material based on a binocular camera;

[0008] performing fusion processing on the first image and the second image to obtain an image to be analyzed;

[0009] Performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0010] Extracting 3D information based on the marked first image and the marked second image, and determining the material position coordinates and all obstacle position coordinates based on the 3D information;

[0011] Read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain the planned path;

[0012] The robot arm is controlled based on the planned path to grab materials.

[0013] Further, specifically, the first image and the second image are fused to obtain an image to be analyzed, including

[0014] Perform a grayscale operation on the first image to obtain a first grayscale image, calculate the average grayscale value of all pixels in the first grayscale image to obtain avg_gray1, traverse the first grayscale image, and perform the following operations on each pixel of the first grayscale image:

[0015] Subtract avg_gray1 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0 to obtain an updated first grayscale image;

[0016] Perform a grayscale operation on the second image to obtain a second grayscale image, calculate the average grayscale value of all pixels in the second grayscale image to obtain avg_gray2, traverse the second grayscale image, and perform the following operations on each pixel of the second grayscale image:

[0017] Subtract avg_gray2 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0 to obtain an updated second grayscale image;

[0018] The pixel points at the same position of the updated first grayscale image and the updated second grayscale image are averaged, and the pixel points at all positions and their associated average values ​​together constitute the image to be analyzed.

[0019] Further, specifically, edge detection is performed on the image to be analyzed, and interference edges are eliminated to obtain the material edge, including:

[0020] Performing edge detection on the image to be analyzed to obtain an edge image;

[0021] Execute the connected domain segmentation algorithm on the edge image to obtain all connected regions in the edge image as suspected material edge regions;

[0022] The suspected material edge area is subjected to double dynamic small area elimination to obtain the material edge.

[0023] Furthermore, performing double dynamic small area elimination on the suspected material edge area to obtain the material edge includes:

[0024] Calculating the area values ​​of all suspected material edge regions, and then calculating the first average area value of all suspected material edge regions;

[0025] Eliminate all suspected material edge regions whose area values ​​are smaller than the first average area value to obtain an optimized material edge region;

[0026] Then calculate the second average area value of all edge regions in the optimized material edge region;

[0027] At this time, the updated average area value is obtained by subtracting the balance coefficient from the second average area value, and the balance coefficient is a positive number set manually;

[0028] The area smaller than the updated average area value in the optimized material edge area is removed to obtain the material edge.

[0029] Further, specifically, extracting 3D information based on the marked first image and the marked second image includes:

[0030] Image segmentation and feature extraction are performed on the marked first image and the marked second image respectively, and then 3D information is obtained through stereo matching, depth map generation and 3D object reconstruction.

[0031] Furthermore, specifically, edge detection is performed on the image to be analyzed by an edge detection algorithm based on a Canny operator to obtain an edge image.

[0032] Furthermore, path planning is performed based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain a planned path, including:

[0033] First, when only considering the coordinates of the robot arm and the material position coordinates, the preliminary path is planned by the Dijkstra algorithm;

[0034] Then add all obstacle position coordinates to the preliminary path;

[0035] When the preliminary path passes through the coordinates of any obstacle, it is avoided through the obstacle avoidance algorithm and the planned path is finally obtained.

[0036] Furthermore, specifically, when the preliminary path passes through the coordinates of any obstacle, the obstacle avoidance algorithm is used to avoid the obstacle, including:

[0037] Project the robot arm coordinates and all obstacle position coordinates onto the xy plane, i.e. z = 0. The projection points are denoted as the robot arm projection point and the obstacle projection point i, where i represents any obstacle number.

[0038] Simulate the movement of the robotic arm along the preliminary path and calculate the actual distance Li between the robotic arm projection point h and any obstacle projection point i in real time;

[0039] If Li is less than the safety threshold, find the nearest obstacle projection point in the current moving direction of the robot arm except the obstacle projection point i and record it as j;

[0040] Connect h and i to obtain straight line hi, and connect h and j to obtain straight line hj. There is a first angle between hi and hj. Compensate the first angle to obtain a second angle, which is the first angle + α, where α is a preset small angle.

[0041] Then the robot arm is controlled to rotate from hi to hj by a second angle until Li is not less than the safety threshold;

[0042] Repeat the above method until the robot reaches the material position, and the updated path is the planned path.

[0043] The present invention also proposes a robot adaptive material grasping device based on machine vision, which is characterized by comprising the following:

[0044] An image acquisition module, configured to acquire a first image and a second image of a target material based on a binocular camera;

[0045] An image fusion module, configured to fuse the first image and the second image to obtain an image to be analyzed;

[0046] an edge detection module, configured to perform edge detection on the image to be analyzed, eliminate interference edges to obtain material edges, and mark the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0047] a 3D information extraction module, configured to extract 3D information based on the marked first image and the marked second image, and determine the material position coordinates and the position coordinates of all obstacles based on the 3D information;

[0048] The path planning module is used to read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path;

[0049] The material grabbing module is used to control the robotic arm to grab materials based on the planned path.

[0050] The beneficial effects of the present invention are:

[0051] The present invention proposes a robot adaptive material grasping method and device based on machine vision. First, after the robot obtains images through a binocular camera, image fusion is performed to obtain the image to be analyzed to eliminate the interference problem caused by the difference between the binocular images. Then, edge detection is performed on the image to be analyzed and interference edges are eliminated to obtain accurate material edges to facilitate subsequent 3D information extraction. Then, when performing path planning, the optimal path is obtained based on the actual obstacles in the 3D information. This can not only meet the robot's adaptive material grasping needs, but also improve the success rate and efficiency of the robot arm's material grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other features of the present disclosure will become more apparent through a detailed description of the embodiments shown in conjunction with the accompanying drawings. The same reference numerals in the drawings of the present disclosure represent the same or similar elements. Obviously, the drawings described below are only some embodiments of the present disclosure. It is possible for a person skilled in the art to derive other drawings based on these drawings without inventive effort. In the drawings:

[0053] Figure 1 Shown is a flow chart of the robot adaptive material grasping method based on machine vision of the present invention;

[0054] Figure 2 The figure shows a flow chart of fusing the first image and the second image to obtain the image to be analyzed;

[0055] Figure 3 The figure shows a flow chart of performing edge detection on the image to be analyzed and eliminating interference edges to obtain the edge of the material;

[0056] Figure 4 The figure shows a flow chart of performing double dynamic small area elimination on the suspected material edge area to obtain the material edge. DETAILED DESCRIPTION

[0057] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of this application can be combined with each other unless there is a conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0058] Example 1, with reference to Figure 1 The present invention proposes a robot adaptive material grasping method based on machine vision, which includes the following:

[0059] Step 110: Acquire a first image and a second image of the target material based on a binocular camera;

[0060] Step 120: performing fusion processing on the first image and the second image to obtain an image to be analyzed;

[0061] Step 130: performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0062] Step 140: extracting 3D information based on the marked first image and the marked second image, and determining the material position coordinates and all obstacle position coordinates based on the 3D information;

[0063] Step 150: Read the coordinates of the robot arm, and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path;

[0064] Step 160: Control the robotic arm to grab the material based on the planned path.

[0065] In this embodiment 1, after the robot obtains the image through the binocular camera, the image fusion is performed to obtain the image to be analyzed to remove the interference problem caused by the binocular image difference, and then the edge detection is performed on the image to be analyzed and the interference edge is eliminated to obtain the accurate material edge to facilitate the subsequent 3D information extraction; then, when performing path planning, the optimal path is obtained based on the actual obstacles in the 3D information, which can not only meet the robot's adaptive material grasping, but also improve the success rate and efficiency of the robot arm's material grasping.

[0066] Reference Figure 2 As a preferred embodiment of the present invention, specifically, the first image and the second image are fused to obtain an image to be analyzed, including

[0067] Step 210: Perform a grayscale operation on the first image to obtain a first grayscale image, calculate the average grayscale value of all pixels in the first grayscale image to obtain avg_gray1, traverse the first grayscale image, and perform the following operations on each pixel of the first grayscale image:

[0068] Step 220: Subtract avg_gray1 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, thereby obtaining an updated first grayscale image;

[0069] Step 230: Perform a grayscale operation on the second image to obtain a second grayscale image, calculate the average grayscale value of all pixels in the second grayscale image to obtain avg_gray2, traverse the second grayscale image, and perform the following operations on each pixel of the second grayscale image:

[0070] Step 240: Subtract avg_gray2 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale values ​​of the updated grayscale values ​​that are less than 0 to 0, thereby obtaining an updated second grayscale image.

[0071] Step 250 : averaging the pixels at the same position in the updated first grayscale image and the updated second grayscale image. The pixels at all positions and their associated average values ​​together constitute the image to be analyzed.

[0072] In this preferred embodiment, on the one hand, the two 2D images obtained by the binocular camera are respectively subjected to grayscale adjustment to preliminarily remove possible dust or other factors that may affect the results; then, considering the possible differences between the binocular cameras, the differences are balanced by averaging, making it easier to find the edges of the images to be analyzed.

[0073] Reference Figure 3 As a preferred embodiment of the present invention, specifically, edge detection is performed on the image to be analyzed, and interference edges are eliminated to obtain the material edge, including:

[0074] Step 310: Perform edge detection on the image to be analyzed to obtain an edge image;

[0075] Step 320: Execute a connected domain segmentation algorithm on the edge image to obtain all connected regions in the edge image and record them as suspected material edge regions;

[0076] Step 330: Perform double dynamic small area elimination on the suspected material edge area to obtain the material edge.

[0077] Reference Figure 4 As a preferred embodiment of the present invention, the suspected material edge area is subjected to double dynamic small area elimination to obtain the material edge, including:

[0078] Step 410: Calculate the area values ​​of all suspected material edge regions, and then calculate the first average area value of all suspected material edge regions;

[0079] Step 420: Eliminate all suspected material edge regions whose area values ​​are smaller than the first average area value to obtain an optimized material edge region;

[0080] Step 430: Calculate the second average area value of all edge regions in the optimized material edge region;

[0081] Step 440: Subtract the balance coefficient from the second average area value to obtain an updated average area value, where the balance coefficient is a positive number set manually.

[0082] Step 450: Eliminate the area smaller than the updated average area value in the optimized material edge area to obtain the material edge.

[0083] In this preferred embodiment, considering the possible interference edge problem, the interference edge is proposed as much as possible through the double dynamic small area elimination algorithm to obtain the material edge.

[0084] As a preferred embodiment of the present invention, specifically, extracting 3D information based on the marked first image and the marked second image includes:

[0085] Image segmentation and feature extraction are performed on the marked first image and the marked second image respectively, and then 3D information is obtained through stereo matching, depth map generation and 3D object reconstruction.

[0086] In this preferred embodiment, since the edges of the material to be grasped are marked by the marked first image and the marked second image, 3D information is obtained through mature stereo matching technology, depth map generation and 3D object reconstruction.

[0087] As a preferred embodiment of the present invention, specifically, edge detection is performed on the image to be analyzed by an edge detection algorithm based on a Canny operator to obtain an edge image.

[0088] In this preferred embodiment, considering that the Canny operator is a commonly used edge detection algorithm in image processing, it has several obvious advantages.

[0089] Multi-stage processing: The Canny operator uses a multi-stage processing method, including:

[0090] Noise Suppression: First, a Gaussian filter is applied to smooth the image to reduce the effect of noise. This helps avoid noise being mistakenly detected as edges.

[0091] Gradient calculation: Calculate the gradient and gradient direction for each pixel to determine where the brightness changes fastest in the image, which are usually edges.

[0092] Non-maximum suppression: By suppressing the non-maximum of the edge, we ensure that only the points with the local maximum gradient are retained as edges, thus reducing false edges.

[0093] Dual-threshold detection and edge connection: The Canny operator uses two thresholds for edge connection, namely a high threshold and a low threshold, which can effectively retain strong edges while connecting weak edges.

[0094] Edge detection accuracy: The Canny operator can accurately detect the exact location of the edge by accurately determining the gradient direction and magnitude. This means that it can find very detailed edge information in the image.

[0095] Noise robustness: The Canny algorithm uses a Gaussian filter for smoothing, which can effectively suppress the noise in the image, thereby reducing the impact of noise on edge detection results, enabling it to achieve better edge detection results even in images with high noise.

[0096] Edge continuity: Through dual threshold processing, the Canny operator can effectively maintain edge continuity. In the image, strong edge points are clearly marked as edges, while weak edges are connected into continuous edges, avoiding broken edge lines.

[0097] Automatic threshold setting: Part of the Canny operator is the process of automatically setting the threshold (such as double threshold detection), which reduces the trouble of manually setting parameters and improves the applicability and versatility of the algorithm.

[0098] Therefore, the Canny operator is used for edge detection to improve accuracy.

[0099] As a preferred embodiment of the present invention, path planning is performed based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain a planned path, including:

[0100] First, when only considering the coordinates of the robot arm and the material position coordinates, the preliminary path is planned by the Dijkstra algorithm;

[0101] Then add all obstacle position coordinates to the preliminary path;

[0102] When the preliminary path passes through the coordinates of any obstacle, it is avoided through the obstacle avoidance algorithm and the planned path is finally obtained.

[0103] As a preferred embodiment of the present invention, specifically, when the preliminary path passes through the coordinates of any obstacle, the obstacle avoidance algorithm is used to avoid the obstacle, including:

[0104] Project the robot arm coordinates and all obstacle position coordinates onto the xy plane, i.e. z = 0. The projection points are denoted as the robot arm projection point and the obstacle projection point i, where i represents any obstacle number.

[0105] Simulate the movement of the robotic arm along the preliminary path and calculate the actual distance Li between the robotic arm projection point h and any obstacle projection point i in real time;

[0106] If Li is less than the safety threshold, find the nearest obstacle projection point in the current moving direction of the robot arm except the obstacle projection point i and record it as j;

[0107] Connect h and i to obtain straight line hi, and connect h and j to obtain straight line hj. There is a first angle between hi and hj. Compensate the first angle to obtain a second angle, which is the first angle + α, where α is a preset small angle.

[0108] Then the robot arm is controlled to rotate from hi to hj by a second angle until Li is not less than the safety threshold;

[0109] Repeat the above method until the robot reaches the material position, and the updated path is the planned path.

[0110] The robot arm coordinates, material position coordinates, and all obstacle position coordinates are their equivalent coordinates (generally the center of mass or end coordinates).

[0111] In this preferred embodiment, considering that existing algorithms often fail to comprehensively consider all obstacles when planning paths, through the above method, after the mature path planning algorithm obtains the preliminary path, the preliminary path is fine-tuned for all possible obstacles on the preliminary path and then the path is updated. The planned path obtained in this way is optimized for the possible paths of the optimal path, and can reach the material to be grabbed as quickly as possible while avoiding obstacles as much as possible.

[0112] The present invention also proposes a robot adaptive material grasping device based on machine vision, which is characterized by comprising the following:

[0113] An image acquisition module, configured to acquire a first image and a second image of a target material based on a binocular camera;

[0114] An image fusion module, configured to fuse the first image and the second image to obtain an image to be analyzed;

[0115] an edge detection module, configured to perform edge detection on the image to be analyzed, eliminate interference edges to obtain material edges, and mark the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0116] a 3D information extraction module, configured to extract 3D information based on the marked first image and the marked second image, and determine the material position coordinates and the position coordinates of all obstacles based on the 3D information;

[0117] The path planning module is used to read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path;

[0118] The material grabbing module is used to control the robotic arm to grab materials based on the planned path.

[0119] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0120] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0121] Although the present invention has been described in considerable detail and with particularity with respect to several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be construed as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively encompassing the intended scope of the invention. In addition, the invention has been described above in terms of embodiments foreseen by the inventors for the purpose of providing a useful description, and those insubstantial modifications of the invention that are not currently foreseen may still represent equivalent modifications of the invention.

[0122] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. As long as the technical effects of the present invention are achieved by the same means, they shall fall within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. A robot adaptive material grasping method based on machine vision, characterized in that: These include: Acquire a first image and a second image of a target material based on a binocular camera; performing fusion processing on the first image and the second image to obtain an image to be analyzed; Performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image; Extracting 3D information based on the marked first image and the marked second image, and determining the material position coordinates and all obstacle position coordinates based on the 3D information; Read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain the planned path; Control the robotic arm to grab materials based on the planned path; Specifically, the first image and the second image are fused to obtain an image to be analyzed, including Perform a grayscale operation on the first image to obtain a first grayscale image, calculate the average grayscale value of all pixels in the first grayscale image to obtain avg_gray1, traverse the first grayscale image, and perform the following operations on each pixel of the first grayscale image: Subtract avg_gray1 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0 to obtain an updated first grayscale image; Perform a grayscale operation on the second image to obtain a second grayscale image, calculate the average grayscale value of all pixels in the second grayscale image to obtain avg_gray2, traverse the second grayscale image, and perform the following operations on each pixel of the second grayscale image: Subtract avg_gray2 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0 to obtain an updated second grayscale image; Calculating the average value of the pixels at the same position of the updated first grayscale image and the updated second grayscale image, so that the pixels at all positions and their associated average values ​​together constitute the image to be analyzed; Specifically, edge detection is performed on the image to be analyzed, and interference edges are eliminated to obtain the material edge, including: Performing edge detection on the image to be analyzed to obtain an edge image; Execute the connected domain segmentation algorithm on the edge image to obtain all connected regions in the edge image as suspected material edge regions; Performing double dynamic small area elimination on the suspected material edge area to obtain the material edge; Performing double dynamic small area elimination on the suspected material edge area to obtain the material edge, including: Calculating the area values ​​of all suspected material edge regions, and then calculating the first average area value of all suspected material edge regions; Eliminate all suspected material edge regions whose area values ​​are smaller than the first average area value to obtain an optimized material edge region; Then calculate the second average area value of all edge regions in the optimized material edge region; At this time, the updated average area value is obtained by subtracting the balance coefficient from the second average area value, and the balance coefficient is a positive number set manually; The area smaller than the updated average area value in the optimized material edge area is removed to obtain the material edge.

2. The machine vision-based robot adaptive material grasping method according to claim 1, characterized in that: Specifically, extracting 3D information based on the marked first image and the marked second image includes: Image segmentation and feature extraction are performed on the marked first image and the marked second image respectively, and then 3D information is obtained through stereo matching, depth map generation and 3D object reconstruction.

3. The method for robot adaptive material grasping based on machine vision according to claim 1, characterized in that: Specifically, edge detection is performed on the image to be analyzed by an edge detection algorithm based on the Canny operator to obtain an edge image.

4. The method for robot adaptive material grasping based on machine vision according to claim 1, characterized in that: Path planning is performed based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain the planned path, including: First, when only considering the coordinates of the robot arm and the material position coordinates, the preliminary path is planned using the Dijkstra algorithm; Then add all obstacle position coordinates to the preliminary path; When the preliminary path passes through the coordinates of any obstacle, it is avoided through the obstacle avoidance algorithm and the planned path is finally obtained.

5. The method for robot adaptive material grasping based on machine vision according to claim 4, characterized in that: Specifically, when the preliminary path passes through the coordinates of any obstacle, the obstacle avoidance algorithm is used to avoid it, including: Project the coordinates of the robot arm and all obstacle positions onto the xy plane, i.e. z=0. The projection points are denoted as the robot arm projection point and the obstacle projection point i, where i represents any obstacle number. Simulate the movement of the robotic arm along the preliminary path and calculate the actual distance Li between the robotic arm projection point h and any obstacle projection point i in real time; If Li is less than the safety threshold, find the nearest obstacle projection point in the current moving direction of the robot arm except the obstacle projection point i and record it as j; Connect h and i to get straight line hi, connect h and j to get straight line hj, then there is a first angle between hi and hj, and the first angle is compensated to get a second angle, the second angle = the first angle + α, where α is a preset small angle; Then the robot arm is controlled to rotate from hi to hj by a second angle until Li is not less than the safety threshold; Repeat the above method until the robot reaches the material position, and the updated path is the planned path.

6. A robot-adaptive material grabbing device based on machine vision, characterized in that: The steps of the method for robot adaptive material grasping based on machine vision according to any one of claims 1 to 5 are applied, and the device comprises the following: An image acquisition module, configured to acquire a first image and a second image of a target material based on a binocular camera; An image fusion module, configured to fuse the first image and the second image to obtain an image to be analyzed; an edge detection module, configured to perform edge detection on the image to be analyzed, eliminate interference edges to obtain material edges, and mark the material edges in the first image and the second image to obtain a marked first image and a marked second image; a 3D information extraction module, configured to extract 3D information based on the marked first image and the marked second image, and determine the material position coordinates and the position coordinates of all obstacles based on the 3D information; The path planning module is used to read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path; The material grabbing module is used to control the robotic arm to grab materials based on the planned path.

Images