Robot self-adaptive material grabbing method and device based on machine vision

Abstract

The invention relates to a robot self-adaptive material grabbing method and device based on machine vision. The robot self-adaptive material grabbing method comprises the following steps that a first image and a second image of a target material are obtained based on a binocular camera; performing fusion processing on the first image and the second image to obtain a to-be-analyzed image; performing edge detection on the to-be-analyzed image, performing interference edge elimination to obtain a material edge, and marking the material edge in the first image and the second image to obtain a marked first image and a marked second image; extracting 3D information based on the marked first image and the marked second image; robot mechanical arm coordinates are read, and path planning is carried out based on the robot mechanical arm coordinates, the material position coordinates and all obstacle position coordinates to obtain a planned path; and controlling the mechanical arm to grab materials based on the planned path. According to the mechanical arm, self-adaptive material grabbing of the robot can be met, and the material grabbing success rate and efficiency of the mechanical arm can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field related to intelligent robots, and in particular to a robot adaptive material grasping method and device based on machine vision. Background Art

[0002] With the development of industrial automation and intelligence, traditional automation systems usually rely on fixed-shaped materials or predefined grasping tasks. However, in actual applications, the complex shapes, sizes, and posture changes of objects increase the difficulty of grasping tasks. How to perform adaptive grasping for materials with complex and irregular shapes is one aspect that needs to be solved.

[0003] In addition, after determining the edge shape of the material, path planning for the robot's mechanical arm to grab the material is another problem that needs to be solved. This is because there may be some obstacles in the three-dimensional space where the material exists. If the path is simply planned based on the shortest distance, it often cannot meet actual needs. Summary of the invention

[0004] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a robot adaptive material grasping method and device based on machine vision.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] Specifically, a robot adaptive material grasping method based on machine vision is proposed, including the following:

[0007] Acquire a first image and a second image of a target material based on a binocular camera;

[0008] Performing fusion processing on the first image and the second image to obtain an image to be analyzed;

[0009] Performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0010] Extracting 3D information based on the marked first image and the marked second image, and determining the material position coordinates and all obstacle position coordinates based on the 3D information;

[0011] Read the robot arm coordinates, and perform path planning based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain the planned path;

[0012] Control the robotic arm to grab materials based on the planned path.

[0013] Further, specifically, the first image and the second image are fused to obtain an image to be analyzed, including:

[0014] Perform a grayscale operation on the first image to obtain a first grayscale image, calculate the average grayscale value of all pixels in the first grayscale image to obtain avg_gray1, traverse the first grayscale image, and perform the following operations on each pixel of the first grayscale image:

[0015] Subtract avg_gray1 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, to obtain an updated first grayscale image;

[0016] Perform grayscale operation on the second image to obtain a second grayscale image, calculate the average grayscale value of all pixels in the second grayscale image to obtain avg_gray2, traverse the second grayscale image, and perform the following operations on each pixel of the second grayscale image:

[0017] Subtract avg_gray2 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, to obtain an updated second grayscale image;

[0018] The pixel points at the same position of the updated first grayscale image and the updated second grayscale image are averaged, and the pixel points at all positions and their associated average values ​​together constitute the image to be analyzed.

[0019] Further, specifically, edge detection is performed on the image to be analyzed, and interference edges are eliminated to obtain material edges, including:

[0020] Performing edge detection on the image to be analyzed to obtain an edge image;

[0021] Execute the connected domain segmentation algorithm on the edge image to obtain all connected regions in the edge image as suspected material edge regions;

[0022] The suspected material edge area is subjected to double dynamic small area elimination to obtain the material edge.

[0023] Further, performing double dynamic small area elimination on the suspected material edge area to obtain the material edge includes:

[0024] Calculate the area values ​​of all suspected material edge regions, and then calculate the first average area values ​​of all suspected material edge regions;

[0025] Eliminate all suspected material edge regions whose area values ​​are smaller than the first average area value to obtain an optimized material edge region;

[0026] Then calculate the second average area value of all edge regions in the edge region of the optimized material;

[0027] At this time, the updated average area value is obtained by subtracting the balance coefficient from the second average area value, and the balance coefficient is a positive number set manually;

[0028] The area in the optimized material edge area that is smaller than the updated average area value is removed to obtain the material edge.

[0029] Further, specifically, extracting 3D information based on the marked first image and the marked second image includes:

[0030] Image segmentation and feature extraction are performed on the marked first image and the marked second image respectively, and then 3D information is obtained through stereo matching, depth map generation and 3D object reconstruction.

[0031] Further, specifically, edge detection is performed on the image to be analyzed by an edge detection algorithm based on a Canny operator to obtain an edge image.

[0032] Furthermore, path planning is performed based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain a planned path, including:

[0033] First, when only the robot arm coordinates and material position coordinates are considered, the preliminary path is planned by the Dijkstra algorithm;

[0034] Then add all obstacle position coordinates to the preliminary path;

[0035] When the preliminary path passes through the coordinates of any obstacle, it is avoided through the obstacle avoidance algorithm and finally the planned path is obtained.

[0036] Further, specifically, when the preliminary path passes through the coordinates of any obstacle position, the obstacle avoidance algorithm is used to avoid the obstacle, including:

[0037] Project the robot arm coordinates and all obstacle position coordinates to the xy plane, i.e. z=0. The projection points are recorded as the robot arm projection point and the obstacle projection point i, where i represents any obstacle number.

[0038] Simulate the movement of the robotic arm along the preliminary path and calculate the actual distance Li between the robotic arm projection point h and any obstacle projection point i in real time;

[0039] If Li is less than the safety threshold, find the nearest obstacle projection point in the current moving direction of the robot arm except the obstacle projection point i and record it as j;

[0040] Connect h and i to get straight line hi, connect h and j to get straight line hj, then there is a first angle between hi and hj, and the first angle is compensated to get a second angle, the second angle = first angle + α, α is a preset small angle;

[0041] Then the robot arm is controlled to rotate from hi to hj by a second angle until Li is not less than the safety threshold;

[0042] Repeat the above method until the robot reaches the material position, and the updated path is the planned path.

[0043] The present invention also proposes a robot adaptive material grabbing device based on machine vision, which is characterized by comprising the following:

[0044] An image acquisition module, used for acquiring a first image and a second image of a target material based on a binocular camera;

[0045] An image fusion module, used for fusing the first image and the second image to obtain an image to be analyzed;

[0046] An edge detection module, used for performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0047] A 3D information extraction module, used to extract 3D information based on the marked first image and the marked second image, and determine the material position coordinates and all obstacle position coordinates based on the 3D information;

[0048] The path planning module is used to read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path;

[0049] The material grabbing module is used to control the robotic arm to grab materials based on the planned path.

[0050] The beneficial effects of the present invention are:

[0051] The present invention proposes a robot adaptive material grasping method and device based on machine vision. First, after the robot acquires images through a binocular camera, image fusion is performed to obtain an image to be analyzed to remove the interference problem caused by the binocular image difference. Then, edge detection is performed on the image to be analyzed and interference edges are eliminated to obtain accurate material edges to facilitate subsequent 3D information extraction. Then, when performing path planning, the optimal path is obtained based on the actual obstacles in the 3D information, which can not only meet the robot's adaptive material grasping, but also improve the success rate and efficiency of the robot arm's material grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0053] Figure 1 Shown is a flow chart of a robot adaptive material grasping method based on machine vision of the present invention;

[0054] Figure 2 The figure shows a flow chart of fusing the first image and the second image to obtain the image to be analyzed;

[0055] Figure 3 The figure shows a flow chart of performing edge detection on the image to be analyzed and eliminating interference edges to obtain the edge of the material;

[0056] Figure 4 The flowchart shown is a process of performing double dynamic small area elimination on the suspected material edge area to obtain the material edge. DETAILED DESCRIPTION

[0057] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0058] Example 1, reference Figure 1 The present invention proposes a robot adaptive material grasping method based on machine vision, comprising the following:

[0059] Step 110: acquiring a first image and a second image of a target material based on a binocular camera;

[0060] Step 120: fusing the first image and the second image to obtain an image to be analyzed;

[0061] Step 130: perform edge detection on the image to be analyzed, eliminate interference edges to obtain material edges, and mark the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0062] Step 140: extracting 3D information based on the marked first image and the marked second image, and determining the material position coordinates and all obstacle position coordinates based on the 3D information;

[0063] Step 150, read the robot arm coordinates, and perform path planning based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain a planned path;

[0064] Step 160: Control the robotic arm to grab the material based on the planned path.

[0065] In this embodiment 1, after the robot acquires the image through the binocular camera, image fusion is first performed to obtain the image to be analyzed to remove the interference problem caused by the binocular image difference, and then edge detection is performed on the image to be analyzed and the interference edge is eliminated to obtain the accurate material edge to facilitate the subsequent 3D information extraction; then, when performing path planning, the optimal path is obtained based on the actual obstacles in the 3D information, which can not only meet the robot's adaptive material grasping, but also improve the success rate and efficiency of the robot's material grasping.

[0066] Reference Figure 2 As a preferred embodiment of the present invention, specifically, the first image and the second image are fused to obtain an image to be analyzed, including

[0067] Step 210: Perform a grayscale operation on the first image to obtain a first grayscale image, calculate the average grayscale value of all pixels in the first grayscale image to obtain avg_gray1, traverse the first grayscale image, and perform the following operations on each pixel of the first grayscale image:

[0068] Step 220, subtract avg_gray1 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, to obtain an updated first grayscale image;

[0069] Step 230: perform a grayscale operation on the second image to obtain a second grayscale image, calculate the average grayscale value of all pixels in the second grayscale image to obtain avg_gray2, traverse the second grayscale image, and perform the following operations on each pixel of the second grayscale image:

[0070] Step 240, subtract avg_gray2 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, to obtain an updated second grayscale image;

[0071] Step 250: averaging the pixels at the same position of the updated first grayscale image and the updated second grayscale image. The pixels at all positions and their associated average values ​​together constitute the image to be analyzed.

[0072] In this preferred embodiment, on the one hand, the two 2D images acquired by the binocular camera are respectively subjected to grayscale adjustment to preliminarily remove possible dust or other factors that may affect the results; then, taking into account possible differences in the binocular cameras, the differences are balanced by averaging, so that the edges of the images to be analyzed are easier to find.

[0073] Reference Figure 3 As a preferred embodiment of the present invention, specifically, edge detection is performed on the image to be analyzed, and interference edges are eliminated to obtain material edges, including:

[0074] Step 310: Perform edge detection on the image to be analyzed to obtain an edge image;

[0075] Step 320, executing a connected domain segmentation algorithm on the edge image to obtain all connected regions in the edge image as suspected material edge regions;

[0076] Step 330: Perform double dynamic small area elimination on the suspected material edge area to obtain the material edge.

[0077] Reference Figure 4 As a preferred embodiment of the present invention, the suspected material edge area is subjected to double dynamic small area elimination to obtain the material edge, including:

[0078] Step 410, calculating the area values ​​of all suspected material edge regions, and then calculating the first average area values ​​of all suspected material edge regions;

[0079] Step 420, eliminating all suspected material edge regions whose area values ​​are smaller than the first average area value to obtain an optimized material edge region;

[0080] Step 430, recalculate the second average area value of all edge regions in the optimized material edge region;

[0081] Step 440: at this time, the second average area value is subtracted from the balance coefficient to obtain an updated average area value, wherein the balance coefficient is a positive number set manually;

[0082] Step 450: Remove the area smaller than the updated average area value in the optimized material edge area to obtain the material edge.

[0083] In this preferred embodiment, considering the possible interference edge problem, the interference edge is proposed as much as possible to obtain the material edge through a double dynamic too small area elimination algorithm.

[0084] As a preferred embodiment of the present invention, specifically, extracting 3D information based on the marked first image and the marked second image includes:

[0085] Image segmentation and feature extraction are performed on the marked first image and the marked second image respectively, and then 3D information is obtained through stereo matching, depth map generation and 3D object reconstruction.

[0086] In this preferred embodiment, since the edges of the material to be grasped are marked by marking the first image and marking the second image, 3D information is obtained through mature stereo matching technology, depth map generation and 3D object reconstruction.

[0087] As a preferred implementation manner of the present invention, specifically, edge detection is performed on the image to be analyzed by an edge detection algorithm based on a Canny operator to obtain an edge image.

[0088] In this preferred embodiment, considering that the Canny operator is a commonly used edge detection algorithm in image processing, it has several obvious advantages.

[0089] Multi-stage processing: The Canny operator uses a multi-stage processing method, including:

[0090] Noise Suppression: First, apply a Gaussian filter to smooth the image and reduce the effect of noise. This helps avoid noise being misdetected as edges.

[0091] Gradient calculation: Calculate the gradient and gradient direction for each pixel to determine where the brightness changes most rapidly in the image, which are usually edges.

[0092] Non-maximum suppression: By suppressing the non-maximum of the edge, we ensure that only the points with the local maximum gradient are retained as edges, thus reducing false edges.

[0093] Dual threshold detection and edge connection: The Canny operator uses two thresholds for edge connection, namely a high threshold and a low threshold, which can effectively retain strong edges while connecting weak edges.

[0094] Edge detection accuracy: The Canny operator can detect the exact position of the edge by accurately determining the gradient direction and size. This means that it can find very detailed edge information in the image.

[0095] Noise robustness: The Canny algorithm uses a Gaussian filter for smoothing, which can effectively suppress the noise in the image, thereby reducing the impact of noise on edge detection results, so that it can achieve better edge detection results even in images with high noise.

[0096] Edge continuity: Through double threshold processing, the Canny operator can effectively maintain edge continuity. In the image, strong edge points are clearly marked as edges, while weak edges are connected into continuous edges, avoiding broken edge lines.

[0097] Automatic threshold setting: Part of the Canny operator is the process of automatically setting the threshold (such as double threshold detection), which reduces the trouble of manually setting parameters and improves the applicability and versatility of the algorithm.

[0098] Therefore, the Canny operator is used for edge detection to improve accuracy.

[0099] As a preferred embodiment of the present invention, path planning is performed based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain a planned path, including:

[0100] First, when only the robot arm coordinates and material position coordinates are considered, the preliminary path is planned by the Dijkstra algorithm;

[0101] Then add all obstacle position coordinates to the preliminary path;

[0102] When the preliminary path passes through the coordinates of any obstacle, it is avoided through the obstacle avoidance algorithm and finally the planned path is obtained.

[0103] As a preferred embodiment of the present invention, specifically, when the preliminary path passes through the coordinates of any obstacle position, the obstacle avoidance algorithm is used to avoid the obstacle, including:

[0104] Project the robot arm coordinates and all obstacle position coordinates to the xy plane, i.e. z=0. The projection points are recorded as the robot arm projection point and the obstacle projection point i, where i represents any obstacle number.

[0105] Simulate the movement of the robotic arm along the preliminary path and calculate the actual distance Li between the robotic arm projection point h and any obstacle projection point i in real time;

[0106] If Li is less than the safety threshold, find the nearest obstacle projection point in the current moving direction of the robot arm except the obstacle projection point i and record it as j;

[0107] Connect h and i to get straight line hi, connect h and j to get straight line hj, then there is a first angle between hi and hj, and the first angle is compensated to get a second angle, the second angle = first angle + α, α is a preset small angle;

[0108] Then the robot arm is controlled to rotate from hi to hj by a second angle until Li is not less than the safety threshold;

[0109] Repeat the above method until the robot reaches the material position, and the updated path is the planned path.

[0110] The robot arm coordinates, material position coordinates, and all obstacle position coordinates are their equivalent coordinates (generally the center of mass or end coordinates).

[0111] In this preferred embodiment, considering that the existing algorithms often fail to comprehensively consider all obstacles when planning the path, through the above method, after the mature path planning algorithm obtains the preliminary path, the preliminary path is fine-tuned for all possible obstacles in the preliminary path and then the path is updated. The planned path thus obtained is optimized for the possible paths of the optimal path, and can reach the material to be grabbed as quickly as possible while avoiding obstacles as much as possible.

[0112] The present invention also proposes a robot adaptive material grabbing device based on machine vision, which is characterized by comprising the following:

[0113] An image acquisition module, used for acquiring a first image and a second image of a target material based on a binocular camera;

[0114] An image fusion module, used for fusing the first image and the second image to obtain an image to be analyzed;

[0115] An edge detection module, used for performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image;

[0116] A 3D information extraction module, used to extract 3D information based on the marked first image and the marked second image, and determine the material position coordinates and all obstacle position coordinates based on the 3D information;

[0117] The path planning module is used to read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path;

[0118] The material grabbing module is used to control the robotic arm to grab materials based on the planned path.

[0119] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.

[0120] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0121] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.

[0122] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A robot adaptive material grasping method based on machine vision, characterized in that: Includes the following: Acquire a first image and a second image of a target material based on a binocular camera; Performing fusion processing on the first image and the second image to obtain an image to be analyzed; Performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image; Extracting 3D information based on the marked first image and the marked second image, and determining the material position coordinates and all obstacle position coordinates based on the 3D information; Read the robot arm coordinates, and perform path planning based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain the planned path; Control the robotic arm to grab materials based on the planned path.

2. The method for robot adaptive material grasping based on machine vision according to claim 1, characterized in that: Specifically, the first image and the second image are fused to obtain an image to be analyzed, including: Perform a grayscale operation on the first image to obtain a first grayscale image, calculate the average grayscale value of all pixels in the first grayscale image to obtain avg_gray1, traverse the first grayscale image, and perform the following operations on each pixel of the first grayscale image: Subtract avg_gray1 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, to obtain an updated first grayscale image; Perform grayscale operation on the second image to obtain a second grayscale image, calculate the average grayscale value of all pixels in the second grayscale image to obtain avg_gray2, traverse the second grayscale image, and perform the following operations on each pixel of the second grayscale image: Subtract avg_gray2 from the grayscale value of the pixel to obtain an updated grayscale value, and adjust the grayscale value of the updated grayscale value that is less than 0 to 0, to obtain an updated second grayscale image; The pixel points at the same position of the updated first grayscale image and the updated second grayscale image are averaged, and the pixel points at all positions and their associated average values ​​together constitute the image to be analyzed.

3. The method for robot adaptive material grasping based on machine vision according to claim 1, characterized in that: Specifically, edge detection is performed on the image to be analyzed, and interference edges are eliminated to obtain material edges, including: Performing edge detection on the image to be analyzed to obtain an edge image; Execute the connected domain segmentation algorithm on the edge image to obtain all connected regions in the edge image as suspected material edge regions; The suspected material edge area is subjected to double dynamic small area elimination to obtain the material edge.

4. The method for robot adaptive material grasping based on machine vision according to claim 3 is characterized in that: Performing double dynamic small area elimination on the suspected material edge area to obtain the material edge, including: Calculate the area values ​​of all suspected material edge regions, and then calculate the first average area values ​​of all suspected material edge regions; Eliminate all suspected material edge regions whose area values ​​are smaller than the first average area value to obtain an optimized material edge region; Then calculate the second average area value of all edge regions in the edge region of the optimized material; At this time, the updated average area value is obtained by subtracting the balance coefficient from the second average area value, and the balance coefficient is a positive number set manually; The area in the optimized material edge area that is smaller than the updated average area value is removed to obtain the material edge.

5. The method for robot adaptive material grasping based on machine vision according to claim 1, characterized in that: Specifically, extracting 3D information based on the marked first image and the marked second image includes: Image segmentation and feature extraction are performed on the marked first image and the marked second image respectively, and then 3D information is obtained through stereo matching, depth map generation and 3D object reconstruction.

6. The method for robot adaptive material grasping based on machine vision according to claim 3 is characterized in that: Specifically, edge detection is performed on the image to be analyzed by an edge detection algorithm based on a Canny operator to obtain an edge image.

7. The method for robot adaptive material grasping based on machine vision according to claim 1, characterized in that: The path planning is performed based on the robot arm coordinates, material position coordinates, and all obstacle position coordinates to obtain the planned path, including: First, when only the robot arm coordinates and material position coordinates are considered, the preliminary path is planned by the Dijkstra algorithm; Then add all obstacle position coordinates to the preliminary path; When the preliminary path passes through the coordinates of any obstacle, it is avoided through the obstacle avoidance algorithm and finally the planned path is obtained.

8. The method for robot adaptive material grasping based on machine vision according to claim 7, characterized in that: Specifically, when the preliminary path passes through the coordinates of any obstacle, the obstacle avoidance algorithm is used to avoid it, including: Project the robot arm coordinates and all obstacle position coordinates to the xy plane, i.e. z=0. The projection points are recorded as the robot arm projection point and the obstacle projection point i, where i represents any obstacle number. Simulate the movement of the robotic arm along the preliminary path and calculate the actual distance Li between the robotic arm projection point h and any obstacle projection point i in real time; If Li is less than the safety threshold, find the nearest obstacle projection point in the current moving direction of the robot arm except the obstacle projection point i and record it as j; Connect h and i to get straight line hi, connect h and j to get straight line hj, then there is a first angle between hi and hj, and the first angle is compensated to get a second angle, the second angle = first angle + α, α is a preset small angle; Then the robot arm is controlled to rotate from hi to hj by a second angle until Li is not less than the safety threshold; Repeat the above method until the robot reaches the material position, and the updated path is the planned path.

9. A robot adaptive material grabbing device based on machine vision, characterized in that: Includes the following: An image acquisition module, used for acquiring a first image and a second image of a target material based on a binocular camera; An image fusion module, used for fusing the first image and the second image to obtain an image to be analyzed; An edge detection module, used for performing edge detection on the image to be analyzed, eliminating interference edges to obtain material edges, and marking the material edges in the first image and the second image to obtain a marked first image and a marked second image; A 3D information extraction module, used to extract 3D information based on the marked first image and the marked second image, and determine the material position coordinates and all obstacle position coordinates based on the 3D information; The path planning module is used to read the coordinates of the robot arm and perform path planning based on the coordinates of the robot arm, the material position coordinates, and the position coordinates of all obstacles to obtain a planned path; The material grabbing module is used to control the robotic arm to grab materials based on the planned path.

Images