Automatic material grabbing robot control system based on machine vision

Abstract

The invention discloses an automatic material grabbing robot control system based on machine vision, and relates to the technical field of machine control. When the system runs, posture image information, position image information and shape image information of materials are obtained through a material collecting module to form a material image information set; after analysis, extraction and processing, a first data set, a second data set and a third data set are obtained, a material grabbing track model is established through a material processing module, training and recording are carried out, and a material grabbing adjustment index Tzzs, a material grabbing adjustment index Tzzs, a material grabbing adjustment index Tzzs and a material grabbing adjustment index Tzzs are obtained; and finally, a preset material grabbing track adjustment threshold value T is compared with a material grabbing adjustment index Tzzs through an evaluation module, a material grabbing track adjustment strategy scheme is obtained, the optimal material grabbing track adjustment strategy scheme is automatically determined, robot grabbing self-adaptive adjustment is achieved, the grabbing success rate is increased, and the working efficiency is improved. The purposes of improving the production efficiency and reducing the production cost are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine control, and specifically to a control system for a material automatic grasping robot based on machine vision. Background Art

[0002] With the continuous improvement of industrial automation, the traditional material grasping method shows some deficiencies when facing an increasingly complex production environment. In many industrial fields, especially in production line assembly and material handling, there are problems with unstable grasping postures in manual operations. Due to limitations in human vision and reaction speed, manual operations are easily affected by the production environment, resulting in unstable grasping processes, high grasping failure rates, or situations where materials fall during the movement after grasping.

[0003] Among them, the traditional material automatic grasping system has some significant deficiencies, including unstable grasping postures: traditional automatic grasping systems often lack the ability to accurately perceive and adjust the posture of materials, resulting in unstable grasping postures and easily causing material damage or dropping; poor adaptability: traditional automatic grasping systems are usually designed for specific shapes or sizes of materials and are difficult to adapt to the diversity and rapid changes in the form of materials on the production line. This inability to quickly adapt to the position or posture of materials will further reduce production efficiency and increase production costs. Summary of the Invention

[0004] (1) Technical Problems to be Solved

[0005] In view of the deficiencies of the prior art, the present invention provides a control system for a material automatic grasping robot based on machine vision, which solves the problems mentioned in the background art.

[0006] (2) Technical Solutions

[0007] To achieve the above objectives, the present invention is realized through the following technical solutions: A control system for a material automatic grasping robot based on machine vision, including a material acquisition module, an attitude analysis module, a position analysis module, a shape analysis module, a material processing module, and an evaluation module;

[0008] The material acquisition module obtains the attitude image information, position image information, and shape image information of the material through the set image sensor, and forms a material image information group;

[0009] The attitude analysis module extracts the attitude image information from the material image information group, uses deep learning technology for analysis, and forms a first data set;

[0010] The position parsing module parses the material image information group, obtains the position image information of the material, uses an image processing algorithm to obtain the three-dimensional information of the material position, establishes a three-dimensional model of the material and conducts geometric analysis, and forms a second data set;

[0011] The shape parsing module extracts the material image information group, obtains the shape image information of the material, uses an image edge detection algorithm and conducts shape feature marking, and forms a third data set;

[0012] The material processing module preprocesses the first data set, the second data set and the third data set, establishes a material grasping trajectory model, conducts training and recording, and obtains: the material grasping adjustment index Tzzs;

[0013] The evaluation module compares the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain a material grasping trajectory adjustment strategy plan.

[0014] Preferably, the material acquisition module includes a positioning unit and a transmission unit;

[0015] The positioning unit obtains the image information of several positions of the material through several infrared sensors arranged in the material to-be-grasped area, locates and identifies the position information of the material through the positioning unit, and marks the relative positions of the image information and the material, so as to obtain the attitude image information of the to-be-grasped material, the position image information of the material and the shape image information of the material;

[0016] The transmission unit integrates the attitude image information of the material, the position image information of the material and the shape image information of the material, forms a material image information group, sends it to the attitude parsing module, the position parsing module and the shape parsing module, synchronously marks the total weight value of the to-be-grasped material as: the material weight value Wlzl, and transmits it to the material processing module.

[0017] Preferably, the attitude parsing module includes an attitude extraction unit;

[0018] The attitude extraction unit extracts the obtained material image information group using deep learning technology, obtains the attitude image information of the material, and uses convolutional neural network technology for parsing and processing to form a first data set;

[0019] Among them, the first data set includes: the rotation angle value Xzjz, the material feature point value Tzdz, the material pitch angle value Fyjz, the material yaw angle value Phjz and the material roll angle value Gzjz.

[0020] Preferably, the position parsing module includes a material three-dimensional unit;

[0021] The three-dimensional unit of the material analyzes the obtained material image information group using an image processing algorithm, obtains the position image information of the material, and uses stereo vision and structured light technology to establish a three-dimensional model of the material, and then performs geometric analysis to form a second data set;

[0022] Among them, the second data set includes: X-axis coordinate value Xz, Y-axis coordinate value Yz, Z-axis coordinate value Zz, material volume value Tjz, and material geometric center value Jhzx.

[0023] Preferably, the shape analysis module includes a shape recognition unit;

[0024] The shape recognition unit extracts the obtained material image information group using an image processing algorithm, obtains the shape image information of the material, and uses an edge detection algorithm to extract the shape features of the material to form a third data set;

[0025] Among them, the third data set includes: the number of edge fold angles Zjsl, the material contour curve rate Qxlz, the ratio of concave and convex hull areas of the material Atbl, the maximum width value Kmax of the material, the minimum width value Kmin of the material, the maximum length value Cmax of the material, and the minimum length value Cmin of the material.

[0026] Preferably, the material processing module includes a processing unit and a trajectory establishment unit;

[0027] The processing unit preprocesses the first data set, the second data set, and the third data set, including denoising, outlier processing, and data format conversion, and then performs dimensionless processing to make them under the same dimension;

[0028] The trajectory establishment unit establishes a material grasping trajectory model for the processed first data set, second data set, and third data set, conducts grasping trajectory training, and obtains: the material attitude coefficient Ztxs, the material position coefficient Wzxs, and the material shape evaluation coefficient Xzxs, and then fits the recorded material attitude coefficient Ztxs, material position coefficient Wzxs, and material shape evaluation coefficient Xzxs to obtain: the material grasping adjustment index Tzzs;

[0029] The material grasping adjustment index Tzzs is obtained through the following calculation formula:

[0030]

[0031] In the formula, Ztxs represents the material attitude coefficient, Wzxs represents the material position coefficient, Xzxs represents the material shape evaluation coefficient, Wlzl represents the material weight value, and t1, t2, t3, and t4 respectively represent the proportionality coefficients of the material attitude coefficient Ztxs, the material position coefficient Wzxs, the material shape evaluation coefficient Xzxs, and the material weight value Wlzl, and C represents the first correction constant;

[0032] Among them, 0.23 ≤ t1 ≤ 0.31, 0.21 ≤ t2 ≤ 0.29, 0.18 ≤ t3 ≤ 0.24, 0.11 ≤ t4 ≤ 0.16, and t1 + t2 + t3 + t4 ≤ 1.0.

[0033] Preferably, the material attitude coefficient Ztxs is obtained through the following calculation formula:

[0034]

[0035] In the formula, Xzjz represents the rotation angle value, Tzdz represents the material feature point value, Fyjz represents the material pitch angle value, Phjz represents the material yaw angle value, Gzjz represents the material roll angle value, and z1, z2, z3, z4, and z5 respectively represent the proportionality coefficients of the rotation angle value Xzjz, the material feature point value Tzdz, the material pitch angle value Fyjz, the material yaw angle value Phjz, and the material roll angle value Gzjz, and E represents the second correction constant;

[0036] Among them, 0.11 ≤ z1 ≤ 0.19, 0.11 ≤ z2 ≤ 0.18, 0.12 ≤ z3 ≤ 0.21, 0.12 ≤ z4 ≤ 0.21, 0.12 ≤ z5 ≤ 0.21, and z1 + z2 + z3 + z4 + z5 ≤ 1.0.

[0037] Preferably, the material position coefficient Wzxs is obtained through the following calculation formula:

[0038]

[0039] In the formula, Xz represents the X-axis coordinate value, Yz represents the Y-axis coordinate value, Zz represents the Z-axis coordinate value, Tjz represents the material volume value, Jhzx represents the material geometric center value, and w1, w2, w3, w4, and w5 respectively represent the proportionality coefficients of the X-axis coordinate value Xz, the Y-axis coordinate value Yz, the Z-axis coordinate value Zz, the material volume value Tjz, and the material geometric center value Jhzx, and F represents the third correction constant;

[0040] Among them, 0.11 ≤ w1 ≤ 0.19, 0.12 ≤ w2 ≤ 0.19, 0.11 ≤ w3 ≤ 0.19, 0.13 ≤ w4 ≤ 0.18, 0.15 ≤ w5 ≤ 0.25, and w1 + w2 + w3 + w4 + w5 ≤ 1.0.

[0041] Preferably, the material shape evaluation coefficient Xzxs is obtained through the following calculation formula:

[0042]

[0043] In the formula, Zjsl represents the value of the number of edge folding angles, Qxlz represents the value of the material contour curve rate, Atbl represents the proportion value of the concave-convex area of the material, Kmax represents the maximum width value of the material, Kmin represents the minimum width value of the material, Cmax represents the maximum length value of the material, Cmin represents the minimum length value of the material, x1, x2, and x3 respectively represent the proportional coefficients of the value of the number of edge folding angles Zjsl, the value of the material contour curve rate Qxlz, and the proportion value of the concave-convex area of the material Atbl, x4 represents the proportional coefficient of the proportion values of the maximum length value Cmax, the minimum length value Cmin, the maximum width value Kmax, and the minimum width value Kmin of the material, and H represents the fourth correction constant;

[0044] Among them, 0.12 ≤ x1 ≤ 0.23, 0.13 ≤ x2 ≤ 0.24, 0.15 ≤ x3 ≤ 0.26, 0.14 ≤ x4 ≤ 0.27, and x1 + x2 + x3 + x4 ≤ 1.0.

[0045] Preferably, the evaluation module includes a storage unit and a matching unit;

[0046] The storage unit is used to store the material grasping trajectory adjustment threshold T, the material grasping trajectory adjustment strategy plan, and the notification content for relevant personnel;

[0047] The matching unit is used to match the preset relevant information with the required comparison value, including comparing the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain the material grasping trajectory adjustment strategy plan:

[0048] When the material grasping adjustment index Tzzs < the material grasping trajectory adjustment threshold T, the result of no adjustment of the material grasping trajectory is obtained;

[0049] When the material grasping adjustment index Tzzs ≥ the material grasping trajectory adjustment threshold T, the result of adjusting the material grasping trajectory is obtained. The adjustment includes: adjusting the position, posture, and grasping point of the material grasping robot. When the material grasping adjustment index Tzzs ≥ twice the material grasping trajectory adjustment threshold T, the result of warning of the material grasping trajectory is obtained, and relevant staff are notified to inspect and detect foreign objects for the material to be grasped, and further adjust the grasping strategy of the robot or manually intervene in the operation.

[0050] (III) Beneficial effects

[0051] The present invention provides a control system for an automatic material grasping robot based on machine vision, which has the following beneficial effects:

[0052] (1) When the system is running, the attitude image information, position image information, and shape image information of the material are obtained through the material acquisition module to form a group of material image information. After parsing, extraction, and processing, the first data set, the second data set, and the third data set are obtained. The material grasping trajectory model is established through the material processing module, trained and recorded to obtain the material grasping adjustment index Tzzs. Finally, the evaluation module compares the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain the material grasping trajectory adjustment strategy plan, and automatically determines the optimal material grasping trajectory adjustment strategy plan. This enables the system to have the ability of self-optimization and intelligent adjustment, further improving the grasping success rate and the adaptability of the production line, and achieving the purpose of improving production efficiency and reducing production costs.

[0053] (2) Through the attitude image information, position image information, and shape image information of the material, it can provide accurate basic parameters for dynamically adjusting the grasping trajectory, enabling the robot to accurately grasp the material, comprehensively reflecting the state of the material to be grasped, and providing support for improving the grasping success rate, reducing the number of grasping failures and retries, and thus improving production efficiency.

[0054] (3) By comparing the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain the material grasping trajectory adjustment strategy plan, it can effectively realize the real-time monitoring and adjustment of the grasping process, making the operation of the production line more stable, reducing the production line downtime and production delays caused by grasping failures or material drops, and improving production efficiency and stability. Brief Description of the Drawings

[0055] Figure 1 It is a schematic diagram of the block diagram process of a control system for an automatic material grasping robot based on machine vision according to the present invention. Detailed Embodiments

[0056] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0057] With the continuous improvement of industrial automation, the traditional material grasping method shows some deficiencies when facing the increasingly complex production environment. In many industrial fields, especially in the fields of production line assembly and material handling, there are problems with unstable grasping postures in manual operations. Due to limitations in human vision and reaction speed, manual operations are easily affected by the production environment, resulting in unstable grasping processes, high grasping failure rates, or situations where materials fall during the movement after grasping.

[0058] Among them, the traditional automatic material grasping system has some significant deficiencies, including unstable grasping postures: The traditional automatic grasping system often lacks the ability to accurately perceive and adjust the posture of the material, resulting in unstable grasping postures, which easily cause material damage or dropping. Poor adaptability: The traditional automatic grasping system is usually designed for materials with specific shapes or sizes and is difficult to adapt to the diversity and rapid changes of material forms on the production line. This inability to quickly adapt to the position or posture of the material will further reduce production efficiency and increase production costs.

[0059] Embodiment 1

[0060] The present invention provides a control system for an automatic material grasping robot based on machine vision. Please refer to Figure 1 : It includes a material acquisition module, a posture analysis module, a position analysis module, a shape analysis module, a material processing module, and an evaluation module;

[0061] The material acquisition module obtains the posture image information, position image information, and shape image information of the material through the set image sensor, and forms a group of material image information;

[0062] The posture analysis module extracts the group of material image information, obtains the posture image information of the material, uses deep learning technology for analysis, and forms a first data set;

[0063] The position analysis module analyzes the group of material image information, obtains the position image information of the material, uses image processing algorithms, obtains the three-dimensional information of the material position, establishes a three-dimensional model of the material and conducts geometric analysis, and forms a second data set;

[0064] The shape analysis module extracts the group of material image information, obtains the shape image information of the material, uses image edge detection algorithms and conducts shape feature marking, and forms a third data set;

[0065] The material processing module preprocesses the first data set, the second data set, and the third data set, establishes a material grasping trajectory model, conducts training and recording, and obtains: the material grasping adjustment index Tzzs;

[0066] The evaluation module compares the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain a material grasping trajectory adjustment strategy plan.

[0067] In this embodiment, the material acquisition module obtains the attitude image information, position image information, and shape image information of the material to form a material image information group. After parsing, extracting, and processing, the first data set, the second data set, and the third data set are obtained. The material processing module establishes a material grasping trajectory model, trains and records it to obtain the material grasping adjustment index Tzzs. Finally, the evaluation module compares the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain a material grasping trajectory adjustment strategy plan, automatically determining the optimal material grasping trajectory adjustment strategy plan. This enables the system to have self-optimization and intelligent adjustment capabilities, further improving the grasping success rate and the adaptability of the production line, achieving the purpose of improving production efficiency and reducing production costs.

[0068] Embodiment 2

[0069] This embodiment is an explanatory description based on Embodiment 1. Please refer to Figure 1 , specifically: The material acquisition module includes a positioning unit and a transmission unit;

[0070] The positioning unit obtains the image information of several positions of the material through several infrared sensors set in the material to-be-grasped area, locates and identifies the position information of the material through the positioning unit, and marks the relative positions of the image information and the material to obtain the attitude image information, position image information, and shape image information of the to-be-grasped material.

[0071] The transmission unit integrates the attitude image information, position image information, and shape image information of the material to form a material image information group, sends it to the attitude parsing module, the position parsing module, and the shape parsing module, synchronously marks the total weight value of the to-be-grasped material as: material weight value Wlzl, and transmits it to the material processing module.

[0072] Embodiment 3

[0073] This embodiment is an explanatory description based on Embodiment 1. Please refer to Figure 1 , specifically: The attitude parsing module includes an attitude extraction unit;

[0074] The attitude extraction unit uses deep learning technology to extract the obtained material image information group to obtain the attitude image information of the material, and uses convolutional neural network technology for parsing and processing to form the first data set;

[0075] Among them, the first data set includes: rotation angle value Xzjz, material feature point value Tzdz, material pitch angle value Fyjz, material yaw angle value Phjz, and material roll angle value Gzjz.

[0076] The position analysis module includes a material three-dimensional unit;

[0077] The material three-dimensional unit analyzes the acquired group of material image information using an image processing algorithm to obtain the position image information of the material, and uses stereo vision and structured light technology to establish a three-dimensional model of the material, and then performs geometric analysis to form a second data set;

[0078] Among them, the second data set includes: X-axis coordinate value Xz, Y-axis coordinate value Yz, Z-axis coordinate value Zz, material volume value Tjz, and material geometric center value Jhzx.

[0079] The shape analysis module includes a shape recognition unit;

[0080] The shape recognition unit extracts the acquired group of material image information using an image processing algorithm to obtain the shape image information of the material, and uses an edge detection algorithm to extract the shape features of the material to form a third data set;

[0081] Among them, the third data set includes: the number of edge fold angles value Zjsl, the material contour curve rate value Qxlz, the material concave-convex area ratio value Atbl, the maximum material width value Kmax, the minimum material width value Kmin, the maximum material length value Cmax, and the minimum material length value Cmin.

[0082] In this embodiment, through the attitude image information of the material, the position image information of the material, and the shape image information of the material, it is possible to provide accurate basic parameters for dynamically adjusting the grasping trajectory, enabling the robot to accurately grasp the material, comprehensively reflecting the state of the material to be grasped, and providing support for improving the grasping success rate, reducing the number of grasping failures and retries, and thus improving production efficiency.

[0083] Embodiment 4

[0084] This embodiment is an explanatory description based on Embodiment 1. Please refer to Figure 1 , specifically: The material processing module includes a processing unit and a trajectory establishment unit;

[0085] The processing unit preprocesses the first data set, the second data set, and the third data set, including denoising, outlier processing, and data format conversion, and then performs dimensionless processing to make them under the same dimension;

[0086] The trajectory establishment unit establishes a material grasping trajectory model for the processed first data set, second data set, and third data set, conducts grasping trajectory training, and obtains: the material attitude coefficient Ztxs, the material position coefficient Wzxs, and the material shape evaluation coefficient Xzxs. Then, it fits the recorded material attitude coefficient Ztxs, material position coefficient Wzxs, and material shape evaluation coefficient Xzxs to obtain: the material grasping adjustment index Tzzs;

[0087] The material grasping adjustment index Tzzs is obtained through the following calculation formula:

[0088]

[0089] In the formula, Ztxs represents the material attitude coefficient, Wzxs represents the material position coefficient, Xzxs represents the material shape evaluation coefficient, Wlzl represents the material weight value, t1, t2, t3, and t4 respectively represent the proportionality coefficients of the material attitude coefficient Ztxs, material position coefficient Wzxs, material shape evaluation coefficient Xzxs, and material weight value Wlzl, and C represents the first correction constant;

[0090] Among them, 0.23 ≤ t1 ≤ 0.31, 0.21 ≤ t2 ≤ 0.29, 0.18 ≤ t3 ≤ 0.24, 0.11 ≤ t4 ≤ 0.16, and t1 + t2 + t3 + t4 ≤ 1.0.

[0091] The material attitude coefficient Ztxs is obtained through the following calculation formula:

[0092]

[0093] In the formula, Xzjz represents the rotation angle value, Tzdz represents the material feature point value, Fyjz represents the material pitch angle value, Phjz represents the material yaw angle value, Gzjz represents the material roll angle value, z1, z2, z3, z4, and z5 respectively represent the proportionality coefficients of the rotation angle value Xzjz, material feature point value Tzdz, material pitch angle value Fyjz, material yaw angle value Phjz, and material roll angle value Gzjz, and E represents the second correction constant;

[0094] Through the calculation of the rotation angle value Xzjz, material feature point value Tzdz, material pitch angle value Fyjz, material yaw angle value Phjz, and material roll angle value Gzj, the material attitude coefficient Ztxs is obtained, which effectively reflects the attitude state of the material, that is, the direction and angle of the material in three-dimensional space. The material attitude coefficient Ztxs can describe the rotation, tilt, and flip states of the material, which helps to determine the orientation and attitude of the material;

[0095] Among them, 0.11 ≤ z1 ≤ 0.19, 0.11 ≤ z2 ≤ 0.18, 0.12 ≤ z3 ≤ 0.21, 0.12 ≤ z4 ≤ 0.21, 0.12 ≤ z5 ≤ 0.21, and z1 + z2 + z3 + z4 + z5 ≤ 1.0.

[0096] The material position coefficient Wzxs is obtained through the following calculation formula:

[0097]

[0098] In the formula, Xz represents the X-axis coordinate value, Yz represents the Y-axis coordinate value, Zz represents the Z-axis coordinate value, Tjz represents the material volume value, Jhzx represents the geometric center value of the material, w1, w2, w3, w4, and w5 respectively represent the proportionality coefficients of the X-axis coordinate value Xz, the Y-axis coordinate value Yz, the Z-axis coordinate value Zz, the material volume value Tjz, and the geometric center value Jhzx of the material, and F represents the third correction constant;

[0099] Through the calculation of the X-axis coordinate value Xz, the Y-axis coordinate value Yz, the Z-axis coordinate value Zz, the material volume value Tjz, and the geometric center value Jhzx of the material, the material position coefficient Wzxs is obtained, which effectively reflects the position and coordinates of the material in space. The material position coefficient Wzxs can describe the position of the material relative to the reference point or reference coordinate system, which helps to determine the accurate position and spatial distribution of the material;

[0100] Among them, 0.11 ≤ w1 ≤ 0.19, 0.12 ≤ w2 ≤ 0.19, 0.11 ≤ w3 ≤ 0.19, 0.13 ≤ w4 ≤ 0.18, 0.15 ≤ w5 ≤ 0.25, and w1 + w2 + w3 + w4 + w5 ≤ 1.0.

[0101] The material shape evaluation coefficient Xzxs is obtained through the following calculation formula:

[0102]

[0103] In the formula, Zjsl represents the value of the number of edge folding angles, Qxlz represents the material contour curve rate value, Atbl identifies the ratio value of the concave-convex package area of the material, Kmax represents the maximum width value of the material, Kmin represents the minimum width value of the material, Cmax represents the maximum length value of the material, Cmin represents the minimum length value of the material, x1, x2, and x3 respectively represent the proportionality coefficients of the value of the number of edge folding angles Zjsl, the material contour curve rate value Qxlz, and the ratio value of the concave-convex package area of the material Atbl, x4 represents the proportionality coefficient of the ratio values of the maximum length value Cmax, the minimum length value Cmin, the maximum width value Kmax, and the minimum width value Kmin of the material, and H represents the fourth correction constant;

[0104] By calculating the edge folding angle quantity value Zjsl, the material contour curve rate value Qxlz, the material concave-convex area ratio value Atbl, the maximum material length value Cmax, the minimum material length value Cmin, the maximum material width value Kmax, and the minimum material width value Kmin, the following is obtained: the material shape evaluation coefficient Xzxs, which can effectively reflect the shape characteristics of the material. The material shape evaluation coefficient Xzxs can describe the external shape, contour, and size-related characteristics of the material, helping to identify and classify materials of different shapes and determine the material grasping method.

[0105] Among them, 0.12 ≤ x1 ≤ 0.23, 0.13 ≤ x2 ≤ 0.24, 0.15 ≤ x3 ≤ 0.26, 0.14 ≤ x4 ≤ 0.27, and x1 + x2 + x3 + x4 ≤ 1.0.

[0106] Example 5

[0107] This example is an explanatory note based on Example 1. Please refer to Figure 1 , specifically: The evaluation module includes a storage unit and a matching unit;

[0108] The storage unit is used to store the material grasping trajectory adjustment threshold T, the material grasping trajectory adjustment strategy plan, and the relevant personnel notification content;

[0109] The matching unit is used to match the preset relevant information with the required comparison values, including comparing the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain the material grasping trajectory adjustment strategy plan:

[0110] When the material grasping adjustment index Tzzs < the material grasping trajectory adjustment threshold T, the result of no adjustment of the material grasping trajectory is obtained;

[0111] When the material grasping adjustment index Tzzs ≥ the material grasping trajectory adjustment threshold T, the result of adjusting the material grasping trajectory is obtained, and the adjustment includes: adjusting the position, posture, and grasping point of the material grasping robot;

[0112] Adjusting the position of the grasping robot: includes adjusting the X-axis coordinate value Xz, Y-axis coordinate value Yz, Z-axis coordinate value Zz of the robot and the running speed of the robot;

[0113] Adjusting the posture of the grasping robot: includes adjusting the rotation angle value Xzjz, the material pitch angle value Fyjz, the material yaw angle value Phjz, and the material roll angle value Gzjz of the robot to adjust the orientation, tilt angle, and rotation angle of the robot;

[0114] Adjust the grasping point of the grasping robot: including the robot's recognition and adjustment of the geometric center value Jhzx of the material, the area ratio value Atbl of the concave and convex packages of the material, the maximum width value Kmax of the material, the minimum width value Kmin of the material, the maximum length value Cmax of the material, and the minimum length value Cmin of the material, to adjust the grasping position of the material, making it more accurate and stable. By adjusting the grasping point, ensure that the robot accurately grasps the specified position of the material, improving the grasping success rate and the stability during the grasping process;

[0115] When the material grasping adjustment index Tzzs ≥ twice the material grasping trajectory adjustment threshold T, obtain the warning result of the material grasping trajectory, and notify the relevant staff to inspect and detect foreign objects for the material to be grasped, and further adjust the grasping strategy of the robot or perform manual intervention operations.

[0116] In this embodiment, by comparing the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs, obtain the material grasping trajectory adjustment strategy plan, which can effectively realize the real-time monitoring and adjustment of the grasping process. This will make the operation of the production line more stable, reduce the production line downtime and production delays caused by grasping failures or material drops, and improve production efficiency and stability.

[0117] Specific example: An automatic material grasping robot control system based on machine vision used in a certain manufacturing production line will use some specific parameters and values to demonstrate how to calculate: the material grasping adjustment index Tzzs, the material attitude coefficient Ztxs, the material position coefficient Wzxs, and the material shape evaluation coefficient Xzxs;

[0118] Suppose the following parameter values are available:

[0119] Material weight value Wlzl: 54;

[0120] The first data set includes: rotation angle value Xzjz: 45, material feature point value Tzdz: 50, material pitch angle value Fyjz: 15, material yaw angle value Phjz: 20, and material roll angle value Gzjz: 1;

[0121] The second data set includes: X-axis coordinate value Xz: 100, Y-axis coordinate value Yz: 80, Z-axis coordinate value Zz: 30, material volume value Tjz: 13, and material geometric center value Jhzx: 70;

[0122] The third data set includes: the number of edge fold angles value Zjsl: 8, the material contour curve rate value Qxlz: 54, the area ratio value Atbl of the concave and convex packages of the material: 56, the maximum width value Kmax of the material: 65, the minimum width value Kmin of the material: 25, the maximum length value Cmax of the material: 30, and the minimum length value Cmin of the material: 15;

[0123] Proportionality coefficients: t1: 0.28, t2: 0.26, t3: 0.21, t4: 0.15, z1: 0.18, z2: 0.16, z3: 0.19, z4: 0.18, z5: 0.17, w1: 0.18, w2: 0.17, w3: 0.18, w4: 0.16, w5: 0.24, x1: 0.22, x2: 0.23, x3: 0.25, and x4: 0.26;

[0124] Correction constants: First correction constant C: 0.81, second correction constant E: 0.28, third correction constant F: 0.12, fourth correction constant H: 0.69,

[0125] Obtained according to the calculation formula of the material attitude coefficient Ztxs:

[0126] Ztxs = [(0.18 * 45) + (0.16 * 50) + (0.19 * 15) + (0.18 * 20) + (0.17 * 1)] + 0.28 = 23;

[0127] Obtained according to the calculation formula of the material position coefficient Wzxs:

[0128] Wzxs = [(0.18 * 100) + (0.17 * 80) + (0.18 * 30)] + [(0.16 * 13) + (0.24 * 70)] + 0.12 = 56;

[0129] Obtained according to the calculation formula of the material shape evaluation coefficient Xzxs:

[0130] Xzxs = [(0.22 * 8) + (0.23 * 54) + (0.25 * 56)] + [0.26 * (65 / 25 - 30 / 15)] + 0.69 = 29;

[0131] Obtained according to the calculation formula of the material grasping adjustment index Tzzs:

[0132] Tzzs = [(0.28 * 23) + (0.26 * 56) + (0.21 * 29) + (0.15 * 54)] + 0.81 = 36;

[0133] Set the material grasping trajectory adjustment threshold T to 49 and match it with the material grasping adjustment index Tzzs to obtain: Obtain the result of no adjustment of the material grasping trajectory.

[0134] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An automatic material grasping robot control system based on machine vision, characterized in that: It includes a material acquisition module, an attitude analysis module, a position analysis module, a shape analysis module, a material processing module, and an evaluation module; The material acquisition module obtains the attitude image information, position image information, and shape image information of the material through the set image sensor, and forms a material image information group; The attitude analysis module extracts the material image information group, obtains the attitude image information of the material, uses deep learning technology for analysis, and forms a first data set; The position analysis module analyzes the material image information group, obtains the position image information of the material, uses image processing algorithms to obtain the three-dimensional information of the material position, establishes a three-dimensional model of the material and conducts geometric analysis, and forms a second data set; The shape analysis module extracts the material image information group, obtains the shape image information of the material, uses image edge detection algorithms and conducts shape feature marking, and forms a third data set; The material processing module preprocesses the first data set, the second data set, and the third data set, establishes a material grasping trajectory model, conducts training and recording, and obtains: the material grasping adjustment index Tzzs; The evaluation module compares the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain a material grasping trajectory adjustment strategy plan.

2. The control system of an automatic material grasping robot based on machine vision according to claim 1, characterized in that: The material acquisition module includes a positioning unit and a transmission unit; The positioning unit obtains the image information of several positions of the material through several infrared sensors set in the material to-be-grasped area, locates and identifies the position information of the material through the positioning unit, and marks the relative positions of the image information and the material, so as to obtain the attitude image information, position image information, and shape image information of the to-be-grasped material; The transmission unit integrates the attitude image information, position image information, and shape image information of the material, forms a material image information group, sends it to the attitude analysis module, the position analysis module, and the shape analysis module, synchronously marks the total weight value of the to-be-grasped material as: the material weight value Wlzl, and transmits it to the material processing module.

3. The control system of a material automatic grasping robot based on machine vision according to claim 1, characterized in that: The attitude analysis module includes an attitude extraction unit; The attitude extraction unit uses deep learning technology to extract the obtained material image information group, obtains the attitude image information of the material, and uses convolutional neural network technology for analysis and processing, and forms a first data set; Among them, the first data set includes: the rotation angle value Xzjz, the material feature point value Tzdz, the material pitch angle value Fyjz, the material yaw angle value Phjz, and the material roll angle value Gzjz.

4. The control system of a material automatic grasping robot based on machine vision according to claim 1, characterized in that: The position analysis module includes a material three-dimensional unit; The material three-dimensional unit analyzes the obtained material image information group using image processing algorithms, obtains the position image information of the material, and uses stereo vision and structured light technology to establish a three-dimensional model of the material, and then conducts geometric analysis, and forms a second data set; Among them, the second data set includes: the X-axis coordinate value Xz, the Y-axis coordinate value Yz, the Z-axis coordinate value Zz, the material volume value Tjz, and the material geometric center value Jhzx.

5. The control system of a material automatic grasping robot based on machine vision according to claim 1, characterized in that: The shape analysis module includes a shape recognition unit; The shape recognition unit extracts the acquired material image information group using an image processing algorithm to obtain the shape image information of the material, and uses an edge detection algorithm to extract the shape features of the material, forming a third data set; Among them, the third data set includes: the number of edge fold angles Zjsl, the material contour curve rate Qxlz, the material concave and convex hull area ratio Atbl, the maximum width value Kmax of the material, the minimum width value Kmin of the material, the maximum length value Cmax of the material, and the minimum length value Cmin of the material.

6. The control system of a material automatic grasping robot based on machine vision according to claim 1, characterized in that: The material processing module includes a processing unit and a trajectory establishment unit; The processing unit preprocesses the first data set, the second data set, and the third data set, including denoising, outlier processing, and data format conversion, and then performs dimensionless processing to make them under the same dimension; The trajectory establishment unit establishes a material grasping trajectory model for the processed first data set, second data set, and third data set, conducts grasping trajectory training, and obtains: the material attitude coefficient Ztxs, the material position coefficient Wzxs, and the material shape evaluation coefficient Xzxs, and then fits the recorded material attitude coefficient Ztxs, material position coefficient Wzxs, and material shape evaluation coefficient Xzxs to obtain: the material grasping adjustment index Tzzs; The material grasping adjustment index Tzzs is obtained through the following calculation formula: In the formula, Ztxs represents the material attitude coefficient, Wzxs represents the material position coefficient, Xzxs represents the material shape evaluation coefficient, Wlzl represents the material weight value, and t1, t2, t3, and t4 respectively represent the proportional coefficients of the material attitude coefficient Ztxs, the material position coefficient Wzxs, the material shape evaluation coefficient Xzxs, and the material weight value Wlzl, and C represents the first correction constant; Among them, 0.23 ≤ t1 ≤ 0.31, 0.21 ≤ t2 ≤ 0.29, 0.18 ≤ t3 ≤ 0.24, 0.11 ≤ t4 ≤ 0.16, and t1 + t2 + t3 + t4 ≤ 1.

0.

7. The control system of a material automatic grasping robot based on machine vision according to claim 6, characterized in that: The material attitude coefficient Ztxs is obtained through the following calculation formula: In the formula, Xzjz represents the rotation angle value, Tzdz represents the material feature point value, Fyjz represents the material pitch angle value, Phjz represents the material yaw angle value, Gzjz represents the material roll angle value, and z1, z2, z3, z4, and z5 respectively represent the proportional coefficients of the rotation angle value Xzjz, the material feature point value Tzdz, the material pitch angle value Fyjz, the material yaw angle value Phjz, and the material roll angle value Gzjz, and E represents the second correction constant; Among them, 0.11 ≤ z1 ≤ 0.19, 0.11 ≤ z2 ≤ 0.18, 0.12 ≤ z3 ≤ 0.21, 0.12 ≤ z4 ≤ 0.21, 0.12 ≤ z5 ≤ 0.21, and z1 + z2 + z3 + z4 + z5 ≤ 1.

0.

8. The control system of an automatic material grasping robot based on machine vision according to claim 6, characterized in that: The material position coefficient Wzxs is obtained through the following calculation formula: Wherein, Xz represents the X-axis coordinate value, Yz represents the Y-axis coordinate value, Zz represents the Z-axis coordinate value, Tjz represents the material volume value, Jhzx represents the geometric center value of the material, and w1, w2, w3, w4, and w5 respectively represent the proportionality coefficients of the X-axis coordinate value Xz, the Y-axis coordinate value Yz, the Z-axis coordinate value Zz, the material volume value Tjz, and the geometric center value Jhzx of the material, and F represents the third correction constant; Among them, 0.11 ≤ w1 ≤ 0.19, 0.12 ≤ w2 ≤ 0.19, 0.11 ≤ w3 ≤ 0.19, 0.13 ≤ w4 ≤ 0.18, 0.15 ≤ w5 ≤ 0.25, and w1 + w2 + w3 + w4 + w5 ≤ 1.

0.

9. The control system of a material automatic grasping robot based on machine vision according to claim 6, characterized in that: The material shape evaluation coefficient Xzxs is obtained through the following calculation formula: Wherein, Zjsl represents the number value of the edge folding angle, Qxlz represents the material contour curve rate value, Atbl identifies the ratio value of the concave-convex package area of the material, Kmax represents the maximum width value of the material, Kmin represents the minimum width value of the material, Cmax represents the maximum length value of the material, Cmin represents the minimum length value of the material, x1, x2, and x3 respectively represent the proportionality coefficients of the number value of the edge folding angle Zjsl, the material contour curve rate value Qxlz, and the ratio value of the concave-convex package area of the material Atbl, x4 represents the proportionality coefficient of the ratio values of the maximum length value Cmax, the minimum length value Cmin, the maximum width value Kmax, and the minimum width value Kmin of the material, and H represents the fourth correction constant; Among them, 0.12 ≤ x1 ≤ 0.23, 0.13 ≤ x2 ≤ 0.24, 0.15 ≤ x3 ≤ 0.26, 0.14 ≤ x4 ≤ 0.27, and x1 + x2 + x3 + x4 ≤ 1.

0.

10. The control system of a material automatic grasping robot based on machine vision according to claim 1, characterized in that: The evaluation module includes a storage unit and a matching unit; The storage unit is used to store the material grasping trajectory adjustment threshold T, the material grasping trajectory adjustment strategy plan, and the notification content for relevant personnel; The matching unit is used to match through the preset relevant information with the required comparison value, including comparing the preset material grasping trajectory adjustment threshold T with the material grasping adjustment index Tzzs to obtain the material grasping trajectory adjustment strategy plan: When the material grasping adjustment index Tzzs < the material grasping trajectory adjustment threshold T, obtain the result that the material grasping trajectory is not adjusted; When the material grasping adjustment index Tzzs ≥ the material grasping trajectory adjustment threshold T, obtain the result of adjusting the material grasping trajectory. The adjustment includes: adjusting the position, posture, and grasping point of the material grasping robot. When the material grasping adjustment index Tzzs ≥ twice the material grasping trajectory adjustment threshold T, obtain the material grasping trajectory warning result, notify the relevant staff to inspect and detect foreign objects for the material to be grasped, and further adjust the grasping strategy of the robot or perform manual intervention operations.

Images