Feeding control system and method based on mechanical arm

Abstract

The invention relates to the technical field of mechanical arms, and discloses a feeding control system and method based on a mechanical arm, and the method comprises the steps: building a first three-dimensional image for the environment of the mechanical arm, extracting an action path feature vector according to the first three-dimensional image, determining a material target position according to the action path feature vector, and obtaining a material target position; when the mechanical arm reaches a material target position, preprocessing the first three-dimensional image and establishing an image first coordinate system, calculating a feature value of the material based on the image first coordinate system, extracting an image optimization factor based on the feature value, and optimizing a space vector according to the image optimization factor to obtain a target vector of the material; and an image second coordinate system is established according to the second three-dimensional image, coordinates of a grabbing point where the material makes contact with the mechanical arm are extracted according to the image second coordinate system, the coordinates of the grabbing point are optimized according to the image coordinate optimization factors, and feeding coordinates are obtained. The grabbing mechanism has the characteristic of good grabbing adjustment.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arms, and in particular to a feeding control system and a control method based on a robotic arm. Background Art

[0002] With the advancement of science and technology, CNC computer-controlled machine tools have developed rapidly. In CNC machine tools, the traditional robotic arm relies on the preset program planning of the robotic arm to grab the material during the feeding control process. However, without the support and processing of three-dimensional images, the robotic arm cannot make appropriate adjustments, resulting in the robotic arm being unable to accurately grab the material during the feeding process, thus affecting the efficiency and success rate of grabbing.

[0003] Traditional control of the robot arm relies on the force feedback and preset programs of the robot arm. Although it can complete some basic tasks, it faces problems such as inaccurate positioning and grasping failure when dealing with feeding tasks that require high precision. At the same time, there is a lack of real-time image feedback from visual sensors to adjust the grasping strategy, and it is impossible to analyze and optimize the actual situation of the material, resulting in instability in the feeding process.

[0004] Therefore, how to provide a feeding control system and control method based on a robotic arm is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention proposes a feeding control system and control method based on a robotic arm, aiming to solve the problem of instability in the feeding process due to the lack of real-time image feedback to adjust the grasping strategy and the inability to analyze and optimize the actual situation of the material.

[0006] In one aspect, the present invention proposes a feeding control method based on a robotic arm, comprising: Establishing a first three-dimensional image of the environment of the robot arm, extracting a motion path feature vector according to the first three-dimensional image, and determining a target position of the material according to the motion path feature vector; When the robot arm reaches the target position of the material, the first three-dimensional image is preprocessed and a first coordinate system of the image is established, a characteristic value of the material is calculated based on the first coordinate system of the image, the characteristic value is similarly processed with the characteristic value in the historical set, and a material grabbing position is determined according to the similarity; Establishing a second three-dimensional image, extracting a spatial vector of a material according to the second three-dimensional image, extracting an image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor to obtain a target vector of the material, and establishing a rotation matrix of the material based on the target vector; The robotic arm grabs the material according to the material grabbing position, the target vector and the rotation matrix; When grasping a material, a second image coordinate system is established according to the second three-dimensional image, and the coordinates of the grasping point where the material and the robot arm are in contact are extracted according to the second image coordinate system; Acquire an analysis vector of the material grabbing position according to the first three-dimensional image, and obtain a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; An image coordinate optimization factor is calculated according to the first axial coefficient, the second axial coefficient and the third axial coefficient, and the grabbing point coordinates are optimized according to the image coordinate optimization factor to obtain a loading coordinate.

[0007] Furthermore, when calculating the characteristic value of the material based on the first coordinate system of the image, it includes: Extracting the three-dimensional coordinates of the target position of the material from the first coordinate system of the image, and respectively counting all x-axis coordinates, y-axis coordinates and z-axis coordinates in the three-dimensional coordinates; Counting the number of all coordinates in the three-dimensional coordinates, and calculating the x-coordinate feature quantity, y-coordinate feature quantity and z-coordinate feature quantity of the material; The first weight coefficient is obtained by taking the absolute values ​​of the differences between all the x-axis coordinates and the x-coordinate feature quantity and adding them, the second weight coefficient is obtained by taking the absolute values ​​of the differences between all the y-axis coordinates and the y-coordinate feature quantity and adding them, and the third weight coefficient is obtained by taking the absolute values ​​of the differences between all the z-axis coordinates and the z-coordinate feature quantity and adding them; The feature value is calculated based on the x-coordinate feature quantity, the y-coordinate feature quantity, the z-coordinate feature quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient.

[0008] Further, when the characteristic value is calculated according to the x-coordinate characteristic quantity, the y-coordinate characteristic quantity, the z-coordinate characteristic quantity, the first weight coefficient, the second weight coefficient and the third weight coefficient, it includes: The characteristic value is calculated according to the following formula: ; Wherein, V represents the eigenvalue, w 1 represents the first weight coefficient, w 2 represents the second weight coefficient, w 3 represents the third weight coefficient, x represents the x-coordinate feature value, y represents the y-coordinate feature value, and z represents the z-coordinate feature value.

[0009] Further, similarity processing is performed on the feature value and the feature value in the history set, and when the material grabbing position is determined according to the similarity, it includes: The number of feature values ​​in the history set that are the same as the feature value is counted, and the similarity is calculated according to the following formula: ; Wherein, S represents the similarity, V 1 Indicates the number of eigenvalues ​​in the history set that are the same as the eigenvalue, V 2 Represents the number of all eigenvalues ​​in the history set; When S is greater than or equal to 70%, the historical material grab position is used; When S is less than 70%, the material grabbing position is preset based on the first coordinate system of the image; The material grabbing position includes the historical material grabbing position and the preset material grabbing position.

[0010] Further, when extracting the spatial vector of the material according to the second three-dimensional image, extracting the image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor, and obtaining the target vector of the material, the method includes: Collecting initial point cloud data of the second three-dimensional image, and performing refinement processing on the initial point cloud data to obtain terminal point cloud data; Extracting characteristic points of the material according to the terminal point cloud data, and performing local fitting on the characteristic points to obtain the space vector; The target vector is obtained according to the following formula: ; ; Among them, C opt represents the image optimization factor, V represents the eigenvalue, K represents the space vector, and Z represents the target vector.

[0011] Furthermore, when establishing the rotation matrix of the material based on the target vector, it includes: The rotation matrix is ​​established according to the following formula: ; ; ; ; Wherein, θ represents the rotation angle, Z represents the target vector, |Z| represents the modulus of the target vector, r represents the rotation axis vector in the first coordinate system of the image, and r x represents the component of the rotation axis in the x-axis direction, ry represents the component of the rotation axis in the y-axis direction, r z represents the component of the rotation axis in the z-axis direction, [r] represents the antisymmetric matrix of the rotation axis vector r, and R represents the rotation matrix.

[0012] Furthermore, it is characterized in that, when obtaining the analysis vector of the material grabbing position according to the first three-dimensional image, and deriving the first axial coefficient, the second axial coefficient and the third axial coefficient according to the analysis vector, it includes: acquiring the analysis vector according to the first three-dimensional image; Count the first vector parameters of all x-axes of the analysis vector, count the second vector parameters of all y-axes of the analysis vector, count the third vector parameters of all z-axes of the analysis vector, calculate the first axial coefficient according to the first vector parameters, calculate the second axial coefficient according to the second vector parameters, and calculate the third axial coefficient according to the third vector parameters.

[0013] Further, when calculating the first axial coefficient according to the first vector parameter, calculating the second axial coefficient according to the second vector parameter, and calculating the third axial coefficient according to the third vector parameter, it includes: ; ; ; Wherein, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, and n 1 represents the number of the first vector parameters containing the x-axis in the analysis vector, n 2 represents the number of the second vector parameters containing the y-axis in the analysis vector, n 3 represents the number of the third vector parameters containing the z-axis in the analysis vector, P i represents any one of the first vector parameters, Q i represents any one of the second vector parameters, L i Represents any one of the third vector parameters.

[0014] Further, when the image coordinate optimization factor is calculated according to the first axial coefficient, the second axial coefficient and the third axial coefficient, the grabbing point coordinates are optimized according to the image coordinate optimization factor to obtain the loading coordinates, it includes: The image coordinate optimization factor is obtained according to the following formula: ; Wherein, M represents the image coordinate optimization factor, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, α represents the weight of the first axial coefficient, β represents the weight of the second axial coefficient, γ represents the weight of the third axial coefficient, and α+β+γ=1.5; The loading coordinates are the product of the grabbing point coordinates and the image coordinate optimization factor.

[0015] Compared with the prior art, the beneficial effect of the present invention is that by establishing the first three-dimensional image, the target position of the material in the surrounding environment of the robot arm can be accurately perceived and analyzed. The positioning error in the traditional method is avoided. After the robot arm reaches the target position of the material, the characteristic value of the material is calculated based on the first coordinate system of the image, and the characteristic value in the historical set is similarly analyzed to ensure that the robot arm can find the material grabbing position, extract the spatial vector of the material according to the second three-dimensional image, optimize the spatial vector using the image optimization factor, generate the target vector and calculate the rotation matrix of the material, ensure that the robot arm can adjust the correct posture when grabbing the material, and improve the accuracy of the grabbing operation. In the grabbing process, by establishing the second coordinate system of the image, the coordinates of the grabbing point where the material and the robot arm contact are extracted in real time, and the material loading coordinates are obtained according to the analysis vector, so as to accurately adjust the grabbing of the robot arm, ensure the material grabbing accuracy, improve production efficiency, reduce manual intervention, and realize the automated operation of the loading process.

[0016] On the other hand, the present application also provides a feeding control system based on a robotic arm, which is used to apply the above-mentioned feeding control method based on a robotic arm, including: A first camera module is used to create a first three-dimensional image of the environment of the robot arm, extract a motion path feature vector according to the first three-dimensional image, and determine a material target position according to the motion path feature vector; A first calculation module, when the robot arm reaches the target position of the material, is used to pre-process the first three-dimensional image and establish a first coordinate system of the image, calculate the characteristic value of the material based on the first coordinate system of the image, perform similarity processing on the characteristic value and the characteristic value in the historical set, and determine the material grabbing position according to the similarity; A second camera module is used to establish a second three-dimensional image, extract a spatial vector of a material according to the second three-dimensional image, extract an image optimization factor based on the eigenvalue, optimize the spatial vector according to the image optimization factor, obtain a target vector of the material, and establish a rotation matrix of the material based on the target vector; A power module, used for providing power to the robot arm when it reaches the target position of the material and when the robot arm grabs the material according to the material grabbing position, the target vector and the rotation matrix; A second calculation module, when grabbing a material, is used to establish a second image coordinate system according to the second three-dimensional image, extract the coordinates of the grabbing point where the material and the robotic arm are in contact according to the second image coordinate system, obtain an analysis vector of the material grabbing position according to the first three-dimensional image, and derive a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; The optimization module is used to calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient and the third axial coefficient, and optimize the grabbing point coordinates according to the image coordinate optimization factor to obtain the loading coordinates.

[0017] It can be understood that the above-mentioned feeding control system and control method based on a robotic arm have the same beneficial effects, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A flow chart of a feeding control method based on a robotic arm provided in an embodiment of the present invention; Figure 2 A functional block diagram of a feeding control system based on a robotic arm provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] See also Figure 1 As shown, in some embodiments of the present application, this embodiment provides a feeding control method based on a robotic arm, comprising: S100: creating a first three-dimensional image of the environment of the robot arm, extracting a motion path feature vector according to the first three-dimensional image, and determining a target position of a material according to the motion path feature vector; S200: When the robot arm reaches the target position of the material, pre-process the first three-dimensional image and establish a first coordinate system of the image, calculate the characteristic value of the material based on the first coordinate system of the image, perform similarity processing on the characteristic value and the characteristic value in the historical set, and determine the material grabbing position according to the similarity; S300: establishing a second three-dimensional image, extracting a spatial vector of a material according to the second three-dimensional image, extracting an image optimization factor based on a eigenvalue, optimizing the spatial vector according to the image optimization factor, obtaining a target vector of the material, and establishing a rotation matrix of the material based on the target vector; The robot grabs the material based on the material grabbing position, target vector and rotation matrix; S400: when grabbing a material, establishing a second image coordinate system according to the second three-dimensional image, extracting the coordinates of the grabbing point where the material and the robot arm are in contact according to the second image coordinate system, obtaining an analysis vector of the material grabbing position according to the first three-dimensional image, and obtaining a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; S500: Calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient and the third axial coefficient, optimize the grabbing point coordinates according to the image coordinate optimization factor, and obtain the loading coordinates.

[0021] Specifically, the robot arm is located on the left side of a CNC machine tool and the right side of a CNC machine tool. The robot arm contains a visual sensor, which can generate a first three-dimensional image of the environment in which the robot arm is located in real time, extract the feature points of the robot arm and the material in the first three-dimensional image, aggregate the feature points of the same type to construct a motion path feature vector, fit the motion path feature vector into a straight path, and the end point of the straight path is set as the target position of the material. When the robot arm is driven to reach the target position of the material, the first three-dimensional image is preprocessed, and the preprocessing includes removing duplicate images and exposing images. The working platform of the CNC machine tool on the left side of the first three-dimensional image is set as the first coordinate system of the image, and the same is true for the right side. The characteristic value of the material is calculated according to the first coordinate system of the image to determine the material grabbing position of the robot arm. The visual sensor generates a second three-dimensional image, collects the point cloud data of the second three-dimensional image, extracts the feature points of the material according to the point cloud data, and fits to obtain a spatial vector. The spatial vector is used to express the shape characteristics of the material, and the spatial vector is optimized by the characteristic value to obtain the target vector of the material. The target vector is expressed as the posture and shape characteristics of the material. The rotation matrix is ​​calculated based on the target vector. The rotation matrix is ​​the rotation posture data of the robot arm when grasping the material. When the robot arm contacts the surface of the material, the material in the second three-dimensional image is used as the reference, the second coordinate system of the image is set, and the coordinates of the grasping point where the material and the robot arm are in contact are extracted. The analysis vector of the material grasping position is obtained based on the first three-dimensional image, and the first axial coefficient, the second axial coefficient and the third axial coefficient are calculated. The image coordinate optimization factor is calculated based on the first axial coefficient, the second axial coefficient and the third axial coefficient. The image coordinate optimization factor optimizes the grasping point coordinates and obtains the loading coordinates. The loading coordinates are the coordinates of the material actually grasped by the robot arm. The grasping point coordinates are optimized by the image coordinate optimization factor, so that the robot arm can accurately grasp the material and complete the loading operation.

[0022] It can be understood that based on the first three-dimensional image and the second three-dimensional image obtained by the visual sensor and the calculated image coordinate optimization factor, the robotic arm can coordinate the movement state and dynamically adjust the grasping of the robotic arm according to the loading coordinates, thereby ensuring the accuracy of material grasping, improving the stability and efficiency of loading, realizing the automated operation of the loading process, and reducing the need for manual intervention.

[0023] In some embodiments of the present application, when calculating the characteristic value of a material based on the first coordinate system of the image, it includes: extracting the three-dimensional coordinates of the target position of the material from the first coordinate system of the image, respectively counting all x-axis coordinates, y-axis coordinates and z-axis coordinates in the three-dimensional coordinates, counting all the coordinate quantities in the three-dimensional coordinates, and calculating the x-coordinate characteristic quantity, y-coordinate characteristic quantity and z-coordinate characteristic quantity of the material, subtracting all x-axis coordinates from the x-coordinate characteristic quantity, taking the absolute value and adding them to obtain a first weight coefficient, subtracting all y-axis coordinates from the y-coordinate characteristic quantity, taking the absolute value and adding them to obtain a second weight coefficient, subtracting all z-axis coordinates from the z-coordinate characteristic quantity, taking the absolute value and adding them to obtain a third weight coefficient, and calculating the characteristic value according to the x-coordinate characteristic quantity, y-coordinate characteristic quantity, z-coordinate characteristic quantity, the first weight coefficient, the second weight coefficient and the third weight coefficient.

[0024] It can be understood that by extracting the three-dimensional coordinates of the target position of the material, the position of the material in space can be accurately determined, ensuring the accuracy of the material grabbing and reducing the risk of misgrasping or wrong grabbing. The x-coordinate feature quantity is the average of all x-axis coordinates. The y-coordinate feature quantity and the z-coordinate feature quantity are determined in the same way as the x-coordinate feature quantity, so they will not be repeated here. By calculating the difference between the coordinates of each axis and the feature quantity and taking the absolute value, the accumulation of errors can be effectively reduced, the influence of coordinate deviation can be reduced, the spatial distribution characteristics of the material can be refined, and data support can be provided for the adjustment of the material grabbing.

[0025] In some embodiments of the present application, when calculating the characteristic value according to the x-coordinate characteristic quantity, the y-coordinate characteristic quantity, the z-coordinate characteristic quantity, the first weight coefficient, the second weight coefficient and the third weight coefficient, it includes: The eigenvalue is calculated according to the following formula: ; Wherein, V represents the eigenvalue, w 1 represents the first weight coefficient, w 2 represents the second weight coefficient, w 3 represents the third weight coefficient, x represents the x-coordinate feature value, y represents the y-coordinate feature value, and z represents the z-coordinate feature value.

[0026] It can be understood that calculating the eigenvalue based on the x-coordinate feature quantity, y-coordinate feature quantity, z-coordinate feature quantity, first weight coefficient, second weight coefficient and third weight coefficient ensures the rigor of the eigenvalue calculation and avoids one-sided calculation results due to insufficient data.

[0027] In some embodiments of the present application, when similarity processing is performed on the feature value and the feature value in the history set, and the material grabbing position is determined according to the similarity, it includes: counting the number of feature values ​​in the history set and the feature value that are the same, and calculating the similarity according to the following formula: ; Wherein, S represents the similarity, V 1 Indicates the number of eigenvalues ​​in the history set that are the same as the eigenvalue, V 2 Represents the number of all eigenvalues ​​in the historical set; when S is greater than or equal to 70%, the historical material grabbing position is adopted; when S is less than 70%, the material grabbing position is preset based on the first coordinate system of the image, and the material grabbing position includes the historical material grabbing position and the preset material grabbing position.

[0028] It can be understood that by comparing with the feature values ​​in the historical set, the change or deviation of the material position can be identified. In the case of similarity of most feature values, the use of historical material grabbing positions ensures the accuracy and stability of grabbing, reducing errors and the adjustment time of the robot arm. When the similarity is greater than or equal to 70%, the historical material grabbing position is directly used to optimize the feeding production rhythm. When the similarity is less than 70%, a grabbing position is preset according to the first coordinate system of the image to ensure the adaptive ability of the robot arm. The use of historical material grabbing positions by the robot arm can maintain the continuity and stability of the grabbing operation, reduce the cost of operation adjustment, and reduce the material grabbing position deviation caused by errors or errors, thereby improving the stability and consistency of feeding.

[0029] In some embodiments of the present application, when extracting the spatial vector of the material according to the second three-dimensional image, extracting the image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor, and obtaining the target vector of the material, it includes: collecting the initial point cloud data of the second three-dimensional image, refining the initial point cloud data to obtain the terminal point cloud data, extracting the characteristic points of the material according to the terminal point cloud data, locally fitting the characteristic points to obtain the spatial vector, and obtaining the target vector according to the following formula: ; ; Among them, C opt represents the image optimization factor, V represents the eigenvalue, K represents the space vector, and Z represents the target vector.

[0030] It is understandable that the initial point cloud data of the second three-dimensional image is collected and refined, and the refinement includes: removing redundancy, reducing noise and enhancing features. The amount of redundant data is removed, thereby realizing the need for accurate calculation. The noise reduction process can effectively remove noise points in the point cloud data and retain feature points and material edges. Enhanced features identify and highlight feature points, which helps to highlight the structural features of the material and enhance the outline of the material. The feature points of the material are extracted, and linear regression analysis is performed on the feature points to obtain the fitting straight line of the feature points in the second three-dimensional image. The direction vector of the fitting straight line is a space vector, and the space vector is used to represent the shape characteristics of the material. Image optimization factors are extracted based on eigenvalues. The calculation of eigenvalues ​​takes into account the difference between the coordinates of each axis and the feature quantity and takes the absolute value, which can reduce the influence of individual data on coordinate deviation, thereby improving the accuracy of image optimization factors. According to the image optimization factors, the distribution characteristics of the space vector can be clearly represented, the accuracy of the robot arm's grasping is enhanced, and the feeding efficiency of the robot arm is guaranteed.

[0031] In some embodiments of the present application, when establishing a rotation matrix of a material based on a target vector, the following steps are included: Create a rotation matrix according to the following formula: ; ; ; ; Wherein, θ represents the rotation angle, Z represents the target vector, |Z| represents the modulus of the target vector, r represents the rotation axis vector in the first coordinate system of the image, and r x represents the component of the rotation axis in the x-axis direction, r y represents the component of the rotation axis in the y-axis direction, r z represents the component of the rotation axis in the z-axis direction, [r] represents the antisymmetric matrix of the rotation axis vector r, and R represents the rotation matrix.

[0032] It can be understood that establishing a rotation matrix can identify the rotation posture of the material after grasping, reduce the error accumulation caused by inaccurate posture, ensure the accuracy of posture adjustment, thereby reducing redundant movement of the robot arm, reducing the computational burden, enhancing the real-time processing capability of the robot arm, and improving the efficiency of feeding.

[0033] In some embodiments of the present application, when obtaining an analysis vector of the material grabbing position according to a first three-dimensional image, and deriving a first axial coefficient, a second axial coefficient and a third axial coefficient according to the analysis vector, it includes: obtaining an analysis vector according to the first three-dimensional image, statistically analyzing the first vector parameters of all x-axes of the analysis vector, statistically analyzing the second vector parameters of all y-axes of the analysis vector, statistically analyzing the third vector parameters of all z-axes of the analysis vector, calculating the first axial coefficient according to the first vector parameters, calculating the second axial coefficient according to the second vector parameters, and calculating the third axial coefficient according to the third vector parameters.

[0034] It can be understood that the method of obtaining the analysis vector of the material grabbing position according to the first three-dimensional image is the same as the method of the space vector, which will not be repeated here. The analysis vector is represented as the vector of the material grabbing position, and the three-axis vector parameters of the analysis vector are statistically analyzed and the axial coefficient is calculated, which reduces the complexity of data processing and enables the robot arm to efficiently complete the recognition and optimization of the material grabbing position by relying on the visual sensor.

[0035] In some embodiments of the present application, when calculating the first axial coefficient according to the first vector parameter, calculating the second axial coefficient according to the second vector parameter, and calculating the third axial coefficient according to the third vector parameter, it includes: ; ; ; Wherein, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, and n 1 represents the number of the first vector parameters containing the x-axis in the analysis vector, n 2 represents the number of the second vector parameters containing the y-axis in the analysis vector, n 3 represents the number of the third vector parameters containing the z-axis in the analysis vector, P i represents any one of the first vector parameters, Q i represents any one of the second vector parameters, L i Represents any one of the third vector parameters.

[0036] It can be understood that the first axial coefficient reflects the spatial characteristics of the material grabbing position in the x-axis direction, the second axial coefficient reflects the spatial characteristics of the material grabbing position in the y-axis direction, and the third axial coefficient reflects the spatial characteristics of the material grabbing position in the z-axis direction. During the calculation process, the calculation of each axial coefficient depends on the number of related axial vector parameters. For example, if the number of first vector parameters containing multiple x-axes in the analysis vector is large, there will be more factors affecting the spatial characteristics in the x-axis direction, while if the number of second vector parameters containing y-axes is small, there will be fewer factors affecting the spatial characteristics in the y-axis direction. The calculated axial coefficients will provide accurate spatial information for the adjustment and positioning of the grabbed materials, ensuring the accuracy and stability of the material grabbing and loading process.

[0037] In some embodiments of the present application, when the image coordinate optimization factor is calculated according to the first axial coefficient, the second axial coefficient and the third axial coefficient, the grab point coordinates are optimized according to the image coordinate optimization factor to obtain the loading coordinates, it includes: The image coordinate optimization factor is obtained according to the following formula: ; Among them, M represents the image coordinate optimization factor, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, α is the weight of the first axial coefficient, β is the weight of the second axial coefficient, γ is the weight of the third axial coefficient, and α+β+γ=1.5. The feeding coordinate is the product of the grabbing point coordinate and the image coordinate optimization factor.

[0038] It can be understood that the image coordinate optimization factor is a comprehensive data used to optimize the coordinates of the grasping point during the material grasping process, the first axial coefficient, the second axial coefficient and the third axial coefficient. These axial coefficients reflect the spatial characteristics of the material in the x, y and z directions respectively. On the basis of these axial coefficients, the image coordinate optimization factor takes into account the importance of each axial coefficient and assigns a weight coefficient to each axial coefficient, which is the weight of the first axial coefficient, the weight of the second axial coefficient and the weight of the third axial coefficient. Through weighted calculation, it is ensured that the influence of different axes on material grasping is reasonably reflected. The coordinates of the grasping point extracted in the second coordinate system of the image are multiplied by the image coordinate optimization factor to obtain the loading coordinates, so that the robot arm can be dynamically adjusted, which improves the accuracy of the robot arm in the grasping process, realizes the automation of the loading process, reduces the need for manual intervention, shortens the processing time of a single piece of material, and ensures the accuracy and efficiency of loading.

[0039] In summary, the beneficial effect of the present invention is that by establishing a first three-dimensional image, the target position of the material in the surrounding environment of the robot arm can be accurately perceived and analyzed. The positioning error in the traditional method is avoided. After the robot arm reaches the target position of the material, the characteristic value of the material is calculated based on the first coordinate system of the image, and the characteristic value in the historical set is similarly analyzed to ensure that the robot arm can find the material grabbing position, extract the spatial vector of the material according to the second three-dimensional image, optimize the spatial vector using the image optimization factor, generate the target vector and calculate the rotation matrix of the material, ensure that the robot arm can adjust the correct posture when grabbing the material, and improve the accuracy of the grabbing operation. In the grabbing process, by establishing the second coordinate system of the image, the coordinates of the grabbing point where the material and the robot arm contact are extracted in real time, and the loading coordinates are obtained according to the analysis vector, so as to accurately adjust the grabbing of the robot arm, ensure the grabbing accuracy of the material, improve production efficiency, reduce manual intervention, and realize the automation of the loading process.

[0040] In another preferred embodiment based on the above embodiment, refer to Figure 2 As shown, this embodiment provides a feeding control system based on a robot arm, which is used to apply the above-mentioned feeding control method based on a robot arm, including: A first camera module is used to create a first three-dimensional image of the environment of the robot arm, extract a motion path feature vector according to the first three-dimensional image, and determine a material target position based on the motion path feature vector; The first calculation module is used to pre-process the first three-dimensional image and establish a first coordinate system of the image when the robot arm reaches the target position of the material, calculate the characteristic value of the material based on the first coordinate system of the image, perform similarity processing on the characteristic value and the characteristic value in the historical set, and determine the material grabbing position according to the similarity; A second camera module is used to establish a second three-dimensional image, extract a spatial vector of the material according to the second three-dimensional image, extract an image optimization factor based on the eigenvalue, optimize the spatial vector according to the image optimization factor, obtain a target vector of the material, and establish a rotation matrix of the material based on the target vector; The power module is used to provide power for the robot arm to reach the target position of the material and grab the material according to the material grabbing position, target vector and rotation matrix; The second calculation module is used to establish a second coordinate system of the image according to the second three-dimensional image when grabbing the material, extract the coordinates of the grabbing point where the material and the robot arm are in contact according to the second coordinate system of the image, obtain the analysis vector of the material grabbing position according to the first three-dimensional image, and obtain the first axial coefficient, the second axial coefficient and the third axial coefficient according to the analysis vector; The optimization module is used to calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient and the third axial coefficient, optimize the grabbing point coordinates according to the image coordinate optimization factor, and obtain the loading coordinates.

[0041] It is understandable that the first camera module extracts the action path feature vector and determines the target position of the material by establishing the first three-dimensional image around the robot arm, so that the robot arm can accurately identify the target position of the material. When the robot arm reaches the target position of the material, the first calculation module preprocesses the first three-dimensional image and establishes the first coordinate system of the image to determine the material grabbing position. The second camera module is used to extract the spatial vector of the material and calculate the image optimization factor based on the eigenvalue to ensure that the robot arm can grab the material in the correct posture. The power module provides power to the robot arm to ensure that it can accurately grab the material. The second calculation module is used to establish the second coordinate system of the image for the second three-dimensional image, extract the coordinates of the grabbing point, and calculate the axial coefficients of the three axes according to the analysis vector to provide data support for subsequent adjustments. The optimization module optimizes the grabbing point coordinates to obtain the loading coordinates, improves the grabbing accuracy of the robot arm, and ensures the efficient and stable operation of the system. Thereby, efficient and accurate grabbing of materials is achieved, thereby realizing the automation of the loading process.

[0042] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0043] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0044] These computer program instructions may also be stored in a computer-readable storage device that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable storage device produce a product including an instruction device that implements the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0045] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A feeding control method based on a robotic arm, characterized in that: include: Establishing a first three-dimensional image of the environment of the robot arm, extracting a motion path feature vector according to the first three-dimensional image, and determining a target position of the material according to the motion path feature vector; When the robot arm reaches the target position of the material, the first three-dimensional image is preprocessed and a first coordinate system of the image is established, a characteristic value of the material is calculated based on the first coordinate system of the image, the characteristic value is similarly processed with the characteristic value in the historical set, and a material grabbing position is determined according to the similarity; Establishing a second three-dimensional image, extracting a spatial vector of a material according to the second three-dimensional image, extracting an image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor to obtain a target vector of the material, and establishing a rotation matrix of the material based on the target vector; The robotic arm grabs the material according to the material grabbing position, the target vector and the rotation matrix; When grasping a material, a second image coordinate system is established according to the second three-dimensional image, and the coordinates of the grasping point where the material and the robot arm are in contact are extracted according to the second image coordinate system; Acquire an analysis vector of the material grabbing position according to the first three-dimensional image, and obtain a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; An image coordinate optimization factor is calculated according to the first axial coefficient, the second axial coefficient and the third axial coefficient, and the grabbing point coordinates are optimized according to the image coordinate optimization factor to obtain a loading coordinate.

2. The feeding control method based on a robotic arm according to claim 1 is characterized in that: When calculating the characteristic value of the material based on the first coordinate system of the image, it includes: Extracting the three-dimensional coordinates of the target position of the material from the first coordinate system of the image, and respectively counting all x-axis coordinates, y-axis coordinates and z-axis coordinates in the three-dimensional coordinates; Counting the number of all coordinates in the three-dimensional coordinates, and calculating the x-coordinate feature quantity, y-coordinate feature quantity and z-coordinate feature quantity of the material; The first weight coefficient is obtained by taking the absolute values ​​of the differences between all the x-axis coordinates and the x-coordinate feature quantity and adding them, the second weight coefficient is obtained by taking the absolute values ​​of the differences between all the y-axis coordinates and the y-coordinate feature quantity and adding them, and the third weight coefficient is obtained by taking the absolute values ​​of the differences between all the z-axis coordinates and the z-coordinate feature quantity and adding them; The feature value is calculated based on the x-coordinate feature quantity, the y-coordinate feature quantity, the z-coordinate feature quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient.

3. The feeding control method based on a robotic arm according to claim 2 is characterized in that: When the feature value is calculated according to the x-coordinate feature quantity, the y-coordinate feature quantity, the z-coordinate feature quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient, the method includes: The characteristic value is calculated according to the following formula: ； Among them, V represents the characteristic value, w1 represents the first weight coefficient, w2 represents the second weight coefficient, w3 represents the third weight coefficient, x represents the x-coordinate feature value, y represents the y-coordinate feature value, and z represents the z-coordinate feature value.

4. The feeding control method based on a robotic arm according to claim 3 is characterized in that: The characteristic value and the characteristic value in the history set are similarly processed, and the material grabbing position is determined according to the similarity, including: The number of feature values ​​in the history set that are the same as the feature value is counted, and the similarity is calculated according to the following formula: ； Wherein, S represents the similarity, V1 represents the number of feature values ​​in the history set that are the same as the feature value, and V2 represents the number of all feature values ​​in the history set; When S is greater than or equal to 70%, the historical material grab position is used; When S is less than 70%, the material grabbing position is preset based on the first coordinate system of the image; The material grabbing position includes the historical material grabbing position and the preset material grabbing position.

5. The feeding control method based on a robotic arm according to claim 3 is characterized in that: When extracting the spatial vector of the material according to the second three-dimensional image, extracting the image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor, and obtaining the target vector of the material, the method includes: Collecting initial point cloud data of the second three-dimensional image, and performing refinement processing on the initial point cloud data to obtain terminal point cloud data; Extracting characteristic points of the material according to the terminal point cloud data, and performing local fitting on the characteristic points to obtain the space vector; The target vector is obtained according to the following formula: ； ； Among them, C opt represents the image optimization factor, V represents the eigenvalue, K represents the space vector, and Z represents the target vector.

6. The feeding control method based on a robot arm according to claim 5 is characterized in that: When establishing the rotation matrix of the material based on the target vector, it includes: The rotation matrix is ​​established according to the following formula: ； ； ； ； Wherein, θ represents the rotation angle, Z represents the target vector, |Z| represents the modulus of the target vector, r represents the rotation axis vector in the first coordinate system of the image, and r x represents the component of the rotation axis in the x-axis direction, r y represents the component of the rotation axis in the y-axis direction, r z represents the component of the rotation axis in the z-axis direction, [r] represents the antisymmetric matrix of the rotation axis vector r, and R represents the rotation matrix.

7. The feeding control method based on a robot arm according to claim 1 is characterized in that: When an analysis vector of the material grabbing position is acquired according to the first three-dimensional image, and a first axial coefficient, a second axial coefficient and a third axial coefficient are obtained according to the analysis vector, the method includes: acquiring the analysis vector according to the first three-dimensional image; Count the first vector parameters of all x-axes of the analysis vector, count the second vector parameters of all y-axes of the analysis vector, count the third vector parameters of all z-axes of the analysis vector, calculate the first axial coefficient according to the first vector parameters, calculate the second axial coefficient according to the second vector parameters, and calculate the third axial coefficient according to the third vector parameters.

8. The feeding control method based on a robot arm according to claim 7 is characterized in that: When calculating the first axial coefficient according to the first vector parameter, calculating the second axial coefficient according to the second vector parameter, and calculating the third axial coefficient according to the third vector parameter, it includes: ； ； ； Wherein, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, n1 represents the number of the first vector parameters containing the x-axis in the analysis vector, n2 represents the number of the second vector parameters containing the y-axis in the analysis vector, n3 represents the number of the third vector parameters containing the z-axis in the analysis vector, P i represents any one of the first vector parameters, Q i represents any one of the second vector parameters, L i Represents any one of the third vector parameters.

9. The feeding control method based on a robot arm according to claim 1, characterized in that: When the image coordinate optimization factor is calculated according to the first axial coefficient, the second axial coefficient and the third axial coefficient, the grabbing point coordinates are optimized according to the image coordinate optimization factor to obtain the loading coordinates, the method includes: The image coordinate optimization factor is obtained according to the following formula: ； Wherein, M represents the image coordinate optimization factor, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, α represents the weight of the first axial coefficient, β represents the weight of the second axial coefficient, γ represents the weight of the third axial coefficient, and α+β+γ=1.5; The loading coordinates are the product of the grabbing point coordinates and the image coordinate optimization factor.

10. A feeding control system based on a robot arm, applied to the feeding control method based on a robot arm as claimed in any one of claims 1 to 9, characterized in that: include: A first camera module is used to create a first three-dimensional image of the environment of the robot arm, extract a motion path feature vector according to the first three-dimensional image, and determine a material target position according to the motion path feature vector; A first calculation module, when the robot arm reaches the target position of the material, is used to pre-process the first three-dimensional image and establish a first coordinate system of the image, calculate the characteristic value of the material based on the first coordinate system of the image, perform similarity processing on the characteristic value and the characteristic value in the historical set, and determine the material grabbing position according to the similarity; A second camera module is used to establish a second three-dimensional image, extract a spatial vector of a material according to the second three-dimensional image, extract an image optimization factor based on the eigenvalue, optimize the spatial vector according to the image optimization factor, obtain a target vector of the material, and establish a rotation matrix of the material based on the target vector; A power module, used for providing power to the robot arm when it reaches the target position of the material and when the robot arm grabs the material according to the material grabbing position, the target vector and the rotation matrix; A second calculation module, when grabbing a material, is used to establish a second image coordinate system according to the second three-dimensional image, extract the coordinates of the grabbing point where the material and the robotic arm are in contact according to the second image coordinate system, obtain an analysis vector of the material grabbing position according to the first three-dimensional image, and derive a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; The optimization module is used to calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient and the third axial coefficient, and optimize the grabbing point coordinates according to the image coordinate optimization factor to obtain the loading coordinates.

Images