A feeding control system and control method based on a robotic arm

Abstract

The present invention relates to the technical field of robotic arms, and discloses a feeding control system and a control method based on a robotic arm. The method includes: establishing a first three-dimensional image of the environment of the robotic arm, extracting an action path feature vector according to the first three-dimensional image, determining a material target position for the action path feature vector, when the robotic arm reaches the material target position, preprocessing the first three-dimensional image and establishing a first image coordinate system, calculating the eigenvalue of the material based on the first image coordinate system, extracting an image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor to obtain the target vector of the material, when grasping the material, establishing a second image coordinate system according to the second three-dimensional image, extracting the coordinates of the grasping point where the material and the robotic arm are in contact according to the second image coordinate system, and optimizing the coordinates of the grasping point according to the image coordinate optimization factor to obtain the feeding coordinates. The present invention has good grasping adjustment characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arms, and more particularly, to a feeding control system and control method based on a robotic arm. Background Art

[0002] With the progress of technology, CNC computer-controlled machine tools have developed rapidly. In CNC machine tools, in the traditional feeding control process of robotic arms, the grasping of materials depends on the pre-set program planning of the robotic arms. However, without the support and processing of three-dimensional images, the robotic arms cannot make appropriate adjustments, resulting in the inability of the robotic arms to accurately grasp materials during the feeding process, thus affecting the efficiency and success rate of grasping.

[0003] The traditional control of robotic arms depends on the force feedback and pre-set programs of the robotic arms. Although some basic tasks can be completed, when dealing with feeding tasks with high precision requirements, problems such as inaccurate positioning and grasping failure are faced. At the same time, there is also a lack of real-time image feedback of visual sensors to adjust the grasping strategy, and the actual situation of the materials cannot be analyzed and optimized, resulting in the instability of the feeding process.

[0004] Therefore, how to provide a feeding control system and control method based on a robotic arm is an urgent technical problem to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention proposes a feeding control system and control method based on a robotic arm, aiming to solve the problem of the instability of the feeding process caused by the lack of real-time image feedback to adjust the grasping strategy and the inability to analyze and optimize the actual situation of the materials.

[0006] On the one hand, the present invention proposes a feeding control method based on a robotic arm, including:

[0007] Establish a first three-dimensional image of the environment of the robotic arm, extract the action path feature vector according to the first three-dimensional image, and determine the material target position for the action path feature vector;

[0008] When the robotic arm reaches the material target position, preprocess the first three-dimensional image and establish an image first coordinate system, calculate the characteristic values of the material based on the image first coordinate system, perform similarity processing on the characteristic values and the characteristic values in the historical set, and determine the material grasping position according to the similarity;

[0009] Build a second three-dimensional image, extract the spatial vector of the material according to the second three-dimensional image, extract the image optimization factor based on the eigenvalue, optimize the spatial vector according to the image optimization factor to obtain the target vector of the material, and establish the rotation matrix of the material based on the target vector;

[0010] The robotic arm grabs the material according to the material grasping position, the target vector, and the rotation matrix;

[0011] When grabbing the material, establish an image second coordinate system according to the second three-dimensional image, and extract the coordinates of the grasping point where the material and the robotic arm are in contact according to the image second coordinate system;

[0012] Obtain the analysis vector of the material grasping position according to the first three-dimensional image, and obtain the first axial coefficient, the second axial coefficient, and the third axial coefficient according to the analysis vector;

[0013] Calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient, and the third axial coefficient, and optimize the grasping point coordinates according to the image coordinate optimization factor to obtain the loading coordinates.

[0014] Further, when calculating the eigenvalue of the material based on the image first coordinate system, it includes:

[0015] Extract the three-dimensional coordinates of the target position of the material from the image first coordinate system, and respectively count all the x-axis coordinates, y-axis coordinates, and z-axis coordinates in the three-dimensional coordinates;

[0016] Count all the coordinate quantities in the three-dimensional coordinates, and calculate the x-coordinate characteristic quantity, y-coordinate characteristic quantity, and z-coordinate characteristic quantity of the material;

[0017] Take the absolute value of the difference between all the x-axis coordinates and the x-coordinate characteristic quantity and add them up to obtain the first weight coefficient, take the absolute value of the difference between all the y-axis coordinates and the y-coordinate characteristic quantity and add them up to obtain the second weight coefficient, and take the absolute value of the difference between all the z-axis coordinates and the z-coordinate characteristic quantity and add them up to obtain the third weight coefficient;

[0018] Calculate the eigenvalue according to the x-coordinate characteristic quantity, the y-coordinate characteristic quantity, the z-coordinate characteristic quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient.

[0019] Further, when calculating the eigenvalue according to the x-coordinate characteristic quantity, the y-coordinate characteristic quantity, the z-coordinate characteristic quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient, it includes:

[0020] Calculate the eigenvalue according to the following formula:

[0021] ;

[0022] Among them, V represents the eigenvalue, w 1 represents the first weight coefficient, w 2 represents the second weight coefficient, w 3 represents the third weight coefficient, x represents the x - coordinate feature quantity, y represents the y - coordinate feature quantity, and z represents the z - coordinate feature quantity.

[0023] Furthermore, when performing similarity processing on the eigenvalue and the eigenvalues in the historical set and determining the material grasping position according to the similarity, it includes:

[0024] Count the number of eigenvalues in the historical set that are the same as the eigenvalue, and calculate the similarity according to the following formula:

[0025] ;

[0026] Among them, S represents the similarity, V 1 represents the number of eigenvalues in the historical set that are the same as the eigenvalue, V 2 represents the total number of all eigenvalues in the historical set;

[0027] When S is greater than or equal to 70%, adopt the historical material grasping position;

[0028] When S is less than 70%, preset the material grasping position based on the first coordinate system of the image;

[0029] The material grasping position includes the historical material grasping position and the preset material grasping position.

[0030] Furthermore, when extracting the spatial vector of the material according to the second three - dimensional image, extracting the image optimization factor based on the eigenvalue, and optimizing the spatial vector according to the image optimization factor to obtain the target vector of the material, it includes:

[0031] Collect the initial point cloud data of the second three - dimensional image, and perform refinement processing on the initial point cloud data to obtain the end - point cloud data;

[0032] According to the end - point cloud data, extract the feature points of the material, and perform local fitting on the feature points to obtain the spatial vector;

[0033] Obtain the target vector according to the following formula:

[0034] ;

[0035] ;

[0036] Among them, C opt represents the image optimization factor, V represents the eigenvalue, K represents the spatial vector, and Z represents the target vector.

[0037] Furthermore, when establishing the rotation matrix of the material based on the target vector, it includes:

[0038] Establish the rotation matrix according to the following formula:

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] Among them, θ represents the rotation angle, Z represents the target vector, |Z| represents the modulus of the target vector, r represents the rotation axis vector in the first coordinate system of the image, and r x represents the component of the rotation axis in the x-axis direction, and r y represents the component of the rotation axis in the y-axis direction, and r z represents the component of the rotation axis in the z-axis direction, [r] represents the skew-symmetric matrix of the rotation axis vector r, and R represents the rotation matrix.

[0044] Furthermore, it is characterized in that when obtaining the analysis vector of the material grasping position according to the first three-dimensional image and obtaining the first axial coefficient, the second axial coefficient, and the third axial coefficient according to the analysis vector, it includes:

[0045] Obtain the analysis vector according to the first three-dimensional image;

[0046] Statistically analyze the first vector parameter of all x-axes of the vector, statistically analyze the second vector parameter of all y-axes of the vector, statistically analyze the third vector parameter of all z-axes of the vector, calculate the first axial coefficient according to the first vector parameter, calculate the second axial coefficient according to the second vector parameter, and calculate the third axial coefficient according to the third vector parameter.

[0047] Furthermore, when calculating the first axial coefficient according to the first vector parameter, calculating the second axial coefficient according to the second vector parameter, and calculating the third axial coefficient according to the third vector parameter, it includes:

[0048] ;

[0049] ;

[0050] ;

[0051] Wherein, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, and n 1 represents the number of the first vector parameters containing the x-axis in the analysis vector, and n 2 represents the number of the second vector parameters containing the y-axis in the analysis vector, and n 3 represents the number of the third vector parameters containing the z-axis in the analysis vector, P i represents any one of the first vector parameters, Q i represents any one of the second vector parameters, L i represents any one of the third vector parameters.

[0052] Further, when calculating the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient, and the third axial coefficient, and optimizing the grasping point coordinates according to the image coordinate optimization factor to obtain the loading coordinates, it includes:

[0053] The image coordinate optimization factor is obtained according to the following formula:

[0054] ;

[0055] Wherein, M represents the image coordinate optimization factor, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, α is the weight of the first axial coefficient, β is the weight of the second axial coefficient, γ is the weight of the third axial coefficient, and α + β + γ = 1.5;

[0056] The loading coordinates are the product value of the grasping point coordinates and the image coordinate optimization factor.

[0057] Compared with the prior art, the beneficial effects of the present invention are as follows: By establishing the first three-dimensional image, it is possible to accurately perceive and analyze the position of the material target in the environment around the robotic arm. Avoiding the positioning error in the traditional method, after the robotic arm reaches the position of the material target, the characteristic values of the material are calculated based on the first coordinate system of the image, and a similarity analysis is performed on the characteristic values in the historical set to ensure that the robotic arm can find the material grasping position. According to the second three-dimensional image, the spatial vector of the material is extracted, and the spatial vector is optimized using the image optimization factor to generate the target vector and calculate the rotation matrix of the material, ensuring that the robotic arm can adjust the correct posture when grasping the material and improving the accuracy of the grasping operation. During the grasping process, by establishing the second coordinate system of the image, the coordinates of the grasping point where the material contacts the robotic arm are extracted in real time, and the feeding coordinates are obtained according to the analysis vector, thereby precisely adjusting the grasping of the robotic arm, ensuring the grasping accuracy of the material, improving the production efficiency, reducing manual intervention, and realizing the automated operation of the feeding process.

[0058] On the other hand, the present application also provides a feeding control system based on a robotic arm for applying the above-mentioned feeding control method based on a robotic arm, including:

[0059] The first camera module is used to establish the first three-dimensional image of the environment of the robotic arm, extract the action path feature vector according to the first three-dimensional image, and determine the position of the material target for the action path feature vector;

[0060] The first calculation module is used to preprocess the first three-dimensional image and establish the first coordinate system of the image when the robotic arm reaches the position of the material target, calculate the characteristic values of the material based on the first coordinate system of the image, perform similarity processing on the characteristic values and the characteristic values in the historical set, and determine the material grasping position according to the similarity;

[0061] The second camera module is used to establish the second three-dimensional image, extract the spatial vector of the material according to the second three-dimensional image, extract the image optimization factor based on the characteristic values, optimize the spatial vector according to the image optimization factor to obtain the target vector of the material, and establish the rotation matrix of the material based on the target vector;

[0062] The power module is used to provide power when the robotic arm reaches the position of the material target and the robotic arm grasps the material according to the material grasping position, the target vector and the rotation matrix;

[0063] A second calculation module, when grasping materials, is used to establish an image second coordinate system according to the second three-dimensional image, extract the coordinates of the grasping points where the materials and the robotic arm are in contact according to the image second coordinate system, obtain the analysis vector of the material grasping position according to the first three-dimensional image, and obtain the first axial coefficient, the second axial coefficient, and the third axial coefficient according to the analysis vector;

[0064] An optimization module is used to calculate an image coordinate optimization factor according to the first axial coefficient, the second axial coefficient, and the third axial coefficient, optimize the grasping point coordinates according to the image coordinate optimization factor, and obtain the loading coordinates.

[0065] It can be understood that the above-mentioned feeding control system and control method based on a robotic arm have the same beneficial effects, which will not be elaborated here. Description of the Drawings

[0066] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0067] Figure 1 is a flowchart of a feeding control method based on a robotic arm provided by an embodiment of the present invention;

[0068] Figure 2 is a functional block diagram of a feeding control system based on a robotic arm provided by an embodiment of the present invention. Detailed Embodiments

[0069] The following will describe the exemplary embodiments of the present disclosure in more detail with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be fully conveyed to those skilled in the art. It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0070] Refer to Figure 1 As shown, in some embodiments of the present application, this embodiment provides a feeding control method based on a robotic arm, including:

[0071] S100: Establish a first three-dimensional image of the environment of the robotic arm, extract the action path feature vector according to the first three-dimensional image, and determine the material target position for the action path feature vector;

[0072] S200: When the robotic arm reaches the material target position, preprocess the first 3D image and establish the first image coordinate system. Calculate the characteristic values of the material based on the first image coordinate system, perform similarity processing on the characteristic values and the characteristic values in the historical set, and determine the material grasping position according to the similarity.

[0073] S300: Establish the second 3D image, extract the spatial vector of the material according to the second 3D image, extract the image optimization factor based on the characteristic values, optimize the spatial vector according to the image optimization factor to obtain the target vector of the material, and establish the rotation matrix of the material based on the target vector.

[0074] The robotic arm grasps the material according to the material grasping position, the target vector, and the rotation matrix.

[0075] S400: When grasping the material, establish the second image coordinate system according to the second 3D image, extract the coordinates of the grasping points where the material and the robotic arm are in contact according to the second image coordinate system, obtain the analysis vector of the material grasping position according to the first 3D image, and obtain the first axial coefficient, the second axial coefficient, and the third axial coefficient according to the analysis vector.

[0076] S500: Calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient, and the third axial coefficient, and optimize the grasping point coordinates according to the image coordinate optimization factor to obtain the loading coordinates.

[0077] Specifically, the robotic arm is located on the left side of a CNC machine tool and on the right side of a CNC machine tool. The robotic arm is equipped with a vision sensor. The vision sensor can generate a first three-dimensional image of the environment where the robotic arm is located in real time, extract the feature points of the robotic arm and the material in the first three-dimensional image, aggregate the same type of feature points to construct an action path feature vector, fit the action path feature vector into a straight line path, and set the end point of the straight line path as the material target position. When driving the robotic arm to reach the material target position, preprocess the first three-dimensional image, and the preprocessing includes removing duplicate images and exposed images. Set the working platform of the left CNC machine tool in the first three-dimensional image as the first image coordinate system, and the same for the right side. Calculate the eigenvalue of the material according to the first image coordinate system to determine the material grasping position of the robotic arm. The vision sensor generates a second three-dimensional image, collects the point cloud data of the second three-dimensional image, extracts the feature points of the material based on the point cloud data for fitting to obtain a spatial vector. The spatial vector is used to express the shape feature of the material, and the spatial vector is optimized through the eigenvalue to obtain the target vector of the material. The target vector expresses the posture and shape features of the material. Calculate the rotation matrix based on the target vector. The rotation matrix is the rotation posture data when the robotic arm grasps the material. When the robotic arm touches the surface of the material, set the second image coordinate system with the material in the second three-dimensional image as the reference, extract the coordinate of the grasping point where the material and the robotic arm are in contact, obtain the analysis vector of the material grasping position according to the first three-dimensional image, and calculate the first axial coefficient, the second axial coefficient, and the third axial coefficient. Calculate the image coordinate optimization factor based on the first axial coefficient, the second axial coefficient, and the third axial coefficient. The image coordinate optimization factor optimizes the grasping point coordinate to obtain the loading coordinate. The loading coordinate is the coordinate where the robotic arm actually grasps the material. By optimizing the grasping point coordinate through the image coordinate optimization factor, the robotic arm can accurately grasp the material and complete the loading operation.

[0078] It can be understood that based on the first three-dimensional image and the second three-dimensional image obtained by the vision sensor and calculating the image coordinate optimization factor, the robotic arm can coordinate its motion state, dynamically adjust the grasping of the robotic arm according to the loading coordinate, ensure the accuracy of material grasping, improve the stability and efficiency of loading, realize the automated operation of the loading process, and reduce the need for manual intervention.

[0079] In some embodiments of the present application, when calculating the characteristic value of the material based on the first image coordinate system, it includes: extracting the three-dimensional coordinates of the target position of the material from the first image coordinate system, respectively counting all the x-axis coordinates, y-axis coordinates, and z-axis coordinates in the three-dimensional coordinates, counting all the coordinate quantities in the three-dimensional coordinates, and calculating the x-coordinate characteristic quantity, y-coordinate characteristic quantity, and z-coordinate characteristic quantity of the material. Taking the absolute value of the difference between all the x-axis coordinates and the x-coordinate characteristic quantity and adding them up to obtain the first weight coefficient, taking the absolute value of the difference between all the y-axis coordinates and the y-coordinate characteristic quantity and adding them up to obtain the second weight coefficient, taking the absolute value of the difference between all the z-axis coordinates and the z-coordinate characteristic quantity and adding them up to obtain the third weight coefficient, and calculating the characteristic value according to the x-coordinate characteristic quantity, y-coordinate characteristic quantity, z-coordinate characteristic quantity, first weight coefficient, second weight coefficient, and third weight coefficient.

[0080] It can be understood that by extracting the three-dimensional coordinates of the target position of the material, the position of the material in space can be accurately determined, ensuring the accuracy of material grasping and reducing the risk of mis-grasping or wrong-grasping. The x-coordinate characteristic quantity is the mean value obtained by adding up all the x-axis coordinates. The determination methods of the y-coordinate characteristic quantity and the z-coordinate characteristic quantity are the same as that of the x-coordinate characteristic quantity, and will not be repeated here. By calculating the absolute value of the difference between the coordinates of each axis and the characteristic quantity, the accumulation of errors can be effectively reduced, the influence of coordinate deviation can be reduced, the spatial distribution characteristics of the material can be refined, and data support is provided for the adjustment of grasping the material.

[0081] In some embodiments of the present application, when calculating the characteristic value according to the x-coordinate characteristic quantity, y-coordinate characteristic quantity, z-coordinate characteristic quantity, first weight coefficient, second weight coefficient, and third weight coefficient, it includes:

[0082] Calculating the characteristic value according to the following formula:

[0083] ;

[0084] where V represents the characteristic value, w 1 represents the first weight coefficient, w 2 represents the second weight coefficient, w 3 represents the third weight coefficient, x represents the x-coordinate characteristic quantity, y represents the y-coordinate characteristic quantity, and z represents the z-coordinate characteristic quantity.

[0085] It can be understood that calculating the characteristic value according to the x-coordinate characteristic quantity, y-coordinate characteristic quantity, z-coordinate characteristic quantity, first weight coefficient, second weight coefficient, and third weight coefficient ensures the rigor of the calculation of the characteristic value and avoids the one-sidedness of the calculation result caused by insufficient data.

[0086] In some embodiments of the present application, when performing similarity processing on the eigenvalue and the eigenvalues in the historical set and determining the material grasping position according to the similarity, it includes: counting the number of eigenvalues that are the same as the eigenvalue in the historical set, and calculating the similarity according to the following formula:

[0087] ;

[0088] where S represents the similarity, V 1 represents the number of eigenvalues that are the same as the eigenvalue in the historical set, and V 2 represents the total number of eigenvalues in the historical set; when S is greater than or equal to 70%, the historical material grasping position is adopted, and when S is less than 70%, a preset material grasping position is set based on the first coordinate system of the image. The material grasping position includes the historical material grasping position and the preset material grasping position.

[0089] It can be understood that by comparing with the eigenvalues in the historical set, the change or deviation of the material position can be identified. In the case where most eigenvalues are similar, using the historical material grasping position ensures the accuracy and stability of grasping, reduces errors and the adjustment time of the robotic arm. When the similarity is greater than or equal to 70%, the historical material grasping position is directly adopted, optimizing the feeding production rhythm. When the similarity is less than 70%, a grasping position is preset according to the first coordinate system of the image, ensuring the adaptive ability of the robotic arm. The robotic arm using the historical material grasping position can maintain the continuity and stability of the grasping operation, reduce the operation adjustment cost, and reduce the deviation of the material grasping position caused by errors or inaccuracies, thereby improving the stability and consistency of feeding.

[0090] In some embodiments of the present application, when extracting the spatial vector of the material according to the second three-dimensional image, extracting the image optimization factor based on the eigenvalue, and optimizing the spatial vector according to the image optimization factor to obtain the target vector of the material, it includes: collecting the initial point cloud data of the second three-dimensional image, performing refinement processing on the initial point cloud data to obtain the end point cloud data, extracting the feature points of the material according to the end point cloud data, locally fitting the feature points to obtain the spatial vector, and obtaining the target vector according to the following formula:

[0091] ;

[0092] ;

[0093] where C opt represents the image optimization factor, V represents the eigenvalue, K represents the spatial vector, and Z represents the target vector.

[0094] It is understandable that the initial point cloud data of the second three-dimensional image is collected and refined. The refinement process includes: removing redundancy, reducing noise, and enhancing features. The redundant data volume is removed, thus meeting the requirement of accurate calculation. The noise reduction process can effectively remove the noise points in the point cloud data and retain the feature points and the edges of the material. Enhancing features identifies and highlights the feature points, which helps to highlight the structural features of the material and enhance the contour of the material. The feature points of the material are extracted, and linear regression analysis is performed on the feature points to obtain the fitting line of the feature points in the second three-dimensional image. The direction vector of the fitting line is a spatial vector, and the spatial vector is used to represent the shape features of the material. Based on the eigenvalues, an image optimization factor is extracted. The calculation of the eigenvalues takes into account the differences between the coordinates of each axis and the feature quantity and takes the absolute value, which can reduce the influence of individual data on the coordinate deviation, thereby improving the accuracy of the image optimization factor. According to the image optimization factor, the distribution characteristics of the spatial vector can be clearly represented, enhancing the accuracy of the robotic arm's grasping and ensuring the feeding efficiency of the robotic arm.

[0095] In some embodiments of the present application, when establishing the rotation matrix of the material based on the target vector, it includes:

[0096] Establish the rotation matrix according to the following formula:

[0097] ;

[0098] ;

[0099] ;

[0100] ;

[0101] where, θ represents the rotation angle, Z represents the target vector, |Z| represents the modulus of the target vector, r represents the rotation axis vector in the first coordinate system of the image, r x represents the component of the rotation axis in the x-axis direction, r y represents the component of the rotation axis in the y-axis direction, r z represents the component of the rotation axis in the z-axis direction, [r] represents the skew-symmetric matrix of the rotation axis vector r, and R represents the rotation matrix.

[0102] It is understandable that establishing the rotation matrix can identify the rotation posture of the material after grasping, reduce the error accumulation caused by inaccurate postures, ensure the accuracy of posture adjustment, thereby reducing the redundant movement of the robotic arm, reducing the computational burden, enhancing the real-time processing ability of the robotic arm, and improving the feeding efficiency.

[0103] In some embodiments of the present application, when obtaining the analysis vector of the material grasping position according to the first three-dimensional image and obtaining the first axial coefficient, the second axial coefficient, and the third axial coefficient based on the analysis vector, it includes: obtaining the analysis vector according to the first three-dimensional image, counting the first vector parameters of all x-axes of the analysis vector, counting the second vector parameters of all y-axes of the analysis vector, counting the third vector parameters of all z-axes of the analysis vector, calculating the first axial coefficient according to the first vector parameters, calculating the second axial coefficient according to the second vector parameters, and calculating the third axial coefficient according to the third vector parameters.

[0104] It can be understood that the method of obtaining the analysis vector of the material grasping position according to the first three-dimensional image is the same as that of the spatial vector, and will not be repeated here. The analysis vector represents the vector of the material grasping position. By separately counting the three-axis vector parameters of the analysis vector and calculating the axial coefficients, the complexity of data processing is reduced, enabling the robotic arm to efficiently complete the recognition and optimization of the material grasping position relying on the vision sensor.

[0105] In some embodiments of the present application, when calculating the first axial coefficient according to the first vector parameters, calculating the second axial coefficient according to the second vector parameters, and calculating the third axial coefficient according to the third vector parameters, it includes:

[0106] ;

[0107] ;

[0108] ;

[0109] where A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, n 1 represents the number of the first vector parameters containing the x-axis in the analysis vector, n 2 represents the number of the second vector parameters containing the y-axis in the analysis vector, n 3 represents the number of the third vector parameters containing the z-axis in the analysis vector, P i represents any one of the first vector parameters, Q i represents any one of the second vector parameters, L i represents any one of the third vector parameters.

[0110] It can be understood that the first axial coefficient reflects the spatial characteristics of the material grasping position in the x-axis direction, the second axial coefficient reflects the spatial characteristics of the material grasping position in the y-axis direction, and the third axial coefficient reflects the spatial characteristics of the material grasping position in the z-axis direction. During the calculation process, the calculation of each axial coefficient depends on the number of relevant axial vector parameters. For example, if the analysis vector contains a large number of first vector parameters on the x-axis, there are more factors affecting the spatial characteristics in the x-axis direction, while if the number of second vector parameters on the y-axis is small, there are fewer factors that can affect the spatial characteristics in the y-axis direction. The calculated axial coefficients will provide accurate spatial information for the adjustment and positioning of the grasped material, ensuring the accuracy and stability of the material grasping and feeding process.

[0111] In some embodiments of the present application, when calculating the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient, and the third axial coefficient, and optimizing the grasping point coordinates according to the image coordinate optimization factor to obtain the feeding coordinates, it includes:

[0112] The image coordinate optimization factor is obtained according to the following formula:

[0113] ;

[0114] where M represents the image coordinate optimization factor, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, α is the weight of the first axial coefficient, β is the weight of the second axial coefficient, γ is the weight of the third axial coefficient, and α + β + γ = 1.5. The feeding coordinates are the product value of the grasping point coordinates and the image coordinate optimization factor.

[0115] It can be understood that the image coordinate optimization factor is a comprehensive data used to optimize the grasping point coordinates during the material grasping process. The first axial coefficient, the second axial coefficient, and the third axial coefficient respectively reflect the spatial characteristics of the material in the x, y, and z directions. Based on these axial coefficients, the image coordinate optimization factor takes into account the importance of each axial coefficient and assigns a weight coefficient to each axial coefficient, namely the weight of the first axial coefficient, the weight of the second axial coefficient, and the weight of the third axial coefficient. Through weighted calculation, it ensures that the influence of different axes on material grasping is reasonably reflected. The grasping point coordinates extracted in the second image coordinate system are multiplied by the image coordinate optimization factor to obtain the feeding coordinates, enabling the robotic arm to dynamically adjust, improving the accuracy of the robotic arm during the grasping process, realizing the automation of the feeding process, reducing the need for manual intervention, shortening the processing time of a single piece of material, and ensuring the accuracy and efficiency of feeding.

[0116] In summary, the beneficial effects of the present invention are as follows: By establishing the first three-dimensional image, the material target position in the environment around the robotic arm can be accurately perceived and analyzed, avoiding the positioning error in the traditional method. After the robotic arm reaches the material target position, the characteristic values of the material are calculated based on the first coordinate system of the image, and the similarity analysis is performed on the characteristic values in the historical set to ensure that the robotic arm can find the material grasping position. According to the second three-dimensional image, the spatial vector of the material is extracted, and the spatial vector is optimized by using the image optimization factor to generate the target vector and calculate the rotation matrix of the material, ensuring that the robotic arm can adjust the correct posture when grasping the material and improving the accuracy of the grasping operation. During the grasping process, by establishing the second coordinate system of the image, the coordinates of the grasping points where the material contacts the robotic arm are extracted in real time, and the feeding coordinates are obtained according to the analysis vector, so as to accurately adjust the grasping of the robotic arm, ensuring the grasping accuracy of the material, improving the production efficiency, reducing the manual intervention, and realizing the automatic operation of the feeding process.

[0117] In another preferred embodiment based on the above embodiments, referring to Figure 2 as shown, this embodiment provides a feeding control system based on a robotic arm for applying the above-mentioned feeding control method based on a robotic arm, including:

[0118] The first camera module is used to establish the first three-dimensional image of the environment of the robotic arm, extract the action path feature vector according to the first three-dimensional image, and determine the material target position for the action path feature vector;

[0119] The first calculation module is used to preprocess the first three-dimensional image and establish the first coordinate system of the image when the robotic arm reaches the material target position, calculate the characteristic values of the material based on the first coordinate system of the image, perform similarity processing on the characteristic values and the characteristic values in the historical set, and determine the material grasping position according to the similarity;

[0120] The second camera module is used to establish the second three-dimensional image, extract the spatial vector of the material according to the second three-dimensional image, extract the image optimization factor based on the characteristic values, optimize the spatial vector according to the image optimization factor to obtain the target vector of the material, and establish the rotation matrix of the material based on the target vector;

[0121] The power module is used to provide power when the robotic arm reaches the material target position and the robotic arm grasps the material according to the material grasping position, the target vector and the rotation matrix;

[0122] The second calculation module is used to establish the second coordinate system of the image according to the second three-dimensional image when grasping the material, extract the coordinates of the grasping points where the material contacts the robotic arm according to the second coordinate system of the image, obtain the analysis vector of the material grasping position according to the first three-dimensional image, and obtain the first axial coefficient, the second axial coefficient and the third axial coefficient according to the analysis vector;

[0123] An optimization module is used to calculate an image coordinate optimization factor according to a first axial coefficient, a second axial coefficient, and a third axial coefficient, and optimize the coordinates of the grasping point according to the image coordinate optimization factor to obtain the loading coordinates.

[0124] It can be understood that the first camera module establishes a first three-dimensional image around the robotic arm, extracts the action path feature vector and determines the material target position, enabling the robotic arm to accurately identify the material target position. When the robotic arm reaches the material target position, the first calculation module preprocesses the first three-dimensional image and establishes the first image coordinate system to determine the material grasping position. The second camera module is used to extract the spatial vector of the material and calculate the image optimization factor based on the eigenvalue to ensure that the robotic arm can grasp the material in the correct posture. The power module provides power for the robotic arm to ensure that it can accurately grasp the material. The second calculation module is used to establish the second image coordinate system for the second three-dimensional image, extract the coordinates of the grasping point, and calculate the axial coefficients of the three axes according to the analysis vector to provide data support for subsequent adjustments. The optimization module optimizes the coordinates of the grasping point to obtain the loading coordinates, improving the grasping accuracy of the robotic arm and ensuring the efficient and stable operation of the system. Thus, efficient and precise grasping of the material is achieved, and the automation of the loading process is realized.

[0125] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the specified functions in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0127] These computer program instructions can also be stored in a computer-readable storage device that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable storage device produce a manufacture including an instruction device that implements the functions specified in one or more of the processes Figure 1 one or more processes and / or blocks Figure 1 specified in the block or blocks.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the processes Figure 1 one or more processes and / or blocks Figure 1 specified in the block or blocks.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A feeding control method based on a robotic arm, characterized in that: include: Establishing a first three-dimensional image of the environment of the robot arm, extracting a motion path feature vector according to the first three-dimensional image, and determining a target position of the material according to the motion path feature vector; When the robot arm reaches the target position of the material, the first three-dimensional image is preprocessed and a first coordinate system of the image is established, a characteristic value of the material is calculated based on the first coordinate system of the image, the characteristic value is similarly processed with the characteristic value in the historical set, and a material grabbing position is determined according to the similarity; When calculating the characteristic value of the material based on the first coordinate system of the image, it includes: Extracting the three-dimensional coordinates of the target position of the material from the first coordinate system of the image, and respectively counting all x-axis coordinates, y-axis coordinates and z-axis coordinates in the three-dimensional coordinates; Counting the number of all coordinates in the three-dimensional coordinates, and calculating the x-coordinate feature quantity, y-coordinate feature quantity and z-coordinate feature quantity of the material; The first weight coefficient is obtained by taking the absolute values ​​of the differences between all the x-axis coordinates and the x-coordinate feature quantity and adding them, the second weight coefficient is obtained by taking the absolute values ​​of the differences between all the y-axis coordinates and the y-coordinate feature quantity and adding them, and the third weight coefficient is obtained by taking the absolute values ​​of the differences between all the z-axis coordinates and the z-coordinate feature quantity and adding them; Calculating the feature value based on the x-coordinate feature quantity, the y-coordinate feature quantity, the z-coordinate feature quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient; When the feature value is calculated according to the x-coordinate feature quantity, the y-coordinate feature quantity, the z-coordinate feature quantity, the first weight coefficient, the second weight coefficient, and the third weight coefficient, the method includes: The characteristic value is calculated according to the following formula: ； Wherein, V represents the characteristic value, w1 represents the first weight coefficient, w2 represents the second weight coefficient, w3 represents the third weight coefficient, x represents the x-coordinate characteristic quantity, y represents the y-coordinate characteristic quantity, and z represents the z-coordinate characteristic quantity; Establishing a second three-dimensional image, extracting a spatial vector of a material according to the second three-dimensional image, extracting an image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor to obtain a target vector of the material, and establishing a rotation matrix of the material based on the target vector; The robotic arm grabs the material according to the material grabbing position, the target vector and the rotation matrix; When grasping a material, a second image coordinate system is established according to the second three-dimensional image, and the coordinates of the grasping point where the material and the robot arm are in contact are extracted according to the second image coordinate system; Acquire an analysis vector of the material grabbing position according to the first three-dimensional image, and obtain a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; Calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient and the third axial coefficient, optimize the grabbing point coordinates according to the image coordinate optimization factor, and obtain the loading coordinates; When extracting the spatial vector of the material according to the second three-dimensional image, extracting the image optimization factor based on the eigenvalue, optimizing the spatial vector according to the image optimization factor, and obtaining the target vector of the material, the method includes: Collecting initial point cloud data of the second three-dimensional image, and performing refinement processing on the initial point cloud data to obtain terminal point cloud data; Extracting characteristic points of the material according to the terminal point cloud data, and performing local fitting on the characteristic points to obtain the space vector; The target vector is obtained according to the following formula: ； ； in, represents the image optimization factor, V represents the eigenvalue, K represents the space vector, and Z represents the target vector; When the image coordinate optimization factor is calculated according to the first axial coefficient, the second axial coefficient and the third axial coefficient, the grabbing point coordinates are optimized according to the image coordinate optimization factor to obtain the loading coordinates, the method includes: The image coordinate optimization factor is obtained according to the following formula: ； Wherein, M represents the image coordinate optimization factor, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, α represents the weight of the first axial coefficient, β represents the weight of the second axial coefficient, γ represents the weight of the third axial coefficient, and α+β+γ=1.5; The loading coordinates are the product of the grabbing point coordinates and the image coordinate optimization factor.

2. The feeding control method based on a robotic arm according to claim 1 is characterized in that: The characteristic value and the characteristic value in the history set are similarly processed, and the material grabbing position is determined according to the similarity, including: The number of feature values ​​in the history set that are the same as the feature value is counted, and the similarity is calculated according to the following formula: ； Wherein, S represents the similarity, V1 represents the number of feature values ​​in the history set that are the same as the feature value, and V2 represents the number of all feature values ​​in the history set; When S is greater than or equal to 70%, the historical material grab position is used; When S is less than 70%, the material grabbing position is preset based on the first coordinate system of the image; The material grabbing position includes the historical material grabbing position and the preset material grabbing position.

3. The feeding control method based on a robotic arm according to claim 2 is characterized in that: When establishing the rotation matrix of the material based on the target vector, it includes: The rotation matrix is ​​established according to the following formula: ； ； ； ； Wherein, θ represents the rotation angle, Z represents the target vector, |Z| represents the modulus of the target vector, r represents the rotation axis vector in the first coordinate system of the image, and r x represents the component of the rotation axis in the x-axis direction, r y represents the component of the rotation axis in the y-axis direction, r z represents the component of the rotation axis in the z-axis direction, [r] represents the antisymmetric matrix of the rotation axis vector r, and R represents the rotation matrix.

4. The feeding control method based on a robotic arm according to claim 1, characterized in that: When an analysis vector of the material grabbing position is acquired according to the first three-dimensional image, and a first axial coefficient, a second axial coefficient and a third axial coefficient are obtained according to the analysis vector, the method includes: acquiring the analysis vector according to the first three-dimensional image; Count the first vector parameters of all x-axes of the analysis vector, count the second vector parameters of all y-axes of the analysis vector, count the third vector parameters of all z-axes of the analysis vector, calculate the first axial coefficient according to the first vector parameters, calculate the second axial coefficient according to the second vector parameters, and calculate the third axial coefficient according to the third vector parameters.

5. The feeding control method based on a robot arm according to claim 4 is characterized in that: When calculating the first axial coefficient according to the first vector parameter, calculating the second axial coefficient according to the second vector parameter, and calculating the third axial coefficient according to the third vector parameter, it includes: ； ； ； Wherein, A represents the first axial coefficient, B represents the second axial coefficient, C represents the third axial coefficient, n1 represents the number of the first vector parameters containing the x-axis in the analysis vector, n2 represents the number of the second vector parameters containing the y-axis in the analysis vector, n3 represents the number of the third vector parameters containing the z-axis in the analysis vector, P i represents any one of the first vector parameters, Q i represents any one of the second vector parameters, L i Represents any one of the third vector parameters.

6. A feeding control system based on a robot arm, applied to the feeding control method based on a robot arm as claimed in any one of claims 1 to 5, characterized in that: include: A first camera module is used to create a first three-dimensional image of the environment of the robot arm, extract a motion path feature vector according to the first three-dimensional image, and determine a material target position according to the motion path feature vector; A first calculation module, when the robot arm reaches the target position of the material, is used to pre-process the first three-dimensional image and establish a first coordinate system of the image, calculate the characteristic value of the material based on the first coordinate system of the image, perform similarity processing on the characteristic value and the characteristic value in the historical set, and determine the material grabbing position according to the similarity; A second camera module is used to establish a second three-dimensional image, extract a spatial vector of a material according to the second three-dimensional image, extract an image optimization factor based on the eigenvalue, optimize the spatial vector according to the image optimization factor, obtain a target vector of the material, and establish a rotation matrix of the material based on the target vector; A power module, used for providing power to the robot arm when it reaches the target position of the material and when the robot arm grabs the material according to the material grabbing position, the target vector and the rotation matrix; A second calculation module, when grabbing a material, is used to establish a second image coordinate system according to the second three-dimensional image, extract the coordinates of the grabbing point where the material and the robotic arm are in contact according to the second image coordinate system, obtain an analysis vector of the material grabbing position according to the first three-dimensional image, and derive a first axial coefficient, a second axial coefficient, and a third axial coefficient according to the analysis vector; The optimization module is used to calculate the image coordinate optimization factor according to the first axial coefficient, the second axial coefficient and the third axial coefficient, and optimize the grabbing point coordinates according to the image coordinate optimization factor to obtain the loading coordinates.

Images