An intelligent control method, system and medium for a robotic arm based on machine vision

Abstract

The present invention relates to the field of robot control technology, and specifically relates to an intelligent control method, system and medium for a robotic arm based on machine vision. The method includes: acquiring a sequence of surface images when the robotic arm welds a sequence of welding areas of a welding target according to a preset control point sequence; generating a sequence of solder joint areas based on the sequence of surface images, and calculating a sequence of defocus deviations based on the sequence of welding areas and the sequence of solder joint areas; generating a sequence of solder joint centers based on the sequence of solder joint areas, generating a sequence of center offsets based on the sequence of control points and the sequence of solder joint centers, and generating a sequence of angular deviation values based on the sequence of solder joint areas and the sequence of solder joint centers; determining a path tracking error based on the sequence of defocus deviations, the sequence of angular deviation values and the sequence of center offsets, adjusting the welding path of the robotic arm based on the path tracking error, and controlling the robotic arm to weld the welding target based on the adjusted welding path; the present invention can improve welding accuracy and stability.

Description

Technical Field

[0001] The present invention relates to the technical fields of machine vision and robot control, and particularly relates to an intelligent control method, system and medium for a robotic arm based on machine vision. Background Art

[0002] The intelligent control of robotic arms is crucial for processes such as component assembly and welding in vehicle manufacturing. With the integration of artificial intelligence and robot technology, robotic arms have stronger autonomous decision-making and adaptation capabilities. By integrating technologies such as vision systems and sensors, robotic arms can perceive and understand the working environment, automatically adjust working parameters and actions, and improve production efficiency and quality.

[0003] During the vehicle manufacturing process, the accuracy of robotic arm welding technology directly affects the quality of the final product. In related technologies, the vision system of the robotic arm needs to capture the welding position and posture in real time, and adjust the welding parameters through algorithm analysis to ensure that the weld seam is uniform and defect-free. Traditional methods rely on manual parameter setting, which is inefficient and error-prone. Therefore, it is necessary to provide a solution that can dynamically optimize welding parameters, improve welding accuracy and stability, reduce manual intervention, and achieve high-efficiency and high-quality automated production. Summary of the Invention

[0004] To solve the above problems, the present invention provides an intelligent control method, system and medium for a robotic arm based on machine vision, aiming to achieve automated intelligent control through machine vision and significantly improve welding accuracy and stability through adaptive welding compensation adjustment.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] On the one hand, an embodiment of the present invention provides an intelligent control method for a robotic arm based on machine vision, and the method includes the following steps:

[0007] Obtain a sequence of surface images when the robotic arm welds a sequence of welding areas of a welding target according to a preset sequence of control points;

[0008] Generate a sequence of solder joint areas based on the sequence of surface images, and calculate a sequence of defocus deviations based on the sequence of welding areas and the sequence of solder joint areas;

[0009] Generate a sequence of solder joint centers based on the sequence of solder joint areas, generate a sequence of center offsets based on the sequence of control points and the sequence of solder joint centers, and generate a sequence of angular deviation values based on the sequence of solder joint areas and the sequence of solder joint centers;

[0010] Determine the path tracking error based on the defocus deviation sequence, the angle deviation value sequence, and the center offset sequence, adjust the welding path of the robotic arm based on the path tracking error, and control the robotic arm to weld the welding target based on the adjusted welding path.

[0011] Optionally, the generating the solder joint area sequence based on the surface image sequence and calculating the defocus deviation sequence based on the welding area sequence and the solder joint area sequence includes:

[0012] Preprocess the surface images in the surface image sequence into grayscale images;

[0013] Use the Canny algorithm to perform edge detection on the grayscale image, identify the edge contour area in the grayscale image, traverse and search each pixel in the edge contour area, connect the pixels with the largest gradient magnitude to form an edge contour line, and use the area enclosed by the edge contour line as the solder joint area;

[0014] Calculate the area ratio of the solder joint area and the corresponding welding area to obtain a plurality of area ratios corresponding to the solder joint area sequence;

[0015] Arrange the area ratios of each solder joint area in order to form a defocus deviation sequence.

[0016] Optionally, the traversing and searching each pixel in the edge contour area and connecting the pixels with the largest gradient magnitude to form an edge contour line includes:

[0017] Determine the gradient value of each pixel in the edge contour area, calculate the variance of the gradient values of each pixel in the edge contour area to obtain the area variance;

[0018] Obtain the reference size of the sliding window, calculate the variance of the gradient values of each pixel in the sliding window to obtain the window variance; calculate the variance ratio of the window variance and the area variance, and adjust the reference size of the sliding window based on the variance ratio to obtain the size of the current sliding window;

[0019] Traverse the edge contour area using the sliding window, perform smoothing processing on each pixel in the sliding window using a two-dimensional Gaussian function, and construct a Hessian matrix;

[0020] Calculate the eigenvalues of the Hessian matrix, use the direction with the largest eigenvalue as the normal direction, and determine multiple auxiliary lines perpendicular to the normal direction within the sliding window;

[0021] Calculate the gradient of the pixels on the auxiliary lines, mark the pixel with the largest gradient value within the sliding window as an edge point, and fit the edge points on multiple auxiliary lines with a quadratic polynomial to form a local curve of the current sliding window;

[0022] After completing the traversal of the edge contour region, the local curves of each sliding window are connected to form an edge contour line.

[0023] Optionally, generating a solder joint center sequence based on the solder joint region sequence, and generating a center offset sequence based on the control point sequence and the solder joint center sequence, includes:

[0024] For each pixel point on the edge contour line of the solder joint region, calculate the first moment of the pixel point in the x direction and the y direction respectively;

[0025] Sum the pixel points of all edge contour lines to obtain the zero moment, and calculate the ratios of the first moments of the pixel point in the x direction and the y direction to the zero moment respectively to obtain the coordinates of the center point of the solder joint region;

[0026] Connect the center points of each solder joint region in the solder joint region sequence in sequence to form a solder joint center sequence;

[0027] Calculate the distance deviation between each solder joint center in the solder joint center sequence and the corresponding control point, divide the distance deviation by the radius of the welding region to obtain the center offset of the corresponding solder joint center, and arrange the center offsets of each solder joint center in order to form a center offset sequence.

[0028] Optionally, generating an angular deviation value sequence based on the solder joint region sequence and the solder joint center sequence, includes:

[0029] Select multiple pixel points at equal intervals on the edge contour line of the solder joint region, calculate the curvature of each pixel point respectively, and connect the two pixel points with the largest curvature to form a first straight line;

[0030] Determine a second straight line passing through the solder joint center and perpendicular to the first straight line, use the intersection point of the second straight line and the edge contour line as a feature point, and calculate the distance ratio from the intersection point of the first straight line and the second straight line to the two feature points respectively;

[0031] Calculate the angular deviation of the solder joint region according to the distance ratio, and form an angular deviation value sequence with the angular deviations of each solder joint region.

[0032] Optionally, determining the path tracking error based on the defocus deviation sequence, the angular deviation value sequence and the center offset sequence, and adjusting the welding path of the robotic arm based on the path tracking error, includes:

[0033] Obtain the step amount of the robotic arm in each solder joint region, use the difference between the step amount of the current solder joint region and the step amount of the previous solder joint region as the step difference of the current solder joint region, and arrange the step differences of each solder joint region in order to form a step deviation sequence; wherein, the step amount includes the vertical step amount in the depth direction, the angular step amount in the normal direction and the horizontal step amount in the plane direction;

[0034] Determine a ratio sequence based on the pose deviation sequence and the corresponding step deviation sequence, and determine the predicted ratio of the next solder joint area based on the ratio sequence; wherein, the pose deviation sequence includes a defocus deviation sequence, an angle deviation value sequence, and a center offset sequence, and the ratio sequence includes a defocus deviation ratio, an angle deviation ratio, and a center offset ratio;

[0035] Predict the path tracking error of the next solder joint area based on the pose deviation sequence, and determine the step adjustment amount based on the path tracking error and the predicted ratio;

[0036] Adjust the welding path of the robotic arm according to the step adjustment amount to obtain the target step amount of the robotic arm in the next solder joint area.

[0037] On the other hand, an embodiment of the present invention provides a robotic arm intelligent control system based on machine vision, including:

[0038] At least one processor;

[0039] At least one memory for storing at least one program;

[0040] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0041] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, in which a processor-executable program is stored, and the processor-executable program is used to execute the above method when executed by a processor.

[0042] The beneficial effects of the present invention are as follows: The present invention discloses a robotic arm intelligent control method, system and medium based on machine vision. By accurately calculating the geometric features of the solder joint area and combining multi-dimensional deviation sequences, the welding path is adjusted in real time, significantly improving the accuracy and stability of the robotic arm operation, optimizing the welding quality, reducing production errors, and realizing efficient and intelligent production. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of a robotic arm intelligent control method based on machine vision according to an embodiment of the present invention;

[0045] Figure 2It is a schematic structural diagram of an intelligent control system for a robotic arm based on machine vision according to an embodiment of the present invention. Detailed implementation manners

[0046] The following will clearly and completely describe the concept, specific structure and technical effects generated by the present invention disclosed in combination with embodiments and drawings, so as to fully understand the purpose, solution and effects of the present invention disclosed. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0047] Refer to Figure 1 , as Figure 1 shown is an intelligent control method for a robotic arm based on machine vision provided by an embodiment of the present invention. The method includes the following steps:

[0048] S100, acquiring a sequence of surface images when the robotic arm welds a sequence of welding areas of a welding target according to a preset control point sequence;

[0049] Specifically, before welding the welding target, a control point sequence of the welding target is set. The welding target can be two workpieces to be welded, and the surface of the workpiece can be a plane or a curved surface; the control point sequence contains a plurality of control points arranged in sequence. A control point refers to a point position used to indicate the robotic arm to weld the welding area. The robotic arm welds each control point in the control point sequence in turn, in order to form a solder joint area consistent with the welding area; during the welding process of the robotic arm, multiple frames of depth images containing welding points are continuously collected along the welding path of the robotic arm to form a sequence of surface images.

[0050] S200, generating a sequence of solder joint areas based on the sequence of surface images, and calculating a defocus deviation sequence based on the sequence of welding areas and the sequence of solder joint areas;

[0051] Specifically, the sequence of solder joint areas contains a plurality of solder joint areas. A solder joint area refers to the action area formed after welding according to the control points, indicating the area range formed after actual welding; during the welding process, the control point sequence can be regarded as a welding path of the robotic arm. Since it is difficult to ensure that the welding path of the robotic arm and the surface of the workpiece are absolutely parallel during the actual welding process, there is a deviation between the actual solder joints welded by the robotic arm and the control points. There is a defocus amount between the actual solder joints and the control points. After welding the workpiece according to the center of the solder joint, a solder joint area will be formed on the surface of the workpiece. The defocus amount deviation represents the difference between the defocus amount of the solder joint and the defocus amount of the control point, and the angle deviation value represents the difference between the welding direction of the robotic arm and the welding direction of the control point.

[0052] The defocus amount reflects the position deviation of the focus; the defocus amount is divided into positive defocus and negative defocus. Positive defocus means that the laser focus is above the workpiece surface, and the formed welding area is small. Negative defocus means that the laser focus is below the workpiece surface (inside the workpiece), with a larger penetration depth and a larger solder joint area. By comparing the sizes of the welding area and the solder joint area, the positive and negative and size of the defocus amount can be reflected, reflecting the deviation direction and degree of the solder joint in depth.

[0053] The angle deviation value reflects the deviation of the welding direction; due to the angle deviation, the solder joint area is deformed, the distances from the edge line to the center point are inconsistent, and there is a deviation from the welding area. This causes uneven stress in all directions.

[0054] S300, generating a solder joint center sequence based on the solder joint area sequence, generating a center offset sequence based on the control point sequence and the solder joint center sequence, and generating an angle deviation value sequence based on the solder joint area sequence and the solder joint center sequence;

[0055] Specifically, after calculating the center points of each solder joint area, multiple solder joint centers are obtained, and the multiple solder joint centers are formed into a solder joint center sequence according to the welding order. By performing image processing on the solder joint areas on the workpiece surface, solder joint centers approximate to the actual solder joints can be generated, and then the center offset sequence can be calculated based on the control point sequence and the solder joint center sequence.

[0056] S400, determining the path tracking error based on the defocus deviation sequence, the angle deviation value sequence, and the center offset sequence, adjusting the welding path of the robotic arm based on the path tracking error, and controlling the robotic arm to weld the welding target based on the adjusted welding path.

[0057] In the subsequent calibration of the subsequent welding path through the deviation influence index, the defocus amount and the center offset are reduced, and the angle deviation from the control point sequence is reduced to achieve an ideal welding effect.

[0058] In the embodiments provided by the present invention, the precise positioning of the solder joints is ensured through real-time data feedback, improving the welding quality. During the welding process, the temperature change of the weld seam is monitored in real time, and the welding parameters are automatically adjusted to obtain the optimal welding effect.

[0059] In some embodiments, in S200, the generating the solder joint area sequence based on the surface image sequence and calculating the defocus deviation sequence based on the welding area sequence and the solder joint area sequence includes:

[0060] S210, preprocessing the surface images in the surface image sequence into grayscale images;

[0061] Specifically, the preprocessing includes denoising and contrast enhancement. The solder joint area formed by welding will contain multiple circles of edge contour lines. The edge contour lines are segmented by using the Canny algorithm based on an adaptive threshold.

[0062] S220. Use the Canny algorithm to perform edge detection on the grayscale image, identify the edge contour area in the grayscale image, traverse and search each pixel in the edge contour area, connect the pixels with the largest gradient magnitude to form an edge contour line, and use the area enclosed by the edge contour line as the solder joint area.

[0063] S230. Calculate the area ratio of the solder joint area to the corresponding welding area to obtain multiple area ratios corresponding to the solder joint area sequence.

[0064] Specifically, the larger the area ratio, the larger the solder joint area is compared to the welding area, and the deeper the solder joint is. That is to say, there is a correlation between the area ratio and the defocus amount. Based on this relationship, the positive and negative and magnitude changes of the defocus amount can be reflected by the change of the area ratio.

[0065] S240. Arrange the area ratios of each solder joint area in order to form a defocus deviation sequence.

[0066] In some embodiments, in S220, the traversing and searching each pixel in the edge contour area and connecting the pixels with the largest gradient magnitude to form an edge contour line includes:

[0067] S221. Determine the gradient value of each pixel in the edge contour area, calculate the variance of the gradient values of each pixel in the edge contour area to obtain the area variance.

[0068] S222. Obtain the reference size of the sliding window, calculate the variance of the gradient values of each pixel in the sliding window to obtain the window variance; calculate the variance ratio of the window variance to the area variance, and adjust the reference size of the sliding window based on the variance ratio to obtain the size of the current sliding window.

[0069] S223. Traverse the edge contour area with the sliding window, perform smoothing processing on each pixel in the sliding window by using a two-dimensional Gaussian function, and construct a Hessian matrix.

[0070] S224. Calculate the eigenvalues of the Hessian matrix, use the direction with the largest eigenvalue as the normal direction, and determine multiple auxiliary lines perpendicular to the normal direction within the sliding window.

[0071] S225. Calculate the gradient of the pixels on the auxiliary lines, mark the pixel with the largest gradient value within the sliding window as an edge point, and use a quadratic polynomial to fit the edge points on multiple auxiliary lines into the local curve of the current sliding window.

[0072] S226, after completing the traversal of the edge contour region, connect the local curves of each sliding window to form an edge contour line.

[0073] Specifically, the reference size of the sliding window is preset according to the size of the welding area. The larger the variance ratio, the smaller the sliding window, so that details can be captured more precisely in areas with complex textures, reducing misidentification; conversely, in areas with simple textures, the sliding window is larger, improving the calculation efficiency and ensuring the coherence and accuracy of the edge contour line. The dynamic adjustment of the sliding window size not only optimizes the accuracy of edge detection but also enhances the adaptability of the overall algorithm, enabling the efficient and accurate extraction of edge contours in welding areas with different texture features, thereby providing a reliable data basis for subsequent solder joint depth analysis and defocus amount evaluation.

[0074] In some embodiments, in S300, generating a solder joint center sequence based on the solder joint area sequence and generating a center offset sequence based on the control point sequence and the solder joint center sequence includes:

[0075] S301, for each pixel point of the edge contour line of the solder joint area, calculate the first moment of the pixel point in the x direction and the y direction respectively;

[0076] S302, sum up the pixel points of all edge contour lines to obtain the zero moment, and calculate the ratios of the first moments of the pixel point in the x direction and the y direction to the zero moment respectively to obtain the coordinates of the center point of the solder joint area;

[0077] S303, sort the center points of each solder joint area in the solder joint area sequence in sequence to form a solder joint center sequence;

[0078] S304, calculate the distance deviation between each solder joint center in the solder joint center sequence and the corresponding control point, divide the distance deviation by the radius of the welding area to obtain the center offset of the corresponding solder joint center, and arrange the center offsets of each solder joint center in order to form a center offset sequence.

[0079] Specifically, analyze the distance deviation of the solder joint position through the center offset sequence, and combine the accuracy of the edge contour line to further optimize the welding process parameters, ensure the stable quality of the solder joints, and improve the automation level and production efficiency of the welding process.

[0080] In some embodiments, in S300, generating an angle deviation value sequence based on the solder joint area sequence and the solder joint center sequence includes:

[0081] S311, select multiple pixel points at equal intervals on the edge contour line of the solder joint area, and calculate the curvature of each pixel point respectively;

[0082] S312. Connect the two pixels with the maximum curvature to form a first straight line, and determine a second straight line passing through the center of the solder joint and perpendicular to the first straight line;

[0083] S313. Take the two intersection points of the second straight line and the edge contour line as feature points, and calculate the distance ratio from the intersection point of the first straight line and the second straight line to the two feature points respectively;

[0084] S314. Use the distance ratio as the angular deviation of the solder joint area, and arrange the angular deviations of each solder joint area in order to form an angular deviation value sequence.

[0085] It should be noted that in an ideal state, when the robotic arm is welding the workpiece directly, the obtained solder joint area is a perfect circle; if there is a deviation between the welding angle and the workpiece, the contour of the welding area will be a closed curve formed by splicing two semi-ellipses with inconsistent minor semi-axes; the angular direction extends from the longer minor semi-axis to the shorter minor semi-axis; the distance ratio of the two minor semi-axes can reflect the magnitude of the angular deviation; the larger the distance ratio, the greater the angular deviation. The angular deviation of each solder joint area can be used to evaluate the offset trend of the overall solder joint center, and then adjust the welding process parameters to optimize the welding quality.

[0086] In some embodiments, in S400, determining the path tracking error based on the defocus deviation sequence, the angular deviation value sequence, and the center offset amount sequence, and adjusting the welding path of the robotic arm based on the path tracking error includes:

[0087] S410. Obtain the step amount of the robotic arm in each solder joint area, take the difference between the step amount of the current solder joint area and the previous solder joint area as the step difference of the current solder joint area, and arrange the step differences of each solder joint area in order to form a step deviation sequence; wherein, the step amount includes the vertical step amount in the depth direction, the angular step amount in the normal direction, and the horizontal step amount in the plane direction;

[0088] Specifically, the step difference has a directionality. Taking the vertical step amount as an example, if the depth step difference value is positive, it means that the step amount gradually increases and the solder joint gradually deepens; otherwise, the solder joint gradually becomes shallower. The angular step amount and the horizontal step amount are similar.

[0089] S420. Determine a ratio sequence based on the pose deviation sequence and the corresponding step deviation sequence, and determine the predicted ratio of the next solder joint area based on the ratio sequence; wherein, the pose deviation sequence includes the defocus deviation sequence, the angular deviation value sequence, and the center offset amount sequence, and the ratio sequence includes the defocus deviation ratio, the angular deviation ratio, and the center offset amount ratio;

[0090] Specifically, after determining the pose deviation sequence and the corresponding step deviation sequence, calculate the ratio based on the ratio of the pose deviation to the step deviation. If the variance of the ratios in the ratio sequence is small, it is considered that the ratio is a fixed value, and the average value of each ratio is taken as the predicted ratio; if the variance is large, it indicates that the ratio is not fixed, and the most recent ratio is taken as the predicted ratio.

[0091] S430, predict the path tracking error of the next solder joint area based on the pose deviation sequence, and determine the step adjustment amount based on the path tracking error and the predicted ratio;

[0092] S440, adjust the welding path of the robotic arm according to the step adjustment amount to obtain the target step amount of the robotic arm in the next solder joint area.

[0093] Exemplarily, calculate the path deviation vector according to the angle deviation value sequence and the center offset amount sequence, and adjust the welding path of the robotic arm in combination with the target number of steps to ensure the accurate alignment of the solder joints, improve the welding accuracy and stability. Through real-time monitoring and dynamic adjustment, the system can effectively reduce the error accumulation during the welding process, significantly improve the consistency and reliability of the solder joints, and thus meet the high-precision welding requirements.

[0094] Reference Figure 2 , an embodiment of the present invention further provides an intelligent control system for a robotic arm based on machine vision, including:

[0095] At least one processor;

[0096] At least one memory for storing at least one program;

[0097] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0098] The content in the above method embodiments is applicable to this embodiment. The functions specifically implemented in this embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments, and will not be elaborated here.

[0099] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program that implements the above method when executed by a processor.

[0100] It can be understood that the content in the above method embodiments is applicable to this storage medium embodiment. The functions specifically implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those in the above method embodiments.

[0101] Although the description of the present disclosure has been rather detailed and has particularly described several of the described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but rather should be regarded as effectively covering the intended scope of the present disclosure by reference to the appended claims, considering the prior art to provide a broad interpretation of these claims. In addition, the present disclosure has been described above in terms of embodiments foreseeable by the inventors for the purpose of providing a useful description, and non-substantive changes to the present disclosure that are not currently foreseeable may still represent equivalent changes to the present disclosure.

Claims

1. A method for intelligent control of a robotic arm based on machine vision, characterized in that: The method comprises the following steps: Acquire a surface image sequence when the robot arm welds a welding area sequence of a welding target according to a preset control point sequence; Generate a welding spot area sequence based on the surface image sequence, and calculate a defocus deviation sequence based on the welding area sequence and the welding spot area sequence; Generate a solder joint center sequence based on the solder joint area sequence, generate a center offset sequence based on the control point sequence and the solder joint center sequence, and generate an angle deviation value sequence based on the solder joint area sequence and the solder joint center sequence; the angle deviation value reflects the deviation of the welding direction; Determining a path tracking error based on the defocus deviation sequence, the angle deviation value sequence, and the center offset sequence, adjusting a welding path of the robot arm based on the path tracking error, and controlling the robot arm to weld the welding target based on the adjusted welding path; The step of generating the angle deviation value sequence based on the solder joint area sequence and the solder joint center sequence includes: Selecting multiple pixel points at equal intervals on the edge contour line of the solder joint area, calculating the curvature of each pixel point respectively, and connecting the two pixel points with the largest curvature to form a first straight line; Determine a second straight line passing through the center of the weld spot and perpendicular to the first straight line, take the intersection of the second straight line and the edge contour line as a feature point, and calculate the distance ratios from the intersection of the first straight line and the second straight line to the two feature points respectively; The angle deviation of the solder joint area is calculated according to the distance ratio, and the angle deviations of each solder joint area are formed into an angle deviation value sequence.

2. The method according to claim 1, characterized in that The step of generating a welding spot area sequence based on the surface image sequence, and calculating a defocus deviation sequence based on the welding area sequence and the welding spot area sequence, comprises: Preprocessing the surface images in the surface image sequence into grayscale images; The Canny algorithm is used to perform edge detection on the grayscale image, the edge contour area in the grayscale image is identified, each pixel in the edge contour area is traversed and searched, the pixels with the largest gradient amplitude are connected to form an edge contour line, and the area contained in the edge contour line is used as the welding point area; Calculating an area ratio between the soldering point area and the corresponding welding area to obtain a plurality of area ratios corresponding to the soldering point area sequence; The area ratios of each solder joint area are arranged in order to form a defocus deviation sequence.

3. The method according to claim 2, characterized in that The step of searching through each pixel in the edge contour area and connecting pixels with the largest gradient amplitude to form an edge contour line includes: Determine the gradient value of each pixel in the edge contour area, calculate the variance of the gradient value of each pixel in the edge contour area, and obtain the regional variance; Obtaining a reference size of the sliding window, performing variance calculation on the gradient value of each pixel in the sliding window to obtain the window variance; calculating a variance ratio of the window variance to the regional variance, and adjusting the reference size of the sliding window based on the variance ratio to obtain the size of the current sliding window; A sliding window is used to traverse the edge contour area, and a two-dimensional Gaussian function is used to smooth each pixel in the sliding window to construct a Hessian matrix; Calculate the eigenvalues ​​of the Hessian matrix, take the direction with the largest eigenvalue as the normal direction, and determine a plurality of auxiliary lines perpendicular to the normal direction in the sliding window; Calculate the gradient of the pixels on the auxiliary lines, mark the pixels with the largest gradient value in the sliding window as edge points, and use quadratic polynomials to fit the edge points on multiple auxiliary lines into the local curve of the current sliding window; After completing the traversal of the edge contour area, the local curves of each sliding window are connected to form an edge contour line.

4. The method according to claim 1, characterized in that: The step of generating a solder joint center sequence based on the solder joint area sequence and generating a center offset sequence based on the control point sequence and the solder joint center sequence comprises: For each pixel point of the edge contour line of the solder joint area, the first-order moment of the pixel point in the x direction and the y direction is calculated respectively; The zero-order moment is obtained by summing up the pixel points of all edge contour lines, and the ratio of the first-order moment to the zero-order moment of the pixel point in the x-direction and the y-direction is calculated respectively to obtain the coordinates of the center point of the solder joint area; Connecting the center points of each solder joint area in the solder joint area sequence in sequence to form a solder joint center sequence; The distance deviation between each solder joint center and the corresponding control point in the solder joint center sequence is calculated, and the distance deviation is divided by the radius of the welding area to obtain the center offset of the corresponding solder joint center. The center offsets of each solder joint center are arranged in order to form a center offset sequence.

5. The method according to claim 1, characterized in that: The method of determining a path tracking error based on the defocus deviation sequence, the angle deviation value sequence and the center offset sequence, and adjusting the welding path of the robot arm based on the path tracking error comprises: Obtain the stepping amount of the robot arm in each solder joint area, take the difference between the stepping amount of the current solder joint area and the previous solder joint area as the stepping difference of the current solder joint area, and arrange the stepping difference of each solder joint area in sequence to form a stepping deviation sequence; wherein the stepping amount includes the vertical stepping amount in the depth direction, the angular stepping amount in the normal direction, and the horizontal stepping amount in the plane direction; Determine a ratio sequence based on a posture deviation sequence and a corresponding step deviation sequence, and determine a predicted ratio of a next solder joint area based on the ratio sequence; wherein the posture deviation sequence includes a defocus deviation sequence, an angle deviation value sequence, and a center offset sequence, and the ratio sequence includes a defocus deviation ratio, an angle deviation ratio, and a center offset ratio; Predicting a path tracking error of a next solder joint area based on the posture deviation sequence, and determining a step adjustment amount based on the path tracking error and the prediction ratio; The welding path of the robot arm is adjusted according to the step adjustment amount to obtain the target step amount of the robot arm in the next welding point area.

6. A robot arm intelligent control system based on machine vision, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 5.

7. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 5 when executed by the processor.

Images