Automatic deviation correction system and deviation correction method for welding seams of brazed cutting parts

Abstract

The invention discloses an automatic deviation correction system and a deviation correction method for the weld of a brazed and cut part, comprising: calibrating a camera; calibrating a laser sensor; image acquisition and weld detection; feature point extraction and weld positioning; contour line fitting; extraction of the weld center axis; and weld trajectory planning. The beneficial effect is that the automatic deviation correction system and the deviation correction method for the weld of a brazed and cut part, by capturing the original image and processing it through bilateral filtering, and then using the synergistic effect of a spatial distance weight function and a pixel value similarity weight function, effectively suppress the noise salt and pepper effect, sharpen the weld boundary features, and avoid boundary misjudgment caused by noise interference; boundary detection follows the convolution operation logic, and the horizontal and vertical convolution kernels interact deeply with the image matrix, thereby realizing quantized gradient amplitude and direction; by traversing the grayscale threshold combination and maximizing the between-class variance calculation, the image is binarized and the main contour of the weld is highlighted.

Description

Technical Field

[0001] The invention belongs to the technical field of welding, and in particular relates to an automatic deviation correction system and a deviation correction method for a weld seam of a brazed cut piece. Background Art

[0002] In the manufacturing industry, especially in the production process involving brazing, welding quality plays a vital role in the overall performance and reliability of the product. The quality of the weld of brazed cut parts directly affects key indicators such as connection strength, sealing and product service life. However, traditional welding methods often rely on manual operation or relatively simple automatic control. When faced with complex weld shapes and high-precision welding requirements, problems such as weld position deviation and unstable welding quality are prone to occur. Summary of the invention

[0003] In order to solve the problems raised in the above background technology, the present invention provides an automatic deviation correction system and deviation correction method for the weld seam of brazed cutting parts.

[0004] To achieve the above object, the present invention provides the following technical solution: An automatic deviation correction system for the weld of a brazed cutting part comprises:

[0005] A welding robot having at least one freely movable joint for performing welding tasks;

[0006] Industrial camera; installed on the robot or designated location to capture images of the weld area;

[0007] Laser sensor; installed at the end of the robot to monitor the weld position in real time;

[0008] Camera calibration module; used to calibrate industrial cameras;

[0009] Laser sensor calibration module; used to calibrate the laser sensor;

[0010] Image acquisition and weld detection module; used for image acquisition and weld detection;

[0011] Feature point extraction and weld positioning module; used to extract the outer contour of the weld from the welding area image, and perform contour fitting, extract the weld center axis and feature point detection based on the extracted outer contour;

[0012] Weld trajectory planning module; used for weld trajectory planning;

[0013] Real-time deviation correction control module; used to correct the weld position;

[0014] Experiment and optimization module; used to optimize the correction effect.

[0015] Further: The industrial camera adopts a high-resolution, low-distortion lens, and the lens has automatic focus and aperture adjustment functions. The autofocus function is based on an image clarity evaluation algorithm, and determines the focus position by analyzing the high-frequency components of a specific area in the image. The aperture adjustment function automatically adjusts the aperture size based on feedback information from the ambient light intensity sensor to ensure the appropriate amount of light entering.

[0016] Further: When acquiring the original image, the image acquisition and weld detection module can automatically adjust the camera's exposure time and gain parameters according to preset image quality evaluation indicators. The image quality evaluation indicators include a weighted comprehensive evaluation of image contrast, signal-to-noise ratio, and weld boundary clarity parameters.

[0017] An automatic deviation correction method for the weld of a brazed cut piece, used to implement the above system, comprises the following steps:

[0018] S1. Implement the camera calibration process and the laser sensor calibration process;

[0019] S2. Acquire images and detect welds in the weld area to generate weld images;

[0020] S3, extracting feature points in the weld image and determining the weld position;

[0021] S4. According to the feature point extraction results and the weld positioning status, contour fitting is performed to obtain the fitted contour line;

[0022] S5. Extract the weld center axis from the fitted contour line; plan the weld trajectory based on the recognition of the characteristic points of the starting end, the end end and the turning point of the weld center axis;

[0023] S6. Construct a real-time deviation correction control model to describe the deviation between the actual trajectory of the robot end effector and the preset trajectory; based on the deviation, perform deviation correction processing on the weld.

[0024] Further: Among them, the camera calibration process and the laser sensor calibration process, the method is:

[0025] For camera calibration, a multi-view imaging calibration technique based on a standard chessboard is used to achieve coordinate transformation from the image coordinate system to the camera coordinate system and from the camera coordinate system to the robot base coordinate system.

[0026] Prepare a standard checkerboard calibration pattern with a square length of l, and place the checkerboard in the field of view of the industrial camera; use the industrial camera to take n images of the checkerboard pattern from multiple angles and distances, and record the position and posture information of the industrial camera during each shot;

[0027] Based on the camera model: Use the calibrateCamera function to solve the camera intrinsic parameter f x 、f y 、c x 、c y ,γ;

[0028] Based on the known camera intrinsic parameters, the coordinates of the pixel point (u, v) in the image coordinate system are converted into the point coordinates (X c ,Y c ,Z c );

[0029] s is the scale factor; it is used in the camera model to convert length units in the physical world to pixels in the image plane.

[0030] uv represents the horizontal and vertical coordinates of the pixel in the image plane. The origin of the image coordinate system is usually located in the upper left corner of the image, with the u axis pointing rightward and the v axis pointing downward. During camera calibration, the various parameters of the camera are calculated using the pixel coordinates of the checkerboard intersections at different positions in the image.

[0031] X c Represents the horizontal coordinate of the chessboard in three-dimensional space, Y c Represents the vertical coordinate Z of the chessboard in three-dimensional space c Represents the vertical coordinate of the chessboard in three-dimensional space. The above coordinates are used to describe the position of the chessboard in real three-dimensional space. During the calibration process, since the size of the chessboard is known, the coordinates of the intersection of the chessboard in the world coordinate system can be accurately determined, which provides a reference for the calculation of the camera's intrinsic and extrinsic parameters. The camera calibration is achieved by establishing a connection between the pixel points in the image and the actual points in the world coordinate system.

[0032] f x 、f y Represent the focal length of the camera in the and directions respectively.

[0033] γ represents the tilt factor; the tilt factor is used to describe the non-orthogonality between the coordinate axes of the camera image plane. Ideally, the coordinate axes of the camera image plane are perpendicular to each other, but due to factors such as camera manufacturing errors or installation deviations, the coordinate axes may be tilted. The tilt factor is a parameter used to quantify the degree of tilt. During the camera calibration process, the tilt factor can better correct the image and improve the accuracy of the conversion between the image and the physical world coordinates.

[0034] c x 、c y Respectively represent the horizontal and vertical coordinates of the pixel at the center of the image in the image plane.

[0035] Among them, the conversion formula based on the inverse operation of the intrinsic parameter matrix to convert it into the point coordinates in the camera coordinate system is:

[0036] In the known camera coordinate system point coordinates (X c ,Y c ,Z c ), based on the external parameter relationship equation: Calculate the point coordinates (X r ,Y r ,Z r ), where R is the rotation matrix of the camera coordinate system point; T is the translation vector; both R and T are captured by the camera;

[0037] X r , Y r , Z r They respectively represent the horizontal, vertical and vertical coordinates of the pixel point in the camera coordinate system.

[0038] After the camera calibration is completed, the laser sensor is calibrated to calibrate the accuracy of the laser sensor's measurement value to the actual coordinates;

[0039] For the laser sensor calibration process, a calibration device is placed at a point with known coordinates, and the detection value of the laser sensor is recorded; the error correction parameter is calculated based on the detection value and the coordinate value of the known coordinate system; the method for calculating the error adjustment parameter is based on the formula:

[0040] Calculate the error function E; where (X mi ,Y mi ,Z mi ) are the coordinates of the known coordinate point, (X wi ,Y wi ,Z wi ) is the coordinate of the measuring point, n is the total number of measuring points; E is the overall deviation between the measured value and the actual value;

[0041] According to the calculated E, the parameters of the laser sensor are adjusted, and the laser sensor calibration is completed on the basis of the minimum E value.

[0042] Further: after the laser sensor is calibrated, image acquisition and weld detection are performed;

[0043] Among them, the image acquisition and weld detection methods are:

[0044] In a constant lighting environment, obtain the original image;

[0045] Based on bilateral filtering function: Calculate filtered pixel values ​​to remove noise and preserve weld boundary information;

[0046] Where i and j represent the horizontal and vertical coordinates of the pixel point in the image after filtering, respectively;

[0047] k and l represent the horizontal and vertical coordinates of the pixel points in the original image respectively;

[0048] Among them, the boundary detection method is: use the Sobel operator to perform boundary detection operation, use the cv2.Sobel function in the OpenCV library to solve the gradient amplitude of the image, and identify the boundary pixels by comparing the gradient amplitude; connect adjacent boundary pixels to extract the weld boundary.

[0049] g(i,j) is the pixel value at the coordinate (i,j) after filtering, f(k,l) is the pixel value at the coordinate k,l in the original image, w s (i,j,k,l) ​​is the weight function based on spatial distance, w r (i,j,k,l) ​​is a weight function based on the similarity of pixel values.

[0050] On the basis of removing noise and retaining weld boundary information, boundary detection is performed;

[0051] The boundary detection method is: apply the Sobel operator for boundary detection, use the cv2.Sobel function in OpenCV to calculate the gradient of the image in the horizontal and vertical directions, and the horizontal gradient G x and the vertical gradient G y Through the corresponding Sobel convolution kernel k x and k y The convolution operation is performed and the calculation formula is as follows:

[0052]

[0053] According to the formula Calculate the gradient amplitude G to highlight the weld boundary; where G represents the gradient amplitude; G x , G y Represent the gradient amplitude of the image in the horizontal and vertical directions respectively; f(i+m,j+n) represents the pixel value of the pixel at the coordinate (i+m,j+n); k represents half of the convolution kernel size; k x and k yRespectively represent the values ​​of the Sobel convolution kernel in the horizontal and vertical directions at the coordinates (m, n); i, j represent the horizontal and vertical coordinates of the pixel point in the image after filtering; m, n represent the horizontal and vertical offsets of the pixel point in the convolution kernel relative to the central pixel point; by calculating the gradient amplitude, a boundary image is obtained, in which the pixel value at the weld boundary is higher because the gray value change rate at the weld boundary is larger, while the pixel values ​​in other areas are lower. In this way, we can highlight and extract the weld boundary through methods such as threshold processing.

[0054] After the boundary detection is completed, the image is binarized; the image is converted into black and white to highlight the weld.

[0055] The image binarization method is: use Otsu's method to automatically calculate the binarization threshold, and calculate the inter-class variance by traversing all possible thresholds; the inter-class variance formula is:

[0056]

[0057] Where w0 and w1 are the ratios of foreground pixels and background pixels respectively; represents the inter-class variance; μ0 and μ1 are the average grayscale values ​​of the pixels; μ T is the total average gray value of the image; When the value is the maximum, it is determined as the optimal threshold, and the image is converted into black and white to highlight the weld area;

[0058] After the image binarization is completed, morphological processing is performed;

[0059] Based on image acquisition and weld detection, feature points are extracted and the weld position is determined; wherein, the method for extracting feature points and determining the weld position is:

[0060] Extract the outer contour lines of the weld from the welding area image; perform convolution operation with the image based on the Sobel operator to obtain the gradient amplitude of each pixel in the image.

[0061] By comparing the gradient amplitudes, the boundary pixels are marked; the adjacent boundary pixels are connected to extract the outer contour lines of the weld.

[0062] Further: The method of contour fitting is:

[0063] Extract points on the contour line and select the fitting equation according to the weld shape;

[0064] According to the fitting equation, the error function is constructed to obtain the curve describing the weld path;

[0065] Based on the fitted outer contour line, the central axis of the weld is extracted for fitting;

[0066] The method for calculating the center axis of the weld is as follows: according to the coordinate data of the fitted outer contour of the weld, the point with the smallest y coordinate under the corresponding x coordinate is determined to be the leftmost point and the point with the largest y coordinate is determined to be the rightmost point, the midpoint coordinates of the left and right points are calculated, and the midpoint coordinates obtained are connected to obtain the center axis;

[0067] Based on the acquired central axis, feature point detection is performed;

[0068] Among them, the feature point detection method is:

[0069] Detect the mutation points in the central axis image, which correspond to the starting point, end point and inflection point of the weld;

[0070] Get the point set (x1,y1) on the determined central axis...(x i ,y i ), calculate the slope k between adjacent points on the central axis ll ;

[0071] Calculate the slope change rate Δk based on the slope i , the slope change rate Δk i and the preset threshold value |Δk i |Compare;

[0072] If the slope change rate is greater than the threshold |Δk i |, the corresponding point is determined as the mutation point;

[0073] Based on the positional relationship of the mutation points, the mutation point located at the starting position of the central axis is determined as the starting point of the weld, the mutation point located at the ending position of the central axis is determined as the ending point of the weld, and the mutation point located in the bending direction of the central axis is determined as the inflection point of the weld, thereby completing the identification of the characteristic points of the starting point, the ending point and the inflection point of the weld;

[0074] On the basis of completing the identification of the characteristic points of the starting point, the end point and the turning point of the central axis of the weld, the weld trajectory is planned; the weld trajectory planning includes path planning and trajectory planning;

[0075] Among them, the method for planning the weld trajectory is:

[0076] The coordinates of the characteristic points of the weld are collected, including the starting point, the ending point, and the inflection point; the weld radius is fitted using the cubic spline interpolation method to clarify the specific morphological characteristics of each curve segment, and a smooth weld path fitting curve that matches the actual shape of the weld path is constructed; the coordinates of the points on the weld path fitting curve are input into the welding robot; the robot kinematic model is used to obtain the angle changes of each joint to achieve a smooth welding trajectory.

[0077] On the basis of trajectory planning, a real-time deviation correction control model is established to describe the deviation between the actual path of the robot end effector and the ideal path;

[0078] Among them, the operation method of building a real-time correction control model is:

[0079] During the welding process, the laser sensor continuously scans and monitors the position of the weld according to the set sampling frequency of collecting data every 10 milliseconds. Based on the reflection principle of laser, the robot joint angle is adjusted and corrected, and the weld position is corrected to the ideal set position.

[0080] Based on the completion of real-time deviation correction, experiments are carried out and the processing flow is optimized.

[0081] Among them, the experimental and optimization methods are:

[0082] According to the optimization algorithm: Calculate the deviation value e(t); where k p is the proportional gain, k z is the integral gain, u(t) is the output of the deviation controller, and T is the integral variable representing the change in the time interval [0, t]. By adjusting k p , k z , so that the e(t) value is minimized to optimize the correction effect;

[0083] A computer-readable storage medium stores program instructions therein. When the program instructions are executed by a processor, an automatic deviation correction method for a weld seam of a brazed cut part is provided.

[0084] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0085] 1. The automatic correction system and correction method for the weld of brazed cutting parts captures the original image and processes it through bilateral filtering, and then uses the synergy of the spatial distance weight function and the pixel value similarity weight function to effectively suppress the noise salt and pepper effect, sharpen the weld boundary features, and avoid boundary misjudgment caused by noise interference;

[0086] Boundary detection follows the convolution operation logic, and the horizontal and vertical convolution kernels interact deeply with the image matrix to achieve quantized gradient amplitude and direction. By traversing the grayscale threshold combination and maximizing the inter-class variance calculation, the image is converted into a binary value to highlight the main contour of the weld.

[0087] The morphological closing operation uses expansion and corrosion to operate in an orderly manner, adapting the set structural elements to repair the microscopic holes inside the weld, bridge the boundary discontinuities, and reshape the continuous and complete outer contour of the weld;

[0088] Compared with the prior art, the welding positioning accuracy is improved.

[0089] 2. The automatic correction system and correction method for the weld of brazed cutting parts use a standard checkerboard pattern to collect images from multiple angles, based on the camera imaging geometry model, and realize the accurate conversion of the image to the base coordinate system through the coordinate transformation matrix; the laser sensor verifies the measurement value with known coordinate points, and finely adjusts the parameters according to the error function to minimize the measurement error;

[0090] During the experimental optimization phase, the PI controller gain parameters were dynamically adjusted, and after multiple rounds of iterations, the system maintained optimal correction performance under multiple working conditions.

[0091] Compared with the prior art, the adaptability and stability of welding are improved.

[0092] 3. The automatic deviation correction system and deviation correction method for the weld of brazed cutting parts have improved positioning accuracy, enhanced deviation correction time, precise adaptation of trajectory planning, and closed-loop control of system calibration optimization;

[0093] Compared with the prior art, the welding defect rate is reduced and the welding efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0095] Figure 1 It is a schematic diagram of the process of the present invention;

[0096] Figure 2 It is a schematic diagram of the process in the present invention;

[0097] Figure 3 It is a schematic diagram of the process in the present invention; DETAILED DESCRIPTION

[0098] The present invention will now be further described in detail with reference to the accompanying drawings and embodiments. These drawings are simplified schematic diagrams that only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0099] Application Overview:

[0100] The traditional method of processing welds of brazed cut parts mainly relies on manual operation or simple automated equipment, which leads to the problem of insufficient positioning accuracy and the weld position is prone to deviation. At the same time, the efficiency of manual operation is relatively low, which is difficult to meet the needs of large-scale and high-efficiency production, making low processing efficiency another significant problem. In addition, in complex working conditions or multi-material welding, traditional methods are difficult to ensure the stability of welding quality, and the quality of welds is uneven, which further affects the overall quality of the product. Traditional methods lack intelligent control and monitoring systems, and cannot detect weld position and deviation in real time, and make real-time adjustments according to actual conditions, which limits the further improvement and intelligent development of welding technology.

[0101] Therefore, in view of the problems in these background technologies, this application aims to overcome the above difficulties and propose an innovative automatic correction system and method for the weld seam of brazed and cut parts. The system integrates a welding robot, an industrial camera and a laser sensor to build an intelligent control architecture.

[0102] In the camera calibration process, a standard chessboard pattern is used for multi-angle shooting, and professional algorithms are used to accurately solve the camera's internal and external parameters, so as to achieve accurate conversion between the image coordinate system and the robot base coordinate system, laying a solid foundation for the subsequent accurate use of image information. Laser sensor calibration verifies the measurement value at a known coordinate point and adjusts the parameters according to rigorous error calculation to ensure that its measurement data is accurate and reliable.

[0103] Image acquisition is carried out under stable lighting. After bilateral filtering, Sobel operator boundary detection, Otsu method binarization and morphological closed operation series processing, the outer contour of the weld is accurately extracted, noise and image quality interference are overcome, and the positioning accuracy reaches sub-millimeter level. On this basis, according to the weld shape fitting curve, the central axis and feature points are calculated, and the cubic spline interpolation method is used to plan a smooth welding path that fits the actual weld, and the robot kinematic model is used to drive the welding action.

[0104] During the entire welding process, the laser sensor monitors the weld position with high-frequency sampling, calculates the offset vector in real-time comparison with the ideal path, and quickly adjusts the robot joint angle with the PI controller to achieve millisecond-level correction response to ensure accurate and stable welding trajectory. The experimental and optimization process is based on scientific algorithms to dynamically adjust controller parameters. After iteration, the system adapts to multiple materials and multiple working conditions to ensure the best correction effect.

[0105] This application significantly improves the welding accuracy, efficiency and stability of brazed cutting parts by virtue of the above-mentioned systematic innovative design.

[0106] like Figure 1 An automatic deviation correction system for brazing a weld of a cut part includes:

[0107] A welding robot having at least one freely movable joint for performing welding tasks;

[0108] Industrial camera; installed on the robot or designated location to capture images of the weld area;

[0109] Laser sensor; installed at the end of the robot to monitor the weld position in real time;

[0110] Camera calibration module; used to calibrate industrial cameras;

[0111] Laser sensor calibration module; used to calibrate the laser sensor;

[0112] Image acquisition and weld detection module; used for image acquisition and weld detection;

[0113] Feature point extraction and weld positioning module; used to extract the outer contour of the weld from the welding area image, and perform contour fitting, extract the weld center axis and feature point detection based on the extracted outer contour;

[0114] Weld trajectory planning module; used for weld trajectory planning;

[0115] Real-time deviation correction control module; used to correct the weld position;

[0116] Experiment and optimization module; used to optimize the correction effect.

[0117] Further: The industrial camera adopts a high-resolution, low-distortion lens, and the lens has automatic focus and aperture adjustment functions. The autofocus function is based on an image clarity evaluation algorithm, and determines the focus position by analyzing the high-frequency components of a specific area in the image. The aperture adjustment function automatically adjusts the aperture size based on feedback information from the ambient light intensity sensor to ensure the appropriate amount of light entering.

[0118] Further: When acquiring the original image, the image acquisition and weld detection module can automatically adjust the camera's exposure time and gain parameters according to preset image quality evaluation indicators. The image quality evaluation indicators include a weighted comprehensive evaluation of image contrast, signal-to-noise ratio, and weld boundary clarity parameters.

[0119] An automatic deviation correction method for the weld of a brazed cut piece, used to implement the above system, comprises the following steps:

[0120] S1. Implement the camera calibration process and the laser sensor calibration process;

[0121] S2. Acquire images and detect welds in the weld area to generate weld images;

[0122] S3, extracting feature points in the weld image and determining the weld position;

[0123] S4. According to the feature point extraction results and the weld positioning status, contour fitting is performed to obtain the fitted contour line;

[0124] S5. Extract the weld center axis from the fitted contour line; plan the weld trajectory based on the recognition of the characteristic points of the starting end, the end end and the turning point of the weld center axis;

[0125] S6. Construct a real-time deviation correction control model to describe the deviation between the actual trajectory of the robot end effector and the preset trajectory; based on the deviation, perform deviation correction processing on the weld.

[0126] Further: Among them, the camera calibration process and the laser sensor calibration process, the method is:

[0127] For camera calibration, a multi-view imaging calibration technique based on a standard chessboard is used to achieve coordinate transformation from the image coordinate system to the camera coordinate system and from the camera coordinate system to the robot base coordinate system.

[0128] Prepare a standard checkerboard calibration pattern with a square length of l, and place the checkerboard in the field of view of the industrial camera; use the industrial camera to take n images of the checkerboard pattern from multiple angles and distances, and record the position and posture information of the industrial camera during each shot;

[0129] Based on the camera model: Use the calibrateCamera function to solve the camera intrinsic parameter f x 、f y 、c x 、c y ,γ;

[0130] Based on the known camera intrinsic parameters, the coordinates of the pixel point (u, v) in the image coordinate system are converted into the point coordinates (X c ,Y c ,Z c );

[0131] s is the scale factor; it is used in the camera model to convert length units in the physical world to pixels in the image plane.

[0132] uv represents the horizontal and vertical coordinates of the pixel in the image plane. The origin of the image coordinate system is usually located in the upper left corner of the image, with the u axis pointing rightward and the v axis pointing downward. During camera calibration, the various parameters of the camera are calculated using the pixel coordinates of the checkerboard intersections at different positions in the image.

[0133] X c Represents the horizontal coordinate of the chessboard in three-dimensional space, Y cRepresents the vertical coordinate Z of the chessboard in three-dimensional space c Represents the vertical coordinate of the chessboard in three-dimensional space. The above coordinates are used to describe the position of the chessboard in real three-dimensional space. During the calibration process, since the size of the chessboard is known, the coordinates of the intersection of the chessboard in the world coordinate system can be accurately determined, which provides a reference for the calculation of the camera's intrinsic and extrinsic parameters. The camera calibration is achieved by establishing a connection between the pixel points in the image and the actual points in the world coordinate system.

[0134] f x 、f y Represent the focal length of the camera in the and directions respectively.

[0135] γ represents the tilt factor; the tilt factor is used to describe the non-orthogonality between the coordinate axes of the camera image plane. Ideally, the coordinate axes of the camera image plane are perpendicular to each other, but due to factors such as camera manufacturing errors or installation deviations, the coordinate axes may be tilted. The tilt factor is a parameter used to quantify the degree of tilt. During the camera calibration process, the tilt factor can better correct the image and improve the accuracy of the conversion between the image and the physical world coordinates.

[0136] c x 、c y Respectively represent the horizontal and vertical coordinates of the pixel at the center of the image in the image plane.

[0137] Among them, the conversion formula based on the inverse operation of the intrinsic parameter matrix to convert it into the point coordinates in the camera coordinate system is:

[0138] In the known camera coordinate system point coordinates (X c ,Y c ,Z c ), based on the external parameter relationship equation: Calculate the point coordinates (X r ,Y r ,Z r ), where R is the rotation matrix of the camera coordinate system point; T is the translation vector; both R and T are captured by the camera;

[0139] X r , Y r , Z r They respectively represent the horizontal, vertical and vertical coordinates of the pixel point in the camera coordinate system.

[0140] After the camera calibration is completed, the laser sensor is calibrated to calibrate the accuracy of the laser sensor's measurement value to the actual coordinates;

[0141] For the laser sensor calibration process, a calibration device is placed at a point with known coordinates, and the detection value of the laser sensor is recorded; the error correction parameter is calculated based on the detection value and the coordinate value of the known coordinate system; the method for calculating the error adjustment parameter is based on the formula:

[0142] Calculate the error function E; where (X mi ,Y mi ,Z mi ) are the coordinates of the known coordinate point, (X wi ,Y wi ,Z wi ) is the coordinate of the measuring point, n is the total number of measuring points; E is the overall deviation between the measured value and the actual value;

[0143] According to the calculated E, the parameters of the laser sensor are adjusted, and the laser sensor calibration is completed on the basis of the minimum E value.

[0144] Further: after the laser sensor is calibrated, image acquisition and weld detection are performed;

[0145] Among them, the image acquisition and weld detection methods are:

[0146] In a constant lighting environment, obtain the original image;

[0147] Based on bilateral filtering function: Calculate the filtered pixel values ​​to remove noise and preserve the weld edge information;

[0148] Where i and j represent the horizontal and vertical coordinates of the pixel point in the image after filtering, respectively;

[0149] k and l represent the horizontal and vertical coordinates of the pixel points in the original image respectively;

[0150] Among them, the boundary detection method is: use the Sobel operator to perform boundary detection operation, use the cv2.Sobel function in the OpenCV library to solve the gradient amplitude of the image, and identify the boundary pixels by comparing the gradient amplitude; connect adjacent boundary pixels to extract the weld boundary.

[0151] g(i,j) is the pixel value at the coordinate (i,j) after filtering, f(k,l) is the pixel value at the coordinate k,l in the original image, w s (i,j,k,l) ​​is the weight function based on spatial distance, w r (i,j,k,l) ​​is a weight function based on the similarity of pixel values.

[0152] On the basis of removing noise and retaining the weld edge information, edge detection is performed;

[0153] The edge detection method is: apply the Sobel operator for edge detection, use the cv2.Sobel function in OpenCV to calculate the gradient of the image in the horizontal and vertical directions, and the horizontal gradient G x and the vertical gradient G y Through the corresponding Sobel convolution kernel k x and k y The convolution operation is performed and the calculation formula is as follows:

[0154]

[0155] According to the formula Calculate the gradient amplitude G to highlight the edge of the weld; where G represents the gradient amplitude; G x , G y Represent the gradient amplitude of the image in the horizontal and vertical directions respectively; f(i+m,j+n) represents the pixel value of the pixel at the coordinate (i+m,j+n); k represents half of the convolution kernel size; k x and k y Respectively represent the values ​​of the Sobel convolution kernel in the horizontal and vertical directions at the coordinates (m, n); i, j represent the horizontal and vertical coordinates of the pixel point in the image after filtering; m, n represent the horizontal and vertical offsets of the pixel point in the convolution kernel relative to the central pixel point; by calculating the gradient amplitude, an edge image is obtained, in which the pixel value at the edge of the weld is higher because the gray value change rate at the edge of the weld is larger, while the pixel values ​​in other areas are lower. In this way, we can highlight and extract the weld edge through methods such as threshold processing.

[0156] After edge detection is completed, image binarization is performed; the image is converted into black and white to highlight the weld.

[0157] The image binarization method is: use Otsu's method to automatically calculate the binarization threshold, and calculate the inter-class variance by traversing all possible thresholds; the inter-class variance formula is:

[0158]

[0159] Where w0 and w1 are the ratios of foreground pixels and background pixels respectively; represents the inter-class variance; μ0 and μ1 are the average grayscale values ​​of the pixels; μ T is the total average gray value of the image; When the value is the maximum, it is determined as the optimal threshold, and the image is converted into black and white to highlight the weld area;

[0160] After the image binarization is completed, morphological processing is performed;

[0161] Based on image acquisition and weld detection, feature points are extracted and the weld position is determined; wherein, the method for extracting feature points and determining the weld position is:

[0162] Extract the outer contour lines of the weld from the welding area image; perform convolution operation with the image based on the Sobel operator to obtain the gradient amplitude of each pixel in the image.

[0163] By comparing the gradient amplitudes, the boundary pixels are marked; the adjacent boundary pixels are connected to extract the outer contour lines of the weld.

[0164] Further: The method of contour fitting is:

[0165] Extract points on the contour line and select the fitting equation according to the weld shape;

[0166] According to the fitting equation, the error function is constructed to obtain the curve describing the weld path;

[0167] Based on the fitted outer contour line, the central axis of the weld is extracted for fitting;

[0168] The method for calculating the center axis of the weld is as follows: according to the coordinate data of the fitted outer contour of the weld, the point with the smallest y coordinate under the corresponding x coordinate is determined to be the leftmost point and the point with the largest y coordinate is determined to be the rightmost point, the midpoint coordinates of the left and right points are calculated, and the midpoint coordinates obtained are connected to obtain the center axis;

[0169] Based on the acquired central axis, feature point detection is performed;

[0170] Among them, the feature point detection method is:

[0171] Detect the mutation points in the central axis image, which correspond to the starting point, end point and inflection point of the weld;

[0172] Get the point set (x1,y1) on the determined central axis...(x i ,y i ), calculate the slope k between adjacent points on the central axis ll ;

[0173] Calculate the slope change rate Δk based on the slope i , the slope change rate Δk i and the preset threshold value |Δk i |Compare;

[0174] If the slope change rate is greater than the threshold |Δk i |, the corresponding point is determined as the mutation point;

[0175] Based on the positional relationship of the mutation points, the mutation point located at the starting position of the central axis is determined as the starting point of the weld, the mutation point located at the ending position of the central axis is determined as the ending point of the weld, and the mutation point located in the bending direction of the central axis is determined as the inflection point of the weld, thereby completing the identification of the characteristic points of the starting point, the ending point and the inflection point of the weld;

[0176] On the basis of completing the identification of the characteristic points of the starting point, the end point and the turning point of the central axis of the weld, the weld trajectory is planned; the weld trajectory planning includes path planning and trajectory planning;

[0177] Among them, the method for planning the weld trajectory is:

[0178] The coordinates of the characteristic points of the weld are collected, including the starting point, the ending point, and the inflection point; the weld radius is fitted using the cubic spline interpolation method to clarify the specific morphological characteristics of each curve segment, and a smooth weld path fitting curve that matches the actual shape of the weld path is constructed; the coordinates of the points on the weld path fitting curve are input into the welding robot; the robot kinematic model is used to obtain the angle changes of each joint to achieve a smooth welding trajectory.

[0179] On the basis of trajectory planning, a real-time deviation correction control model is established to describe the deviation between the actual path of the robot end effector and the ideal path;

[0180] Among them, the operation method of building a real-time correction control model is:

[0181] During the welding process, the laser sensor continuously scans and monitors the position of the weld according to the set sampling frequency of collecting data every 10 milliseconds. Based on the reflection principle of laser, the robot joint angle is adjusted and corrected, and the weld position is corrected to the ideal set position.

[0182] Based on the completion of real-time deviation correction, experiments are carried out and the processing flow is optimized.

[0183] Among them, the experimental and optimization methods are:

[0184] According to the optimization algorithm: Calculate the deviation value e(t); where k p is the proportional gain, k z is the integral gain, u(t) is the output of the deviation controller, and T is the integral variable representing the change in the time interval [0, t]. By adjusting k p , k z , so that the e(t) value is minimized to optimize the correction effect;

[0185] A computer-readable storage medium stores program instructions therein. When the program instructions are executed by a processor, an automatic deviation correction method for a weld seam of a brazed cut part is provided.

[0186] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes the steps described in the above-mentioned method part of this specification according to various embodiments of the present application.

[0187] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0188] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps described in the above method section of this specification according to various embodiments of the present application.

[0189] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0190] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0191] In the description of this specification, the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" refer to the specific features, structures, materials or characteristics described in combination with the embodiment or example being included in at least one embodiment or example of the present invention.

[0192] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An automatic deviation correction system for brazing and cutting welds, characterized in that: The system comprises: A welding robot having at least one freely movable joint for performing welding tasks; Industrial camera; installed on the robot or designated location to capture images of the weld area; Laser sensor; installed at the end of the robot to monitor the weld position in real time; Camera calibration module; used to calibrate industrial cameras; Laser sensor calibration module; used to calibrate the laser sensor; Image acquisition and weld detection module; used for image acquisition and weld detection; Feature point extraction and weld positioning module; used to extract the outer contour of the weld from the welding area image, and perform contour fitting, extract the weld center axis and feature point detection based on the extracted outer contour; Weld trajectory planning module; used for weld trajectory planning; Real-time deviation correction control module; used to correct the weld position; Experiment and optimization module; used to optimize the correction effect.

2. The automatic deviation correction system for the weld of brazed cut parts according to claim 1, characterized in that: The industrial camera uses a high-resolution, low-distortion lens, and the lens has automatic focus and aperture adjustment functions. The autofocus function is based on an image clarity evaluation algorithm and determines the focus position by analyzing the high-frequency components of a specific area in the image. The aperture adjustment function automatically adjusts the aperture size based on feedback from the ambient light intensity sensor to ensure an appropriate amount of light entering.

3. The automatic deviation correction system for the weld of brazed cut parts according to claim 1, characterized in that: When acquiring the original image, the image acquisition and weld detection module can automatically adjust the camera's exposure time and gain parameters according to preset image quality evaluation indicators. The image quality evaluation indicators include a weighted comprehensive evaluation of image contrast, signal-to-noise ratio, and weld boundary clarity parameters.

4. An automatic deviation correction method for brazing and cutting welds, used to implement the system of claim 1, characterized in that: The following steps are involved: S1, implement the camera calibration process and the laser sensor calibration process; S2. Acquire images and detect welds in the weld area to generate weld images; S3, extracting feature points in the weld image and determining the weld position; S4. According to the feature point extraction results and the weld positioning status, contour fitting is performed to obtain the fitted contour line; S5. Extract the weld center axis from the fitted contour line; plan the weld trajectory based on the recognition of the characteristic points of the starting end, the end end and the turning point of the weld center axis; S6. Construct a real-time deviation correction control model to describe the deviation between the actual trajectory of the robot end effector and the preset trajectory; based on the deviation, perform deviation correction processing on the weld.

5. The automatic deviation correction method for the weld of a brazed cut part according to claim 4, characterized in that: in, The camera calibration process and laser sensor calibration process are as follows: For camera calibration, a multi-view imaging calibration technique based on a standard chessboard is used to achieve coordinate transformation from the image coordinate system to the camera coordinate system and from the camera coordinate system to the robot base coordinate system. For the laser sensor calibration process, a calibration device is placed at a point with known coordinates, and the detection value of the laser sensor is recorded; the error correction parameter is calculated based on the detection value and the coordinate value of the known coordinate system; the method for calculating the error adjustment parameter is based on the formula: Calculate the error function E; where (X mi ,Y mi ,Z mi ) are the coordinates of the known coordinate point, (X wi ,Y wi ,Z wi ) is the coordinate of the measuring point, n is the total number of measuring points; E is the overall deviation between the measured value and the actual value; According to the calculated E, the parameters of the laser sensor are adjusted, and the laser sensor calibration is completed on the basis of the minimum E value.

6. The automatic deviation correction method for the weld of a brazed cut part according to claim 4, characterized in that: in, The image acquisition and weld detection methods are: In a constant lighting environment, the original image is obtained; the noise is filtered out and the weld boundary information is retained by using the bilateral filtering algorithm for the original image, and boundary detection is performed based on the weld boundary information; The boundary detection method is as follows: using the Sobel operator to perform boundary detection operations, using the cv2.Sobel function in the Open CV library to solve the gradient amplitude of the image, and identifying the boundary pixels by comparing the gradient amplitudes; connecting adjacent boundary pixels to extract the weld boundary; After the boundary detection is completed, image binarization and morphological processing are performed; Among them, the method of extracting feature points and determining the weld position is: Extract the outer contour of the weld from the welding area image; perform convolution operation with the image based on the Sobel operator to obtain the gradient amplitude of each pixel in the image; By comparing the gradient amplitudes, the boundary pixels are marked; the adjacent boundary pixels are connected to extract the outer contour lines of the weld.

7. The automatic deviation correction method for the weld of a brazed cut part according to claim 4, characterized in that: The method of contour fitting is: Extract points on the contour line and select the fitting equation according to the weld shape; According to the fitting equation, the error function is constructed to obtain the curve describing the weld path; Based on the fitted outer contour line, the central axis of the weld is extracted for fitting; The method for calculating the center axis of the weld is as follows: according to the coordinate data of the fitted outer contour of the weld, the point with the smallest coordinate under the corresponding coordinate is determined as the leftmost point and the point with the largest coordinate is determined as the rightmost point, the midpoint coordinates of the left and right points are calculated, and the midpoint coordinates obtained are connected to obtain the center axis; Based on the acquired central axis, feature point detection is performed; Among them, the feature point detection method is: The mutation points in the central axis image are detected, and the mutation points correspond to the starting point, the ending point and the inflection point of the weld; thereby completing the detection of the characteristic points of the starting point, the ending point and the inflection point of the weld.

8. The automatic deviation correction method for the weld of a brazed cut part according to claim 4, Features: Among them, the method for planning the weld trajectory is: Collect the coordinates of the characteristic points of the weld, including the starting point, the end point, and the inflection point; use the cubic spline interpolation method to fit the weld radius, clarify the specific morphological characteristics of each curve segment, and construct a smooth weld path fitting curve that matches the actual shape of the weld path; input the point coordinates on the weld path fitting curve into the welding robot; The robot kinematics model is used to obtain the angle changes of each joint to achieve a smooth welding trajectory.

9. The automatic deviation correction method for the weld of a brazed cut part according to claim 4, characterized in that: in, The operation method of building a real-time correction control model is: During the welding process, the laser sensor continuously scans and monitors the position of the weld according to the set sampling frequency of collecting data every 10 milliseconds. Based on the reflection principle of laser, the robot joint angle is adjusted and corrected, and the weld position is corrected to the ideal set position. Based on the completion of real-time deviation correction work, experiments are carried out and the processing flow is optimized.

10. A computer-readable storage medium having program instructions stored therein, characterized in that: When the program instructions are executed by the processor, an automatic deviation correction method for a weld seam of a brazed cut part is implemented as described in any one of claims 4 to 9.

Images