A welding point autonomous positioning method, device, equipment and medium

Abstract

The application discloses a welding point autonomous positioning method and device, equipment and medium, relates to the welding fume collection technical field, and is applied to the positioning equipment, including the monocular camera located above the welding surface, the mechanical arm located above the monocular camera and provided with the smoke suction disc, and comprises the following steps: the monocular camera collects a welding light arc picture, and the light arc center point pixel coordinates of the welding light arc picture are extracted based on a target welding light arc detection model; the monocular camera is a camera after camera calibration; the light arc center point pixel coordinates are converted into the actual coordinates of the welding point of the welding surface based on an affine matrix; the affine matrix is a matrix supporting the coordinate conversion between the pixel coordinates of the corner point and the actual coordinates of the corner point in the welding surface; the pixel coordinates of the corner point are obtained based on the growth of the chessboard corner point detection algorithm; the welding surface is obtained by splicing a plurality of chessboards of the same size covering different regions; and the mechanical arm is regulated based on the actual coordinates to suck the welding fume by using the smoke suction disc. The welding fume can be effectively sucked.

Description

Technical Field

[0001] This invention relates to the field of welding fume collection technology, and in particular to a method, device, equipment and medium for autonomous positioning of welding points. Background Technology

[0002] Currently, welding fumes are harmful smoke and particulate matter produced during the welding process. Long-term inhalation by workers may cause fume poisoning or pneumoconiosis.

[0003] Existing welding fume treatment methods still have significant limitations: top-mounted fume collectors are inefficient and lack flexibility, making them unsuitable for complex and changing welding environments; handheld devices are complex to operate, increasing the physical burden on workers and posing safety hazards; while portable welding fume purifiers are convenient, they are limited by their small processing capacity and high maintenance costs. Currently, large welding plants mainly use suction arm-type treatment solutions. Although these can achieve horizontal swinging and hovering collection of welding fumes, workers need to frequently adjust the suction arm position to meet different workpiece sizes and welding requirements, making the operation cumbersome and time-consuming, severely impacting work efficiency and operational smoothness. In summary, existing technologies struggle to effectively balance the core needs of collection efficiency and reducing worker burden, and their applicability in complex environments is limited, resulting in ineffective welding fume treatment.

[0004] In summary, how to effectively extract welding fumes is a problem that needs to be addressed. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for autonomous positioning of welding points, which can effectively absorb welding fumes. The specific solution is as follows:

[0006] In a first aspect, this application discloses an autonomous positioning method for welding points, applied to a positioning device. The positioning device includes a monocular camera positioned above the welding surface and having a field of view covering the welding surface, and a robotic arm positioned above the monocular camera and equipped with a fume extraction plate. The method includes:

[0007] The monocular camera is used to acquire welding arc images in real time, and the pixel coordinates of the arc center point in the welding arc image are extracted based on the pre-trained target welding arc detection model; the monocular camera is a calibrated camera obtained after camera calibration.

[0008] The pixel coordinates of the center point of the light arc are converted into the actual coordinates of the welding point located on the welding surface based on the affine matrix; the affine matrix is ​​a matrix that supports coordinate transformation between the pixel coordinates of corner points in the camera virtual plane and the actual coordinates of corner points in the welding surface; the pixel coordinates of the corner points are obtained based on the corner point detection algorithm of the growing checkerboard; the welding surface is obtained by splicing together several checkerboard grids of the same size covering different areas.

[0009] The robotic arm is adjusted based on the actual coordinates of the welding point to extract welding fumes using the fume extraction tray.

[0010] Optionally, the robotic arm includes a horizontally rotatable and retractable arm, a vertically retractable smoking arm connected to the arm, and a smoking tray connected to the smoking arm.

[0011] Optionally, before acquiring welding arc images in real time using the monocular camera, the method further includes:

[0012] The monocular camera is used to acquire several images of a calibration board of known size at different angles; the calibration board is a checkerboard with several corner points.

[0013] The camera parameters of the monocular camera are calculated based on the calibration board image; the camera parameters include an intrinsic parameter matrix, an extrinsic parameter matrix, and distortion coefficients.

[0014] The calibration accuracy is verified by the reprojection error method and corrected by the distortion model to obtain camera parameters that meet the accuracy requirements, thus completing the camera calibration and obtaining the calibrated camera.

[0015] Optionally, before converting the pixel coordinates of the light arc center point to the actual coordinates of the welding point located on the welding surface based on the affine matrix, the method further includes:

[0016] The coordinates of the first corner point of several corner points are determined, and the coordinates of the second corner point of several corner points are detected; the corner point is a corner point in the chessboard grid; the coordinates of the first corner point are the coordinates of the corner point in the chessboard grid coordinate system corresponding to the chessboard grid; the coordinates of the second corner point are the coordinates of the corner point in the camera virtual plane coordinate system.

[0017] Based on the first and second corner coordinates of each corner point, determine the coordinate transformation relationship between the pixel coordinates of the corner point in the planar coordinate system and the actual coordinates of the corner point in the welding surface coordinate system;

[0018] The affine matrix is ​​constructed based on the coordinate transformation relationship.

[0019] Optionally, detecting the coordinates of the second corner point of the plurality of corner points includes:

[0020] The coordinates of the second corner point are detected from a number of seed corner points; the seed corner points are corner points whose confidence scores meet the confidence requirements.

[0021] The coordinates of the second corner point of the aforementioned seed corner points are used as the base coordinates, and the coordinates of the second corner point corresponding to the next corner point are determined based on the base coordinates.

[0022] The coordinates of the second corner point corresponding to the next corner point are used as the new base coordinates, and the process jumps to the step of determining the coordinates of the second corner point corresponding to the next corner point based on the base coordinates, until the coordinates of the second corner point of all corner points in the chessboard are obtained.

[0023] Optionally, determining the coordinates of the second corner point corresponding to the next corner point based on the base coordinates includes:

[0024] The prediction method predicts the coordinates of the next corner point based on the prediction method and two adjacent base coordinates; the prediction method is that the first distance and the second distance between the two adjacent base coordinates are equal, and the angle of the target angle is a right angle; the second distance is the distance between the reference coordinate and the next corner point; the reference coordinate is the corner point that is closer to the next corner point among the two adjacent base coordinates; the target angle is the angle formed by the next corner point and the two adjacent base coordinates with the reference coordinate as the vertex;

[0025] The second corner coordinates of the predicted coordinates are accurately determined according to the sub-pixel method; the sub-pixel method represents the gray-level gradient of the vector between the second corner coordinates and the coordinates of the adjacent pixels, which is perpendicular to the camera virtual plane.

[0026] Optionally, before extracting the pixel coordinates of the arc center point located in the welding arc image based on the pre-trained target welding arc detection model, the method further includes:

[0027] The monocular camera is used to acquire images of the light arc to be trained, and the images of the light arc to be trained are processed into grayscale to obtain grayscale images;

[0028] The grayscale image is processed using a contrast-limited adaptive histogram equalization algorithm to obtain a processed image, and the boundary information of the welding arc region in the processed image is extracted using the Canny edge detection algorithm.

[0029] Based on the boundary information, the welding arc region is annotated to obtain an annotated image;

[0030] The target welding arc detection model is obtained by training an initial model using the labeled images.

[0031] Secondly, this application discloses an autonomous welding point positioning device, applied to a positioning equipment. The positioning equipment includes a monocular camera located above the welding surface and having a field of view covering the welding surface, and a robotic arm located above the monocular camera and equipped with a fume extraction plate. The device includes:

[0032] The coordinate detection module is used to acquire welding arc images in real time using the monocular camera, and extract the pixel coordinates of the arc center point located in the welding arc image based on the pre-trained target welding arc detection model; the monocular camera is a calibrated camera obtained after camera calibration.

[0033] The coordinate transformation module is used to convert the pixel coordinates of the center point of the light arc into the actual coordinates of the welding point located on the welding surface based on an affine matrix; the affine matrix is ​​a matrix that supports coordinate transformation between the pixel coordinates of corner points in the camera's virtual plane and the actual coordinates of corner points in the welding surface; the pixel coordinates of the corner points are obtained based on a growing checkerboard corner detection algorithm; the welding surface is obtained by splicing together several checkerboard grids of the same size covering different areas;

[0034] A welding fume extraction module is used to adjust the robotic arm based on the actual coordinates of the welding point to extract welding fumes using the fume extraction pan.

[0035] Thirdly, this application discloses an electronic device, including:

[0036] Memory, used to store computer programs;

[0037] A processor is used to execute the computer program to implement the aforementioned disclosed method for autonomous positioning of welding points.

[0038] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for autonomous positioning of welding points.

[0039] As can be seen, this application utilizes the monocular camera to acquire welding arc images in real time, and extracts the pixel coordinates of the arc center point located in the welding arc image based on a pre-trained target welding arc detection model; the monocular camera is a calibrated camera obtained after camera calibration; the pixel coordinates of the arc center point are converted into the actual coordinates of the welding point located on the welding surface based on an affine matrix; the affine matrix is ​​a matrix that supports coordinate transformation between the pixel coordinates of corner points in the camera's virtual plane and the actual coordinates of corner points in the welding surface; the pixel coordinates of the corner points are obtained based on a growing checkerboard corner detection algorithm; the welding surface is obtained by splicing several checkerboard grids of the same size covering different areas; the robotic arm is controlled based on the actual coordinates of the welding points to use the fume extraction tray to extract welding fumes. Therefore, this application combines the target welding arc detection model and the affine matrix to improve the welding point positioning accuracy. It utilizes a self-controlled robotic arm to extract welding fumes from the welding point, eliminating the need for manual intervention and achieving automated welding fume collection. The application uses the affine matrix between the welding surface and the camera's virtual plane to perform coordinate transformation from the camera's virtual plane coordinate system to the welding surface coordinate system—that is, a two-dimensional to two-dimensional coordinate transformation, rather than a three-dimensional to two-dimensional projection transformation. This treats the welding surface as a two-dimensional plane, solving the problem that a monocular camera cannot directly acquire depth information, simplifying the process, and improving accuracy and efficiency. Furthermore, this application uses a growing chessboard corner detection algorithm for affine transformation, overcoming the limitation of traditional chessboard corner detection methods that cannot simultaneously identify the coordinates of multiple chessboard corners, thus improving the accuracy of the affine matrix and enhancing the accuracy and stability of welding point positioning. As can be seen, this application can automatically extract welding fumes while ensuring extraction efficiency and accuracy. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0041] Figure 1 This is a flowchart of a welding point autonomous positioning method disclosed in this application;

[0042] Figure 2 This is a schematic diagram of an autonomous welding fume collection device disclosed in this application;

[0043] Figure 3 This is a schematic diagram of a traditional chessboard corner detection method that includes all chessboard squares, as disclosed in this application.

[0044] Figure 4 This is a schematic diagram of a growth-based chessboard corner detection method that includes all chessboard squares, as disclosed in this application.

[0045] Figure 5 This is a schematic diagram of a growth-based chessboard corner detection method for a single chessboard grid disclosed in this application;

[0046] Figure 6 This is a schematic diagram of an optimized affine transformation process disclosed in this application;

[0047] Figure 7 This is a practical diagram illustrating an optimized affine transformation process disclosed in this application;

[0048] Figure 8 This is a schematic diagram of an initial corner point parallel to the coordinate axis disclosed in this application;

[0049] Figure 9 This is a schematic diagram of an initial corner point at a 45° angle to the coordinate axis disclosed in this application;

[0050] Figure 10 This is a schematic diagram of a prototype containing two target corner points disclosed in this application;

[0051] Figure 11 This is a schematic diagram of a chessboard grid growth disclosed in this application;

[0052] Figure 12 This is a schematic diagram of the structure of a welding point autonomous positioning device disclosed in this application;

[0053] Figure 13 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Existing welding fume treatment methods still have significant limitations: top-mounted fume collectors are inefficient and lack flexibility, making them unsuitable for complex and changing welding environments; handheld devices are complex to operate, increasing the physical burden on workers and posing safety hazards; while portable welding fume purifiers are convenient, they are limited by their small processing capacity and high maintenance costs. Currently, large welding plants mainly use suction arm-type treatment solutions. Although these can achieve horizontal swinging and hovering collection of welding fumes, workers need to frequently adjust the suction arm position to meet different workpiece sizes and welding requirements, making the operation cumbersome and time-consuming, severely impacting work efficiency and operational smoothness. In summary, existing technologies struggle to effectively balance the core needs of collection efficiency and reducing worker burden, and their applicability in complex environments is limited, resulting in ineffective welding fume treatment.

[0056] Therefore, this application proposes an autonomous positioning scheme for welding points, which can effectively absorb welding fumes.

[0057] This application discloses an autonomous positioning method for welding points, applied to a positioning device. The positioning device includes a monocular camera positioned above the welding surface and covering the welding surface, and a robotic arm positioned above the monocular camera and equipped with a fume extraction device. See [link to relevant documentation]. Figure 1 As shown, the method includes:

[0058] Step S11: Use the monocular camera to acquire welding arc images in real time, and extract the pixel coordinates of the arc center point in the welding arc image based on the pre-trained target welding arc detection model; the monocular camera is the calibrated camera obtained after camera calibration.

[0059] In this embodiment, the robotic arm includes a horizontally rotatable and extendable support arm, a vertically extendable smoking arm connected to the support arm, and a smoking tray connected to the smoking arm. See details. Figure 2 The diagram shows a schematic of an autonomous welding fume collection device, consisting of a camera and a robotic arm. A monocular camera with a certain tilt angle is mounted on a side column above the welding area, its field of view covering the welding working plane. The robotic arm or electronic control device is mounted above the camera, possessing three degrees of freedom. It consists of a horizontally rotating and retractable lateral support arm and a vertically retractable fume extraction arm. The lateral support arm can flexibly adjust its working angle and horizontal range through horizontal rotation and extension, improving adaptability; the fume extraction arm can precisely adjust its height through vertical extension and retraction to optimize welding fume collection. This design enhances the functionality and flexibility of the device, making it suitable for complex working environments.

[0060] In this embodiment, before acquiring welding arc images in real time using the monocular camera, the method further includes: acquiring several calibration plate images at different angles corresponding to a calibration plate of known size using the monocular camera; the calibration plate is a checkerboard with several corner points; calculating the camera parameters of the monocular camera based on the calibration plate images; the camera parameters include an intrinsic parameter matrix, an extrinsic parameter matrix, and distortion coefficients; verifying the calibration accuracy using the reprojection error method and correcting it using a distortion model to obtain camera parameters that meet the accuracy requirements, thereby completing the calibration of the calibrated camera.

[0061] It is important to note that camera calibration is a fundamental step in achieving accurate image measurement and object detection in the field of computer vision. Its purpose is to eliminate various distortions and aberrations in the image by calculating and adjusting the camera's intrinsic and extrinsic parameters, thereby ensuring an accurate correspondence between the image and the 3D object. When shooting with an uncalibrated camera, especially with a wide-angle lens, the edges of the image often exhibit distortions such as bending, stretching, or shrinkage. This error affects the accuracy of the actual measurement results and the effectiveness of subsequent object detection applications. Therefore, camera calibration is a necessary prerequisite for overcoming these problems. The simplified steps of camera calibration are as follows:

[0062] First, image acquisition: Use a checkerboard of known size and specifications as the calibration board, ensuring that the checkerboard has multiple corner points for easy identification in the image. Take multiple images of the calibration board from different angles and distances to ensure coverage of different fields of view and directions.

[0063] Second, solve for the camera parameters: Extract the corner positions on the calibration board using a sub-pixel level corner detection algorithm. Then, using the Zhang Zhengyou calibration method, calculate the camera's intrinsic parameter matrix K, extrinsic parameter matrix [R|t], and distortion coefficient d based on the corner positions in the image and their corresponding spatial positions. The formulas are shown below:

[0064] ;

[0065] in, and It is the focal length of the camera in the X and Y directions (in pixels). and These are the coordinates of the principal point.

[0066] ;

[0067] in, Represents the values ​​of each component in the rotation matrix. It is the amount of translation of the camera in the X, Y, and Z axes.

[0068] ;

[0069] in, It is the radial distortion coefficient, which describes the nonlinear deformation of the image from the center to the edge; It is the tangential distortion coefficient, which describes the tilt or offset effect of the image.

[0070] Third, verification and optimization: The reprojection error, i.e., the error of the observed image points, is calculated using the following formula. Points calculated by projection from three-dimensional space The Euclidean distance between them. The formula is:

[0071] ;

[0072] in, , It represents a point in three-dimensional space.

[0073] The accuracy of the calibration is verified by reprojection error, ensuring the accuracy of the intrinsic and extrinsic parameter matrices and distortion coefficients. The formulas are normalized, radial distortion formula and tangential distortion formula are calculated, and the corrected coordinate formula is obtained. The intrinsic parameter matrix and distortion coefficients that meet the accuracy requirements are used to correct the image.

[0074] Normalization formula:

[0075] ;

[0076] Formula for calculating radial distortion:

[0077] ;

[0078] ;

[0079] Formula for calculating tangential distortion:

[0080] ;

[0081] The corrected coordinate formula is obtained as follows:

[0082] .

[0083] It should be noted that this application proposes to regard the relatively fixed welding working surface as a plane, and to simplify the three-dimensional to two-dimensional projection transformation relationship into an affine transformation between the two-dimensional coordinates of the welding working surface and the pixel coordinates of the camera plane. This effectively solves the problem that monocular cameras cannot directly obtain depth information, while depth cameras cannot accurately capture the depth of the light arc. This simplifies the calculation process and improves the geometric description accuracy and target detection real-time performance.

[0084] In this embodiment, the pre-training process of the target welding arc detection model is as follows: The monocular camera is used to acquire images of the arc to be trained; these images are then converted to grayscale to obtain a grayscale image; the grayscale image is processed using a contrast-limited adaptive histogram equalization algorithm to obtain a processed image; and the Canny edge detection algorithm is used to extract the boundary information of the welding arc region in the processed image; the welding arc region is labeled based on the boundary information to obtain a labeled image; and the initial model is trained using the labeled image to obtain the target welding arc detection model.

[0085] It should be noted that the welding arc is a high-brightness local spot or region during the welding process, and its target detection requires extracting the accurate edges of the arc in a high-brightness background and complex scene. Combining the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm and the Canny operator can effectively enhance the edge features of the arc and improve detection accuracy. The specific pre-training process of the target welding arc detection model is as follows:

[0086] First, data collection: A monocular camera was used to collect welding arc datasets. The dataset should cover images under different welding environments, including different lighting conditions, welding angles, shooting angles and distances, different welding types (arc welding, gas welding, etc.), and different welded workpiece conditions, to ensure the diversity of the dataset. During shooting, changes in camera parameters such as exposure time and focal length were considered to capture the different performance of the welding arc under different conditions, ensuring that the model has good adaptability under different working conditions.

[0087] Second, data preprocessing: Welding arc images often have characteristics such as uneven illumination and large brightness variations. In particular, the welding area may be overexposed or lose details. The following preprocessing is required to enhance the arc area:

[0088] ① Grayscale conversion: Convert the input color image to a grayscale image so that the Canny algorithm can better detect edges;

[0089] ②CLAHE algorithm for contrast enhancement: The core idea of ​​this algorithm is to divide the image into multiple local regions and perform histogram equalization in the local regions to enhance the contrast. By setting reasonable size and contrast limits for the local regions, it can adaptively enhance the local brightness and avoid uniform adjustment of the entire image.

[0090] ③ Canny edge detection algorithm processing: First, potential edges are identified by calculating the gradient magnitude and direction of each pixel in the enhanced image. Areas with high gradient values ​​usually represent edges, and the algorithm identifies areas with significant changes in the image in this way. Then, non-maximum suppression is used to refine the edges, removing insignificant pixels and retaining only the most significant edges, thus making the edges smoother. Finally, the assignment of high and low thresholds in the Canny algorithm is continuously optimized through experiments to distinguish between strong and weak edges. Strong edge pixels are directly retained, while weak edge pixels are only retained when connected to strong edge pixels. The threshold is continuously adjusted based on the edge extraction effect of the initially set low threshold (50) and high threshold (150). When the boundary of the welding arc is obvious, the low threshold can be lowered to extract more edges; if the boundary is blurry or complex, the low threshold should be increased to avoid false detection and ensure accurate edge extraction. At the same time, it is tested on multiple welding arc images to ensure that the threshold adapts to different image features. Through the above preprocessing, the boundary information of the welding arc region can be accurately extracted, providing clear edge contours for subsequent target detection and image analysis.

[0091] Third, data annotation: The Labelme annotation tool was used to annotate the welding arcs. Since the welding arc usually appears as a concentrated bright area, the entire arc area should be selected by stretching the bounding box, rather than annotating a single point of light. The four corners of the bounding box define the outer range of the welding arc. After selection, Labelme automatically generates the corresponding coordinate information, recording the position and size of the bounding box in the image. Finally, all annotation results were saved as a .json (JavaScript Object Notation, a lightweight data exchange format) file. This file contains the bounding box data for each welding arc. To facilitate subsequent model training, this annotation file was converted into a .txt (Plain Text File) format annotation file required for training the target welding arc detection model.

[0092] Training the target welding arc detection model: The labeled dataset was divided into training and validation sets in a 7:3 ratio. The YOLO (You Only Look Once, Object Detection) algorithm from deep learning was then used to train the model, learning the features and location information of welding arcs in the image. During training, the model's performance on both the training and validation sets was monitored, and adjustments and optimizations were made to ensure the model could automatically detect welding arcs and output the pixel coordinates of their center point. Using the trained model for welding arc detection, the pixel coordinates of the welding arc in the image can be obtained.

[0093] It should be noted that this application combines the CLAHE algorithm, the Canny algorithm, and real-time detection of the welding arc. By performing grayscale conversion, local contrast enhancement, and edge extraction on the welding image, the edge features of the welding arc are effectively enhanced, and the arc boundary is accurately extracted in complex and high-brightness backgrounds. Combined with the YOLO deep learning algorithm for target detection, the center point of the welding arc can be located in real time and accurately, exhibiting good adaptability and generalization ability, making it suitable for diverse welding environments.

[0094] It's worth noting that CLAHE is an adaptive histogram equalization method widely used for image contrast enhancement, especially effective when processing images with non-uniform brightness distributions. Unlike traditional histogram equalization, CLAHE avoids excessive contrast enhancement across the entire image through local processing. Its basic idea is to perform histogram equalization within local regions while limiting contrast enhancement to prevent noise amplification. The Canny operator is a classic edge detection algorithm widely used in image processing and computer vision. Its basic working principle is to accurately extract edges by calculating image gradients and using non-maximum suppression, and then retaining salient edges through double thresholding. The main characteristics of the Canny operator are high edge detection accuracy and strong noise resistance, and it is often used for tasks such as target edge detection, object contour extraction, and feature point detection. Object detection is an important task in computer vision, aiming to detect and locate target objects from images or videos. Its output is usually a bounding box that labels the position and size of the target object. Current object detection algorithms mainly rely on deep learning technology. By accurately labeling bounding boxes, they achieve end-to-end detection from candidate region generation to object classification and localization, and support tasks such as classification, localization and instance segmentation.

[0095] Step S12: Convert the pixel coordinates of the center point of the light arc into the actual coordinates of the welding point located on the welding surface based on the affine matrix; the affine matrix is ​​a matrix that supports coordinate system transformation between the camera virtual plane and the checkerboard grid in the welding surface; the affine matrix is ​​a matrix constructed using the corner coordinates obtained by the growth-based checkerboard grid corner detection algorithm; the welding surface is a plane obtained by stitching together the checkerboard grid of several target sizes covering different areas.

[0096] In this embodiment, the target size can be specifically set according to the actual situation, and can be determined based on the size of the welding surface. The welding surface does not refer to the area including the target chessboard, but also includes blank areas, that is, areas not covered by the target chessboard.

[0097] It's important to note the concepts of affine transformation and affine matrices: Affine transformations are commonly used to achieve coordinate system transformations between two-dimensional planes. They can be used to describe operations such as translation, flipping, scaling, rotation, and shearing of geometric objects. Affine transformations possess linearity and parallelism. Their core principle is to establish a system of linear equations based on the coordinate correspondences of three or more known points to calculate the affine matrix M, thus achieving the coordinate system transformation. However, since the camera produces different affine matrices at different positions and viewpoints, it's usually necessary to fix the camera position to ensure consistency. (Assuming a point...) Mapping from the original image coordinate system to In the real coordinate system, the affine transformation can be expressed as:

[0098] ;

[0099] in, For affine matrices, These are coefficients that control transformations such as rotation, scaling, and shearing. It is the translation amount.

[0100] In this embodiment, coordinate transformation refers to converting pixel coordinates in the image into actual welding plane coordinates, a crucial step to ensure accurate operation of the robotic arm. The pixel coordinates are converted into the actual coordinates of the welding surface using a transformation formula, which is then transmitted to the robotic arm to ensure precise positioning of the welding arc and execution of the operation. Through optimized affine transformation, the actual position of the welding arc guides the robotic arm for precise positioning and motion control, enabling automated welding fume collection, ensuring precise collaboration between the camera and the robotic arm, and improving the efficiency and accuracy of welding operations. The transformation formula is shown below:

[0101] ;

[0102] Represents an affine matrix. Represents pixel coordinates, Indicates the actual coordinates.

[0103] In this embodiment, the affine matrix construction process is as follows: Before converting the pixel coordinates of the center point of the light arc into the actual coordinates of the welding point on the welding surface based on the affine matrix, the process further includes: determining the first corner coordinates of several corner points and detecting the second corner coordinates of several corner points; the corner points are corner points in the checkerboard grid; the first corner coordinates are the coordinates of the corner point in the checkerboard grid coordinate system corresponding to the checkerboard grid; the second corner coordinates are the coordinates of the corner point in the camera virtual plane coordinate system; based on the first corner coordinates and the second corner coordinates of each corner point, the coordinate transformation relationship between the pixel coordinates of the corner point in the plane coordinate system and the actual coordinates of the corner point in the welding surface coordinate system is determined; the affine matrix is ​​constructed based on the coordinate transformation relationship.

[0104] In this embodiment, detecting the second corner coordinates of several corner points includes: detecting the second corner coordinates of several seed corner points; the seed corner points are corner points whose confidence levels meet the confidence requirements; using the second corner coordinates of the several seed corner points as base coordinates, and determining the second corner coordinates corresponding to the next corner point based on the base coordinates; using the second corner coordinates corresponding to the next corner point as new base coordinates, and jumping to the step of determining the second corner coordinates corresponding to the next corner point based on the base coordinates, until the second corner coordinates of all corner points in the chessboard are obtained.

[0105] In this embodiment, determining the coordinates of the second corner point corresponding to the next corner point based on the base coordinates includes: predicting the predicted coordinates of the next corner point based on a prediction method and two adjacent base coordinates; the prediction method is that the first distance and the second distance between the two adjacent base coordinates are equal, and the angle of the target angle is a right angle; the second distance is the distance between the reference coordinate and the next corner point; the reference coordinate is the corner point that is closer to the next corner point among the two adjacent base coordinates; the target angle is the angle formed by the next corner point and the two adjacent base coordinates with the reference coordinate as the vertex; accurately determining the coordinates of the second corner point based on the predicted coordinates using a sub-pixel method; the sub-pixel method represents the grayscale gradient of the vector between the coordinates of the second corner point and the coordinates of the adjacent pixel points, which is perpendicular to the camera virtual plane.

[0106] It should be noted that factory welding areas are typically large. While traditional methods of calibrating affine matrices using a single checkerboard pattern can guarantee local accuracy, they struggle to cover the entire welding area, leading to significantly increased positioning errors in areas far from the checkerboard. This is especially true when the field of view is wide, making it difficult for traditional methods to accurately recover the distance and positional relationships of the entire area. Furthermore, creating a large-scale checkerboard pattern covering the welding area is practically impossible and unsuitable for welding areas of varying sizes. Additionally, traditional checkerboard corner detection methods are only applicable to extracting corner coordinates from a single checkerboard pattern, failing to detect multiple corners simultaneously across multiple checkerboard patterns. Moreover, the accuracy of checkerboard corner detection is low at long distances. Figure 3 The diagram illustrates a traditional checkerboard corner detection method that includes all checkerboard squares. This method suffers from two key drawbacks: first, insufficient accuracy at long distances, failing to precisely acquire the pixel coordinates of each corner point; and second, when facing a plane with multiple checkerboard squares coexisting, it can only identify the corner coordinates of a single square, unable to simultaneously identify multiple squares, severely limiting its application. To address these issues, an affine transformation-based checkerboard corner detection method based on growth is proposed. This method constructs a virtual, large-scale plane by stitching together multiple small-sized checkerboard squares covering different areas. See [link to details] for further information. Figure 4 and Figure 5 As shown, Figure 4 This is a schematic diagram of a growth-based chessboard corner detection method that includes all chessboard squares. Figure 5 This is a schematic diagram of a growth-based checkerboard corner detection method that includes a single checkerboard grid. The growth-based checkerboard corner detection method, with its local-to-global recognition logic of "seed corner + iterative verification", overcomes the limitations of global feature matching in traditional methods. It can accurately define the checkerboard grid through the detection box and identify the pixel coordinates of each corner point. While solving the problems of low accuracy in long-distance recognition and inability to simultaneously identify multiple checkerboard corner points, it can also improve the robustness of detection in complex environments and significantly broaden the applicable scenarios of the technology. Figure 6 This is a schematic diagram of an optimized affine transformation process. Figure 7 This document presents a practical demonstration of an optimized affine transformation process. It employs a growing checkerboard corner detection method to read the coordinates of each checkerboard corner point, and then solves for the affine matrix based on this virtual plane. This method effectively overcomes the limitations of a single large checkerboard and the inability of traditional checkerboard corner detection methods to simultaneously detect the coordinates of multiple checkerboard corner points, achieving high-precision positioning of the entire welding area. Figure 4 As shown.

[0107] It should be pointed out that, Figure 6 In the affine transformation of the growth-based chessboard corner detection method, within a virtual large-scale image plane composed of multiple small chessboard squares, assuming the target plane of the 𝒊th small chessboard is... That would require at least three sets of point correspondences to find the coordinate system of the target plane. To the image plane coordinate system The optimal affine matrix is ​​obtained. Within the target plane, the transformation relationship between different small checkerboard grids and the image plane can be used to determine the transformation relationship between different checkerboard grid coordinate systems, ultimately transforming each small checkerboard grid to the checkerboard grid coordinate system (welding surface coordinate system). The following derivation combines the coordinates of the image plane into a large-size checkerboard grid: After obtaining the coordinate transformation relationship between the image plane and a single small checkerboard grid according to the first formula, the first formula is:

[0108] ;

[0109] The relative positional relationship between two different small chessboard squares can be calculated using the following second formula:

[0110] ;

[0111] Therefore, all points in a single small chessboard can be accessed through... Transform to the checkerboard coordinate system (welding surface coordinate system), and then they can be merged into a large planar coordinate system: .

[0112] The optimized affine transformation matrix accurately maps pixel coordinates in the image to the actual welding plane coordinate system, reducing errors during coordinate transformation. Through this process, the detected welding arc can be precisely located, providing reliable data for subsequent robotic arm control. The successful implementation of this affine transformation relies on a checkerboard corner detection method to simultaneously detect the corner coordinates of multiple checkerboard squares in a single image.

[0113] It should be noted that the growth-based checkerboard corner detection method is specifically designed for checkerboard corner detection. Its implementation steps are as follows: First, seed corner detection: A small number of high-confidence corners are initially obtained as seeds using the traditional Harris corner detector; selection criteria: corner response values ​​are higher than the confidence threshold, and local gradients conform to checkerboard intersection characteristics. Second, corner growth: Based on the arrangement direction of the detected corners, the possible position of the next corner is predicted; for example, given two adjacent corners E and F, the predicted position of G should satisfy EF≈FG and ∠EFG≈90°. Third, region search: Within a small window near the predicted position, the new corner is precisely located using a sub-pixel method. Fourth, energy function optimization: Define the energy function. , This is the negative value of the total number of corner points on the current chessboard. This indicates that the matching degree of the third corner point is predicted using two known adjacent corner points. The structural energy is calculated for the three adjacent corner points in each row and column, and the maximum value is taken as the structural energy of the chessboard. Fifth, iterative optimization: Repeat the above steps, continuously expanding new corner points and optimizing the energy function until no new corner points that meet the constraints can be detected or the chessboard area is completely covered.

[0114] It should be noted that the specific steps and calculation formulas for the above-mentioned implementation method based on the corner detection of a growing chessboard are as follows:

[0115] First, locate the corner points of the checkerboard pattern (corresponding to the first step of seed corner point detection in the above implementation steps). A checkerboard pattern is a regular planar pattern composed of alternating black and white squares. Its corner points are located at the boundaries between the black and white squares, and are prominent features in the image. Identifying and locating these corner points in complex backgrounds is the first step in checkerboard detection. Checkerboard corner points can be divided into two typical types based on direction: one type is right-angled corner points parallel to the coordinate axes (such as corner points formed by the perpendicular intersection of horizontal and vertical lines), whose local features are distributed along the horizontal and vertical directions; the other type is oblique corner points at 45° to the coordinate axes (including oblique structures of the checkerboard itself or oblique corner points caused by shooting or distortion), whose local features are distributed obliquely along 45°, 135°, etc. Therefore, two initial corner point prototypes are first defined: one initial corner point prototype parallel to the coordinate axes (prototype 1 (m=1)), such as... Figure 8 As shown, the prototype includes Figure 8 The four convolution kernels in Figures A, B, C, and D correspond to the four right-angle directions (top left, top right, bottom right, and bottom left), respectively, and can completely cover all corner points in the chessboard that are parallel to the coordinate axes; another initial corner point prototype (prototype 2 (m=2)) is at a 45° angle to the coordinate axes, as shown in Figure D. Figure 9 As shown, the prototype includes Figure 9 The four convolutional kernels in Figures A, B, C, and D correspond to the four diagonal directions of 45°, 135°, 225°, and 315°, respectively, and can completely cover all corner shapes in the 45° diagonal direction. Each prototype consists of its own four convolutional kernels (A, B, C, and D), which are used to perform convolution operations with the image, thereby calculating the corner probability c of each pixel using the following formula. When the response value of any one of the four convolutional kernels is small, the corner probability is significantly reduced, which helps to eliminate non-checkerboard corners. Finally, non-maximum suppression is used to extract local maxima points as candidate corners.

[0116] The formula combination is as follows:

[0117] ;

[0118] ;

[0119] ;

[0120] in, Represents the convolution kernel A and the prototype The convolution response at a certain pixel; Represents convolution kernel B and prototype The convolution response at a certain pixel; Represents the convolution kernel C and the prototype The convolution response at a certain pixel; Represents the convolution kernel D and the prototype The convolution response at a certain pixel; Representing two possible forms of the prototype m, such as Figure 10 The diagram shown illustrates two target corner point prototypes corresponding to prototype 1, revealing two possible forms corresponding to prototype 1. Figure 10 Figure (a) in the figure is based on Figure 8 Figure A and Figure 8 Figure C in the diagram shows a prototype of a target corner point parallel to the coordinate axes. Figure 10 Figure (b) in the figure is based on Figure 8 Figure B and Figure 8 Figure D shows a prototype of a target corner point that is parallel to the coordinate axes.

[0121] Second, refinement of sub-pixel level corner coordinates and orientations (corresponding to the third step of region search in the above implementation steps). The vector from the pixel corner (second corner coordinates) to the surrounding pixels should be perpendicular to the grayscale gradient of the image. Based on this principle, by minimizing the error function and iterative optimization, the sub-pixel level corner coordinates can be accurately calculated, thereby improving the accuracy of corner positioning.

[0122] Third, optimize the energy function to grow the chessboard grid, ultimately achieving accurate chessboard corner detection (corresponding to the fourth step of optimizing the energy function in the above implementation steps). To globally optimize the chessboard structure, an energy function is defined, which is... and It consists of two parts, among which This represents a negative value indicating the total number of corner points on the current chessboard. This indicates the use of two known adjacent corner points. and To predict the third corner point The degree of matching, in the formula Let each represent the coordinates of three adjacent corner points on the chessboard. This represents the set of three adjacent corner vertices in all rows and columns of the chessboard. The structural energy of each row and column is calculated for each of the three adjacent corner vertices, and the maximum value is taken as the structural energy of the chessboard. This process is iteratively executed, continuously expanding to include new corner vertices and optimizing the energy function, until no new corner vertices meeting the constraints can be detected, or the chessboard area is completely covered. The specific formula is shown below:

[0123] ;

[0124] ;

[0125] ;

[0126] in, It represents the set of three adjacent corner points in all rows and columns on the chessboard.

[0127] This method starts from known corner points and borrows the idea of ​​region growing algorithms. It gradually expands based on image grayscale values ​​and edge features, using similarity metrics to find more corner points. During the expansion process, sub-pixel-level optimization is incorporated to accurately locate corner points, reducing errors caused by noise or deformation and improving detection accuracy and robustness. Simultaneously, by constructing and optimizing an energy function, corner points are gradually grown into a complete checkerboard structure, effectively solving the corner detection problem of multiple checkerboard grids in a single image. The specific checkerboard growth process is as follows: Figure 11 The image shown is a schematic diagram of a chessboard pattern growth. Figure 11 In Figure (a), the seed represents the initially selected corner point. Figure 11 Figures (a), (b), (c), (d), and up to (e) illustrate the specific sequence and process of the gradual growth of the checkerboard area starting from the initial corner point.

[0128] It should be noted that this application proposes to use a growth-based checkerboard corner detection method for affine transformation, transforming multiple small checkerboards into a checkerboard coordinate system (welding surface coordinate system) to form a virtual large-scale plane, and growing multiple checkerboards in a single image. This solves the defect of traditional checkerboard corner detection methods that cannot simultaneously identify the coordinates of multiple checkerboard corners, overcomes the limitations of single checkerboard calibration, effectively improves the accuracy of the affine matrix, and thus enhances the accuracy and stability of welding point positioning.

[0129] It should be noted that traditional methods typically treat the welding work area as a three-dimensional solid, calculating the transformation relationship between the three-dimensional coordinates of the welding surface and the pixel coordinates of the camera plane through projection transformation. However, since the welding surface is usually relatively fixed and can be approximated as a two-dimensional plane at a certain height, a two-dimensional welding fume localization method based on affine transformation is proposed. This method simplifies the three-dimensional to two-dimensional projection transformation relationship into an affine transformation between the two-dimensional coordinates of the welding surface and the pixel coordinates of the camera plane. This method has the following significant advantages: ① It effectively solves the problem that monocular cameras cannot directly acquire depth information, while depth cameras cannot accurately capture the depth of the light arc, reducing the equipment cost for subsequent target detection; ② It reduces the computation of two parameters, and the mathematical expression is simple and can be implemented efficiently; ③ Unlike projection transformation, which can only handle rotation and translation relationships, affine transformation can accurately describe the geometric relationship and deformation (such as local rotation or tilt) of the welding workpiece surface, facilitating modeling and application; ④ Due to the low complexity of two-dimensional affine transformation, compared with three-dimensional transformation, it significantly improves the real-time performance of the system, meeting the requirements for real-time control of devices such as robotic arms in automated welding fume collection technology.

[0130] Step S13: Adjust the robotic arm based on the actual coordinates of the welding point to use the fume extraction tray to extract welding fumes.

[0131] In summary, this application proposes an autonomous welding fume localization method based on affine transformation and deep learning. It employs a monocular camera to detect the welding arc in real time and converts the image coordinates into the actual coordinates of the welding surface using an affine matrix, thereby accurately locating the welding point. This application also proposes an automated welding fume collection technology combining deep learning and robotic arm control. The autonomously located welding fume coordinates guide the robotic arm or electronic control device to move to the welding point for fume collection, achieving fully automated operation from welding point identification to fume collection. This reduces manual intervention and operational complexity, improving the efficiency of welding fume control. The advantage of this application lies in utilizing the high-resolution imaging capability and flexible deployment of a monocular camera, significantly improving the accuracy and efficiency of welding area monitoring while reducing equipment complexity and deployment costs.

[0132] As can be seen, this application utilizes the monocular camera to acquire welding arc images in real time, and extracts the pixel coordinates of the arc center point located in the welding arc image based on a pre-trained target welding arc detection model; the monocular camera is a calibrated camera obtained after camera calibration; the pixel coordinates of the arc center point are converted into the actual coordinates of the welding point located on the welding surface based on an affine matrix; the affine matrix is ​​a matrix that supports coordinate transformation between the pixel coordinates of corner points in the camera's virtual plane and the actual coordinates of corner points in the welding surface; the pixel coordinates of the corner points are obtained based on a growing checkerboard corner detection algorithm; the welding surface is obtained by splicing several checkerboard grids of the same size covering different areas; the robotic arm is controlled based on the actual coordinates of the welding points to use the fume extraction tray to extract welding fumes. Therefore, this application combines the target welding arc detection model and the affine matrix to improve the welding point positioning accuracy. It utilizes a self-controlled robotic arm to extract welding fumes from the welding point, eliminating the need for manual intervention and achieving automated welding fume collection. The application uses the affine matrix between the welding surface and the camera's virtual plane to perform coordinate transformation from the camera's virtual plane coordinate system to the welding surface coordinate system—that is, a two-dimensional to two-dimensional coordinate transformation, rather than a three-dimensional to two-dimensional projection transformation. This treats the welding surface as a two-dimensional plane, solving the problem that a monocular camera cannot directly acquire depth information, simplifying the process, and improving accuracy and efficiency. Furthermore, this application uses a growing chessboard corner detection algorithm for affine transformation, overcoming the limitation of traditional chessboard corner detection methods that cannot simultaneously identify the coordinates of multiple chessboard corners, thus improving the accuracy of the affine matrix and enhancing the accuracy and stability of welding point positioning. As can be seen, this application can automatically extract welding fumes while ensuring extraction efficiency and accuracy.

[0133] Accordingly, this application also discloses an autonomous welding point positioning device, applied to a positioning equipment. The positioning equipment includes a monocular camera located above the welding surface and whose field of view covers the welding surface, and a robotic arm located above the monocular camera and equipped with a fume extraction plate. See [link to relevant documentation]. Figure 12 As shown, the device includes:

[0134] The coordinate detection module 11 is used to acquire welding arc images in real time using the monocular camera, and extract the pixel coordinates of the arc center point located in the welding arc image based on the pre-trained target welding arc detection model; the monocular camera is a calibrated camera obtained after camera calibration.

[0135] The coordinate transformation module 12 is used to convert the pixel coordinates of the center point of the light arc into the actual coordinates of the welding point located on the welding surface based on an affine matrix; the affine matrix is ​​a matrix that supports coordinate transformation between the pixel coordinates of corner points in the camera virtual plane and the actual coordinates of corner points in the welding surface; the pixel coordinates of the corner points are obtained based on a growing checkerboard corner detection algorithm; the welding surface is obtained by splicing together several checkerboard grids of the same size covering different areas.

[0136] The welding fume extraction module 13 is used to adjust the robotic arm based on the actual coordinates of the welding point to extract welding fumes using the fume extraction tray.

[0137] The more specific working process of each of the above modules can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0138] As can be seen, this application utilizes the monocular camera to acquire welding arc images in real time, and extracts the pixel coordinates of the arc center point located in the welding arc image based on a pre-trained target welding arc detection model; the monocular camera is a calibrated camera obtained after camera calibration; the pixel coordinates of the arc center point are converted into the actual coordinates of the welding point located on the welding surface based on an affine matrix; the affine matrix is ​​a matrix that supports coordinate transformation between the pixel coordinates of corner points in the camera's virtual plane and the actual coordinates of corner points in the welding surface; the pixel coordinates of the corner points are obtained based on a growing checkerboard corner detection algorithm; the welding surface is obtained by splicing several checkerboard grids of the same size covering different areas; the robotic arm is controlled based on the actual coordinates of the welding points to use the fume extraction tray to extract welding fumes. Therefore, this application combines the target welding arc detection model and the affine matrix to improve the welding point positioning accuracy. It utilizes a self-controlled robotic arm to extract welding fumes from the welding point, eliminating the need for manual intervention and achieving automated welding fume collection. The application uses the affine matrix between the welding surface and the camera's virtual plane to perform coordinate transformation from the camera's virtual plane coordinate system to the welding surface coordinate system—that is, a two-dimensional to two-dimensional coordinate transformation, rather than a three-dimensional to two-dimensional projection transformation. This treats the welding surface as a two-dimensional plane, solving the problem that a monocular camera cannot directly acquire depth information, simplifying the process, and improving accuracy and efficiency. Furthermore, this application uses a growing chessboard corner detection algorithm for affine transformation, overcoming the limitation of traditional chessboard corner detection methods that cannot simultaneously identify the coordinates of multiple chessboard corners, thus improving the accuracy of the affine matrix and enhancing the accuracy and stability of welding point positioning. As can be seen, this application can automatically extract welding fumes while ensuring extraction efficiency and accuracy.

[0139] Furthermore, embodiments of this application also provide an electronic device. Figure 13 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0140] Figure 13 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the welding point autonomous positioning method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0141] In this embodiment, the power supply 26 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 24 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0142] Furthermore, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored thereon can include computer programs 221, and the storage method can be temporary storage or permanent storage. The computer programs 221 may include, in addition to computer programs capable of performing the welding point autonomous positioning method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, computer programs capable of performing other specific tasks.

[0143] Furthermore, embodiments of this application also disclose a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed method for autonomous positioning of welding points.

[0144] The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0145] The various embodiments in this application are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For the same or similar parts between the various embodiments, refer to each other. As for the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and relevant parts can be referred to in the method section.

[0146] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0147] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0148] Finally, it should be noted that in this document, relational terms such as "first" and "first" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] The above provides a detailed description of the welding point autonomous positioning method, apparatus, equipment, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for autonomous positioning of a welding spot, characterized in that The method is applied to a positioning device, the positioning device comprises a monocular camera above a welding surface and covering a field of view of the welding surface, and a mechanical arm above the monocular camera and provided with a smoke suction disc, and the method comprises the following steps: real-time acquisition of a welding light arc picture by the monocular camera, and extraction of a light arc center point pixel coordinate in the welding light arc picture based on a target welding light arc detection model obtained through pre-training; the monocular camera is a calibrated camera obtained after camera calibration is completed; conversion of the light arc center point pixel coordinate into a welding point actual coordinate on the welding surface based on an affine matrix; the affine matrix is a matrix for coordinate conversion between a pixel coordinate of a corner point in a virtual plane of the camera and an actual coordinate of the corner point in the welding surface; the pixel coordinate of the corner point is a corner point coordinate obtained based on a growth-based checkerboard corner point detection algorithm; the welding surface is obtained by splicing a plurality of checkerboards of the same size covering different areas; adjustment and control of the mechanical arm based on the welding point actual coordinate to suck welding fume by the smoke suction disc; wherein, before the real-time acquisition of the welding light arc picture by the monocular camera and the extraction of the light arc center point pixel coordinate in the welding light arc picture based on the target welding light arc detection model obtained through pre-training, the method further comprises the following steps: acquisition of a to-be-trained light arc picture by the monocular camera, and gray processing of the to-be-trained light arc picture to obtain a gray processed image; processing of the gray processed image by a limited contrast self-adaptive histogram equalization algorithm to obtain a processed picture, and extraction of boundary information of a welding light arc region in the processed picture by a Canny edge detection algorithm; data labeling of the welding light arc region based on the boundary information to obtain a labeled picture; training of an initial model by using the labeled picture to obtain the target welding light arc detection model.

2. The method of claim 1, wherein, the mechanical arm comprises a horizontally rotatable and telescopic support arm, a vertically telescopic vertical smoke suction arm connected to the support arm, and the smoke suction disc connected to the smoke suction arm.

3. The method of claim 1, wherein, before the real-time acquisition of the welding light arc picture by the monocular camera, the method further comprises the following steps: acquisition of a plurality of calibration board images of different angles corresponding to a calibration board of a known size by the monocular camera; the calibration board is a checkerboard provided with a plurality of corner points; calculation of camera parameters of the monocular camera based on the calibration board images; the camera parameters comprise an intrinsic matrix, an extrinsic matrix and a distortion coefficient; verification of calibration accuracy by a re-projection error method, correction by a distortion model, and obtaining of camera parameters with calibration accuracy meeting accuracy requirements to complete camera calibration and obtain the calibrated camera.

4. The method of claim 1, wherein, before the conversion of the light arc center point pixel coordinate into the welding point actual coordinate on the welding surface based on the affine matrix, the method further comprises the following steps: determination of first corner point coordinates of a plurality of corner points, and detection of second corner point coordinates of a plurality of corner points; the corner points are corner points in the checkerboard; the first corner point coordinates are coordinates of the corner points in a checkerboard coordinate system corresponding to the checkerboard; the second corner point coordinates are coordinates of the corner points in a virtual plane coordinate system of the camera; determine a coordinate conversion relationship between the corner point pixel coordinate in the plane coordinate system and the corner point actual coordinate in a welding surface coordinate system based on the first corner point coordinate and the second corner point coordinate of each of the corner points; construct the affine matrix based on the coordinate conversion relationship.

5. The method of claim 4, wherein, The second corner point coordinates of a plurality of corner points are detected, including: The second corner point coordinates of a plurality of seed corner points are detected; the seed corner points are corner points whose corner point confidence meets a confidence requirement; The second corner point coordinates of the plurality of seed corner points are taken as base coordinates, and the second corner point coordinates corresponding to a next corner point are determined based on the base coordinates; The second corner point coordinates corresponding to the next corner point are taken as new base coordinates, and the step of determining the second corner point coordinates corresponding to the next corner point based on the base coordinates is jumped to until the second corner point coordinates of all the corner points in the checkerboard are obtained.

6. The method of autonomous weld pad positioning of claim 5, wherein, The second corner point coordinates corresponding to a next corner point are determined based on the base coordinates, including: a predicted coordinate of the next corner point is predicted according to a prediction method and based on two adjacent base coordinates; the prediction method is that a first distance and a second distance between the two adjacent base coordinates are equal, and an angle of a target corner is a right angle; the second distance is a distance between a reference coordinate and the next corner point; the reference coordinate is a corner point that is closer to the next corner point among the two adjacent base coordinates; the target corner is an angle formed by the next corner point and the two adjacent base coordinates with the reference coordinate as a vertex; the second corner point coordinate of the predicted coordinate is accurately determined according to a sub-pixel method; the sub-pixel method indicates that a vector between the second corner point coordinate and a coordinate of an adjacent pixel point is perpendicular to a gray gradient of the camera virtual plane.

7. A self-positioning device for a weld point, characterized in that The welding point autonomous positioning device is applied to a positioning device, the positioning device includes a monocular camera located above a welding surface and having a field of view covering the welding surface, and a mechanical arm located above the monocular camera and mounted with a smoke suction disc, and the device includes: a coordinate detection module configured to collect a welding light arc picture in real time by using the monocular camera, and extract a light arc center point pixel coordinate located in the welding light arc picture based on a target welding light arc detection model obtained through pre-training; the monocular camera is a calibrated camera obtained after camera calibration is completed; a coordinate conversion module configured to convert the light arc center point pixel coordinate into a welding point actual coordinate located in the welding surface based on an affine matrix; the affine matrix is a matrix supporting coordinate conversion between a corner point pixel coordinate in a camera virtual plane and a corner point actual coordinate in the welding surface; the corner point pixel coordinate is a corner point coordinate obtained based on a growing checkerboard corner point detection algorithm; the welding surface is obtained by splicing a plurality of checkerboards of the same size covering different areas; a welding fume suction module configured to control the mechanical arm based on the welding point actual coordinate to suction welding fume by using the smoke suction disc; The welding point autonomous positioning device further includes: a model construction module configured to collect a to-be-trained light arc picture by using the monocular camera, and obtain a gray image after gray processing by performing gray processing on the to-be-trained light arc picture; The gray-scale post-image is processed by using a limited contrast adaptive histogram equalization algorithm to obtain a processed picture, and a Canny edge detection algorithm is used to extract boundary information of the welding light arc area in the processed picture; Data labeling is performed on the welding light arc area based on the boundary information to obtain a labeled picture; An initial model is trained by using the labeled picture to obtain the target welding light arc detection model.

8. An electronic device, comprising: Comprise: a memory for saving a computer program; a processor for executing the computer program to implement the welding point autonomous positioning method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, for saving a computer program; wherein the computer program is executed by a processor to implement the welding point autonomous positioning method according to any one of claims 1 to 6.

Images