Weld tracking image acquisition device and method based on camera and sharm lens

Abstract

The application discloses a seam tracking image acquisition device and method based on a camera and a Schlieren lens, and the device comprises: a filter device arranged between the camera and the Schlieren lens, and the filter device is installed on the camera; the filter device comprises a pressing ring main body, a fastening ring and a filter screen, the pressing ring main body is annular, the inner side and the outer side of the pressing ring main body are provided with threads, the inner side of the pressing ring main body is further provided with a boss, and the filter screen is arranged on the boss of the pressing ring main body; the pressing ring main body is matched with the inner side thread of the front end of the camera through the outer side thread, so that the pressing ring main body is fixed to the front end of the camera; the fastening ring is annular, the outer side of the fastening ring is provided with a thread, and the fastening ring is matched with the inner side thread of the pressing ring main body through the outer side thread of the fastening ring, so that the filter screen is fixed to the inner side of the pressing ring main body; and the image acquisition device is installed obliquely above the front direction of welding, image shooting is performed on the welding pool area, and tracking of the seam to be welded is realized.

Description

Technical Field

[0001] This invention relates to the field of automated welding technology, and in particular to a weld seam tracking image acquisition device and method using a camera and a SAM lens. Background Technology

[0002] The statements in this section merely refer to the background art related to this invention and do not necessarily constitute prior art.

[0003] Welding, as a permanent joining method, relies heavily on the control strategies of the welding process and the accuracy of weld seam tracking to determine weld quality. With the continuous development of automated welding technology in industry, vision-based weld seam tracking and control technology is becoming increasingly important in fields such as aerospace, shipbuilding, vehicle engineering, nuclear engineering, and marine engineering equipment.

[0004] Compared to other monitoring and sensing technologies, visual sensing technology offers advantages such as rich information, non-contact operation, strong anti-interference capabilities, high versatility, and diverse data processing methods during the welding process. When faced with complex welding environments, visual sensing technology can adapt to different welding conditions through software updates and algorithm optimization, without changing or with minimal changes to sensor settings, based on the characteristics of the welding process. Visual sensing technology has become one of the most direct and widely used welding sensing technologies for reflecting the welding status.

[0005] Traditional vision-based image acquisition devices for welding processes typically consist of a filter, a lens, and a camera. The filter is mounted at the front of the lens, followed by the lens and camera. Furthermore, due to limitations in the welding torch's position and movement requirements, the lens's optical axis is installed at a certain angle to the welding plane. During operation, because of this angle between the welding plane and the camera's imaging plane, while the camera can clearly image the welding arc or molten pool area, it cannot clearly image the leading edge of the molten pool (i.e., it is out of focus).

[0006] During welding, the intensity of the arc light is much greater than that of other parts within the camera's field of view. Therefore, in order to achieve a clear image of the arc, a filter is needed to reduce the light entering the camera. However, this will also lead to severe underexposure in other areas, making it difficult to achieve effective image acquisition.

[0007] Before performing subsequent image processing, calibration based on the optical characteristics of the lens and camera, converting pixel information into real-world coordinates, is a necessary step. However, due to the significant differences in optical characteristics between SAM lenses and traditional lenses, traditional camera calibration methods cannot be directly used. Currently, calibration schemes for SAM lenses or SAM imaging systems are quite complex and poorly adapted to welding conditions. A simple, efficient, and accurate camera calibration scheme is needed. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a weld seam tracking image acquisition device and method based on a camera and a SAM lens. By using a C-port camera and an SM-port SAM lens, as well as a filter device suitable for the C-port camera interface, clear, wide-range focusing and imaging are achieved during the welding process. Specifically, the filter device suitable for the C-port camera interface can fix a semi-circular filter within the threaded interface of the camera without affecting the connection to the lens, thus achieving regional filtering of the image acquisition area.

[0009] On the one hand, a weld seam tracking image acquisition device based on a camera and a SAM lens was provided; A weld seam tracking image acquisition device based on a camera and a SAM lens includes: a camera, a filter device, and a SAM lens; the filter device is disposed between the camera and the SAM lens, and the filter device is mounted on the camera; The filtering device includes a pressure ring body, a fastening ring, and a filter. The pressure ring body is annular, with threads on both its inner and outer sides. A boss is also provided on the inner side of the pressure ring body. The filter is a semi-circular filter, positioned on the boss of the pressure ring body. The pressure ring body is fixed to the front of the camera by its outer thread engaging with the inner thread of the camera's front end. The fastening ring is annular, with threads on its outer side. The fastening ring's outer thread engages with the inner thread of the pressure ring body to fix the filter inside the pressure ring body. The image acquisition device is installed diagonally above the welding advance direction to capture images of the weld pool area in order to track the weld seam to be welded.

[0010] On the other hand, a method for acquiring weld seam tracking images based on a camera and a SAM lens is provided; A method for acquiring weld seam tracking images based on a camera and a SAM lens includes: Read multiple calibration images, wherein the calibration images are black and white grid calibration boards; read the pixel coordinates of each corner point on the black and white grid calibration board, establish a world coordinate system, set the center of the calibration board as the origin of the world coordinate system, and set the actual distance between the corner points on the calibration board as the interval of the coordinate points in the world coordinate system; Based on the world coordinate system and camera coordinate system, the camera intrinsic and extrinsic parameters are obtained by solving homogeneous linear equations. The camera's intrinsic and extrinsic parameters are optimized using a nonlinear least squares method by minimizing the reprojection error; the acquired images are then corrected using the optimized camera intrinsic and extrinsic parameters. Based on the arrangement of corner points in the corrected image, the corner points are converted into a standard positively distributed matrix using a perspective matrix. Calculate the ratio of pixel values ​​in the standard positive direction distribution matrix to the size of the black and white grid calibration board; the calibration is now complete. The calibrated image acquisition device is used to acquire images and track weld seams during the welding process.

[0011] The above technical solution has the following advantages or beneficial effects: 1. The filter is fixed between the camera and the SAM lens, which can avoid the problem of the filter's filtering range shifting due to the offset of the lens optical axis when the SAM lens is adjusted.

[0012] 2. Reduces the problem of large filter transition area and inaccurate filtering range caused by non-parallel light entering the lens, because the light after lens conversion is more parallel than the original light.

[0013] 3. By using a SAM lens instead of a regular industrial lens, clear focusing on the welding surface can be achieved, avoiding the problem of narrow focusing range caused by the tilted arrangement of the camera.

[0014] 4. By solving for the camera's intrinsic and extrinsic parameters and perspective matrix, the image acquisition device was effectively calibrated, while the camera calibration process was simplified.

[0015] 5. The calibration method proposed in this invention is not limited to black and white grid calibration boards, thus expanding the selectivity of calibration boards. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 This is a schematic diagram of the overall structure of the image acquisition device with a C-port camera and an SM-port Sham lens provided in the embodiments of this disclosure; Figure 2 This is a schematic cross-sectional view of the overall structure of the image acquisition device with a C-port camera and an SM-port Sham lens provided in this embodiment of the disclosure; Figure 3 This is a schematic diagram of the pressure ring structure provided in an embodiment of this disclosure; Figure 4 This is a schematic cross-sectional view of the pressure ring structure provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the calibration board in an embodiment of this disclosure; Figure 6 These are verification images of embodiments of this disclosure; Figure 7 This is a flowchart of a camera calibration method provided in an embodiment of this disclosure; Figure 8 This is a schematic diagram showing the position of the image acquisition device provided in an embodiment of this disclosure under an optional welding condition; Figure 9 This is an overall system connection diagram under welding conditions provided in the embodiments of this disclosure; Figure 10 This is a schematic diagram of a matching filter device installation tool provided in an embodiment of this disclosure.

[0018] Explanation of reference numerals in the attached figures: 1. Sham lens; 2. Camera; 3. Filter device; 10. Pressure ring body; 11. Fastening ring; 12. Filter; 20. Image acquisition device; 21. Welding torch; 22. Welding plane; 23. Focusing plane; 24. Imaging plane; 30. Installation tools. Detailed Implementation

[0019] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments of the invention. The terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] Example 1 This embodiment provides a weld seam tracking image acquisition device based on a camera and a Sham lens; like Figure 1 , Figure 2 , Figure 10 As shown, a weld seam tracking image acquisition device based on a camera and a SAM lens includes: a camera 2, a filter device, and a SAM lens 1; the filter device 3 is disposed between the camera 2 and the SAM lens 1, and the filter device is mounted on the camera; like Figure 3 and Figure 4As shown, the filter device 3 includes a pressure ring body 10, a fastening ring 11, and a filter 12. The pressure ring body 10 is annular, with threads on both its inner and outer sides. A boss is also provided on the inner side of the pressure ring body. The filter 12 is a semi-circular filter, and it is mounted on the boss of the pressure ring body. The pressure ring body is fixed to the front of the camera by its outer thread engaging with the inner thread of the camera's front end. The fastening ring 11 is annular, with threads on its outer side. The fastening ring engages with the inner thread of the pressure ring body by its outer thread engaging with the inner thread of the pressure ring body, thus fixing the filter inside the pressure ring body. The image acquisition device 20 is installed diagonally above the welding advance direction to capture images of the weld pool area in order to track the weld seam to be welded.

[0022] Furthermore, the camera port is a C-type or CS-type port, which is manufactured according to the "ANSI B1.1" international standard, with a thread diameter of 1 inch and a distance of 17.526 mm from the mounting reference surface to the focal plane. In particular, the connection method of this type of port is that the camera uses an internal thread and the corresponding lens uses an external thread, and the two are connected by threads.

[0023] Furthermore, the port of the SAM lens is an SM type port, which is an industrial camera interface that does not rely on threads to connect to the camera. Instead, the lens is fixed to the outside of the camera interface by three positioning screws evenly distributed on the interface.

[0024] Furthermore, after the camera and SAM lens are fitted onto the outside of the camera port, the original threads used to connect the camera to other C-port lenses are left unused, forming a space without any optical structure, and the filter device is installed in this space.

[0025] It should be understood that the protrusion inside the pressure ring body serves as a limiting function to prevent the semi-circular filter from moving within the pressure ring.

[0026] Furthermore, recesses for assisting installation or disassembly are provided on the outer sides of both the pressure ring body and the fastening ring. The installation tool 30 provided by this invention is used for disassembly and installation. Both ends of the installation tool 30 are respectively adapted to the grooves in the fastening ring and the pressure ring body.

[0027] It should be understood that the image acquisition device is located in the space above the weld, in front of the weld direction.

[0028] Furthermore, the imaging plane 24 of the camera forms a set angle with the welding plane 22. By adjusting the SAM angle of the SAM lens, the angle between the focusing plane 23 and the welding plane 22 is made smaller than a set threshold range.

[0029] Furthermore, such as Figure 9 As shown, the camera of the image acquisition device is connected to a computer, which is also connected to the welding torch 21. The computer is also connected to a three-dimensional motion platform via a controller. The workpiece to be welded is placed on the three-dimensional motion platform, and the workpiece to be welded has a weld seam to be welded. The three-dimensional motion platform can drive the movement of the workpiece to be welded. The welding torch 21 is mounted on the welding torch bracket, with the end of the welding torch facing the welding molten pool area. The weld seam to be welded is located in the middle of the welding molten pool. The image acquisition device is fixed to the bracket through screw holes and takes pictures of the front of the molten pool and the welding molten pool area from an obliquely above the welding direction to achieve weld seam tracking and avoid welding direction deviation.

[0030] The specific models of C-port cameras and SM-port SAM lenses are not limited. In this embodiment, the C-port camera includes CMOS cameras, CCD cameras, or cameras with other imaging principles, as long as they can provide a C-port and meet the flange distance requirements between the camera and the lens.

[0031] During the welding process, such as Figure 8 As shown, the image acquisition device 20 is located diagonally above the welding direction. The field of view should include the weld pool and the weld seam in front of the weld pool, and the weld pool should be located in the upper half of the image so that the semi-circular filter can filter the welding arc light.

[0032] According to one of the alternative implementation schemes, such as Figure 8 As shown, the image acquisition device is installed diagonally above the welding torch in the welding forward direction. Viewed from the side in the welding forward direction, an installation angle of 10° to 40° from the vertical direction allows for better utilization of the SAM lens characteristics and achieves better image acquisition. However, the specific installation angle is not limited to the recommended range. The focusing plane 23 of the image acquisition device is not required to be strictly parallel to the welding plane 22; the angle between the two planes should be as small as possible within the limits allowed by the structure and framing to obtain a wider focusing range.

[0033] According to an alternative embodiment, the image acquisition device can be installed on the side in the welding direction, and from a top view, the device can be at an angle of 50° to 80° with the weld, and the device is slightly higher than the welding plane.

[0034] Example 2 This embodiment provides a method for acquiring weld seam tracking images based on a camera and a SAM lens; like Figure 7 As shown, the weld seam tracking image acquisition method based on a camera and a SAM lens includes: S201: Read multiple calibration images, wherein the calibration images are black and white grid calibration boards, such as... Figure 5As shown; read the pixel coordinates of each corner point on the black and white grid calibration board, establish a world coordinate system, set the center of the calibration board as the origin of the world coordinate system, and set the actual distance between the corner points on the calibration board as the interval of the coordinate points in the world coordinate system; S202: Based on the world coordinate system and camera coordinate system, the camera intrinsic and extrinsic parameters are obtained by solving homogeneous linear equations; S203: The camera's intrinsic and extrinsic parameters are optimized using a nonlinear least squares method by minimizing the reprojection error; the acquired images are then corrected using the optimized camera intrinsic and extrinsic parameters. S204: Based on the arrangement of corner points in the corrected image, use a perspective matrix to convert the corner points into a standard positively oriented point matrix; S205: Calculate the ratio of pixel values ​​in the standard positive direction distribution matrix to the size of the black and white grid calibration board; the calibration is now complete. S206: Use a calibrated image acquisition device to acquire images and track weld seams during the welding process.

[0035] Furthermore, S202: Based on the world coordinate system and the camera coordinate system, the intrinsic and extrinsic parameters of the camera are obtained by solving homogeneous linear equations, specifically including: The mathematical relationships between the world coordinate system, camera coordinate system, and pixel coordinate system are represented by a unified intrinsic parameter matrix:

[0036] in,( , ) represents the pixel coordinates that are ultimately projected onto the graphic from the world coordinates. , ( ) is the camera focal length to be solved, ( , ) are the coordinates of the primary pixel, which is also the optical center of the camera. , The coordinates of the camera after incorporating the lens distortion model are expressed mathematically as follows:

[0037] Among them, camera homogeneous coordinates It is the original coordinates in the camera coordinate system Calculated, in the formula , , , , , , It is a radial distortion parameter. , The tangential distortion parameter is the parameter to be determined, while the radial distortion parameter and the tangential distortion parameter are the extrinsic parameters to be determined.

[0038]

[0039] Furthermore, S203: By minimizing the reprojection error, the camera intrinsic and extrinsic parameters are optimized using a nonlinear least squares method, specifically including: The Levenberg-Marquardt algorithm (LM algorithm) is used to solve the intrinsic parameter matrix expression constructed in S202. The LM algorithm is a commonly used method for solving camera reprojection models. It finds the optimal solution through iterative optimization. The specific algorithm will not be described in detail here.

[0040] Further, step S204: Based on the arrangement of corner points in the corrected image, using a perspective matrix, converts the corner points into a standard positively distributed corner point matrix, including: The expression for the perspective matrix is:

[0041] In the formula A~H Let be the parameters of the perspective matrix, where A, B, D, E Responsible for linear transformations of images. C, F Control the horizontal translation of the image. G, H Responsible for changes in image viewpoint, the final It is a normalization term for a matrix, used to maintain the homogeneity of matrix transformations.

[0042] For the coordinates of the midpoint in the corrected image ( , The new coordinates of the standard positive-direction distribution of points obtained after perspective transformation. ( , ): ; .

[0043] Further, S205: Calculating the proportional relationship between the pixel values ​​in the standard positive direction distribution dot matrix and the size of the black and white grid calibration board, specifically includes:

[0044]

[0045] In the formula, x and y are the actual lengths of the n grids on the actual calibration plate. , This represents the number of pixels corresponding to n grids in the corrected image, thus establishing the proportional relationship between pixel size and true size.

[0046] The calibration board is not limited to the black and white grid calibration board. Other boards with recurring patterns and the same regularity, and with the patterns arranged according to a fixed logic, can be used for calibration, such as dot matrix and cross markers.

[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A weld seam tracking image acquisition device based on a camera and a SAM lens, characterized in that, include: Camera, filter, and Sham lens; The filter device is disposed between the camera and the Sham lens, and the filter device is mounted on the camera; The filtering device includes a pressure ring body, a fastening ring, and a filter. The pressure ring body is annular, with threads on both its inner and outer sides. A boss is also provided on the inner side of the pressure ring body. The filter is a semi-circular filter, positioned on the boss of the pressure ring body. The pressure ring body is fixed to the front of the camera by its outer thread engaging with the inner thread of the camera's front end. The fastening ring is annular, with threads on its outer side. The fastening ring's outer thread engages with the inner thread of the pressure ring body to fix the filter inside the pressure ring body. The image acquisition device is installed diagonally above the welding advance direction to capture images of the weld pool area in order to track the weld seam to be welded. Among them, the weld seam tracking image acquisition method based on camera and SAM lens includes: Read multiple calibration images, wherein the calibration images are black and white grid calibration boards; read the pixel coordinates of each corner point on the black and white grid calibration board, establish a world coordinate system, set the center of the calibration board as the origin of the world coordinate system, and set the actual distance between the corner points on the calibration board as the interval of the coordinate points in the world coordinate system; Based on the world coordinate system and camera coordinate system, the camera intrinsic and extrinsic parameters are obtained by solving homogeneous linear equations. The camera's intrinsic and extrinsic parameters are optimized using a nonlinear least squares method by minimizing the reprojection error; the acquired images are then corrected using the optimized camera intrinsic and extrinsic parameters. Based on the arrangement of corner points in the corrected image, the corner points are converted into a standard positively distributed matrix using a perspective matrix. Calculate the ratio of pixel values ​​in the standard positive direction distribution matrix to the size of the black and white grid calibration board; the calibration is now complete.

2. The weld seam tracking image acquisition device based on a camera and a SAM lens as described in claim 1, characterized in that, The camera has a C-type or CS-type port, and the SAM lens has an SM-type port.

3. The weld seam tracking image acquisition device based on a camera and a SAM lens as described in claim 1, characterized in that, The camera and the SAM lens are fixed together by bolts, and the SAM lens is sleeved on the outside of the camera port; the outer side of the pressure ring body and the outer side of the fastening ring are both equipped with recesses to assist in installation or disassembly.

4. The weld seam tracking image acquisition device based on a camera and a SAM lens as described in claim 1, characterized in that, The camera's imaging plane forms a set angle with the welding plane. By adjusting the SAM angle of the SAM lens, the angle between the focusing plane and the welding plane is made smaller than a set threshold range.

5. The weld seam tracking image acquisition device based on a camera and a SAM lens as described in claim 1, characterized in that, The camera of the image acquisition device is connected to a computer, which is also connected to a welding torch. The computer is also connected to a three-dimensional motion platform via a controller. The workpiece to be welded is placed on the three-dimensional motion platform, and the workpiece to be welded has a weld seam to be welded. The three-dimensional motion platform can drive the movement of the workpiece to be welded. The welding torch is mounted on a welding torch bracket, with the end of the welding torch facing the welding molten pool area. The weld seam to be welded is located in the middle of the welding molten pool. The image acquisition device is fixed to the bracket through screw holes and takes pictures of the front of the molten pool and the welding molten pool area from an obliquely above the welding direction.

6. A method for acquiring weld seam tracking images based on a camera and a SAM lens, characterized in that, include: Read multiple calibration images, wherein the calibration images are black and white grid calibration boards; Read the pixel coordinates of each corner point on the black and white grid calibration board, establish a world coordinate system, set the center of the calibration board as the origin of the world coordinate system, and set the actual distance between the corner points on the calibration board as the interval of the coordinate points in the world coordinate system. Based on the world coordinate system and camera coordinate system, the camera intrinsic and extrinsic parameters are obtained by solving homogeneous linear equations. The camera's intrinsic and extrinsic parameters are optimized using a nonlinear least squares method by minimizing the reprojection error; the acquired images are then corrected using the optimized camera intrinsic and extrinsic parameters. Based on the arrangement of corner points in the corrected image, the corner points are converted into a standard positively distributed matrix using a perspective matrix. Calculate the ratio of pixel values ​​in the standard positive-direction distribution matrix to the size of the black-and-white grid calibration board; The calibration is now complete. The calibrated image acquisition device is used to acquire images and track weld seams during the welding process.

7. The weld seam tracking image acquisition method based on a camera and a SAM lens as described in claim 6, characterized in that, Based on the world coordinate system and camera coordinate system, the camera intrinsic and extrinsic parameters are obtained by solving homogeneous linear equations, specifically including: The mathematical relationships between the world coordinate system, camera coordinate system, and pixel coordinate system are represented by a unified intrinsic parameter matrix: in,( , ) represents the pixel coordinates that are ultimately projected onto the graphic from the world coordinates. , ( ) is the camera focal length to be solved, ( , () represents the camera coordinates after incorporating the lens distortion model. , The coordinates of the main pixel are the optical center of the camera.

8. The weld seam tracking image acquisition method based on a camera and a SAM lens as described in claim 7, characterized in that, ( , The coordinates of the camera after incorporating the lens distortion model are expressed mathematically as follows: Among them, camera homogeneous coordinates It is the original coordinates in the camera coordinate system Calculated, in the formula , , , , , , It is a radial distortion parameter. , The tangential distortion parameter is the parameter to be determined, while the radial distortion parameter and the tangential distortion parameter are the extrinsic parameters to be determined.

9. The weld seam tracking image acquisition method based on a camera and a SAM lens as described in claim 6, characterized in that, Based on the arrangement of corner points in the corrected image, a perspective matrix is ​​used to convert the corner points into a standard, positively oriented point matrix, including: The expression for the perspective matrix is: Where A~H are the parameters of the perspective matrix, with A, B, D, and E responsible for the linear transformation of the image, C and F controlling the horizontal translation of the image, and G and H responsible for the change of the image viewpoint. It is a normalization term for a matrix, used to maintain the homogeneity of matrix transformations; For the coordinates P(x, y) of a point in the corrected image, the new coordinates of the point matrix obtained after perspective transformation are the standard positive orientation distribution. ( , ): ； 。 10. The weld seam tracking image acquisition method based on a camera and a SAM lens as described in claim 6, characterized in that, Calculate the ratio of pixel values ​​in the standard positive-direction distribution matrix to the size of the black-and-white grid calibration board, specifically including: In the formula, x and y are the actual lengths of the n grids on the actual calibration plate. , This corresponds to the number of pixels in the corrected image, thus establishing the proportional relationship between the pixel size and the actual size.

Images