Automatic tracking system and method for laser processing

Abstract

The present application relates to the field of laser processing technology, and specifically discloses an automatic tracking system and method for laser processing, which collects calibration plate images and calculates internal parameters through a multi-view camera array deployed on a robot, builds a dynamic transformation model of the robot TCP and the camera, and then generates a multi-exposure sequence in combination with the robot posture time series data. It uses a high dynamic range feature aggregation algorithm to fuse the effective information of different exposure images, and introduces a deep learning-based weld analysis model to extract weld centerline features in high dynamic range images. Furthermore, the weld centerline coordinates are aligned with the robot posture, calibration parameters and ideal path in time and space, and finally the path deviation is calculated in real time by a PID controller and motion adjustment instructions are generated. In this way, high-precision tracking of complex scenes such as spatial curved surface welds and welding of dissimilar materials can be achieved, which is beneficial to improving the anti-interference ability and adaptive adjustment efficiency factor of the laser processing process.

Description

Technical Field

[0001] The present application relates to the field of laser processing technology, and more specifically, to an automatic tracking system and method for laser processing. Background Art

[0002] Laser processing technology, with its advantages of high energy density, non-contact processing, high precision, and high efficiency, has been widely used in various industrial fields such as welding, cutting, and cladding. Traditional laser processing relies on preset fixed paths or manual instruction, making it difficult to cope with dynamic factors such as workpiece deformation, assembly errors, and ambient light interference. This can easily lead to weld offset and uneven energy distribution, which in turn affects processing quality and efficiency. To ensure processing quality, improve production efficiency, and reduce scrap rates, visual sensing technology is often used. A fixed-angle monocular camera or structured light system is used to capture weld images and calculate path deviations based on geometric models.

[0003] However, in the existing technology, when faced with complex working conditions such as spatial curved surface welds and welding of dissimilar materials, laser stripes are easily interfered with, resulting in a decrease in feature extraction accuracy. In addition, traditional methods usually rely on a single exposure image, which makes it difficult to take into account the overexposure of highlight areas and the loss of dark details. Especially in the laser molten pool area, in high dynamic range scenes where strong reflections and shadows coexist, the image information is severely distorted. Although probe-type detection can avoid optical interference, it has problems such as complex operation and easy introduction of mechanical deformation errors, and cannot adapt to the needs of high-speed dynamic processing. In a strong interference environment, the image feature blurring due to overexposure, the contrast reduction caused by smoke and dust, the dynamic noise caused by metal splashing, and the change of target shape caused by thermal deformation of the workpiece will cause the tracking stability and accuracy to be greatly reduced.

[0004] Therefore, an optimized automatic tracking system and method for laser processing is desired. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an automatic tracking system and method for laser processing, which collects calibration plate images and calculates internal parameters through a multi-view camera array deployed on the robot, builds a dynamic transformation model of the robot TCP and the camera, and then generates a multi-exposure sequence in combination with the robot posture time series data, uses a high dynamic range feature aggregation algorithm to fuse the effective information of different exposure images, and introduces a deep learning-based weld analysis model to extract the weld centerline features in the high dynamic range image. Furthermore, the weld centerline coordinates are aligned with the robot posture, calibration parameters and ideal path in time and space, and finally the path deviation is calculated in real time by the PID controller and motion adjustment instructions are generated. In this way, high-precision tracking of complex scenes such as spatial curved surface welds and welding of dissimilar materials can be achieved, which is beneficial to improving the anti-interference ability and adaptive adjustment efficiency factor of the laser processing process.

[0006] According to one aspect of the present application, there is provided an automatic tracking method for laser processing, comprising:

[0007] The camera deployed on the robot collects images of the calibration plate at different angles and calculates the intrinsic parameters of the camera after calibration;

[0008] Based on the initial robot TCP and camera transformation matrices and the robot pose data time series, the calibration plate images collected at different angles are analyzed to calculate the robot TCP and camera transformation matrices.

[0009] Generate a time series of laser processing exposure images based on the previous frame analysis results and exposure strategy;

[0010] performing high dynamic range feature aggregation analysis on the time series of the laser processing exposure image to obtain a laser processing exposure high dynamic range image;

[0011] Passing the laser-processed exposed high dynamic range image through a weld analysis model based on a trained neural network model to obtain weld centerline pixel coordinates;

[0012] Determine the path deviation based on the weld centerline pixel coordinates, the calibrated camera intrinsic parameters, the robot TCP and camera transformation matrix, the robot's current pose data, and a predefined ideal weld path;

[0013] The path deviation is passed through a PID controller to generate robot motion adjustment instructions.

[0014] According to another aspect of the present application, there is provided an automatic tracking system for laser processing, comprising:

[0015] The calibration plate image acquisition module is used to collect calibration plate images at different angles through the camera deployed on the robot and calculate the camera intrinsic parameters after calibration;

[0016] The calibration plate image analysis module is used to analyze the calibration plate images collected at different angles based on the initial robot TCP and camera transformation matrix and the robot pose data time series set to calculate the robot TCP and camera transformation matrix;

[0017] An exposure image sequence generation module is used to generate a time sequence of laser processing exposure images based on the previous frame analysis results and exposure strategy;

[0018] a feature aggregation analysis module, configured to perform high dynamic range feature aggregation analysis on the time series of the laser processing exposure image to obtain a laser processing exposure high dynamic range image;

[0019] A weld analysis module, configured to pass the laser processing exposed high dynamic range image through a weld analysis model based on a trained neural network model to obtain pixel coordinates of the weld centerline;

[0020] a path deviation determination module, configured to determine the path deviation based on the weld centerline pixel coordinates, the calibrated camera intrinsic parameters, the robot TCP and camera transformation matrix, the robot's current pose data, and a predefined ideal weld path;

[0021] The motion adjustment instruction generating module is used to generate the robot motion adjustment instruction by passing the path deviation through the PID controller.

[0022] Compared with the prior art, the automatic tracking system and method for laser processing provided by the present application collects calibration plate images and calculates internal parameters through a multi-view camera array deployed on the robot, builds a dynamic transformation model of the robot TCP and the camera, and then generates a multi-exposure sequence in combination with the robot posture time series data. The high dynamic range feature aggregation algorithm is used to fuse the effective information of different exposure images, and a deep learning-based weld analysis model is introduced to extract the weld centerline features in the high dynamic range image. Furthermore, the weld centerline coordinates are aligned with the robot posture, calibration parameters and ideal path in time and space, and finally the path deviation is calculated in real time by the PID controller and motion adjustment instructions are generated. In this way, high-precision tracking of complex scenes such as spatial curved surface welds and welding of dissimilar materials can be achieved, which is beneficial to improving the anti-interference ability and adaptive adjustment efficiency factor of the laser processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0024] Figure 1 Flowchart of an automatic tracking method for laser processing according to an embodiment of the present application.

[0025] Figure 2 4 is a data flow diagram of an automatic tracking method for laser processing according to an embodiment of the present application.

[0026] Figure 3 This is a flowchart of sub-step S3 of the automatic tracking method for laser processing according to an embodiment of the present application.

[0027] Figure 4 This is a flowchart of sub-step S4 of the automatic tracking method for laser processing according to an embodiment of the present application.

[0028] Figure 5 This is a flowchart of sub-step S42 of the automatic tracking method for laser processing according to an embodiment of the present application.

[0029] Figure 6 This is a flowchart of sub-step S7 of the automatic tracking method for laser processing according to an embodiment of the present application.

[0030] Figure 7 4 is a block diagram of an automatic tracking system for laser processing according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0032] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0033] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0034] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0035] It is worth noting that in this application, all actions to obtain data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0036] In response to the technical problems described in the above background technology, the present application proposes an automatic tracking method for laser processing, which collects calibration plate images and calculates internal parameters through a multi-view camera array deployed on the robot, builds a dynamic transformation model of the robot TCP and the camera, and then generates a multi-exposure sequence in combination with the robot posture time series data. The high dynamic range feature aggregation algorithm is used to fuse the effective information of different exposure images, and a deep learning-based weld analysis model is introduced to extract the weld centerline features in the high dynamic range image. Furthermore, the weld centerline coordinates are aligned with the robot posture, calibration parameters and ideal path in time and space, and finally the path deviation is calculated in real time by the PID controller and motion adjustment instructions are generated. In this way, high-precision tracking of complex scenes such as spatial curved surface welds and welding of dissimilar materials can be achieved, which is beneficial to improving the anti-interference ability and adaptive adjustment efficiency factor of the laser processing process.

[0037] Figure 1 Flowchart of an automatic tracking method for laser processing according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of an automatic tracking method for laser processing according to an embodiment of the present application. Figure 1 and Figure 2As shown, the automatic tracking method for laser processing includes the following steps: S1, collecting calibration plate images at different angles by a camera deployed on a robot, and calculating the camera intrinsic parameters after calibration; S2, analyzing the calibration plate images collected at different angles based on the initial robot TCP and camera transformation matrix and the robot posture data time series set to calculate the robot TCP and camera transformation matrix; S3, generating a time series of laser processing exposure images based on the previous frame analysis result and the exposure strategy; S4, performing high dynamic range feature aggregation analysis on the time series of the laser processing exposure images to obtain a laser processing exposure high dynamic range image; S5, passing the laser processing exposure high dynamic range image through a weld analysis model based on a trained neural network model to obtain the weld centerline pixel coordinates; S6, determining the path deviation based on the weld centerline pixel coordinates, the camera intrinsic parameters after calibration, the robot TCP and camera transformation matrix, the robot's current posture data and a predefined ideal weld path; S7, passing the path deviation through a PID controller to generate a robot motion adjustment instruction.

[0038] In the above-mentioned automatic tracking method for laser processing, step S1 involves capturing images of a calibration plate at different angles using a camera deployed on the robot, and calculating the camera's intrinsic parameters after calibration, which include focal length, principal point, and distortion coefficient. It should be understood that due to inherent characteristics of the camera, such as lens distortion, during the imaging process, and that imaging parameters at different angles can affect the accuracy of subsequent processing, to eliminate the interference of these factors, the present application captures images of a calibration plate and calculates camera intrinsic parameters, including focal length, principal point, and distortion coefficient, to establish a precise conversion relationship between camera image coordinates and real-world coordinates, and performs distortion correction on the image, so that the weld image captured by the camera truly restores the actual weld shape and position, thereby providing accurate visual data for the robot's path deviation calculation and improving tracking accuracy.

[0039] Specifically, the non-idealities of camera lenses can cause image distortion, such as barrel or pincushion distortion. These distortions can cause straight lines to appear as curves in the image, severely impacting the accuracy of subsequent feature extraction. Therefore, specific methods must be used to obtain parameters that describe these distortion characteristics, namely camera intrinsic parameters. These include important parameters such as focal length, principal point, and distortion coefficient. Focal length determines the magnification of the image and directly affects the size of the object's projection on the image. The principal point, the intersection of the optical axis and the image plane, is crucial for determining the true size of the object. The distortion coefficient is used to quantify and correct for lens-induced image distortion.

[0040] During implementation, a suitable calibration plate is selected. Ideally, one with high-contrast, regularly arranged feature points facilitates accurate identification and positioning. A black and white checkerboard pattern is commonly used due to its clarity and ease of detection. Such a calibration plate is placed near the workpiece at various positions and angles, and the robot's onboard camera captures it from multiple viewpoints. This allows for the capture of images of the plate in a variety of poses, resulting in a comprehensive dataset.

[0041] After collecting enough calibration plate images, the next step is to analyze them and extract the required parameter information. First, each calibration plate image, captured at different angles, is preprocessed to remove noise, enhance image quality, and ensure accurate location of feature points. Then, based on the known calibration plate geometry, classic computer vision methods such as Zhang Zhengyou's calibration method are applied to calculate the position of the camera relative to the calibration plate, thereby deriving the camera's intrinsic parameters.

[0042] It's worth noting that factors such as lighting variations and mechanical vibrations in the actual working environment can affect calibration results. Therefore, the entire calibration process also requires consideration of how to effectively address these external interferences. For example, increasing the number of calibrations and employing different lighting conditions can improve data robustness. Furthermore, to further enhance calibration accuracy, it's possible to combine information from multiple sensors, such as distance data from a laser rangefinder, to determine calibration parameters.

[0043] Determining the focal length depends on the physical properties of the lens used and is also affected by the shooting distance. By analyzing the relative position changes between feature points in the image of the same calibration plate at different distances, a relatively accurate estimate of the focal length can be achieved. The position of the principal point is determined primarily based on the distribution of the feature points on the calibration plate in the image plane. Furthermore, quality control of the data acquisition process is crucial. For example, ensure that the surface of the calibration plate is clean and undamaged to avoid erroneous feature point recognition caused by stains or scratches. Ensure that the relative motion between the camera and the calibration plate is as smooth and stable as possible to minimize unnecessary jitter.

[0044] In the above-mentioned automatic tracking method for laser processing, the step S2 analyzes the calibration plate images collected at different angles based on the initial robot TCP and camera transformation matrix and the robot posture data time series set to calculate the robot TCP and camera transformation matrix. It should be understood that since the robot's posture will change in real time during actual movement, the relative position relationship between the camera and TCP will also change dynamically. If the initial robot TCP and camera transformation matrix are directly used, there will be a deviation in the conversion between the weld image coordinates collected by the camera and the robot coordinate system, and the coupling of visual information and robot movement cannot be accurately achieved. Therefore, the present application analyzes the multi-angle calibration plate images in combination with the robot posture data time series set, and dynamically optimizes the initial robot TCP and camera transformation matrix to eliminate the coupling error in actual movement. Specifically, by introducing the robot posture time series data and multi-view calibration plate image analysis, the coordinate conversion error caused by the relative position change between the TCP and the camera during the robot movement process can be eliminated.

[0045] Specifically, step S2 includes:

[0046] in, is the robot TCP relative motion matrix between different robot poses, is the relative motion matrix of the calibration plate between different robot postures, is the robot TCP and camera transformation matrix.

[0047] In the specific implementation process, first, the calibration plate is fixed near the workpiece to be processed, and the robot carrying the multi-view camera array is controlled to move to multiple preset postures, such as postures P1, P2, and P3. The calibration plate image is collected at each posture, and the posture data corresponding to each position, including joint angles or Cartesian coordinates, is recorded through the robot control system. For the robot TCP relative motion matrix, the TCP coordinates corresponding to different postures, such as P1 to P2, are read from the robot control system, and the translation and rotation parameters of the TCP between the two postures are calculated, and then the robot TCP relative motion matrix is ​​constructed. , which reflects the motion transformation relationship between TCP in different postures. For the relative motion matrix of the calibration plate, the calibration plate images collected at different postures are processed, and the calibration plate feature points are extracted using computer vision algorithms. The posture of the calibration plate relative to the camera in each image is calculated based on the known geometric dimensions of the calibration plate. Then, through the initial transformation relationship between the camera and the robot, the posture of the calibration plate in different postures is converted to the same coordinate system, and the relative motion matrix of the calibration plate in different postures is obtained. , which reflects the change in the calibration plate’s pose relative to the robot’s motion. In this way, we can obtain the key matrices used to construct the robot TCP and camera dynamic transformation models, laying the foundation for the precise coordinate transformation of subsequent weld tracking.

[0048] In the above-mentioned automatic tracking method for laser processing, step S3 generates a time series of laser processing exposure images based on the previous frame analysis results and exposure strategy. It should be understood that since a single exposure image is prone to overexposure of highlight areas and loss of dark details, resulting in image information distortion and affecting the accuracy of subsequent weld feature extraction, it is necessary to dynamically adjust the exposure parameters based on the previous frame analysis results and exposure strategy. The resulting time series of laser processing exposure images covers scene details at different exposure levels and improves the overall image quality. Among them, Figure 3 FIG. 1 is a flow chart of sub-step S3 of the automatic tracking method for laser processing according to an embodiment of the present application. Figure 3 As shown, the step S3 includes the following steps: S31, calculating the time series set of exposure parameters based on the previous frame analysis result and the exposure strategy; S32, based on the time series set of exposure parameters, taking images by a camera to obtain a time series of laser processing exposure images.

[0049] Specifically, the step S31 calculates a time series set of exposure parameters based on the previous frame analysis result and the exposure strategy. In a specific example of the present application, the previous frame analysis result is the ratio of overexposed / underexposed areas of the previous frame calibration plate image, and the exposure strategy is the target exposure number and the basic exposure time range. It should be understood that the ratio of overexposed / underexposed areas of the previous frame calibration plate image can reflect the adaptability of the current exposure parameters. Combined with the preset target exposure number and the basic exposure time range, the time series set of exposure parameters is dynamically calculated to adjust the subsequent exposure strategy in a targeted manner, ensuring that in the subsequent image sequence, the overexposed areas are clearly presented in the short exposure frames, and the underexposed areas retain details in the long exposure frames, providing multi-dimensional effective data for high dynamic range feature aggregation.

[0050] During the specific implementation process, if the previous frame analysis results show that the overexposed area accounts for 25% and the underexposed area accounts for 15% in the previous frame calibration plate image, the exposure strategy sets the target exposure number to 3 times, and the basic exposure time range is 1 / 2000s to 1 / 100s. First, based on the high proportion of overexposed areas, the initial exposure time is shortened to suppress overexposure. Combined with the basic exposure time range, the first exposure time is set to 1 / 1000s. Secondly, to balance the intermediate brightness area, the second exposure time is adjusted to 1 / 500s. Then, for the underexposed area, the third exposure time is extended to 1 / 200s. Through such dynamic adjustment, the time series set of exposure parameters is obtained as [1 / 1000s, 1 / 500s, 1 / 250s].

[0051] Specifically, step S32 involves capturing images with a camera based on the timing set of exposure parameters to obtain a time series of laser processing exposure images. That is, the camera is controlled to capture images sequentially according to the timing set of exposure parameters to obtain a time series of laser processing exposure images containing different exposure information, so that each frame retains the effective features of different brightness areas in the scene. Specifically, the camera captures three frames of images according to the timing of exposure parameters, such as exposure times of 1 / 1000s, 1 / 500s, and 1 / 250s, respectively. The first frame retains the texture details of the highlight area of ​​the molten pool, the second frame presents a clear outline of the weld edge, and the third frame displays the material characteristics of the dark area. Subsequent feature aggregation can improve the dynamic range of the image and reduce the error in weld centerline extraction.

[0052] In the above-mentioned automatic tracking method for laser processing, step S4 performs high dynamic range feature aggregation analysis on the time series of the laser processing exposure image to obtain a laser processing exposure high dynamic range image. It should be understood that since images with different exposure levels in the time series of the laser processing exposure image retain effective information in different regions, performing high dynamic range feature aggregation analysis on the time series of the laser processing exposure image can fuse the complementary information of multiple frames of images, thereby avoiding distortion of single image information. Figure 4 FIG. 4 is a flow chart of sub-step S4 of the automatic tracking method for laser processing according to an embodiment of the present application. Figure 4 As shown, the step S4 includes the following steps: S41, extracting the laser processing exposure low dynamic features from each laser processing exposure image in the time series of the laser processing exposure image to obtain a set of laser processing exposure low dynamic feature vectors; S42, performing feature compensation dynamic aggregation on the set of laser processing exposure low dynamic feature vectors to obtain a laser processing exposure low dynamic feature aggregation coding vector; S43, passing the laser processing exposure low dynamic feature aggregation coding vector through an AIGC-based laser processing exposure high dynamic map generator to obtain the laser processing exposure high dynamic range image.

[0053] Specifically, step S41 extracts laser processing exposure low-dynamic features from each laser processing exposure image in the time series of the laser processing exposure images to obtain a set of laser processing exposure low-dynamic feature vectors. It should be understood that by extracting low-dynamic features such as edge gradients and textures from each frame of the laser processing exposure image, valid information under different exposure conditions is converted into a set of laser processing exposure low-dynamic feature vectors. This can preserve features such as the outline of the highlight area of ​​the molten pool, the texture of the weld seam, and the material of the dark areas, thereby retaining the valid information of each frame in a numerical form and providing multi-dimensional feature input for subsequent aggregation.

[0054] In the specific implementation, the time series of laser processing exposure images consists of three frames, captured with a short exposure of 1 / 1000s to highlight the highlights of the weld pool, a medium exposure of 1 / 500s to reveal the weld edge, and a long exposure of 1 / 250s to highlight the dark areas. For the first short-exposure image, the Canny edge detection algorithm is used to extract the contour edges of the highlights in the weld pool. The edge gradient direction and intensity are calculated to form an edge feature matrix. A gray-level co-occurrence matrix is ​​used to extract the texture and roughness characteristics of this region. These features are integrated into a 128-dimensional low-dynamic feature vector, preserving the morphological information of the highlights. For the second medium-exposure image, the Sobel operator is used to calculate the horizontal and vertical gradients of the weld region. This is combined with the Hough transform to detect straight edge segments to capture the weld orientation. These geometric and gradient features are converted into a 128-dimensional vector to highlight the continuity of the weld contour. For the third long-exposure image, a local binary pattern is used to extract material texture features in the dark areas. The average grayscale and entropy values ​​of the region are calculated to reflect the brightness distribution. These features are then integrated into a 128-dimensional vector to preserve the dark details. Finally, these three 128-dimensional vectors constitute a set of low-dynamic feature vectors of laser processing exposure, providing multi-dimensional feature inputs of highlights, welds, and dark areas for subsequent aggregation.

[0055] Specifically, the step S42 performs feature compensation dynamic aggregation on the set of the laser processing exposure low dynamic feature vectors to obtain a laser processing exposure low dynamic feature aggregation coding vector. Specifically, since there is a feature offset in each vector in the set of laser processing exposure low dynamic feature vectors, such as the overexposed image feature lacks dark information, the present application fuses the advantages of each laser processing exposure low dynamic feature vector through feature compensation dynamic aggregation. The generated laser processing exposure low dynamic feature aggregation coding vector can simultaneously include highlight area contour features and dark area texture features, which is more comprehensive than the dynamic range covered by a single feature vector, thereby eliminating the limitations of a single feature and providing a complete feature input for the subsequent generation of high dynamic images. Among them, Figure 5 FIG. 4 is a flow chart of sub-step S42 of the automatic tracking method for laser processing according to an embodiment of the present application. Figure 5As shown, the step S42 includes the steps of: S421, inputting the set of laser processing exposure low dynamic feature vectors into the feature baseline learning network to obtain the laser processing exposure low dynamic feature baseline regression coding vector; S422, calculating the feature dynamic compensation factor of each laser processing exposure low dynamic feature vector in the set of laser processing exposure low dynamic feature vectors relative to the laser processing exposure low dynamic feature baseline regression coding vector to obtain a set of feature dynamic compensation factors; S423, performing regularization processing based on the Softmax activation function on the set of feature dynamic compensation factors to obtain a set of feature dynamic compensation weight factors; S424, based on the set of feature dynamic compensation weight factors and the set of laser processing exposure low dynamic feature vectors, dynamically compensating the laser processing exposure low dynamic feature baseline regression coding vector to obtain the laser processing exposure low dynamic feature aggregation coding vector.

[0056] More specifically, step S421 is expressed as follows:

[0057]

[0058] in, represents the set of low dynamic feature vectors of laser processing exposure, and Respectively The first, second, and and Laser processing exposure low dynamic feature vector, express The number of low dynamic feature vectors in laser processing exposure, represents the set of real numbers, Represents the dimension of the low dynamic feature vector of laser processing exposure, represents the feature baseline learning network, represents the sigmoid activation function, represents the learnable parameter matrix, represents the learnable bias vector, Represents the baseline regression encoding vector of the low dynamic features of laser processing exposure.

[0059] That is, a stable benchmark that can characterize the common pattern of input features, namely the statistical central trend, is extracted from the set of laser processing exposure low-dynamic feature vectors to eliminate the offset differences between heterogeneous features. The generated laser processing exposure low-dynamic feature baseline regression coding vector encodes the common essential characteristics of different exposure image features, which can be used as a standard to measure the degree of deviation of each feature vector, providing a stable benchmark framework for the dynamic aggregation of various effective features, thereby improving the accuracy and robustness of feature aggregation.

[0060] More specifically, step S422 is expressed as follows:

[0061]

[0062] in, represents the hyperbolic tangent function, Indicates the calculation of L2 norm, represents vector dot product, express The corresponding characteristic dynamic compensation factor.

[0063] Specifically, the method quantifies the degree of deviation of the unique features of each laser processing low-dynamic exposure feature vector in the set relative to the baseline regression encoding vector of the laser processing low-dynamic exposure feature vector. This captures the dynamic change information in each laser processing low-dynamic exposure feature vector that is not covered by the baseline, providing a quantitative basis for dynamic compensation during subsequent feature aggregation, enabling the aggregation process to specifically integrate the differentiated and effective features of each laser processing low-dynamic exposure feature vector. The resulting set of feature dynamic compensation factors accurately characterizes the uniqueness of each laser processing low-dynamic exposure feature vector, improving the integrity and diversity of the features and providing more comprehensive feature support for high dynamic range image generation.

[0064] More specifically, in step S423, the set of feature dynamic compensation factors is regularized based on the Softmax activation function to obtain a set of feature dynamic compensation weight factors. Here, when evaluating the degree of deviation or contribution difference of each laser processing exposure low dynamic feature vector relative to the established laser processing exposure low dynamic feature baseline regression encoding vector, it is expected that the laser processing exposure low dynamic feature vector to the laser processing exposure low dynamic feature baseline regression encoding vector has a continuous differentiable mapping, so that the feature dynamic compensation weight factors calculated from each laser processing exposure low dynamic feature vector have generalization capabilities. Based on this, in a preferred example of the present application, step S423 includes: first, performing smooth mapping stabilization optimization on the set of feature dynamic compensation factors to obtain a set of optimized feature dynamic compensation factors. Then, the set of optimized feature dynamic compensation factors is regularized based on the Softmax activation function to obtain a set of feature dynamic compensation weight factors.

[0065] Specifically, the displacement gradient is first obtained by targeting the tangent space deviation field from the laser processing exposure low dynamic feature vector to the laser processing exposure low dynamic feature baseline regression encoding vector, that is:

[0066]

[0067] in, represents the calculation of partial derivatives, represents the displacement gradient.

[0068] Then, the displacement gradient vector Construct the correlation change matrix:

[0069]

[0070] Among them, the displacement gradient vector is a column vector, is the transpose of a vector.

[0071] That is, if the differential term of the distance-related mapping based on the laser processing exposure low dynamic feature vector to the laser processing exposure low dynamic feature baseline regression encoding vector is used as a parameterized mapping, then the mapping state will be guided by the displacement gradient. Thus, by solving the displacement gradient vector , and construct the correlation change matrix under symmetric action To ensure no deformation, the associated change matrix To determine the boot stability.

[0072] Therefore, the correlation change matrix is ​​calculated The F norm of , that is, the low-dimensional stability representation, is used as a stability factor to calculate the characteristic dynamic compensation factor, namely:

[0073]

[0074] in, represents the correlation change matrix, Indicates the number of the set of dynamic compensation factors for optimized features. An optimization feature dynamic compensation factor.

[0075] This achieves smooth mapping guidance from the laser processing exposure low-dynamic feature vector to the laser processing exposure low-dynamic feature baseline regression encoding vector, improving the universal accuracy of the feature dynamic compensation weight factor that is amplified by small noise or variation disturbances.

[0076] Then, the set of optimized feature dynamic compensation factors is regularized based on the Softmax activation function to obtain a set of feature dynamic compensation weight factors, which is expressed as follows:

[0077]

[0078] in, Represents the exponential function operation with e as the base, Indicates the preset parameter value, express The corresponding feature dynamic compensation weight factor.

[0079] That is, through the regularization processing of the Softmax activation function, the set of optimized feature dynamic compensation factors is transformed into a set of feature dynamic compensation weight factors that meet the characteristics of the probability distribution. This enables the model to adaptively focus on feature deviations that contribute more to the current aggregation task or contain more information, and realizes the dynamic allocation of feature importance. In this way, the model can dynamically adjust the weights based on the importance of the features, increase the focus on effective features, reduce the interference of irrelevant information, and suppress the influence of noise or irrelevant variation, thereby improving the accuracy and effectiveness of feature aggregation.

[0080] More specifically, the step S424 is expressed as follows:

[0081]

[0082] in, Figure 4 shows the low dynamic feature aggregation coding vector of laser processing exposure.

[0083] That is, the set of laser processing exposure low dynamic feature vectors is weighted using a set of feature dynamic compensation weight factors, and the laser processing exposure low dynamic feature baseline regression coding vector is then fine-tuned to dynamically incorporate the specific and dynamic features of each laser processing exposure low dynamic feature vector while retaining the common feature information represented by the laser processing exposure low dynamic feature baseline regression coding vector. The generated laser processing exposure low dynamic feature aggregate coding vector can comprehensively and adaptively represent the input data, and integrates the unique contributions and relatively important features of each independent laser processing exposure low dynamic feature vector, significantly improving the robustness and expressiveness of the feature representation, and providing a higher-quality feature foundation for subsequent high dynamic range image generation.

[0084] Specifically, in step S43, the laser processing exposure low dynamic feature aggregation coding vector is passed through the laser processing exposure high dynamic map generator based on AIGC to obtain the laser processing exposure high dynamic range image. It should be understood that AIGC is based on the Transformer architecture and can convert the digitized laser processing exposure low dynamic feature aggregation coding vector into a visual laser processing exposure high dynamic range image, thereby realizing the mapping from feature space to image space. Specifically, the present application first inputs the laser processing exposure low dynamic feature aggregation coding vector into the feature mapping layer of the AIGC generator, converts it into a feature map through upsampling convolution, and then reconstructs the features through a multi-layer Transformer decoder. Finally, it is enlarged to the original image size through deconvolution and pixel shuffling operations, and the output laser processing exposure high dynamic range image can simultaneously present the clear contours of the highlight area and the texture details of the dark area, so that the image dynamic range is significantly improved, and the feature integrity of the overexposed and underexposed areas is improved.

[0085] In the aforementioned automatic tracking method for laser processing, step S5 involves passing the laser-processed exposed high-dynamic range image through a weld analysis model based on a trained neural network model to obtain the weld centerline pixel coordinates. Specifically, the neural network model's multi-layer convolutional feature extraction mechanism accurately captures the gradient changes and texture patterns of the weld region in the high-dynamic range image. The convolutional blocks in the model encoder perform hierarchical extraction of multi-scale features, effectively separating the noise signal generated by strong light reflections from the molten pool from the true weld edge features. This enables the model to preserve the continuity of the weld contour when processing highly reflective areas. Furthermore, the decoder fuses shallow detail features with deep semantic features through upsampling and skip connections, ensuring that weld texture features in dark areas are not missed, thereby fully representing the pixel-level boundaries of the weld region in the semantic segmentation results. Finally, after threshold segmentation and skeletonization, the generated weld centerline pixel coordinates accurately reflect subtle changes in weld geometry, providing a precise visual feature reference for subsequent coordinate transformation.

[0086] Specifically, the training process for the neural network-based weld analysis model is as follows: First, a training dataset containing diverse weld scenarios is constructed, covering complex conditions such as curved surface welds and dissimilar material welding. Laser processing images are collected under different exposure conditions and generated into high-dynamic-range images. The pixel coordinates of the weld centerline in each image are manually annotated as labels to ensure that the data covers interference scenarios such as strong reflections, shadows, and smoke. Next, the network adopts an encoder-decoder architecture. The encoder extracts multi-scale image features through multi-layer convolution operations, including low-dynamic features such as weld edge gradients and texture patterns, as well as semantic features in high dynamic range. Batch normalization and activation functions are combined after each convolution layer to enhance feature representation. The decoder gradually restores image resolution through upsampling operations and fuses shallow detail features output by the encoder with deep semantic features through skip connections to accurately locate the weld centerline. Supervised learning is used for training, with a combination of mean squared error (MSE) and Dice loss as the loss function. The MSE loss constrains the positional deviation between the predicted centerline and the annotated coordinates, while the Dice loss improves the segmentation accuracy of the weld area and background. The optimizer uses the Adam algorithm, with an initial learning rate set to 0.001, which gradually decays with training iterations. During the training process, a data augmentation strategy is introduced to perform random rotation, scaling, brightness adjustment, and Gaussian noise on the image to simulate dynamic interference in actual processing and enhance the generalization ability of the model. The training is divided into multiple traversal processes. Each traversal process randomly extracts batch data and inputs it into the network. The forward propagation obtains the predicted pixel coordinates of the weld centerline. The difference with the label is calculated through the loss function, and the network parameters are updated through backpropagation. The model performance is evaluated on the validation set at regular intervals. If the verification accuracy does not improve after multiple consecutive traversal processes, the training is stopped and the optimal model parameters are saved. The final trained model needs to be verified on the test set to ensure that the weld centerline can still be accurately extracted in complex scenarios to meet the accuracy requirements of automatic tracking in laser processing.

[0087] In the aforementioned automatic tracking method for laser machining, step S6 determines the path deviation based on the weld centerline pixel coordinates, calibrated camera intrinsic parameters, the robot TCP and camera transformation matrix, the robot's current pose data, and a predefined ideal weld path. Specifically, by correcting the distortion of the weld centerline pixel coordinates using the calibrated camera intrinsic parameters, mapping the camera and robot TCP coordinate systems using the robot TCP and camera transformation matrix, and unifying the robot's current pose data with the spatiotemporal reference, a precise mapping of the weld centerline features from visual space to robot motion space is achieved. During the coordinate transformation process, the calibrated camera intrinsic parameters eliminate the distortion of the weld geometry caused by lens distortion, restoring the pixel-level centerline features to a curved form in real space. The robot TCP and camera transformation matrix ensures the spatial correlation between the visual features and the laser machining tool point, avoiding feature misalignment due to relative position deviation. The temporal alignment of the robot's current pose data resolves the issue of inconsistent spatiotemporal references during the robot's dynamic motion, enabling semantic-level comparison of weld features at different times with the ideal weld path within the same motion coordinate system. This multi-parameter coordinated coordinate transformation mechanism can maintain the spatial continuity of weld features in spatial surface weld tracking, and provide accurate path deviation for the subsequent PID controller to generate motion adjustment instructions.

[0088] In the implementation process, the weld centerline pixel coordinates (u1, v1), (u2, v2)…(un, vn) are first extracted from the high-dynamic-range image. Using the calibrated camera intrinsic parameters (focal length f = 1000 pixels, principal point cx = 640, cy = 480, and distortion coefficient k1 = -0.03), these pixel coordinates are distortion-corrected to eliminate lens distortion. These coordinates are then converted to three-dimensional coordinates (Xc1, Yc1, Zc1)…(Xcn, Ycn, Zcn) in the camera coordinate system using the perspective projection formula. Next, the robot TCP and camera transformation matrices are used to convert the weld coordinates in the camera coordinate system to coordinates (Xt1, Yt1, Zt1)…(Xtn, Ytn, Ztn) in the robot TCP coordinate system, spatially associating visual features with machining tool points. Then, based on the robot's current pose data, the weld coordinates in the TCP coordinate system are converted to the robot base coordinate system, resulting in (Xb1, Yb1, Zb1)…(Xbn, Ybn, Zbn), unifying the spatiotemporal reference. Finally, the predefined ideal weld path coordinate sequence in the base coordinate system (Xr1, Yr1, Zr1)…(Xrn, Yrn, Zrn) is read. The Euclidean distance between corresponding positions is calculated point by point, resulting in a three-dimensional path deviation that includes tangential progress deviation, normal lateral offset, and vertical height deviation. This provides a precise basis for subsequent PID control adjustments.

[0089] In the aforementioned automatic tracking method for laser processing, step S7 involves passing the path deviation through a PID controller to generate robot motion adjustment instructions. It should be understood that since the PID controller has real-time dynamic adjustment capabilities and can generate continuous control variables based on the proportional, integral, and differential components of the deviation, the PID algorithm performs real-time calculations on the path deviation to generate executable robot motion adjustment instructions, thereby achieving dynamic correction of the robot's posture. Figure 6 FIG. 1 is a flow chart of sub-step S7 of the automatic tracking method for laser processing according to an embodiment of the present application. Figure 6 As shown, the step S7 includes the following steps: S71, generating a robot motion adjustment amount based on the path deviation and PID control strategy parameters; S72, generating a robot motion adjustment instruction based on the motion adjustment amount.

[0090] Specifically, step S71 generates a robot motion adjustment based on the path deviation and PID control strategy parameters. Leveraging the closed-loop control characteristics of the PID controller, the three-dimensional path deviation, including tangential progress deviation, normal lateral offset, and vertical height deviation, is converted into motion parameters of the robot end effector or each joint, such as displacement and velocity. The resulting robot motion adjustment enables real-time compensation for path deviation.

[0091] Specifically, step S72 generates a robot motion adjustment instruction based on the motion adjustment value. That is, the motion adjustment value representing the motion direction and amplitude is converted into a robot motion adjustment instruction supported by the robot programming language. This motion adjustment value can be interpreted by the robot control cabinet and drive the servo motor to perform the corresponding action, thereby achieving a mapping from the deviation signal to the physical motion, ensuring that the laser processing tool points follow the ideal path and ensuring the continuity and accuracy of weld tracking.

[0092] In summary, the automatic tracking method for laser processing based on the embodiment of the present application is explained, which collects the calibration plate image and calculates the internal parameters through the multi-view camera array deployed on the robot, constructs the dynamic transformation model of the robot TCP and the camera, and then generates a multi-exposure sequence in combination with the robot posture time series data. The high dynamic range feature aggregation algorithm is used to fuse the effective information of the different exposure images, and the weld analysis model based on deep learning is introduced to extract the weld centerline features in the high dynamic range image. Furthermore, the weld centerline coordinates are aligned with the robot posture, calibration parameters and ideal path in time and space, and finally the path deviation is calculated in real time by the PID controller and the motion adjustment instruction is generated. In this way, high-precision tracking of complex scenes such as spatial curved surface welds and welding of dissimilar materials can be achieved, which is beneficial to improving the anti-interference ability and adaptive adjustment efficiency factor of the laser processing process.

[0093] Furthermore, an automatic tracking system for laser processing is also provided.

[0094] Figure 7 FIG. 1 is a block diagram of an automatic tracking system for laser processing according to an embodiment of the present application. Figure 7 As shown, the automatic tracking system 100 for laser processing according to the embodiment of the present application includes: a calibration plate image acquisition module 110, which is used to acquire calibration plate images at different angles through a camera deployed on a robot, and calculate the camera internal parameters after calibration; a calibration plate image analysis module 120, which is used to analyze the calibration plate images acquired at different angles based on the initial robot TCP and camera transformation matrix and the robot posture data time series set to calculate the robot TCP and camera transformation matrix; an exposure image sequence generation module 130, which is used to generate a time series of laser processing exposure images based on the previous frame analysis result and exposure strategy; a feature aggregation analysis module 140, which is used to analyze the laser processing exposure image. A high dynamic range feature aggregation analysis is performed on the time series of the light processing exposure image to obtain a laser processing exposure high dynamic range image; a weld analysis module 150 is used to pass the laser processing exposure high dynamic range image through a weld analysis model based on a trained neural network model to obtain the weld centerline pixel coordinates; a path deviation determination module 160 is used to determine the path deviation based on the weld centerline pixel coordinates, the calibrated camera internal parameters, the robot TCP and camera transformation matrix, the robot's current posture data and a predefined ideal weld path; a motion adjustment instruction generation module 170 is used to pass the path deviation through a PID controller to generate a robot motion adjustment instruction.

[0095] Here, those skilled in the art will appreciate that the specific operations of the various modules in the automatic tracking system for laser processing have been described above with reference to Figures 1 to 6 The automatic tracking method for laser processing has been described in detail, and therefore, its repeated description will be omitted.

[0096] The basic principles of the present invention have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in the present invention are merely illustrative and non-limiting, and should not be construed as necessarily possessed by each embodiment of the present invention. Furthermore, the specific details of the above embodiments are provided for illustrative purposes and to facilitate understanding, and are not intended to be limiting. These details do not necessarily limit the present invention to being implemented using these specific details.

[0097] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiment described above is only schematic. For example, the unit division is only a logical function division, and there may be other division methods in actual implementation. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0098] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be encompassed therein. Any reference to a figure in a claim should not be construed as limiting the claim to which it relates.

[0099] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0100] Finally, it should be noted that the above description has been provided for the purpose of illustration and description. In addition, the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to be limiting. Although the technical solutions may be modified or replaced with equivalents with reference to the preferred embodiments, they do not depart from the spirit and scope of the technical solutions of the present invention.

Claims

1. An automatic tracking method for laser processing, characterized in that: include: The camera deployed on the robot collects images of the calibration plate at different angles and calculates the intrinsic parameters of the camera after calibration; Based on the initial robot TCP and camera transformation matrices and the robot pose data time series, the calibration plate images collected at different angles are analyzed to calculate the robot TCP and camera transformation matrices. Generate a time series of laser processing exposure images based on the previous frame analysis results and exposure strategy; Performing high dynamic range feature aggregation analysis on the time series of the laser processing exposure image to obtain a laser processing exposure high dynamic range image includes the following steps: extracting laser processing exposure low dynamic features from each laser processing exposure image in the time series of the laser processing exposure images to obtain a set of laser processing exposure low dynamic feature vectors; Performing feature compensation dynamic aggregation on the set of the laser processing exposure low dynamic feature vectors to obtain a laser processing exposure low dynamic feature aggregation coding vector; Passing the laser processing exposure low dynamic feature aggregation coding vector through the laser processing exposure high dynamic image generator based on AIGC to obtain the laser processing exposure high dynamic range image; The step of performing feature compensation dynamic aggregation on the set of the laser processing exposure low dynamic feature vectors to obtain the laser processing exposure low dynamic feature aggregation coding vector comprises the following steps: Inputting the set of laser processing exposure low dynamic feature vectors into a feature baseline learning network to obtain a laser processing exposure low dynamic feature baseline regression encoding vector; Calculating a characteristic dynamic compensation factor of each laser processing exposure low dynamic feature vector in the set of laser processing exposure low dynamic feature vectors relative to the laser processing exposure low dynamic feature baseline regression encoding vector to obtain a set of characteristic dynamic compensation factors; Performing regularization processing based on a Softmax activation function on the set of feature dynamic compensation factors to obtain a set of feature dynamic compensation weight factors; Based on the set of feature dynamic compensation weight factors and the set of laser processing exposure low dynamic feature vectors, dynamically compensate the laser processing exposure low dynamic feature baseline regression code vector to obtain the laser processing exposure low dynamic feature aggregate code vector; Passing the laser-processed exposed high dynamic range image through a weld analysis model based on a trained neural network model to obtain weld centerline pixel coordinates; Determine the path deviation based on the weld centerline pixel coordinates, the calibrated camera intrinsic parameters, the robot TCP and camera transformation matrix, the robot's current pose data, and a predefined ideal weld path; The path deviation is passed through a PID controller to generate robot motion adjustment instructions.

2. The automatic tracking method for laser processing according to claim 1, characterized in that: The calibrated camera internal parameters include focal length, principal point and distortion coefficient.

3. The automatic tracking method for laser processing according to claim 1, characterized in that: Based on the initial robot TCP and camera transformation matrices and the robot pose data time series, the calibration plate images collected at different angles are analyzed to calculate the robot TCP and camera transformation matrices, including: ; in, is the robot TCP relative motion matrix between different robot poses, The relative motion matrix of the calibration plate between different robot poses, is the robot TCP and camera transformation matrix.

4. The automatic tracking method for laser processing according to claim 1, characterized in that: The previous frame analysis result is the ratio of over-exposure / under-exposure areas of the previous frame calibration plate image, and the exposure strategy is the target number of exposures and the basic exposure time range.

5. The automatic tracking method for laser processing according to claim 4, characterized in that: Based on the previous frame analysis results and exposure strategy, a time series of laser processing exposure images is generated, including: Calculating a time series set of exposure parameters based on the previous frame analysis result and the exposure strategy; Based on the time sequence set of the exposure parameters, images are captured by a camera to obtain a time sequence of laser processing exposure images.

6. The automatic tracking method for laser processing according to claim 1, characterized in that: Regularization processing based on a Softmax activation function is performed on the set of feature dynamic compensation factors to obtain a set of feature dynamic compensation weight factors, including: Performing smooth mapping stabilization optimization on the set of characteristic dynamic compensation factors to obtain a set of optimized characteristic dynamic compensation factors; Regularization processing based on a Softmax activation function is performed on the set of optimized feature dynamic compensation factors to obtain a set of feature dynamic compensation weight factors.

7. The automatic tracking method for laser processing according to claim 1, characterized in that: The path deviation is passed through a PID controller to generate a robot motion adjustment instruction, including: generating a robot motion adjustment based on the path deviation and PID control strategy parameters; Based on the motion adjustment amount, a robot motion adjustment instruction is generated.

8. An automatic tracking system for laser processing, used to implement the automatic tracking method for laser processing according to any one of claims 1 to 7, characterized in that: include: The calibration plate image acquisition module is used to collect calibration plate images at different angles through the camera deployed on the robot and calculate the camera intrinsic parameters after calibration; The calibration plate image analysis module is used to analyze the calibration plate images collected at different angles based on the initial robot TCP and camera transformation matrix and the robot pose data time series set to calculate the robot TCP and camera transformation matrix; An exposure image sequence generation module is used to generate a time sequence of laser processing exposure images based on the previous frame analysis results and exposure strategy; a feature aggregation analysis module, configured to perform high dynamic range feature aggregation analysis on the time series of the laser processing exposure image to obtain a laser processing exposure high dynamic range image; A weld analysis module, configured to pass the laser processing exposed high dynamic range image through a weld analysis model based on a trained neural network model to obtain pixel coordinates of the weld centerline; a path deviation determination module, configured to determine the path deviation based on the weld centerline pixel coordinates, the calibrated camera intrinsic parameters, the robot TCP and camera transformation matrix, the robot's current pose data, and a predefined ideal weld path; The motion adjustment instruction generating module is used to generate the robot motion adjustment instruction by passing the path deviation through the PID controller.

Images