Intelligent welding visual identification robot cooperative control system for steel tube tower assembly

Abstract

The present application relates to the technical field of image analysis, in particular to a steel pipe tower assembly intelligent welding visual identification robot coordination control system, which comprises a groove surface contaminant identification module, which collects images of the groove surface of a steel pipe tower at multiple different polarization angles, calculates the linear polarization component corresponding to each pixel point, simultaneously converts the original light intensity image to a CIELAB color space, extracts the a channel value, fuses the polarization degree value of each pixel point with the corresponding a channel value, and establishes a contaminant pixel mask. In the present application, during the image analysis process, multi-angle polarization images are introduced and color channel values are fused to achieve accurate identification of small contaminated areas, thereby enhancing the spatial perception ability of the groove contaminants in a complex welding environment; by point-by-point comparison and screening of the spatial coordinates of the contaminant area and the weld edge, the overlapping part of the contaminated area and the welding path is located, thereby improving the pertinence and accuracy of the contaminant interference identification.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a smart welding visual recognition robot collaborative control system for steel pipe tower assemblies. Background Technology

[0002] Image analysis technology is a core branch of computer vision, which mainly studies how to extract, identify, understand and quantify meaningful information from images.

[0003] Existing technologies in practical applications mostly rely on single-frame images or images from fixed angles for feature extraction, lacking the ability to continuously perceive dynamic processes, resulting in a slow response to rapidly evolving or subtle disturbances. Under conditions of multi-interference backgrounds or complex material surfaces, existing technologies rely solely on brightness or color changes for identification, making them susceptible to interference from strong light reflections, welding fumes, and other environmental noise, leading to a reduced contamination detection rate. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a smart welding vision recognition robot cooperative control system for steel pipe tower assemblies.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies includes:

[0006] The bevel surface contaminant identification module acquires images of the steel pipe tower bevel surface under multiple different polarization angles, calculates the linear polarization component corresponding to each pixel, converts the original light intensity image to the CIELAB color space, extracts the a channel value, and fuses the polarization degree value of each pixel with the corresponding a channel value to establish a contaminant pixel mask.

[0007] The welding path contamination area localization module detects pixel gradient changes in the bevel area of ​​the steel pipe tower based on the original light intensity image, generates a weld edge pixel set, aligns the weld edge pixel set with the contaminant pixel mask in spatial position, filters pixels that exist in both the weld edge pixel set and the contaminant pixel mask, and determines the coordinates of the contamination area on the welding path.

[0008] The welding process spatter dynamic analysis module acquires continuous image sequences during the welding process in real time, calculates pixel-level motion vectors between consecutive frames, establishes a global optical flow vector field, and separates and extracts spatter particles generated during the welding process based on the global optical flow vector field through the inter-frame difference method to obtain the spatter particle motion trajectory sequence.

[0009] The welding process stability assessment module calculates the average velocity of all particles based on the spatter particle trajectory sequence, obtains a spatter motion quantification index, determines the stability of the welding process, and generates a welding state adjustment signal.

[0010] Preferably, the step of obtaining the contaminant pixel mask is as follows:

[0011] By synchronously switching four polarizers with polarization angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees, light intensity images of the steel pipe tower bevel surface at each angle are acquired. Four sets of intensity values ​​are extracted for each pixel and mapped to a uniform grayscale range. The normalized linear polarization degree of the pixel is calculated by combining the light intensity values ​​in the four directions, and the normalized linear polarization degree value of each pixel is obtained.

[0012] Based on the normalized linear polarization value, the a-channel value of each pixel in the original image in the CIELAB color space is called to calculate the fusion anomaly response value of each pixel.

[0013] Based on the fusion anomaly response value, all pixels in the image are traversed and a fusion anomaly response threshold is set. A set of pixels with fusion anomaly response values ​​higher than the threshold is selected and mapped to the image space to form a contaminant pixel mask.

[0014] Preferably, the step of obtaining the weld edge pixel set is as follows:

[0015] The original light intensity image matrix is ​​extracted in the bevel area of ​​the steel pipe tower. Each pixel is traversed, and the gray-level difference between the pixel and its neighboring pixel in the horizontal and vertical directions is calculated to form gradient distribution maps in two directions. The directional gradient value of each pixel is recorded according to its position index to obtain the directional gradient value map.

[0016] Based on the directional gradient value map, and combined with the local gray-level statistical features within a fixed window around each pixel, the adaptive enhancement gradient magnitude is calculated.

[0017] Based on the adaptive enhancement gradient magnitude, all pixels greater than the preset threshold are extracted, and the pixels are tracked and connected according to their spatial connectivity. Continuous edge pixels are organized into a set of closed or open structures to form a weld edge pixel set.

[0018] Preferably, the steps for obtaining the coordinates of the polluted area are as follows:

[0019] Based on each pixel in the weld edge pixel set, the spatial coordinate information of the pixel in the horizontal and vertical directions in the original image matrix is ​​extracted, and the spatial coordinate information is saved in the order of pixel index to generate a weld edge pixel spatial coordinate set.

[0020] Based on the set of spatial coordinates of weld edge pixels, the spatial position of contaminated pixels marked by the contaminant pixel mask is called, and the spatial coordinates of each pixel in the set of spatial coordinates of weld edge pixels are compared one by one to determine whether the spatial coordinates are consistent. If they are consistent, it is confirmed that the pixel exists in both the set of weld edge pixels and the contaminant pixel mask. The pixel is recorded point by point and a common pixel set of weld contamination area is formed.

[0021] Based on the common pixel set of the weld contamination area, the spatial coordinates of each common pixel are mapped to the welding path coordinate system, the welding path position of the pixel is marked, and the coordinates of the contamination area on the generated welding path are recorded one by one.

[0022] Preferably, the step of obtaining the global optical flow vector field is as follows:

[0023] The welding area of ​​the steel pipe tower bevel is continuously photographed at a fixed sampling frame rate. Image data is acquired frame by frame, and each frame is marked with a unique timestamp according to the time sequence of image capture. All image data are arranged and combined in sequence according to the timestamp to generate a continuous image sequence of the welding process.

[0024] Based on the continuous image sequence of the welding process, each continuous image pair is called frame by frame, and the gray intensity value of each pixel in the two consecutive frames is extracted. All corresponding pixels in the image are compared point by point. According to the difference in gray value and spatial position of each pixel between the two frames, the horizontal displacement and vertical displacement of each pixel between the two frames are calculated. Combined with the spatial position of the pixel in the image, the displacement vector information of each pixel is recorded to form a set of pixel-level motion vectors between continuous image frames.

[0025] Based on the set of pixel-level motion vectors between consecutive image frames, the displacement vector information of each pixel is traversed, the direction angle and displacement length values ​​of the displacement vector of each pixel are extracted, and the pixels are rearranged in sequence according to their actual spatial positions in the original image. Each pixel is then labeled into the coordinate system of the original image to generate a global optical flow vector field.

[0026] Preferably, the step of obtaining the sequence of splash particle trajectories is as follows:

[0027] Based on the global optical flow vector field, the displacement vector of each pixel position in two adjacent frames is extracted frame by frame. The absolute value of the gray value difference of corresponding pixels in the two frames is calculated respectively. The absolute value of the gray value difference exceeds the preset gray value threshold as the discrimination condition. The pixels that meet the discrimination condition are defined as the initial splash particle pixel positions. The splash particle pixel position set is generated frame by frame.

[0028] Based on the set of splash particle pixel positions, the displacement vector direction angle value and displacement vector length value at each splash particle pixel position are extracted. The difference between the displacement vector direction angle and the displacement vector length of the splash particle pixel positions in adjacent consecutive image frames are compared frame by frame. When the difference between the direction angle and the length do not exceed the set angle and length difference thresholds, the splash particles at the corresponding pixel positions in two adjacent frames are determined to be the same particle. The correspondence of each splash particle is determined frame by frame, and the inter-frame association data of splash particles is generated.

[0029] Based on the inter-frame association data of the splash particles, according to the change trajectory of the corresponding pixel position of each splash particle between consecutive image frames from the appearance to the disappearance, all associated pixel positions of each splash particle are sequentially connected in the image space, the motion trajectory coordinates of each splash particle are recorded, and a sequence of splash particle motion trajectories is generated.

[0030] Preferably, the steps for obtaining the splash motion quantification index are as follows:

[0031] Based on the sequence of splash particle motion trajectories, the image capture timestamp of the starting point of each splash particle motion trajectory is extracted one by one. A uniform unit time interval is set, and the total number of newly appearing splash particles in each unit time interval is counted to form a sequence of the number of splash particles generated per unit time.

[0032] Based on the sequence of splash particle trajectories, the total displacement distance between all pixels within the trajectory of each splash particle is calculated. The start and end image capture timestamps of each splash particle's trajectory are extracted. The duration of motion for each splash particle is calculated. The total displacement distance of each splash particle is divided by the duration of motion of the corresponding particle to obtain the motion velocity value of each splash particle. Then, the average value of the motion velocity values ​​of all splash particles is calculated to generate the average motion velocity of all particles.

[0033] Based on the sequence of the number of splash particles generated per unit time and the average velocity of all particles, the product of the two is defined as the splash motion quantification index.

[0034] Preferably, the step of acquiring the welding state adjustment signal is as follows:

[0035] The splash motion quantification index is compared with the stability threshold of the steel pipe tower welding process. If the splash motion quantification index is greater than or equal to the stability threshold of the steel pipe tower welding process, the current steel pipe tower welding process is determined to be in an unstable state. If the splash motion quantification index is less than the stability threshold of the steel pipe tower welding process, the current steel pipe tower welding process is determined to be in a stable state. Based on the determination result, a welding state adjustment signal is generated.

[0036] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0037] In this invention, during image analysis, multi-angle polarization images are introduced and color channel values ​​are fused to achieve accurate identification of minute contamination areas, enhancing the spatial perception capability of bevel contaminants in complex welding environments. By comparing and filtering the spatial coordinates of contaminant areas with the weld edge point by point, the overlapping parts of contamination areas and welding paths are located, improving the targeting and accuracy of contamination interference identification. By constructing pixel-level motion vectors between continuous image frames and combining motion direction and displacement consistency judgment, the tracking of spatter particles in multiple frames is accurately calibrated to obtain complete particle trajectory information, avoiding the omission or misidentification of welding dynamic interference signals. By statistically analyzing the number and speed of spatter particles per unit time, a comprehensive spatter motion quantification index is constructed, and this index is judged against a threshold, transforming it into a quantifiable analytical benchmark for welding stability. This allows dynamic abnormal states to be perceived and control signals to be driven during the formation process, ensuring real-time response and continuous quality adjustment in the welding process. The image feature extraction logic has been expanded from single-frame static recognition to multi-dimensional and multi-frame fusion judgment. The feature data is structured and refined under the dual constraints of spatial and temporal dimensions, enabling image analysis capabilities to break through the surface recognition and delve into the identification of complex interference and the prediction of quality trends, thereby improving the judgment and control capabilities in welding scenarios. Attached Figure Description

[0038] Figure 1 This is a system flowchart of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0040] Please see Figure 1 The present invention provides a technical solution: a smart welding vision recognition robot collaborative control system for steel pipe tower assemblies, comprising:

[0041] The bevel surface contaminant identification module acquires images of the steel pipe tower bevel surface under multiple different polarization angles, calculates the linear polarization component corresponding to each pixel, converts the original light intensity image to the CIELAB color space, extracts the a channel value, and fuses the polarization degree value of each pixel with the corresponding a channel value to establish a contaminant pixel mask.

[0042] The welding path contamination area localization module detects pixel gradient changes in the bevel area of ​​the steel pipe tower based on the original light intensity image, generates a weld edge pixel set, aligns the weld edge pixel set with the contaminant pixel mask in spatial position, filters pixels that exist in both the weld edge pixel set and the contaminant pixel mask, and determines the coordinates of the contamination area on the welding path.

[0043] The welding process spatter dynamic analysis module acquires continuous image sequences during the welding process in real time, calculates pixel-level motion vectors between consecutive frames, establishes a global optical flow vector field, and separates and extracts spatter particles generated during the welding process based on the global optical flow vector field through the inter-frame difference method to obtain the spatter particle motion trajectory sequence.

[0044] The welding process stability assessment module calculates the average velocity of all particles based on the spatter particle trajectory sequence, obtains a spatter motion quantification index, determines the stability of the welding process, and generates a welding state adjustment signal.

[0045] The steps to obtain the contaminant pixel mask are as follows:

[0046] By synchronously switching four polarizers with polarization angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees, light intensity images of the steel pipe tower bevel surface at each angle are acquired. Four sets of intensity values ​​are extracted for each pixel and mapped to a uniform grayscale range. The normalized linear polarization degree of the pixel is calculated by combining the light intensity values ​​in the four directions, and the normalized linear polarization degree value of each pixel is obtained.

[0047] Based on the normalized linear polarization value, the a-channel value of each pixel in the original image in the CIELAB color space is retrieved, and the fusion anomaly response value of each pixel is calculated using the following formula:

[0048] ;

[0049] in, For the first The fusion anomaly response value of each pixel. For the first The a-channel value of each pixel. This is the average a-channel reference value for pixels in the uncontaminated area. The maximum absolute value that channel a can reach. For the first Normalized linear polarization degree of each pixel, A constant factor for adjusting the intensity of polarization interaction;

[0050] Based on the fusion anomaly response value, all pixels in the image are traversed and a fusion anomaly response threshold is set. A set of pixels with fusion anomaly response values ​​higher than the threshold is selected and mapped to the image space to form a contaminant pixel mask.

[0051] Specifically, after acquiring light intensity images of the steel pipe tower bevel surface at four polarization angles (0°, 45°, 90°, and 135°), the system extracts the original light intensity value of each pixel in the images at its corresponding position in these four polarization images. These original light intensity values ​​are directly output by the image sensor, and their numerical range depends on the sensor's bit depth. For example, for a common 8-bit image sensor, the output intensity value range is 0 to 255. To ensure the accuracy and consistency of subsequent polarization degree calculations, these four extracted intensity values, i.e. , , and (in Using the pixel index and combining the light intensity values ​​under these four polarization directions, the Stokes parameter of each pixel is calculated as follows: , , Based on these Stokes parameters, the linear polarization degree of each pixel is further calculated using the following formula: This yields the normalized linear polarization degree value corresponding to the pixel.

[0052] formula: The advantage of this formula lies in its ability to more accurately identify contaminants on beveled surfaces by fusing pixel color information (a-channel values ​​in CIELAB space) and polarization information (normalized linear polarization degree). Color information helps distinguish contaminants with specific color characteristics (such as reddish rust), while polarization information reflects differences in surface material and roughness. Contaminants typically alter the surface's polarization characteristics. Combining these two factors improves the robustness and accuracy of identification, particularly for contaminants with indistinct color but significantly altered polarization characteristics, or contaminants with indistinct polarization characteristics but large color differences. The function introduces a non-linear enhancement effect, which significantly amplifies the response value when both color difference and polarization degree point to anomalies, making it easier to distinguish contaminants from the background. The introduction of factors allows for the adjustment of the relative contribution weights of color information and polarization information based on actual conditions.

[0053] Parameter description:

[0054] For the first The a-channel value of a pixel in the CIELAB color space. In the CIELAB color space, the a-channel represents the red and green components of the color; positive values ​​indicate redness, and negative values ​​indicate greenness. This value is extracted after converting the RGB or grayscale image captured by the camera to the CIELAB space. For example, for a pixel with RGB values ​​of (R: 150, G: 100, B: 80), its corresponding L, a, and b values ​​can be obtained using a standard RGB to CIELAB conversion algorithm. Here, the a-channel value is extracted; for example, after conversion, we get... .

[0055] The average a-channel reference value for pixels in pre-selected uncontaminated areas represents the typical a-channel color characteristics of a clean steel pipe tower bevel surface. This value is obtained as follows: Before welding, several confirmed uncontaminated areas are selected on the steel pipe tower bevel surface, images of these areas are acquired, converted to the CIELAB color space, the a-channel values ​​of pixels within all selected areas are extracted, and the average of these a-channel values ​​is calculated. For example, five clean areas of 100x100 pixels each were selected, totaling 50,000 pixels. The average value of the 'a' channel for these pixels was calculated to obtain... .

[0056] The a channel value is theoretically the maximum absolute value it can reach. In the standard CIELAB color space, the a channel value is typically defined as ranging from -128 to +127. It can be set to 128, which is used to normalize the difference of channel a to the range of 0-1.

[0057] For the first The normalized linear polarization degree of each pixel, calculated in the previous steps, ranges from 0 to 1 and reflects the degree of polarization of the light reflected from that pixel. For example, the polarization degree of a pixel can be calculated using a polarization image. .

[0058] A constant factor for adjusting the intensity of polarization interaction, used to balance the contribution weights of color information and polarization information in the calculation of the fusion anomaly response value. The settings are determined experimentally on a sample image set containing known contaminants and clean surfaces, based on contaminant type and background characteristics, with the goal of adjusting... This makes the polluted area Values ​​and cleaning areas The value has the greatest discriminative power, for example, by testing different values ​​on samples containing different contaminants such as rust and oil. Values ​​(e.g., from 0.5 to 5.0, in steps of 0.1) are used to assess the accuracy and recall of pollutant detection. If it is found that... When the F1 score for pollutant detection is highest, then set .

[0059] Calculation process:

[0060] With a specific pixel For example, let's substitute the parameters and perform the calculation:

[0061] Given:

[0062] ;

[0063] ;

[0064] ;

[0065] ;

[0066] ;

[0067] Calculation steps:

[0068] Calculate the normalized value of the difference in channel a:

[0069] ;

[0070] calculate The function's input parameters:

[0071] ;

[0072] calculate Function value:

[0073] ;

[0074] Calculate the polarization interaction term:

[0075] ;

[0076] ;

[0077] ;

[0078] Calculate the final fusion anomaly response value :

[0079] ;

[0080] ;

[0081] ;

[0082] The result shows that the fusion anomaly response value of this pixel is 1.68078925. This value will be used for subsequent comparison with the set threshold. If the value is higher than the preset fusion anomaly response threshold, the pixel is judged as a contaminant pixel. The higher the fusion anomaly response value, the greater the possibility that the pixel is a contaminant.

[0083] Based on the fusion anomaly response value of each pixel in the image calculated in the previous step The system will examine each pixel in the image one by one, and then determine the appropriate fusion anomaly response threshold based on that threshold. To screen out potential contaminant pixels, the fusion anomaly response threshold is used. The setting of the standard is crucial, and its determination process is as follows: First, prepare an image dataset containing various typical contaminants (such as rust, oil stains, and scale) and clean steel pipe bevel surfaces. Accurately label the contaminant areas in these images manually to form a "gold standard." Then, for each image in the dataset, calculate the density of all its pixels. The value, next, select a series of candidate thresholds, for example, from The value is traversed between the minimum and maximum values ​​with a step size of 0.01. For each candidate threshold, the pixels in the image are divided into contaminants. ) and non-pollutants ( The results were compared with the manually labeled "gold standard" to calculate the corresponding number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Precision (TP = TP / (TP + FP)) and recall (TP / (TP + FN)) were then calculated. Finally, the F1 score (F1 = 2 * (Precision * Recall) / (Precision + Recall)) was calculated, and the candidate threshold that maximized the F1 score was selected as the final fusion anomaly response threshold. For example, after testing and evaluating 100 sample images containing different pollutants, it was found that when the candidate threshold was 0.75, the average F1 score reached 0.91, which was the highest among all test thresholds. Therefore, the threshold was set as follows: After confirming Then, the system iterates through all pixels in the current image to be detected, and sets each pixel... Value and Compare, if If the pixel is classified as a contaminant pixel, its spatial coordinates (row and column number) are recorded to form a set of contaminant pixels. Finally, all pixels in this set are marked in the two-dimensional space of their original image (for example, these pixels are assigned a value of 1 or 255 in the new binary image, and other pixels are assigned a value of 0), thus forming a binary image in the image space that clearly identifies the distribution area of ​​contaminants, i.e., the contaminant pixel mask.

[0084] The steps for obtaining the pixel set of the weld edge are as follows:

[0085] The original light intensity image matrix is ​​extracted in the bevel area of ​​the steel pipe tower. Each pixel is traversed, and the gray-level difference between the pixel and its neighboring pixel in the horizontal and vertical directions is calculated to form gradient distribution maps in two directions. The directional gradient value of each pixel is recorded according to its position index to obtain the directional gradient value map.

[0086] Based on the directional gradient map and combined with the local gray-level statistical features within a fixed window surrounding each pixel, the adaptive enhancement gradient magnitude is calculated using the following formula:

[0087] ;

[0088] in, Indicates the first The adaptive enhancement gradient magnitude of each pixel. For the first The rate of grayscale change in the horizontal direction of each pixel. For the first The rate of grayscale change in the vertical direction of each pixel. For the first The local variance of grayscale values ​​within a fixed neighborhood window centered at a pixel. The average variance of all local windows in the image. To enhance the coefficient, To prevent constants with a denominator of zero;

[0089] Based on the adaptive enhancement gradient magnitude, all pixels greater than the preset threshold are extracted, and the pixels are traced and connected according to their spatial connectivity. Continuous edge pixels are organized into a set of closed or open structures to form a set of weld edge pixels.

[0090] Specifically, based on the original light intensity image matrix extracted from the bevel area of ​​the steel pipe tower, the system will perform gradient calculation operations on each pixel in the image matrix. Specifically, for any pixel in the image... ,in and These represent its row and column coordinates in the image, respectively. The system will use the central difference method to estimate its gray-level change rate in the horizontal and vertical directions. The gray-level difference in the horizontal direction... By calculating its right-side adjacent pixels Adjacent pixels to the left It is obtained by taking half the difference in grayscale values ​​between them, that is... Similarly, the gray-level difference in the vertical direction By calculating its adjacent pixels below Adjacent pixels above It is obtained by taking half the difference in grayscale values ​​between them, that is... For pixels at the image boundaries, forward or backward differencing is used, or edge padding (e.g., copying edge pixels or padding with zero values) is employed to ensure that gradient values ​​can be calculated for all pixels. After calculating gradient values ​​for all pixels, two independent gradient component maps are generated: a horizontal gradient distribution map containing the horizontal gradient values ​​of all pixels and a vertical gradient distribution map containing the vertical gradient values ​​of all pixels. These two gradient distribution maps together form the directional gradient values ​​of each pixel required for subsequent processing steps, and are stored using their position index in the original image, ultimately resulting in a directional gradient value map.

[0091] formula: The advantage of this formula lies in its combination of traditional gradient magnitude calculation with an adaptive enhancement factor based on local image statistical properties. Traditional gradient operators are sensitive to noise and generate too many false edges in textured regions, while the enhancement term... Utilizing pixels Local variance within the neighborhood The average local variance of the entire image (or region of interest) The relationship between pixels, when a pixel is located in a region with richer texture ( In a relatively smooth region, if the gradient itself is also strong, the enhancement term will further amplify its gradient magnitude, helping to highlight true edges; conversely, in a relatively smooth region... The smaller the value, the weaker the enhancement effect. This adaptability allows the formula to maintain sensitivity to the real edge while suppressing noise and spurious responses within the smooth area to a certain extent, thus improving the accuracy and robustness of weld edge detection.

[0092] Parameter description:

[0093] For the first The rate of grayscale change of each pixel in the horizontal direction, this value is directly derived from the horizontal gradient component in the directional gradient map generated in the previous step, and it reflects the pixel's... The degree of drastic change in light intensity within its horizontal neighborhood, for example, if the pixel is calculated using the aforementioned center difference method. If the horizontal grayscale change is 50, then .

[0094] For the first The rate of grayscale change of each pixel in the vertical direction, this value is also directly derived from the vertical gradient component in the directional gradient map generated in the previous step, and it reflects the pixel's grayscale value. The degree of drastic change in light intensity within its vertical neighborhood, for example, if the pixel is calculated using the aforementioned center difference method. The vertical grayscale change is -30, then .

[0095] For the first The local variance of grayscale values ​​within a fixed neighborhood window centered on a pixel. The size of this window is preset, for example, a 3x3 or 5x5 window. During the calculation, the grayscale values ​​of all pixels within this window are first extracted, and the average of these grayscale values ​​is calculated. Calculate the local variance. For example, for a 3x3 window containing 9 pixel grayscale values ​​{100, 105, 110, 140, 150, 160, 110, 115, 120}, calculate its average value. Then the local variance .

[0096] To calculate the average variance of all non-overlapping or sliding local windows in the image, the original light intensity image of the entire steel pipe tower bevel region is first divided into several segments, and the variance is calculated. Given fixed neighborhood windows of the same size, calculate the local variance of each window. Then, all these local variance values ​​are averaged. For example, if an image is divided into 1000 3x3 windows, the variance of each window is calculated, and these 1000 variance values ​​are summed and then divided by 1000. If the sum is 200,000, then... .

[0097] The enhancement coefficient, used to adjust the contribution of local variance to gradient enhancement, is set based on experimental evaluation on a sample image set containing typical weld features. Specifically, a series of... Candidate values ​​(e.g., from 0.2 to 1.5, with a step size of 0.1) are used for each image in the sample image set. The adaptive enhancement gradient magnitude is calculated and edge extraction is performed. By comparing the extraction results with manually annotated weld edges ("gold standard"), the F1 score is calculated, and the edge with the highest average F1 score is selected. Values, for example, testing 50 sample images, if when At that time, the average F1 score was 0.91, and this F1 score was the highest among all tests. If the value is the highest, then set it. .

[0098] It should be a small positive number, for example, 1.0, to ensure the stability of numerical calculations.

[0099] Calculation process:

[0100] No. Each pixel has the following relevant parameter values:

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] Calculation steps:

[0108] Calculate the basic gradient magnitude :

[0109] ;

[0110] Calculate the variance ratio term:

[0111] ;

[0112] Calculate the logarithmic term:

[0113] ;

[0114] Calculate the product term in the enhancement factor:

[0115] ;

[0116] Calculate the complete enhancement factor (the part in parentheses):

[0117] ;

[0118] Calculate the final adaptive boosting gradient magnitude. :

[0119] ;

[0120] The result indicates that: The adaptive enhancement gradient magnitude of this pixel is 113.576, which is a significant improvement over its original gradient magnitude of 58.3095. This is due to the local variance of the region where this pixel is located. Much larger than the average local variance of the image This indicates that the pixel is located in an area with rich texture or detail, and a higher [percentage]. The value indicates that the pixel is more likely to be an edge point.

[0121] The adaptive enhancement gradient magnitude of each pixel calculated in the previous step To construct the gradient image, the system first needs a preset gradient threshold. To initially filter candidate edge pixels, this preset threshold The determination method is as follows: For a batch (e.g., 100) of representative original light intensity images of the steel pipe tower bevel, calculate the corresponding adaptive enhancement gradient magnitude map. For each gradient magnitude map, count the values ​​of all pixels. The values ​​are sorted, and a fixed percentile, such as the 90th percentile, is selected as the candidate threshold for the image. The average of the thresholds at the 90th percentile of these 100 images is calculated, and this average is used as the global preset gradient threshold. For example, if the average threshold of the 90th percentile in 100 images is 65.5, then set... The system then iterates through each pixel in the gradient image, adaptively enhancing its gradient magnitude. and Compare, all Pixels with a value greater than 65.5 are marked as potential edge points, forming a binary edge candidate image. Then, connectivity analysis is performed on these marked potential edge points, using the eight-neighbor connectivity criterion (i.e., a pixel is considered connected to its eight neighboring pixels in the horizontal, vertical, and diagonal directions). Starting from an unvisited edge candidate point, the system uses tracing algorithms such as breadth-first search or depth-first search to find and connect all connected edge candidate points until all points in the connected region have been visited, forming a continuous edge chain. This process is repeated until all edge candidate points are assigned to an edge chain. These edge chains can be open curve segments or closed contours. Finally, all these sets of open or closed structures representing possible weld boundaries, composed of continuous edge pixels, are summarized to form the weld edge pixel set.

[0122] The steps to obtain the coordinates of the contaminated area are as follows:

[0123] Based on each pixel in the weld edge pixel set, the spatial coordinate information of the pixel in the horizontal and vertical directions in the original image matrix is ​​extracted, and the spatial coordinate information is saved in the order of pixel index to generate a set of weld edge pixel spatial coordinates.

[0124] Based on the set of spatial coordinates of pixels at the weld edge, the spatial position of the contaminated pixels marked by the contaminated pixel mask is called. The spatial coordinates of each pixel in the set of spatial coordinates of pixels at the weld edge are compared one by one to determine whether the spatial coordinates are consistent. If they are consistent, it is confirmed that the pixel exists in both the set of pixels at the weld edge and the contaminated pixel mask. The pixel is recorded point by point and a common pixel set of the weld contaminated area is formed.

[0125] Based on the common pixel set of the weld contamination area, the spatial coordinates of each common pixel are mapped to the welding path coordinate system, the welding path position of the pixel is marked, and the coordinates of the contamination area on the generated welding path are recorded one by one.

[0126] Specifically, based on the weld edge pixel set obtained in the previous steps, the system processes each pixel in this set. First, it determines the precise position of each selected pixel in the original light intensity image matrix. This involves extracting the pixel's two-dimensional spatial coordinates, i.e., its row number in the image matrix (usually representing the vertical direction, denoted as ). ) and column number (usually representing the horizontal direction, denoted as The spatial coordinates of each pixel point are extracted. The coordinates will be recorded strictly according to their original arrangement or processing order in the weld edge pixel set. For example, if the weld edge pixel set is an ordered list containing N pixels, the extracted coordinates will also form a set containing N coordinate pairs. An ordered list of coordinate information is collected and stored to generate a set of pixel spatial coordinates for the weld edge.

[0127] Based on the set of weld edge pixel spatial coordinates generated in the previous step, and calling the previously generated contaminant pixel mask, which is a two-dimensional data structure with the same size as the original image, marking the state of each pixel as contaminant (e.g., contaminant pixels are marked as 1, and non-contaminant pixels are marked as 0), the system will iterate through the pixel coordinates in the set of weld edge pixel spatial coordinates. For each such coordinate, the system will query the corresponding pixel in the contaminant mask. The location marker value is obtained by directly reading the mask. The value at that location determines whether that spatial coordinate is also marked as a contaminant. If the contaminant pixel mask is at that coordinate... If the value at a location is 1 (indicating that the location is contaminant), then the pixel is confirmed to belong to both the weld edge and the contaminant area. The coordinates of this pixel... This process is then recorded and repeated for all pixels in the pixel space coordinate set of the weld edge. The coordinates of all pixels that are confirmed to meet both conditions are collected to form a common pixel set of the weld contamination area.

[0128] Based on the common pixel set of the weld contamination area obtained in the previous step, which contains the image coordinates of all pixels that are both at the weld edge and identified as contaminated, The system will perform a coordinate transformation operation on the coordinates of each pixel in this set, converting them from a two-dimensional image coordinate system (usually in pixels) to a predefined welding path coordinate system. This welding path coordinate system can be a one-dimensional, two-dimensional, or three-dimensional spatial coordinate system (usually in millimeters or meters) that defines the welding robot's motion trajectory. This mapping process relies on a pre-established and calibrated transformation relationship, obtained through camera calibration. For example, for each image coordinate... , apply one The affine transformation matrix or a more complex nonlinear mapping function Calculate its corresponding position in the welding path coordinate system. or distance along the path The system records the welding path coordinates after the conversion of each common pixel. These recorded path coordinates are collected one by one to generate the coordinates of the contaminated area on the welding path.

[0129] The steps for obtaining the global optical flow vector field are as follows:

[0130] The welding area of ​​the steel pipe tower bevel is continuously photographed at a fixed sampling frame rate. Image data is acquired frame by frame, and each frame is marked with a unique timestamp according to the time sequence of image capture. All image data are arranged and combined in sequence according to the timestamp to generate a continuous image sequence of the welding process.

[0131] Based on the continuous image sequence of the welding process, each continuous image pair is called frame by frame, and the gray intensity value of each pixel in the two consecutive frames is extracted. All corresponding pixels in the image are compared point by point. According to the difference in gray value and spatial position of each pixel between the two frames, the horizontal and vertical displacement of each pixel between the two frames is calculated. Combined with the spatial position of the pixel in the image, the displacement vector information of each pixel is recorded to form a set of pixel-level motion vectors between continuous image frames.

[0132] Based on the set of pixel-level motion vectors between consecutive image frames, the displacement vector information of each pixel is traversed, the direction angle and displacement length of the displacement vector of each pixel are extracted, and the pixels are rearranged in sequence according to their actual spatial positions in the original image. They are then labeled one by one into the coordinate system of the original image to generate a global optical flow vector field.

[0133] Specifically, by configuring an industrial camera at a preset fixed sampling frame rate, such as 100 frames per second (100 FPS), the system continuously captures the dynamic situation of the steel pipe tower bevel during the welding process. Each shooting action captures one frame of image data, which is usually a grayscale or color image. While acquiring each frame, the system immediately reads the current precise time from a clock source (such as a system clock synchronized with Network Time Protocol (NTP) or a dedicated hardware clock) and uses it as the timestamp information for that frame. The timestamp format is usually year-month-day-hour-minute-second-millisecond to ensure its uniqueness and high precision. For example, a frame may be marked as "2023-10-27-10-30-15-500". All captured image data, along with their corresponding timestamp information, are stored sequentially. Subsequently, the system arranges all image frames in a strict time sequence according to the order of these timestamps, forming an ordered image set, i.e., a continuous image sequence of the welding process.

[0134] Based on the continuous image sequence of the welding process generated in the previous step, optical flow estimation algorithms, such as the classic Lucas-Kanade method or deep learning-based optical flow networks (such as FlowNet, PWC-Net, etc.), are used to calculate pixel-level motion vectors. For each pair of consecutive image frames in the sequence, denoted as the _ ... Frame Image and the Frame Image The system first extracts the grayscale intensity value of each corresponding pixel in the two frames of images. Each pixel in (in (based on its image space coordinates), the optical flow algorithm will attempt to... In-neighbor search and Most similar pixels Similarity metrics are typically based on brightness constancy (i.e., the brightness of corresponding pixels remains constant over a short period of time) and spatial consistency (i.e., neighboring pixels have similar motion). The horizontal displacement of each pixel is calculated by minimizing some error function (e.g., minimizing the sum of squared grayscale differences in the Lucas-Kanade method). and vertical displacement Once calculated and This yields the pixels. From the Frame to the Motion vector of a frame This process is performed on all (or selected feature) pixels in the image, and records the original image spatial location of each pixel. and its corresponding displacement vector This information together constitutes the set of inter-frame pixel-level motion vectors between the current consecutive image pairs. This process is applied to all consecutive frame pairs in the entire image sequence, forming a series of such sets.

[0135] Based on the series of consecutive image frame pixel-level motion vector sets calculated in the previous step, the system will process each such set. Each set contains the displacement vector information of each pixel in the image from the previous frame to the current frame. and its spatial location in the original image. For the displacement vector of each pixel The system will further calculate its two key attributes: the direction and angle of the displacement vector. And the length (i.e., magnitude) of the displacement vector. , direction angle It is usually calculated using the arctangent function, for example The atan2 function can handle all quadrants and output values ​​from 0 to 1. or arrive Angle values ​​and displacement lengths within the range This is obtained by calculating the Euclidean distance of the displacement vectors, i.e. After obtaining the displacement vector direction angle and length values ​​of each pixel, the system correlates this information with its actual spatial position in the original image. Correlate them, and then, according to their original image coordinates, these motion vectors with direction and length information are aligned. The order is reorganized and visualized (or stored) on a 2D grid of the same size as the original image, with each grid point... The points are labeled with the motion vectors originating from that point (usually indicated by arrows, the direction of which represents the direction of motion). The length of the arrow represents This generates a complete motion vector map, i.e., a global optical flow vector field, for each pair of consecutive frames, covering all pixels (or feature pixels) of the image.

[0136] The steps for obtaining the trajectory sequence of splash particles are as follows:

[0137] Based on the global optical flow vector field, the displacement vector of each pixel position in two adjacent frames is extracted frame by frame. The absolute value of the gray value difference of corresponding pixels in the two frames is calculated respectively. The absolute value of the gray value difference exceeds the preset gray value threshold as the discrimination condition. The pixels that meet the discrimination condition are defined as the initial splash particle pixel positions. The splash particle pixel position set is generated frame by frame.

[0138] Based on the set of splash particle pixel positions, the displacement vector direction angle value and displacement vector length value at each splash particle pixel position are extracted. The difference between the displacement vector direction angle and the displacement vector length of the splash particle pixel positions in adjacent consecutive image frames are compared frame by frame. When the difference between the direction angle and the length do not exceed the set angle and length difference thresholds, the splash particles at the corresponding pixel positions in two adjacent frames are determined to be the same particle. The correspondence of each splash particle is determined frame by frame, and the inter-frame association data of splash particles is generated.

[0139] Based on the inter-frame correlation data of splash particles, according to the change trajectory of the corresponding pixel position of each splash particle between consecutive image frames from the appearance to the disappearance, all the associated pixel positions of each splash particle are sequentially connected in the image space, the motion trajectory coordinates of each splash particle are recorded, and a sequence of splash particle motion trajectories is generated.

[0140] Specifically, based on the previously generated global optical flow vector field, the system will perform calculations for each pair of adjacent raw image frames (denoted as the 1st ... frame and the frame To process, firstly, for Each pixel position in The system will extract the corresponding displacement vector from the global optical flow vector field. At the same time, the system will calculate the position of the pixel in grayscale values ​​in With it The corresponding position in the middle (i.e.) grayscale value The absolute value of the difference between them, i.e. Then, the absolute value of this grayscale difference is compared with a preset grayscale threshold. The preset grayscale threshold is compared. The setup method is as follows: acquire a series of welding process image sequences containing known spatter particles and background, manually label the spatter particle regions and non-spatter background regions in these sequences, and calculate the absolute value of the average grayscale difference between adjacent frames for all pixels labeled as spatter particle regions. and its standard deviation Simultaneously, calculate the absolute value of the average grayscale difference between adjacent frames for all pixels marked as background regions. and its standard deviation Choose a threshold that can better distinguish between splashes and the background; for example, you can set... ,in It is a constant (e.g.) Alternatively, the optimal balance point can be found through ROC curve analysis. For example, after analyzing 50 welding videos (approximately 100 frames per video), the absolute value of the average grayscale difference of pixels in the background area is 10, with a standard deviation of 5, while the absolute value of the average grayscale difference of pixels in the spatter area is 80, with a standard deviation of 20. If we take... ,but If a certain pixel position Greater than If the value is greater than 25, then the pixel is defined as an initial splash particle pixel position in the current frame. All pixel coordinates that meet this condition are collected, and a set of splash particle pixel positions is generated frame by frame.

[0141] Based on the set of splash particle pixel positions generated frame by frame in the previous step, the system further processes these sets to determine the correspondence between particles. For the current frame (denoted as the ), For each pixel in the set of splash particle pixel positions (frame), the system first extracts its corresponding displacement vector direction angle from the global optical flow vector field. and displacement vector length Then, the system will search for the previous frame (the first frame). The set of splash particle pixel positions for the first frame (frame), for the second frame. A splash particle pixel in the frame In position Its motion vector points to its possible position in the previous frame. The system will be in the first In the set of pixel locations of the splashed particles in the frame, search for candidate corresponding particles within a small neighborhood around this back-projected location. For each candidate Extract its displacement vector direction angle and length ,calculate and The difference in the direction angle of the displacement vector between (The angle difference needs to be considered for periodicity, for example, modulus taking) The minimum difference after the displacement vector length difference) and the difference between the displacement vector lengths The system compares these two differences with a preset angle difference threshold. and length difference threshold The two thresholds were set based on statistical analysis of a large amount of real splash particle trajectory data, for example, It can be set to twice the standard deviation of the average angle change of splash particles between consecutive frames. This can be set to twice the standard deviation of the average velocity change of splash particles between consecutive frames (length represents velocity multiplied by the frame interval). For example, if the statistical value of the average angle change of splash particles between consecutive frames is 5 degrees and the standard deviation is 10 degrees, then... It can be set to 25 degrees; the mean length variation is 2 pixels / frame, and the standard deviation is 3 pixels / frame. It can be set to 8 pixels / frame, when (i.e., less than or equal to 25 degrees) and When (i.e., less than or equal to 8 pixels / frame), then it is determined. and To represent the same splash particle in different frames, if there are multiple If the conditions are met, the one with the smallest difference is selected. In this way, the correspondence of each splash particle is determined frame by frame, generating inter-frame correlation data of splash particles. This data records which particles are the same physical particle observed at consecutive time points.

[0142] Based on the inter-frame correlation data of splash particles generated in the previous step, the system can now track the complete lifecycle of each individual splash particle from its first appearance to its final disappearance. For each identified and successfully tracked splash particle, its inter-frame correlation data provides a time series indicating the pixel position of the particle when it appears in consecutive image frames. The system iterates through this correlation data, mapping the pixel positions of all consecutive frames belonging to the same physical particle. Connecting these points in timestamp order (i.e., frame order) creates a discrete sequence of points in the image space. This sequence represents the trajectory of the splash particle. The system records the two-dimensional image coordinates of all points on this trajectory. For example, if a splash particle A appears in frame k, passes through frames k+1, k+2, ..., k+m, and disappears in frame k+m+1, then its trajectory coordinates are: This process is repeated for all tracked splash particles, resulting in a series of such trajectory coordinate sequences, which together constitute the final splash particle trajectory sequence.

[0143] The steps for obtaining the quantitative indicators of splash motion are as follows:

[0144] Based on the sequence of splash particle motion trajectories, the image capture timestamp of the starting point of each splash particle motion trajectory is extracted one by one. A uniform unit time interval is set, and the total number of newly appearing splash particles in each unit time interval is counted to form a sequence of the number of splash particles generated per unit time.

[0145] Based on the sequence of splash particle trajectories, the total displacement distance between all pixels within the trajectory of each splash particle is calculated. The start and end image capture timestamps of each splash particle's trajectory are extracted. The duration of motion for each splash particle is calculated. The total displacement distance of each splash particle is divided by the duration of motion of the corresponding particle to obtain the motion velocity value of each splash particle. Then, the average value of the motion velocity values ​​of all splash particles is calculated to generate the average motion velocity of all particles.

[0146] Based on the sequence of the number of splash particles generated per unit time and the average velocity of all particles, the product of the two is defined as the splash motion quantification index.

[0147] Specifically, based on the sequence of spatter particle trajectories obtained in the previous step, the system first iterates through each individual spatter particle trajectory in this sequence. For each trajectory, the system extracts the capture timestamp of the image frame corresponding to its starting point. This timestamp precisely records the time when the spatter particle was first detected. Next, a uniform unit time interval is set. The choice of this interval needs to be determined based on the dynamic characteristics of the welding process and the required accuracy of the analysis. For example, it can be set to 0.1 seconds or 1 second. Then, the system divides the duration of the entire welding process into continuous such unit time intervals. For each unit time interval... The system counts the total number of newly appearing splash particles within each time interval. This means counting the number of splash particles whose starting timestamps fall within the current time interval. For example, if the unit time interval is 0.1 seconds, the first time interval is from 0.0 seconds to 0.1 seconds, the second is from 0.1 seconds to 0.2 seconds, and so on. The system will calculate how many new splash particles appeared in 0.0-0.1 seconds, how many new splash particles appeared in 0.1-0.2 seconds, and so on. The number of newly appearing splash particles counted in each unit time interval is arranged in chronological order to form a sequence of the number of splash particles generated per unit time.

[0148] Based on the existing sequence of splash particle trajectories, the system performs a detailed analysis of each individual splash particle trajectory. First, for the trajectory of a single splash particle, the trajectory consists of a series of pixel coordinates arranged in chronological order. The system calculates the cumulative displacement distance between all pixels the particle passes through throughout its trajectory. This is done by calculating and summing the Euclidean distances between adjacent pixels, i.e., the total displacement distance. Next, the system extracts the capture timestamp of the starting image frame of the splash particle's trajectory. and the capture timestamp of the last image frame (i.e., the frame in which the particle was last detected). Calculate the duration of the particle's motion. Then, use the total displacement distance of the particle. Divided by the duration of its motion The average velocity of the splashing particle was obtained. This calculation process is repeated for all particles in the sequence of splash particle trajectories to obtain the velocity value of each splash particle. Finally, the average velocity value of all these independent splash particles is calculated, which is to add up the velocities of all particles and then divide by the total number of particles to generate the average velocity of all particles.

[0149] Based on the sequence of spatter particle generation per unit time and the average velocity of all particles obtained in the first two steps, the system now combines these two quantities to calculate a spatter motion quantification index. The specific calculation process is as follows: each value in the sequence of spatter particle generation per unit time (representing the number of newly generated spatter particles within a specific unit time interval) is multiplied by the calculated average velocity of all particles (a single scalar value). Thus, for each time point (or time period) in the sequence of spatter particle generation per unit time, a corresponding product value is obtained. These product values ​​can form a new time series, or a representative value from the sequence of spatter particle generation per unit time (e.g., the average number of particles generated per unit time throughout the welding process, or the average number generated during a key time period) can be multiplied by the average velocity of all particles to obtain a single spatter motion quantification index that represents the spatter activity level of the entire welding process or a specific stage. For example, if 50 new spatter particles are generated within a certain unit time interval, and the average velocity of all particles is calculated to be 100 pixels / second, then the spatter motion quantification index for that time interval is: Pixels per second (Particles per second) is a metric that combines the frequency of splash generation with the average kinetic energy of the splash (velocity is one aspect of kinetic energy). The product of these two values ​​is defined as the splash motion quantification metric.

[0150] The steps for obtaining the welding status adjustment signal are as follows:

[0151] The spatter motion quantification index is compared with the stability threshold of the steel pipe tower welding process. If the spatter motion quantification index is greater than or equal to the stability threshold of the steel pipe tower welding process, the current steel pipe tower welding process is determined to be in an unstable state. If the spatter motion quantification index is less than the stability threshold of the steel pipe tower welding process, the current steel pipe tower welding process is determined to be in a stable state. Based on the determination result, a welding state adjustment signal is generated.

[0152] Specifically, the splash motion quantification index calculated in the previous step, which is a specific value, such as 5000 pixels per second, is compared with a pre-set stability threshold for the steel pipe tower welding process. Comparison of the stability threshold of the welding process of the steel pipe tower The design is based on extensive experimental data and expert experience. The specific process is as follows: First, welding experiments are conducted with multiple combinations of different welding parameters (such as current, voltage, welding speed, and gas flow rate). In each experiment, the spatter motion quantification index and the weld quality grade, assessed by welding experts or through non-destructive testing of weld quality (such as X-ray inspection or ultrasonic testing), are recorded simultaneously. The weld quality grades are categorized as "Excellent" (stable state), "Pass" (critical state), and "Unqualified" (unstable state). The distribution of the spatter motion quantification index under different quality grades is statistically analyzed. For example, it was found that in the "Excellent" state, the spatter motion quantification index is typically below 3000 pixels / second; in the "Pass" state, the index fluctuates between 3000 and 6000 pixels / second; and in the "Unqualified" state, the index is generally above 6000 pixels / second. To determine a specific threshold (pixels per second), statistical methods can be used. For example, a compromise can be chosen between the maximum value (or 95th percentile) of the spatter motion quantization index under all "excellent" conditions and the minimum value (or 5th percentile) of the spatter motion quantization index under all "unsatisfactory" conditions. Alternatively, the average value of the spatter motion quantization index observed under historically stable welding conditions plus twice the standard deviation can be used as an initial threshold, which can then be adjusted based on weld quality feedback from actual production. Pixels per second, if the currently calculated splash motion quantization value (e.g., 5000 pixels per second) is greater than or equal to (Right now If the current steel pipe tower welding process is unstable, then the welding process is considered to be in an unstable state. Conversely, if the spatter motion quantification index value (e.g., 2500 pixels·particles / second) is less than... (Right now If the current welding process of the steel pipe tower is determined to be in a stable state, a corresponding welding state adjustment signal will be generated based on this determination result (stable or unstable).

Claims

1. A smart welding vision recognition robot collaborative control system for steel pipe tower assemblies, characterized in that: The system includes: The bevel surface contaminant identification module acquires images of the steel pipe tower bevel surface under multiple different polarization angles, calculates the linear polarization component corresponding to each pixel, converts the original light intensity image to the CIELAB color space, extracts the a channel value, and fuses the polarization degree value of each pixel with the corresponding a channel value to establish a contaminant pixel mask. The welding path contamination area localization module detects pixel gradient changes in the bevel area of ​​the steel pipe tower based on the original light intensity image, generates a weld edge pixel set, aligns the weld edge pixel set with the contaminant pixel mask in spatial position, filters pixels that exist in both the weld edge pixel set and the contaminant pixel mask, and determines the coordinates of the contamination area on the welding path. The welding process spatter dynamic analysis module acquires continuous image sequences during the welding process in real time, calculates pixel-level motion vectors between consecutive frames, establishes a global optical flow vector field, and separates and extracts spatter particles generated during the welding process based on the global optical flow vector field through the inter-frame difference method to obtain the spatter particle motion trajectory sequence. The welding process stability assessment module calculates the average velocity of all particles based on the spatter particle trajectory sequence, obtains a spatter motion quantification index, determines the stability of the welding process, and generates a welding state adjustment signal.

2. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the pollutant pixel mask are as follows: By synchronously switching four polarizers with polarization angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees, light intensity images of the steel pipe tower bevel surface at each angle are acquired. Four sets of intensity values ​​are extracted for each pixel and mapped to a uniform grayscale range. The normalized linear polarization degree of the pixel is calculated by combining the light intensity values ​​in the four directions, and the normalized linear polarization degree value of each pixel is obtained. Based on the normalized linear polarization value, the a-channel value of each pixel in the original image in the CIELAB color space is called to calculate the fusion anomaly response value of each pixel. Based on the fusion anomaly response value, all pixels in the image are traversed and a fusion anomaly response threshold is set. A set of pixels with fusion anomaly response values ​​higher than the threshold is selected and mapped to the image space to form a contaminant pixel mask.

3. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the weld edge pixel set are as follows: The original light intensity image matrix is ​​extracted in the bevel area of ​​the steel pipe tower. Each pixel is traversed, and the gray-level difference between the pixel and its neighboring pixel in the horizontal and vertical directions is calculated to form gradient distribution maps in two directions. The directional gradient value of each pixel is recorded according to its position index to obtain the directional gradient value map. Based on the directional gradient value map, and combined with the local gray-level statistical features within a fixed window around each pixel, the adaptive enhancement gradient magnitude is calculated. Based on the adaptive enhancement gradient magnitude, all pixels greater than the preset threshold are extracted, and the pixels are tracked and connected according to their spatial connectivity. Continuous edge pixels are organized into a set of closed or open structures to form a weld edge pixel set.

4. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the coordinates of the polluted area are as follows: Based on each pixel in the weld edge pixel set, the spatial coordinate information of the pixel in the horizontal and vertical directions in the original image matrix is ​​extracted, and the spatial coordinate information is saved in the order of pixel index to generate a weld edge pixel spatial coordinate set. Based on the set of spatial coordinates of weld edge pixels, the spatial position of contaminated pixels marked by the contaminant pixel mask is called, and the spatial coordinates of each pixel in the set of spatial coordinates of weld edge pixels are compared one by one to determine whether the spatial coordinates are consistent. If they are consistent, it is confirmed that the pixel exists in both the set of weld edge pixels and the contaminant pixel mask. The pixel is recorded point by point and a common pixel set of weld contamination area is formed. Based on the common pixel set of the weld contamination area, the spatial coordinates of each common pixel are mapped to the welding path coordinate system, the welding path position of the pixel is marked, and the coordinates of the contamination area on the generated welding path are recorded one by one.

5. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the global optical flow vector field are as follows: The welding area of ​​the steel pipe tower bevel is continuously photographed at a fixed sampling frame rate. Image data is acquired frame by frame, and each frame is marked with a unique timestamp according to the time sequence of image capture. All image data are arranged and combined in sequence according to the timestamp to generate a continuous image sequence of the welding process. Based on the continuous image sequence of the welding process, each continuous image pair is called frame by frame, and the gray intensity value of each pixel in the two consecutive frames is extracted. All corresponding pixels in the image are compared point by point. According to the difference in gray value and spatial position of each pixel between the two frames, the horizontal displacement and vertical displacement of each pixel between the two frames are calculated. Combined with the spatial position of the pixel in the image, the displacement vector information of each pixel is recorded to form a set of pixel-level motion vectors between continuous image frames. Based on the set of pixel-level motion vectors between consecutive image frames, the displacement vector information of each pixel is traversed, the direction angle and displacement length values ​​of the displacement vector of each pixel are extracted, and the pixels are rearranged in sequence according to their actual spatial positions in the original image. Each pixel is then labeled into the coordinate system of the original image to generate a global optical flow vector field.

6. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the sequence of splash particle trajectories are as follows: Based on the global optical flow vector field, the displacement vector of each pixel position in two adjacent frames is extracted frame by frame. The absolute value of the gray value difference of corresponding pixels in the two frames is calculated respectively. The absolute value of the gray value difference exceeds the preset gray value threshold as the discrimination condition. The pixels that meet the discrimination condition are defined as the initial splash particle pixel positions. The splash particle pixel position set is generated frame by frame. Based on the set of splash particle pixel positions, the displacement vector direction angle value and displacement vector length value at each splash particle pixel position are extracted. The difference between the displacement vector direction angle and the displacement vector length of the splash particle pixel positions in adjacent consecutive image frames are compared frame by frame. When the difference between the direction angle and the length do not exceed the set angle and length difference thresholds, the splash particles at the corresponding pixel positions in two adjacent frames are determined to be the same particle. The correspondence of each splash particle is determined frame by frame, and the inter-frame association data of splash particles is generated. Based on the inter-frame association data of the splash particles, according to the change trajectory of the corresponding pixel position of each splash particle between consecutive image frames from the appearance to the disappearance, all associated pixel positions of each splash particle are sequentially connected in the image space, the motion trajectory coordinates of each splash particle are recorded, and a sequence of splash particle motion trajectories is generated.

7. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the splash motion quantification index are as follows: Based on the sequence of splash particle motion trajectories, the image capture timestamp of the starting point of each splash particle motion trajectory is extracted one by one. A uniform unit time interval is set, and the total number of newly appearing splash particles in each unit time interval is counted to form a sequence of the number of splash particles generated per unit time. Based on the sequence of splash particle trajectories, the total displacement distance between all pixels within the trajectory of each splash particle is calculated. The start and end image capture timestamps of each splash particle's trajectory are extracted. The duration of motion for each splash particle is calculated. The total displacement distance of each splash particle is divided by the duration of motion of the corresponding particle to obtain the motion velocity value of each splash particle. Then, the average value of the motion velocity values ​​of all splash particles is calculated to generate the average motion velocity of all particles. Based on the sequence of the number of splash particles generated per unit time and the average velocity of all particles, the product of the two is defined as the splash motion quantification index.

8. The intelligent welding visual recognition robot collaborative control system for steel pipe tower assemblies according to claim 1, characterized in that, The steps for obtaining the welding status adjustment signal are as follows: The splash motion quantification index is compared with the stability threshold of the steel pipe tower welding process. If the splash motion quantification index is greater than or equal to the stability threshold of the steel pipe tower welding process, the current steel pipe tower welding process is determined to be in an unstable state. If the splash motion quantification index is less than the stability threshold of the steel pipe tower welding process, the current steel pipe tower welding process is determined to be in a stable state. Based on the determination result, a welding state adjustment signal is generated.

Images