Intelligent welding visual identification robot cooperative control system for steel tube tower assembly

Abstract

The invention relates to the technical field of image analysis, in particular to a steel tube tower assembly intelligent welding visual identification robot cooperative control system, which comprises a groove surface pollutant identification module, a groove surface pollutant identification module, a linear polarization component calculation module and a control module, meanwhile, the original light intensity image is converted into a CIELAB color space, an a channel value is extracted, the polarization degree value of each pixel point is fused with the corresponding a channel value, and a pollutant pixel mask is established. According to the method, in the image analysis process, the multi-angle polarization image is introduced and the color channel value is fused, so that precise recognition of a tiny pollution area is achieved, and the space sensing capacity of groove pollutants in a complex welding environment is enhanced; the space coordinates of the pollutant area and the welding seam edge are compared and screened point by point, the overlapped part of the pollutant area and the welding path is positioned, and the pertinence and accuracy of pollution interference identification are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image analysis, in particular to a steel pipe tower assembly intelligent welding visual identification robot cooperative control system. BACKGROUND

[0002] The technical field of image analysis belongs to the core branch of computer vision, mainly studying how to extract, identify, understand and quantify meaningful information from images.

[0003] The prior art relies on single-frame images or fixed-angle image information for feature extraction in actual application, lacks continuous perception ability for dynamic processes, and causes delayed response to rapidly evolving or weak interference phenomena. In a multi-interference background or complex material surface condition, the prior art only identifies based on brightness or color changes, is easily disturbed by environmental noise such as strong light reflection and welding smoke, and thus leads to a decrease in pollution identification rate. Therefore, improvement is needed. SUMMARY

[0004] The purpose of the present application is to solve the shortcomings in the prior art and provide a steel pipe tower assembly intelligent welding visual identification robot cooperative control system.

[0005] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme: the steel pipe tower assembly intelligent welding visual identification robot cooperative control system comprises: A bevel surface contaminant identification module acquires images of the bevel surface of a steel pipe tower at multiple different polarization angles, calculates the linear polarization component corresponding to each pixel point, simultaneously converts the original light intensity image to the CIELAB color space, extracts the a channel value, fuses the polarization degree value of each pixel point with the corresponding a channel value, and establishes a contaminant pixel mask; A welding path contamination area positioning module detects the pixel gradient change in the bevel area of the steel pipe tower based on the original light intensity image, generates a weld edge pixel set, aligns the weld edge pixel set and the contaminant pixel mask in spatial position, screens the pixel points existing in both the weld edge pixel set and the contaminant pixel mask, and determines the contamination area coordinates on the welding path; A welding process spatter dynamic analysis module acquires a continuous image sequence in the welding process in real time, calculates the motion vector at the pixel level between consecutive frames, establishes a global optical flow vector field, separates and extracts the spatter particles generated in the welding process based on the global optical flow vector field through an inter-frame difference method, and obtains a spatter particle motion trajectory sequence; A welding process stability evaluation module calculates the average motion speed of all particles based on the spatter particle motion trajectory sequence, obtains a spatter motion quantitative index, determines the stability of the welding process, and generates a welding state adjustment signal.

[0006] Preferably, the step of obtaining the contaminant pixel mask is: By synchronously switching four polarizing angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees, the light intensity images of the steel pipe tower bevel surface under each angle are collected, four groups of intensity values are extracted for each pixel point and mapped to a unified gray scale range, the normalized linear polarization degree of the pixel point is calculated combining the light intensity values in four directions, and the normalized linear polarization degree value of each pixel point is obtained; According to the normalized linear polarization degree value, the a channel value of each pixel of the original image in the CIELAB color space is called, and the fusion abnormal response value of each pixel point is calculated; According to the fusion abnormal response value, all pixel points in the image are traversed and a fusion abnormal response threshold is set, a pixel point set with a fusion abnormal response value higher than the threshold is screened, the pixel point set is mapped to the image space, and a contaminant pixel mask is formed.

[0007] Preferably, the step of obtaining the weld edge pixel set is: The original light intensity image matrix is extracted in the steel pipe tower bevel area, the gray difference between each pixel point and the adjacent pixel in the horizontal direction and the vertical direction is calculated respectively, the gradient distribution graph in two directions is formed, and the directional gradient value of each pixel is recorded according to the position index, and the directional gradient value graph is obtained; According to the directional gradient value graph, the adaptive enhanced gradient modulus value is calculated combining the local gray scale statistical characteristics in the fixed window around each pixel; According to the adaptive enhanced gradient modulus value, all pixel points greater than a preset threshold are extracted, the continuous edge pixels are connected according to the connectivity of the pixels in space, the continuous edge pixels are organized into closed or open structure sets, and a weld edge pixel set is formed.

[0008] Preferably, the step of obtaining the contaminant area coordinates is: Based on each pixel point in the weld edge pixel set, the spatial coordinate information of the pixel point in the horizontal and vertical directions in the original image matrix is extracted respectively, and the spatial coordinate information is sequentially saved according to the pixel point index order to generate a weld edge pixel spatial coordinate set; According to the weld edge pixel spatial coordinate set, the spatial position of the contaminated pixel marked by the contaminant pixel mask is called, the spatial coordinates of each pixel point in the weld edge pixel spatial coordinate set are compared one by one, it is judged whether the spatial coordinates are consistent, if consistent, it is confirmed that the pixel point exists in the weld edge pixel set and the contaminant pixel mask at the same time, the common pixel set of the weld contaminant area is recorded and formed point by point; Based on the common pixel set of the weld contamination area, the spatial coordinates of each common pixel are respectively mapped into the weld path coordinate system, the weld path position where the pixel is located is marked, and the contamination area coordinates on the weld path are recorded.

[0009] Preferably, the step of obtaining the global optical flow vector field is: The steel pipe tower groove welding area is continuously shot at a fixed sampling frame rate, image data is obtained frame by frame, each frame of image is marked with unique timestamp information in time sequence of image capture, all image data is arranged and combined in time sequence according to the timestamp, and a continuous image sequence of the welding process is generated. Based on the continuous image sequence of the welding process, each continuous image pair is called frame by frame, the gray intensity values of each pixel point in the front and back two frames of images are extracted respectively, the corresponding pixel points in the images are compared point by point, the horizontal displacement and the vertical displacement of each pixel point between the two frames of images are calculated according to the gray value difference and the spatial position change relationship between the front and back frames of each pixel point, the displacement vector information of each pixel point is recorded combined with the image spatial position of the pixel point, and a pixel-level motion vector set between continuous image frames is formed. Based on the pixel-level motion vector set between continuous image frames, the displacement vector information of each pixel point is traversed, the direction angle and the displacement length value of the displacement vector of each pixel point are extracted, the original image coordinate system is labeled in sequence according to the actual spatial position of each pixel point in the original image, and a global optical flow vector field is generated.

[0010] Preferably, the step of obtaining the splash particle motion trajectory sequence is: Based on the global optical flow vector field, the displacement vectors of each pixel position in the adjacent two frames of images are extracted frame by frame, the absolute values of the gray value difference of the corresponding pixel points in the front and back two frames of images are calculated respectively, the pixel points meeting the discrimination condition are defined as initial splash particle pixel positions when the absolute value of the gray value difference exceeds the preset gray threshold, and a splash particle pixel position set is generated frame by frame. According to the splash particle pixel position set, the direction angle value and the length value of the displacement vector of each splash particle pixel position are extracted respectively, the direction angle difference and the length difference of the displacement vector of the splash particle pixel position in the adjacent continuous image frames are compared frame by frame, and when the direction angle difference and the length difference do not exceed the set angle and length difference threshold, it is determined that the splash particles of the corresponding pixel positions in the two adjacent frames are the same particle, the corresponding relationship of each splash particle is determined frame by frame, and splash particle inter-frame association data is generated. Based on the interframe association data of the splash particles, the sequential connection of all associated pixel positions of each splash particle in the image space is performed according to the change track of the corresponding pixel position between the continuous image frames during the period from the appearance to the disappearance of each splash particle, the motion track coordinates of each splash particle are recorded, and a splash particle motion track sequence is generated.

[0011] Preferably, the obtaining step of the splash motion quantification index is: Based on the splash particle motion track sequence, the image capture time stamp of the starting point of the motion track of each splash particle is extracted one by one, a uniform unit time interval is set, the total number of newly appeared splash particles in each unit time interval is counted, and a splash particle unit time generation quantity sequence is formed in turn; According to the splash particle motion track sequence, the total displacement distance between all pixel points in the motion track of each splash particle is calculated one by one, the starting image capture time stamp and the ending image capture time stamp of the motion track of each splash particle are extracted one by one, the motion duration of each splash particle is calculated respectively, and the total displacement distance of each splash particle is divided by the motion duration of the corresponding particle to obtain the motion speed value of each splash particle, and then the average value of the motion speed values of all splash particles is calculated to generate the average motion speed of all particles; Based on the splash particle unit time generation quantity sequence and the average motion speed of all particles, the product of the two is calculated and defined as the splash motion quantification index.

[0012] Preferably, the obtaining step of the welding state adjustment signal is: The splash motion quantification index is compared with the steel pipe tower welding process stability threshold value in numerical value, if the splash motion quantification index value is greater than or equal to the steel pipe tower welding process stability threshold value, it is determined that the current steel pipe tower welding process is in an unstable state, if the splash motion quantification index value is less than the steel pipe tower welding process stability threshold value, it is determined that the current steel pipe tower welding process is in a stable state, and the welding state adjustment signal is generated according to the determination result.

[0013] Compared with the prior art, the advantages and positive effects of the present application are: In the present application, during image analysis, multi-angle polarization images are introduced and color channel values are fused to achieve accurate identification of small contaminated areas, enhancing the spatial perception of groove contaminants in complex welding environments. By comparing and screening the spatial coordinates of the contaminated area and the weld edge point by point, the overlapping part of the contaminated area and the welding path is located, improving the pertinence and accuracy of the contamination interference identification. By constructing pixel-level motion vectors between consecutive image frames and combining motion direction and displacement consistency judgment, the tracking of splashing particles in multiple images is accurately calibrated, complete particle trajectory information is obtained, and the omission or misidentification of welding dynamic interference signals is avoided. By counting the number and movement speed of splashing particles per unit time, a comprehensive splashing motion quantitative index is constructed, and the index is judged with a threshold value to convert it into a quantifiable analysis benchmark for welding stability. The dynamic abnormal state is perceived and the control signal is driven during the formation process, ensuring real-time response and continuous quality adjustment of the welding process. The image feature extraction logic is expanded from single-frame static identification to multi-dimensional and multi-frame fusion judgment, and the feature data is structured and refined under the dual constraints of space and time dimensions, breaking through the surface recognition and deepening to the complex interference recognition and quality trend prediction link, improving the judgment and control ability in the welding scene. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 The system flowchart of the present application. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0016] Please refer to Figure 1 The present application provides a technical scheme: a steel pipe tower assembly intelligent welding visual recognition robot cooperative control system comprises: A groove surface contaminant identification module collects images of the groove surface of the steel pipe tower under multiple different polarization angles, calculates the linear polarization component corresponding to each pixel point, simultaneously converts the original light intensity image to the CIELAB color space, extracts the a channel value, fuses the polarization degree value of each pixel point with the corresponding a channel value, and establishes a contaminant pixel mask; A welding path contaminated area positioning module detects the pixel gradient change in the groove area of the steel pipe tower based on the original light intensity image, generates a weld edge pixel set, aligns the weld edge pixel set and the contaminant pixel mask in spatial position, screens the pixel points existing in the weld edge pixel set and the contaminant pixel mask at the same time, and determines the contaminated area coordinates on the welding path; The welding process spatter dynamic analysis module acquires continuous image sequences during the welding process in real time, calculates pixel-level motion vectors between consecutive frames, establishes a global optical flow vector field, and separates and extracts spatter particles generated during the welding process based on the global optical flow vector field through the inter-frame difference method to obtain the spatter particle motion trajectory sequence. The welding process stability assessment module calculates the average velocity of all particles based on the spatter particle trajectory sequence, obtains a spatter motion quantification index, determines the stability of the welding process, and generates a welding state adjustment signal.

[0017] The steps to obtain the contaminant pixel mask are as follows: By synchronously switching four polarizers with polarization angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees, light intensity images of the steel pipe tower bevel surface at each angle are acquired. Four sets of intensity values ​​are extracted for each pixel and mapped to a uniform grayscale range. The normalized linear polarization degree of the pixel is calculated by combining the light intensity values ​​in the four directions, and the normalized linear polarization degree value of each pixel is obtained. Based on the normalized linear polarization value, the a-channel value of each pixel in the original image in the CIELAB color space is retrieved, and the fusion anomaly response value of each pixel is calculated using the following formula: ; in, For the first The fusion anomaly response value of each pixel. For the first The a-channel value of each pixel. This is the average a-channel reference value for pixels in the uncontaminated area. The maximum absolute value that channel a can reach. For the first Normalized linear polarization degree of each pixel, A constant factor for adjusting the intensity of polarization interaction; Based on the fusion anomaly response value, all pixels in the image are traversed and a fusion anomaly response threshold is set. A set of pixels with fusion anomaly response values ​​higher than the threshold is selected and mapped to the image space to form a contaminant pixel mask.

[0018] Specifically, after acquiring light intensity images of the steel pipe tower bevel surface at four polarization angles (0°, 45°, 90°, and 135°), the system extracts the original light intensity value of each pixel in the images at its corresponding position in these four polarization images. These original light intensity values ​​are directly output by the image sensor, and their numerical range depends on the sensor's bit depth. For example, for a common 8-bit image sensor, the output intensity value range is 0 to 255. To ensure the accuracy and consistency of subsequent polarization degree calculations, these four extracted intensity values, i.e. , , and where is the pixel index), combined with the light intensity values in these four polarization directions, the Stokes parameters of each pixel point are calculated, which are calculated as follows: , , Based on these Stokes parameters, the linear polarization degree of each pixel point is further calculated, and the calculation formula is , and the normalized linear polarization degree value corresponding to the pixel point is obtained.

[0019] Formula: The advantages of the formula are: the formula can more accurately identify the contaminants on the bevel surface by fusing the color information (a channel value in CIELAB space) and polarization information (normalized linear polarization degree) of the pixel. The color information helps to distinguish contaminants with specific color characteristics (such as red rust), and the polarization information can reflect the differences in surface material and roughness of the object. Contaminants usually change the polarization characteristics of the surface, and the combination of the two can improve the robustness and accuracy of identification, especially for contaminants with no obvious color but significant changes in polarization characteristics, or contaminants with no obvious polarization characteristics but significant color differences, which can be effectively responded. The function in the formula introduces a nonlinear enhancement effect, so that when both the color difference and the polarization degree point to an anomaly, the response value will be significantly amplified, making it easier to distinguish contaminants from the background, The introduction of the factor makes it possible to adjust the relative contribution weight of color information and polarization information according to actual conditions.

[0020] Parameter description: is the a channel value of the pixel in the CIELAB color space. In the CIELAB color space, the a channel represents the red-green component of the color, with positive values being red and negative values being green. This value is obtained by converting the RGB image or grayscale image captured by the camera to the CIELAB space, for example, for a certain pixel point, its RGB value is (R: 150, G: 100, B: 80), through the standard RGB to CIELAB conversion algorithm, its corresponding L, a, b values can be obtained, here the a channel value is extracted, for example, after conversion, it is obtained .

[0021] is the average a-channel reference value of pre-selected uncontaminated area pixels, representing the typical a-channel color characteristics of a clean steel pipe tower groove surface, which is obtained by: before welding, select several uncontaminated areas on the surface of the steel pipe tower groove, collect the images of these areas, convert them to CIELAB color space, extract the a-channel values of all selected area pixels, and calculate the average value of these a-channel values to obtain For example, 5 clean areas with an area of 100x100 pixels are selected, a total of 50000 pixels, and the average value of the a-channel values of these pixels is calculated to obtain .

[0022] is the maximum absolute value that the a-channel value can theoretically reach, and in the standard CIELAB color space, the a-channel value range is usually defined as -128 to +127, so can be set to 128, which is used to normalize the a-channel difference to the range of 0-1.

[0023] is the normalized linear polarization degree of the th pixel, which is calculated by the previous step and has a value range of 0 to 1, reflecting the polarization degree of the reflected light of the pixel, for example, the of a certain pixel is calculated by the polarization image.

[0024] is a constant factor for adjusting the strength of polarization interaction, used to balance the contribution weight of color information and polarization information in the calculation of fusion abnormal response value, is set according to the type of pollutants and background characteristics, determined by testing on a sample image set containing known pollutants and clean surfaces, the goal is to adjust so that the value of the contaminated area and the value of the clean area have the largest degree of distinction, for example, by testing different values (such as from 0.5 to 5.0, step 0.1) on samples containing different pollutants such as rust and oil stains, and evaluating the accuracy and recall rate of pollutant detection, if it is found that when , the F1 score of pollutant detection is the highest, then set .

[0025] Calculation process: Take a specific pixel as an example, and calculate with parameters: Known: ; ; ; ; ; Computing step: Computing the normalized value of the a-channel difference: ; Computing Input parameters of the function: ; Computing Function value: ; Computing the polarization interaction term: ; ; ; Computing the final fusion anomaly response value : ; ; ; The result shows that the fusion anomaly response value of this pixel point is 1.68078925, which will be used for subsequent comparison with the set threshold value. If the value is higher than the preset fusion anomaly response threshold value, the pixel point is judged as a pollutant pixel. The higher the fusion anomaly response value, the greater the possibility that the pixel point is a pollutant.

[0026] According to the fusion anomaly response value of each pixel point in the image calculated in the previous step matrix, the system will check each pixel in the image one by one, and screen out potential pollutant pixel points according to a preset fusion anomaly response threshold value The setting of the fusion anomaly response threshold value is crucial, and its determination process is as follows: first, prepare an image dataset containing various typical pollutants (such as rust, oil stains, and oxide skin) and clean steel pipe bevel surfaces, and accurately manually label the pollutant areas in these images to form a "golden standard". Then, for each image in the dataset, calculate the value of all pixels in the image. Next, select a series of candidate threshold values, for example, traverse from the minimum value to the maximum value of the value with a step size of 0.01. For each candidate threshold value, divide the pixels in the image into pollutants ( ) and non-pollutants ( The results were compared with the manually labeled "gold standard" to calculate the corresponding number of true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Precision (TP = TP / (TP + FP)) and recall (TP / (TP + FN)) were then calculated. Finally, the F1 score (F1 = 2 * (Precision * Recall) / (Precision + Recall)) was calculated, and the candidate threshold that maximized the F1 score was selected as the final fusion anomaly response threshold. For example, after testing and evaluating 100 sample images containing different pollutants, it was found that when the candidate threshold was 0.75, the average F1 score reached 0.91, which was the highest among all test thresholds. Therefore, the threshold was set as follows: After confirming Then, the system iterates through all pixels in the current image to be detected, and sets each pixel... Value and Compare, if If the pixel is classified as a contaminant pixel, its spatial coordinates (row and column number) are recorded to form a set of contaminant pixels. Finally, all pixels in this set are marked in the two-dimensional space of their original image (for example, these pixels are assigned a value of 1 or 255 in the new binary image, and other pixels are assigned a value of 0), thus forming a binary image in the image space that clearly identifies the distribution area of ​​contaminants, i.e., the contaminant pixel mask.

[0027] The steps for obtaining the pixel set of the weld edge are as follows: The original light intensity image matrix is ​​extracted in the bevel area of ​​the steel pipe tower. Each pixel is traversed, and the gray-level difference between the pixel and its neighboring pixel in the horizontal and vertical directions is calculated to form gradient distribution maps in two directions. The directional gradient value of each pixel is recorded according to its position index to obtain the directional gradient value map. Based on the directional gradient map and combined with the local gray-level statistical features within a fixed window surrounding each pixel, the adaptive enhancement gradient magnitude is calculated using the following formula: ; in, Indicates the first The adaptive enhancement gradient magnitude of each pixel. For the first The rate of grayscale change in the horizontal direction of each pixel. For the first The rate of grayscale change in the vertical direction of each pixel. For the first the local variance of the gray value in a fixed neighborhood window centered on the pixel, the average variance of all local windows in the image, the enhancement coefficient, a constant to prevent the denominator from being zero; According to the adaptive enhancement gradient modulus value, all pixel points greater than a preset threshold are extracted, and the continuous edge pixels are organized into a closed or open structure set according to the connectivity of the pixels in space, to form a weld edge pixel set.

[0028] Specifically, based on the original light intensity image matrix extracted in the groove area of the steel tube tower, the system will perform gradient calculation operation on each pixel point in the image matrix. Specifically, for any pixel point in the image , where and represent the row and column coordinates of the pixel in the image, respectively. The system will use the central difference method to estimate the gray rate of change in the horizontal and vertical directions. The horizontal gray difference is obtained by calculating half of the difference between the gray values of the right adjacent pixel and the left adjacent pixel , that is, Similarly, the vertical gray difference is obtained by calculating half of the difference between the gray values of the lower adjacent pixel and the upper adjacent pixel , that is, For the pixel points at the image boundary, forward or backward difference is used, or image edge padding (such as copying edge pixels or padding with zero values) is used to ensure that all pixel points can calculate the gradient value. After the calculation of all pixel points is completed, two independent gradient component maps are generated: one contains the horizontal gradient distribution of all pixel points, and the other contains the vertical gradient distribution of all pixel points. These two gradient distribution maps together form the directional gradient value of each pixel point required for the subsequent processing step, and are stored with the position index of the pixel in the original image. Finally, the directional gradient value map is obtained.

[0029] Formula: The advantage of the formula is that the formula combines the traditional gradient amplitude calculation with an adaptive enhancement factor based on the local image statistical characteristics. The traditional gradient operator is sensitive to noise and produces too many false edges in complex texture areas, while the enhancement term uses the local variance in the neighborhood of the pixel point and the average local variance of the entire image (or the region of interest).The relationship between pixels, when a pixel is located in a region with richer texture ( In a relatively smooth region, if the gradient itself is also strong, the enhancement term will further amplify its gradient magnitude, helping to highlight the true edges; conversely, in a relatively smooth region... The smaller the value, the weaker the enhancement effect. This adaptability allows the formula to maintain sensitivity to the real edge while suppressing noise and spurious responses within the smooth area to a certain extent, thus improving the accuracy and robustness of weld edge detection.

[0030] Parameter description: For the first The rate of grayscale change of each pixel in the horizontal direction is directly derived from the horizontal gradient component in the directional gradient map generated in the previous step. It reflects the pixel's... The degree of drastic change in light intensity within its horizontal neighborhood, for example, if the pixel is calculated using the aforementioned center difference method. If the horizontal grayscale change is 50, then .

[0031] For the first The rate of grayscale change of each pixel in the vertical direction, this value is also directly derived from the vertical gradient component in the directional gradient map generated in the previous step, and it reflects the pixel's grayscale value. The degree of drastic change in light intensity within its vertical neighborhood, for example, if the pixel is calculated using the aforementioned center difference method. The vertical grayscale change is -30, then .

[0032] For the first The local variance of grayscale values ​​within a fixed neighborhood window centered on a pixel. The size of this window is preset, for example, a 3x3 or 5x5 window. During the calculation, the grayscale values ​​of all pixels within this window are first extracted, and the average of these grayscale values ​​is calculated. Calculate the local variance. For example, for a 3x3 window containing 9 pixel grayscale values ​​{100, 105, 110, 140, 150, 160, 110, 115, 120}, calculate its average value. Then the local variance .

[0033] To calculate the average variance of all non-overlapping or sliding local windows in the image, the original light intensity image of the entire steel pipe tower bevel region is first divided into several segments, and the variance is calculated. Given fixed neighborhood windows of the same size, calculate the local variance of each window. Then all these local variance values are averaged, for example, if an image is divided into 1000 windows of 3x3, after calculating the variance of each window, the 1000 variance values are added and divided by 1000, if the sum is 200000, then .

[0034] is an enhancement coefficient, which is used to adjust the degree of contribution of local variance to gradient enhancement, the value of which is set based on experimental evaluation of a sample image set containing typical weld features, the specific method is: select a series of candidate values (for example, from 0.2 to 1.5, step 0.1), for each image in the sample image set, use different values to calculate the adaptive enhanced gradient modulus and perform edge extraction, by comparing the extraction results with the manually labeled weld edge ("golden standard"), calculate the F1 score, select the value that can make the average F1 score the highest, for example, test 50 sample images, when , the average F1 score is 0.91, and this F1 score is the highest among all test values, set .

[0035] is a small normal number, for example, it can be set to 1.0 to ensure the stability of numerical calculation.

[0036] Calculation process: The pixel, its related parameter values are as follows: ; ; ; ; ; ; Calculation steps: Calculate the basic gradient amplitude : ; Calculate the variance ratio term: ; Calculate the logarithmic term: ; Calculate the product term in the enhancement factor: ; Calculate the complete enhancement factor (the part in parentheses): ; Calculate the final adaptive boosting gradient magnitude. : ; The result indicates that: The adaptive enhancement gradient magnitude of this pixel is 113.576, which is a significant improvement over its original gradient magnitude of 58.3095. This is due to the local variance of the region where this pixel is located. Much larger than the average local variance of the image This indicates that the pixel is located in an area with rich texture or detail, and a higher [percentage]. The value indicates that the pixel is more likely to be an edge point.

[0037] The adaptive enhancement gradient magnitude of each pixel calculated in the previous step To construct the gradient image, the system first needs a preset gradient threshold. To initially filter candidate edge pixels, this preset threshold The determination method is as follows: For a batch (e.g., 100) of representative original light intensity images of the steel pipe tower bevel, calculate the corresponding adaptive enhancement gradient magnitude map. For each gradient magnitude map, count the values ​​of all pixels. The values ​​are sorted, and a fixed percentile, such as the 90th percentile, is selected as the candidate threshold for the image. The average of the thresholds at the 90th percentile of these 100 images is calculated, and this average is used as the global preset gradient threshold. For example, if the average threshold of the 90th percentile in 100 images is 65.5, then set... The system then iterates through each pixel in the gradient image, adaptively enhancing its gradient magnitude. and Compare, all Pixel points greater than 65.5 are marked as potential edge points, forming a binary edge candidate image, then, connectivity analysis is performed on these marked potential edge points, using eight-neighbor connectivity criteria (i.e. a pixel point and its eight adjacent pixels in horizontal, vertical and diagonal directions are considered to be connected), the system starts from an unvisited edge candidate point, and through a breadth-first search or depth-first search algorithm, all connected edge candidate points are found and connected, until all points in the connected region are visited, forming a continuous edge chain, repeating this process until all edge candidate points are attributed to a certain edge chain, these edge chains can be open curve segments or closed contours, finally, all these open or closed structures composed of continuous edge pixels, representing the possible boundaries of the weld, are collected to form the weld edge pixel set.

[0038] The step of obtaining the coordinates of the contaminated area is: Based on each pixel point in the weld edge pixel set, the spatial coordinate information of the pixel point in the horizontal and vertical directions in the original image matrix is extracted respectively, and the spatial coordinate information is saved in order according to the pixel point index, to generate a weld edge pixel spatial coordinate set; According to the weld edge pixel spatial coordinate set, the spatial position of the contaminated pixel mask marked by the contaminant pixel mask is called, and the spatial coordinates of each pixel point in the weld edge pixel spatial coordinate set are compared one by one to determine whether the spatial coordinates are consistent, if they are consistent, it is confirmed that the pixel point exists in the weld edge pixel set and the contaminant pixel mask at the same time, and the weld contamination area common pixel set is recorded and formed point by point; Based on the weld contamination area common pixel set, the spatial coordinates of each common pixel point are mapped into the welding path coordinate system respectively, the position of the pixel point in the welding path is marked, and the coordinates of the contaminated area on the welding path are recorded one by one.

[0039] Specifically, based on the weld edge pixel set obtained in the previous step, the system processes each pixel point in the set, first determines the exact position of each selected pixel point in the original light intensity image matrix, which involves extracting the two-dimensional spatial coordinates of the pixel point, i.e. its row number (usually representing the vertical direction, denoted as ) and column number (usually representing the horizontal direction, denoted as ) in the image matrix, the spatial coordinates of each pixel point extracted will be recorded strictly according to its original arrangement order or processing order in the weld edge pixel set, for example, if the weld edge pixel set is an ordered list containing N pixels, the extracted coordinates will also form an ordered list containing N coordinate pairs , these coordinate information is collected and stored to generate a weld edge pixel spatial coordinate set.

[0040] According to the set of weld edge pixel spatial coordinates generated in the previous step, and calling the previously generated contaminant pixel mask, which is a two-dimensional data structure with the same size as the original image, marking the state of each pixel as a contaminant (for example, contaminant pixels are marked as 1, and non-contaminant pixels are marked as 0), the system will traverse each pixel coordinate in the set of weld edge pixel spatial coordinates For each such coordinate, the system will query the marked value of the corresponding position in the contaminant pixel mask by directly reading the value of the mask at , to determine whether the spatial coordinate is marked as a contaminant at the same time, if the value of the contaminant pixel mask at the coordinate is 1 (indicating that the position is a contaminant), it is confirmed that the pixel point belongs to both the weld edge and the contaminant area, and the coordinate of the pixel point is recorded, this process is repeated for all pixel points in the set of weld edge pixel spatial coordinates, and all pixel point coordinates that are confirmed to satisfy both conditions are collected to form the common pixel set of the weld contaminant area.

[0041] Based on the common pixel set of the weld contaminant area obtained in the previous step, which contains the image coordinates of all pixel points that are both in the weld edge and identified as contaminants, the system will perform coordinate transformation operations on each pixel point coordinate in this set, converting it from the two-dimensional image coordinate system (units are usually pixels) to the predefined welding path coordinate system, which can be a one-dimensional, two-dimensional or three-dimensional spatial coordinate system (units are usually millimeters or meters) that defines the motion trajectory of the welding robot. This mapping process relies on a pre-established and calibrated conversion relationship, which is obtained through camera calibration. For example, for each image coordinate , an affine transformation matrix or a more complex nonlinear mapping function is applied to calculate its corresponding position or distance along the path in the welding path coordinate system. The system will record the converted welding path coordinates of each common pixel point, and these recorded path coordinates are collected to generate the coordinates of the contaminant area on the welding path.

[0042] The steps for obtaining the global optical flow vector field are as follows: Continuously shoot the steel pipe tower bevel welding area at a fixed sampling frame rate, obtain image data frame by frame, and mark each frame of image with a unique timestamp information according to the time sequence of image capture. Arrange all image data in time sequence according to the timestamp to generate a continuous image sequence of the welding process. Based on the continuous image sequence of the welding process, each continuous image pair is called frame by frame, and the gray intensity value of each pixel in the two consecutive frames is extracted. All corresponding pixels in the image are compared point by point. According to the difference in gray value and spatial position of each pixel between the two frames, the horizontal and vertical displacement of each pixel between the two frames is calculated. Combined with the spatial position of the pixel in the image, the displacement vector information of each pixel is recorded to form a set of pixel-level motion vectors between continuous image frames. Based on the set of pixel-level motion vectors between consecutive image frames, the displacement vector information of each pixel is traversed, the direction angle and displacement length of the displacement vector of each pixel are extracted, and the pixels are rearranged in sequence according to their actual spatial positions in the original image. They are then labeled one by one into the coordinate system of the original image to generate a global optical flow vector field.

[0043] Specifically, by configuring an industrial camera at a preset fixed sampling frame rate, such as 100 frames per second (100 FPS), the system continuously captures the dynamic situation of the steel pipe tower bevel during the welding process. Each shooting action captures one frame of image data, which is usually a grayscale or color image. While acquiring each frame, the system immediately reads the current precise time from a clock source (such as a system clock synchronized with Network Time Protocol (NTP) or a dedicated hardware clock) and uses it as the timestamp information for that frame. The timestamp format is usually year-month-day-hour-minute-second-millisecond to ensure its uniqueness and high precision. For example, a frame may be marked as "2023-10-27-10-30-15-500". All captured image data, along with their corresponding timestamp information, are stored sequentially. Subsequently, the system arranges all image frames in a strict time sequence according to the order of these timestamps, forming an ordered image set, i.e., a continuous image sequence of the welding process.

[0044] Based on the continuous image sequence of the welding process generated in the previous step, optical flow estimation algorithms, such as the classic Lucas-Kanade method or deep learning-based optical flow networks (such as FlowNet, PWC-Net, etc.), are used to calculate pixel-level motion vectors. For each pair of consecutive image frames in the sequence, denoted as the _ ... Frame Image and the Frame Image The system first extracts the grayscale intensity value of each corresponding pixel in the two frames of images. Each pixel in (in (based on its image space coordinates), the optical flow algorithm will attempt to... In-neighbor search and Most similar pixels Similarity metrics are typically based on brightness constancy (i.e., the brightness of corresponding pixels remains constant over a short period of time) and spatial consistency (i.e., neighboring pixels have similar motion). The horizontal displacement of each pixel is calculated by minimizing some error function (e.g., minimizing the sum of squared grayscale differences in the Lucas-Kanade method). and vertical displacement Once calculated and This yields the pixels. From the Frame to the Motion vector of a frame This process is performed on all (or selected feature) pixels in the image, and records the original image spatial location of each pixel. and its corresponding displacement vector This information together constitutes the set of inter-frame pixel-level motion vectors between the current consecutive image pairs. This process is applied to all consecutive frame pairs in the entire image sequence, forming a series of such sets.

[0045] Based on the series of consecutive image frame pixel-level motion vector sets calculated in the previous step, the system will process each such set. Each set contains the displacement vector information of each pixel in the image from the previous frame to the current frame. and its spatial location in the original image. For the displacement vector of each pixel The system will further calculate its two key attributes: the direction and angle of the displacement vector. And the length (i.e., magnitude) of the displacement vector. , direction angle It is usually calculated using the arctangent function, for example The atan2 function can handle all quadrants and output values ​​from 0 to 1. or arrive Angle values ​​and displacement lengths within the range This is obtained by calculating the Euclidean distance of the displacement vectors, i.e. After obtaining the displacement vector direction angle and length values ​​of each pixel, the system correlates this information with its actual spatial position in the original image. Correlate them, and then, according to their original image coordinates, these motion vectors with direction and length information are aligned. The order is reorganized and visualized (or stored) on a 2D grid of the same size as the original image, with each grid point... The points are labeled with the motion vectors originating from that point (usually indicated by arrows, the direction of which represents the direction of motion). , the length of the arrow represents ), so that a complete motion vector field covering all pixels (or feature pixels) of the image is generated for each pair of consecutive frames, i.e. a global optical flow vector field.

[0046] The step of obtaining the sequence of the motion trajectory of the splashing particle is: Based on the global optical flow vector field, the displacement vector of each pixel position in the adjacent two frames of images is extracted frame by frame, and the absolute value of the difference between the gray values of the corresponding pixel points in the front and rear two frames of images is calculated respectively. Taking the absolute value of the gray value difference exceeding a preset gray threshold as a judgment condition, the pixel points meeting the judgment condition are defined as initial splashing particle pixel positions, and a set of splashing particle pixel positions is generated frame by frame; According to the set of splashing particle pixel positions, the direction angle value and the length value of the displacement vector at each splashing particle pixel position are extracted respectively, and the difference values of the direction angle and the length of the displacement vector of the splashing particle pixel positions in the adjacent consecutive image frames are compared frame by frame. When the difference values of the direction angle and the length do not exceed the set angle and length difference threshold values, it is determined that the splashing particles at the corresponding pixel positions in the two adjacent frames are the same particle. The corresponding relationship of each splashing particle is determined frame by frame, and the inter-frame association data of the splashing particle is generated; Based on the inter-frame association data of the splashing particle, the motion trajectory coordinates of each splashing particle are recorded by sequentially connecting all the associated pixel positions of each splashing particle in the image space according to the change trajectory of each splashing particle at the corresponding pixel position during the period from appearance to disappearance in the consecutive image frames, and the sequence of the motion trajectory of the splashing particle is generated.

[0047] Specifically, based on the previously generated global optical flow vector field, the system will process each pair of adjacent original image frames (denoted as the frame and the frame ). First, for each pixel position in the , the system will extract its corresponding displacement vector from the global optical flow vector field. At the same time, the system will calculate the absolute value of the difference between the gray value of the pixel position in the and the gray value of the corresponding position (i.e. ) in the , i.e. . Then, the absolute value of the gray difference is compared with a preset gray threshold . The preset gray threshold The setup method is as follows: acquire a series of welding process image sequences containing known spatter particles and background, manually label the spatter particle regions and non-spatter background regions in these sequences, and calculate the absolute value of the average grayscale difference between adjacent frames for all pixels labeled as spatter particle regions. and its standard deviation Simultaneously, calculate the absolute value of the average grayscale difference between adjacent frames for all pixels marked as background regions. and its standard deviation Choose a threshold that can better distinguish between splashes and the background; for example, you can set... ,in It is a constant (e.g.) Alternatively, the optimal balance point can be found through ROC curve analysis. For example, after analyzing 50 welding videos (approximately 100 frames per video), the absolute value of the average grayscale difference of pixels in the background area is 10, with a standard deviation of 5, while the absolute value of the average grayscale difference of pixels in the spatter area is 80, with a standard deviation of 20. If we take... ,but If a certain pixel position Greater than If the value is greater than 25, then the pixel is defined as an initial splash particle pixel position in the current frame. All pixel coordinates that meet this condition are collected, and a set of splash particle pixel positions is generated frame by frame.

[0048] Based on the set of splash particle pixel positions generated frame by frame in the previous step, the system further processes these sets to determine the correspondence between particles. For the current frame (denoted as the ), For each pixel in the set of splash particle pixel positions (frame), the system first extracts its corresponding displacement vector direction angle from the global optical flow vector field. and displacement vector length Then, the system will search for the previous frame (the first frame). The set of splash particle pixel positions for the first frame (frame), for the second frame. A splash particle pixel in the frame In position Its motion vector points to its possible position in the previous frame. The system will be in the first In the set of pixel locations of the splashed particles in the frame, search for candidate corresponding particles within a small neighborhood around this back-projected location. For each candidate Extract its displacement vector direction angle and length ,calculate and The difference in the direction angle of the displacement vector between (angle difference value needs to consider periodicity, for example, take the minimum difference after modulo ) and displacement vector length difference value , the system compares the two difference values with the preset angle difference threshold value and length difference threshold value , the setting of the two threshold values is based on statistical analysis of a large number of real splash particle trajectory data, for example, can be set to twice the standard deviation of the average angle change of the splash particle between consecutive frames, can be set to twice the standard deviation of the average speed change (length represents speed multiplied by frame interval) of the splash particle between consecutive frames, for example, by statistical analysis, the average angle change of the splash particle between consecutive frames is 5 degrees, and the standard deviation is 10 degrees, then can be set to 25 degrees; the average length change is 2 pixels / frame, and the standard deviation is 3 pixels / frame, then can be set to 8 pixels / frame, when (that is, less than or equal to 25 degrees) and (that is, less than or equal to 8 pixels / frame), then it is determined that and are the same splash particle in different frames, if there are multiple satisfy the condition, then the one with the smallest difference is selected, in this way, the corresponding relationship of each splash particle is determined frame by frame, and the splash particle inter-frame association data is generated, which records which particles are the same physical particle observed at consecutive time points.

[0049] Based on the splash particle inter-frame association data generated in the previous step, the system can now track the complete life cycle of each independent splash particle from its first appearance to its final disappearance. For each identified and successfully tracked splash particle, its inter-frame association data provides a time sequence indicating the pixel position of the particle when it appears in consecutive image frames. The system will traverse these association data and connect all pixel positions in the same physical particle's continuous frames in timestamp order (i.e., frame order) to form a discrete point sequence in the image space, which represents the motion trajectory of the splash particle. The system will record the two-dimensional image coordinates of all points on this trajectory, for example, a splash particle A appears from the k-th frame, after k+1, k+2,..., k+m frames, it disappears in the k+m+1-th frame, then its motion trajectory coordinates are , this operation is performed on all tracked splash particles to obtain a series of such motion trajectory coordinate sequences, which together constitute the final splash particle motion trajectory sequence.

[0050] The steps for obtaining the splash motion quantification indicators are: Based on the sequence of the splashing particle motion trajectories, the image capture time stamp of the starting point of each splashing particle motion trajectory is extracted one by one, a uniform unit time interval is set, the total number of newly appearing splashing particles in each unit time interval is counted, and a sequence of the number of splashing particles generated per unit time is formed in turn; According to the sequence of the splashing particle motion trajectories, the total displacement distance between all pixel points in the motion trajectory of each splashing particle is calculated one by one, the starting image capture time stamp and the ending image capture time stamp of each splashing particle motion trajectory are extracted one by one, the motion duration of each splashing particle is calculated respectively, and the motion speed value of each splashing particle is obtained by dividing the total displacement distance of each splashing particle by the motion duration of the corresponding particle, and then the average value of the motion speed values of all splashing particles is calculated to generate the average motion speed of all particles. Based on the sequence of the number of splashing particles generated per unit time and the average motion speed of all particles, the product of the two is calculated to define the splashing motion quantification index.

[0051] Specifically, based on the sequence of the splashing particle motion trajectories obtained in the previous step, the system first traverses each independent splashing particle trajectory in the sequence. For each trajectory, the system extracts the capture time stamp of the image frame corresponding to the starting point of the trajectory. This time stamp accurately records the time when the splashing particle was first detected. Next, a uniform unit time interval is set. The selection of this interval needs to be determined according to the dynamic characteristics of the welding process and the accuracy requirements of the analysis. For example, it can be set to 0.1 seconds or 1 second. Then, the system divides the duration of the entire welding process into consecutive unit time intervals. For each unit time interval, the system counts the total number of newly appearing splashing particles in that time interval, i.e., counts the number of splashing particles whose starting point time stamp falls within the current time interval. For example, if the unit time interval is 0.1 seconds, the first time interval is 0.0-0.1 seconds, the second is 0.1-0.2 seconds, and so on. The system calculates how many new splashing particles appear in 0.0-0.1 seconds, how many new splashing particles appear in 0.1-0.2 seconds, and so on. The number of newly appearing splashing particles in each unit time interval is arranged in chronological order to form a sequence of the number of splashing particles generated per unit time.

[0052] According to the existing sequence of splashing particle motion trajectories, the system analyzes each independent splashing particle trajectory in detail. First, for the motion trajectory of a single splashing particle, the trajectory is composed of a series of pixel point coordinates arranged in chronological order The system calculates the cumulative displacement distance between all pixel points passed by the particle in its entire motion trajectory. This is obtained by calculating the Euclidean distance between adjacent pixel points and summing them up, i.e., the total displacement distance , then the system extracts the capture timestamp of the starting image frame of the splatter particle's motion trajectory and the capture timestamp of the ending image frame (i.e. the frame where the particle is last detected) , calculates the motion duration of the particle and then divides the total displacement distance of the particle by its motion duration to obtain the average motion velocity of the splatter particle This calculation is repeated for all particles in the sequence of splatter particle motion trajectories to obtain the motion velocity value of each splatter particle. Finally, the average motion velocity of all the independent splatter particles is calculated by summing up the motion velocities of all the particles and dividing by the total number of particles, thereby generating the average motion velocity of all the particles.

[0053] Based on the sequence of splatter particle generation number per unit time and the average motion velocity of all the particles obtained in the previous two steps, the system now combines these two quantities to calculate the splatter motion quantification index. The calculation process is as follows: multiply each value in the sequence of splatter particle generation number per unit time (representing the number of newly generated splatter particles in a specific unit time interval) by the calculated average motion velocity of all the particles (a single scalar value). In this way, for each time point (or time period) in the sequence of splatter particle generation number per unit time, a corresponding product value is obtained. These product values can form a new time sequence, or a single splatter motion quantification index representing the activity level of splatter throughout the entire welding process or a specific stage can be obtained by multiplying a representative value of the sequence of splatter particle generation number per unit time (e.g. the average generation number per unit time throughout the entire welding process, or the average generation number in a certain key time period) by the average motion velocity of all the particles. For example, if 50 splatter particles are newly generated in a unit time interval and the average motion velocity of all the particles is calculated to be 100 pixels / second, then the splatter motion quantification index for that time interval is pixels·particles / second. This calculated product is defined as the splatter motion quantification index.

[0054] The acquisition step of the welding state adjustment signal is: Compare the splatter motion quantification index with the steel pipe tower welding process stability threshold value. If the value of the splatter motion quantification index is greater than or equal to the steel pipe tower welding process stability threshold value, it is determined that the current steel pipe tower welding process is in an unstable state. If the value of the splatter motion quantification index is less than the steel pipe tower welding process stability threshold value, it is determined that the current steel pipe tower welding process is in a stable state. Based on the determination result, a welding state adjustment signal is generated.

[0055] Specifically, the spatter motion quantification index calculated in the previous step, which is a specific numerical value, such as 5000 pixels·particle / second, is compared with a pre-set steel pipe tower welding process stability threshold The setting of the steel pipe tower welding process stability threshold is based on a large amount of experimental data and expert experience. The specific process is as follows: First, perform welding experiments under multiple sets of different welding parameters (such as current, voltage, welding speed, gas flow, etc.) combinations, and simultaneously record the spatter motion quantification index and the weld quality grade evaluated by welding experts or through non-destructive testing of weld quality (such as X-ray detection, ultrasonic detection) in each experiment. The weld quality grade is divided into "excellent" (stable state), "qualified" (critical state), and "unqualified" (unstable state). The distribution of spatter motion quantification index corresponding to different quality grades is counted, for example, it is found that in the "excellent" state, the spatter motion quantification index is usually below 3000 pixels·particle / second, in the "qualified" state, the index fluctuates between 3000 and 6000 pixels·particle / second, and in the "unqualified" state, the index is generally higher than 6000 pixels·particle / second. In order to determine a clear threshold, statistical methods can be used, for example, a compromise point is selected between the maximum value (or the 95th percentile) of the spatter motion quantification index in the "excellent" state and the minimum value (or the 5th percentile) of the spatter motion quantification index in the "unqualified" state. For example, the average value of the spatter motion quantification index observed under stable welding conditions in history is taken as the preliminary threshold, and combined with the feedback of weld quality in actual production, it is adjusted, for example, to determine pixels·particle / second. If the current calculated spatter motion quantification index value (for example, 5000 pixels·particle / second) is greater than or equal to (i.e. ), it is determined that the current steel pipe tower welding process is in an unstable state, otherwise, if the spatter motion quantification index value (for example, 2500 pixels·particle / second) is less than (i.e. ), it is determined that the current steel pipe tower welding process is in a stable state. According to the determination result (stable or unstable), the corresponding welding state adjustment signal is generated.

Claims

1. A steel pipe tower assembly intelligent welding visual identification robot coordination control system, characterized in that, The system comprises: A bevel surface contaminant identification module acquires images of the steel pipe tower bevel surface at multiple different polarization angles, calculates the linear polarization component corresponding to each pixel point, simultaneously converts the original light intensity image to the CIELAB color space, extracts the a channel value, fuses the polarization degree value of each pixel point with the corresponding a channel value, and establishes a contaminant pixel mask; A welding path contamination area positioning module detects the pixel gradient change in the steel pipe tower bevel region based on the original light intensity image, generates a weld edge pixel set, aligns the weld edge pixel set and the contaminant pixel mask in spatial position, screens the pixel points existing in the weld edge pixel set and the contaminant pixel mask at the same time, and determines the contamination area coordinates on the welding path; A welding process spatter dynamic analysis module acquires a continuous image sequence in the welding process in real time, calculates the motion vector at the pixel level between continuous frames, establishes a global optical flow vector field, separates and extracts the spatter particles generated in the welding process based on the global optical flow vector field through the inter-frame difference method, and obtains a spatter particle motion trajectory sequence; A welding process stability evaluation module calculates the average motion speed of all particles based on the spatter particle motion trajectory sequence, obtains a spatter motion quantitative index, judges the stability of the welding process, and generates a welding state adjustment signal.

2. The steel tube tower assembly smart welding visual identification robot collaborative control system of claim 1, wherein, The acquisition step of the contaminant pixel mask is: Four polarization angles of 0 degrees, 45 degrees, 90 degrees and 135 degrees are synchronously switched through a polarization plate, light intensity images of the steel pipe tower bevel surface at each angle are acquired, four groups of intensity values are extracted for each pixel point and mapped to a unified gray scale range, the normalized linear polarization degree of the pixel point is calculated in combination with the light intensity values in the four directions, and the normalized linear polarization degree value of each pixel point is obtained; According to the normalized linear polarization degree value, the a channel value of each pixel in the original image in the CIELAB color space is called, and the fusion abnormal response value of each pixel point is calculated; According to the fusion abnormal response value, all pixel points in the image are traversed and a fusion abnormal response threshold is set, a pixel point set with a fusion abnormal response value higher than the threshold is screened, the pixel point set is mapped to the image space, and a contaminant pixel mask is formed.

3. The steel tube tower assembly smart welding visual identification robot collaborative control system of claim 1, wherein, The acquisition step of the weld edge pixel set is: The original light intensity image matrix in the steel pipe tower bevel region is extracted, each pixel point is traversed, the gray difference between adjacent pixels in the horizontal direction and the vertical direction is calculated respectively, the gradient distribution graphs in the two directions are formed, and the directional gradient value of each pixel is recorded according to the position index, and a directional gradient value graph is obtained; According to the directional gradient value graph, the adaptive enhanced gradient modulus value is calculated in combination with the local gray scale statistical features in the fixed window around each pixel; According to the adaptive enhanced gradient modulus value, all pixel points greater than a preset threshold are extracted, the tracking connection is performed according to the connectivity of the pixels in space, the continuous edge pixels are organized into a closed or open structure set, and a weld edge pixel set is formed.

4. The steel tube tower assembly smart welding visual identification robot collaborative control system of claim 1, wherein, The acquisition step of the contamination area coordinates is: Based on each pixel point in the weld edge pixel set, the spatial coordinate information of the pixel point in the horizontal and vertical directions in the original image matrix is extracted respectively, and the spatial coordinate information is sequentially saved according to the pixel point index order to generate a weld edge pixel spatial coordinate set; According to the weld edge pixel spatial coordinate set, the spatial position of the pollution pixel marked by the pollution pixel mask is called, the spatial coordinates of each pixel point in the weld edge pixel spatial coordinate set are compared one by one, and it is judged whether the spatial coordinates are consistent. If consistent, it is confirmed that the pixel point exists in the weld edge pixel set and the pollution pixel mask at the same time, and the common pixel set of the weld pollution area is recorded and formed point by point; Based on the common pixel set of the weld pollution area, the spatial coordinates of each common pixel point are mapped into the welding path coordinate system respectively, the position of the pixel point in the welding path is marked, and the pollution area coordinates on the welding path are recorded and generated one by one.

5. The steel tube tower assembly smart welding visual identification robot collaborative control system of claim 1, wherein, The acquisition step of the global optical flow vector field is: The steel pipe tower groove welding area is continuously shot at a fixed sampling frame rate, the image data is acquired frame by frame, and the unique timestamp information is marked for each frame image according to the time sequence of image capture. All image data are arranged and combined in time sequence according to the timestamp to generate a continuous image sequence of the welding process; Based on the continuous image sequence of the welding process, each continuous image pair is called frame by frame, the gray intensity values of each pixel point in the front and rear two frames of images are extracted respectively, and the pixel points in the image are compared point by point. According to the gray value difference and spatial position change relationship between the front and rear frames of each pixel point, the horizontal displacement and vertical displacement of each pixel point between the two frames of images are calculated, and the displacement vector information of each pixel point is recorded combined with the image space position of the pixel point to form a pixel-level motion vector set between continuous image frames; Based on the continuous image frame pixel-level motion vector set, the displacement vector information of each pixel point is traversed, the direction angle and displacement length value of the displacement vector of each pixel point are extracted, and the original image is sequentially rearranged according to the actual spatial position of each pixel point. Marked in the original image coordinate system one by one to generate a global optical flow vector field.

6. The steel tube tower assembly smart welding visual recognition robot coordination control system of claim 1, wherein, The acquisition step of the spatter particle motion trajectory sequence is: Based on the global optical flow vector field, the displacement vector of each pixel position in the adjacent two frames of images is extracted frame by frame, the absolute value of the gray value difference of the corresponding pixel points in the front and rear two frames of images is calculated respectively, and the pixel points that meet the discrimination condition are defined as initial spatter particle pixel positions when the absolute value of the gray value difference exceeds the preset gray threshold. The spatter particle pixel position set is generated frame by frame; According to the set of splash particle pixel positions, the direction angle value and the length value of the displacement vector at each splash particle pixel position are extracted respectively, the difference values of the direction angle and the length of the displacement vector of the splash particle pixel positions in the adjacent continuous image frames are compared frame by frame, when the difference values of the direction angle and the length do not exceed the set angle and length difference threshold values, it is determined that the splash particles at the corresponding pixel positions in the two adjacent frames are the same particle, the corresponding relationship of each splash particle is determined frame by frame, and the inter-frame association data of the splash particles is generated; Based on the inter-frame association data of the splash particles, according to the change trajectory of the corresponding pixel positions of each splash particle during its appearance to disappearance in the continuous image frames, all the associated pixel positions of each splash particle are sequentially connected in the image space, the motion trajectory coordinates of each splash particle are recorded, and the motion trajectory sequence of the splash particles is generated.

7. The steel tube tower assembly smart welding visual identification robot collaborative control system of claim 1, wherein, The acquisition step of the splash motion quantification index is: Based on the motion trajectory sequence of the splash particles, the image capture time stamp of the starting point of the motion trajectory of each splash particle is extracted one by one, a uniform unit time interval is set, the total number of newly appeared splash particles in each unit time interval is counted, and a sequence of the number of splash particles generated per unit time is formed in turn; According to the motion trajectory sequence of the splash particles, the total displacement distance between all the pixel points in the motion trajectory of each splash particle is calculated one by one, the starting image capture time stamp and the ending image capture time stamp of the motion trajectory of each splash particle are extracted one by one, the motion duration of each splash particle is calculated respectively, and the total displacement distance of each splash particle is divided by the motion duration of the corresponding particle to obtain the motion speed value of each splash particle, and then the average value of the motion speed values of all the splash particles is calculated to generate the average motion speed of all the particles; Based on the sequence of the number of splash particles generated per unit time and the average motion speed of all the particles, the product of the two is calculated to define the splash motion quantification index.

8. The steel tube tower assembly smart welding visual recognition robot collaborative control system of claim 1, wherein, The acquisition step of the welding state adjustment signal is: The splash motion quantification index is compared with the steel pipe tower welding process stability threshold value in numerical value, if the value of the splash motion quantification index is greater than or equal to the steel pipe tower welding process stability threshold value, it is determined that the current steel pipe tower welding process is in an unstable state, if the value of the splash motion quantification index is less than the steel pipe tower welding process stability threshold value, it is determined that the current steel pipe tower welding process is in a stable state, and the welding state adjustment signal is generated according to the determination result.

Images