A method and system for real-time monitoring of a weld pool morphology of a crane boom welding process

Abstract

The application discloses a molten pool shape real-time monitoring method and system for a crane arm welding process, and the method comprises the following steps: collecting continuous multiple frames of images of a molten pool area during crane arm welding; extracting geometric parameters of the molten pool area in the frame image, and determining a comprehensive deformation index of the frame image based on the geometric parameters; judging whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, a preset deformation threshold, the geometric parameters and a preset geometric threshold; if the frame image is an abnormal molten pool image, determining an abnormal time sequence parameter of the frame image based on the frame image, a neighboring frame image of the frame image and corresponding comprehensive deformation indexes, and dividing the frame image into a spatter abnormal image or a molten pool instability image based on the abnormal time sequence parameter; and if the frame image is a molten pool instability image, outputting a warning signal. The application solves the problem that in the existing molten pool state detection, spatter interference and structural instability are difficult to distinguish on the image, resulting in frequent false alarms and low detection accuracy.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method and system for real-time monitoring of the molten pool morphology during the welding process of a crane arm. Background Technology

[0002] In the welding of large structures such as heavy machinery booms, heat sources such as electric arcs and lasers continuously heat the base metal, melting it to form a molten pool. The morphology of the molten pool reflects the current welding state, and to ensure welding quality, the molten pool state needs to be monitored in real time. Existing technologies typically employ edge detection and feature extraction algorithms. By extracting geometric parameters from the edge contours of a single frame of the molten pool image and comparing them with a preset standard shape, they determine whether the molten pool morphology is abnormal, thereby assessing the stability of the welding process.

[0003] However, in actual welding processes, the surface of the molten pool is easily affected by heat source impact and protective gas flow, resulting in liquid splashing. The splashed metal droplets can cause uneven areas such as bulges or depressions at the edge of the molten pool. When the contour abnormalities caused by splashing are similar to the edge deformations caused by the instability of the molten pool structure itself in terms of image representation, existing edge detection and feature extraction methods are very likely to misjudge splashing as molten pool structural instability, thereby triggering frequent false alarms and reducing the accuracy of molten pool condition detection. Summary of the Invention

[0004] The main objective of this application is to provide a method and system for real-time monitoring of the molten pool morphology during the welding process of a crane boom, aiming to solve the technical problem in existing molten pool condition detection where spatter interference and structural instability are difficult to distinguish on images, leading to frequent false alarms and low detection accuracy.

[0005] To achieve the above objectives, this application provides a method for real-time monitoring of the molten pool morphology during the welding process of a crane boom, comprising: acquiring multiple consecutive frames of images of the molten pool region during crane boom welding; for any frame of the multiple frames, extracting geometric parameters of the molten pool region in the frame image, and determining a comprehensive deformation index of the frame image based on the geometric parameters and preset standard molten pool parameters; determining whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, a preset deformation threshold, the geometric parameters, and the preset geometric threshold; if the frame image is an abnormal molten pool image, determining abnormal time-series parameters of the frame image based on the frame image, adjacent frame images, the comprehensive deformation index of the frame image, and the comprehensive deformation index of adjacent frame images, and classifying the frame image into a spatter abnormal image or a molten pool instability image based on the abnormal time-series parameters; and outputting a warning signal if the frame image is a molten pool instability image.

[0006] Optionally, the warning signal includes multiple levels. If the frame image is a melt pool instability image, the output warning signal includes: determining the melt pool shape prediction deviation of the melt pool region based on the comprehensive deformation index of the frame image, the comprehensive deformation index of each frame image within a preset time window before the frame image, and the comprehensive deformation index of a preset standard image; determining the degree of anomaly of the melt pool region based on the melt pool shape prediction deviation; determining the level of the warning signal based on the degree of anomaly and outputting the warning signal of the specified level.

[0007] Optionally, determining the abnormal time-series parameters of the frame image based on the frame image, adjacent frame images of the frame image, a comprehensive deformation index of the frame image, and a comprehensive deformation index of adjacent frame images of the frame image includes: determining the degree of deformation deviation of the frame image based on the comprehensive deformation index of the frame image and the comprehensive deformation index of adjacent frame images of the frame image; determining a joint determination coefficient of the frame image based on the frame image and adjacent frame images of the frame image; and determining the abnormal time-series parameters of the frame image based on the degree of deformation deviation and the joint determination coefficient.

[0008] Optionally, determining the joint determination coefficient of the frame image based on the frame image and its neighboring frame images includes: determining the persistence similarity and abnormal region concentration of the frame image based on the pixel values ​​of each pixel in the frame image and the pixel values ​​of each pixel in the neighboring frame images; and determining the joint determination coefficient of the frame image based on the persistence similarity and the abnormal region concentration.

[0009] Optionally, determining the molten pool morphology prediction deviation of the molten pool region based on the comprehensive deformation index of the frame image, the comprehensive deformation index of each frame image within a preset time window preceding the frame image, and the comprehensive deformation index of a preset standard image includes: determining the deformation rate of the molten pool region based on the comprehensive deformation index of the frame image and the comprehensive deformation index of adjacent frame images of the frame image; determining the pre-anomaly degree of the molten pool region based on the comprehensive deformation index of each frame image within a preset time window preceding the frame image; determining the standard deformation rate and the standard pre-anomaly degree based on the comprehensive deformation index of the preset standard image; and determining the molten pool morphology prediction deviation of the molten pool region based on the deformation rate, the pre-anomaly degree, the standard deformation rate, and the standard pre-anomaly degree.

[0010] Optionally, the geometric parameters include: molten pool area, aspect ratio, contour smoothness, center of gravity offset, and grayscale uniformity.

[0011] Optionally, the preset geometric threshold includes preset global parameters and preset local parameters. The preset global parameters consist of a preset standard melt pool area and a preset standard aspect ratio. The preset local parameters consist of a preset standard contour smoothness, a preset standard centroid offset, and a preset standard grayscale uniformity.

[0012] Optionally, before extracting multiple geometric parameters of the molten pool region in the frame image, the method further includes: cropping the frame image to obtain a cropped image, wherein the molten pool region in the cropped image is located at the center of the image, and obtaining a binary image and a grayscale image based on the cropped image.

[0013] Optionally, obtaining a binary image and a grayscale image based on the cropped image includes: performing binarization processing on the cropped image to obtain a binary image; and obtaining a grayscale image based on the binary image and the frame image, wherein the melt pool region in both the binary image and the grayscale image is located at the center of the image.

[0014] Furthermore, to achieve the above objectives, this application also provides a real-time monitoring system for the molten pool morphology during crane boom welding, comprising: an image acquisition module for acquiring multiple consecutive frames of images of the molten pool region during crane boom welding; a preliminary anomaly judgment module for extracting geometric parameters of the molten pool region in any frame of the multiple frames, determining a comprehensive deformation index of the frame image based on the geometric parameters and preset standard molten pool parameters, and determining whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, a preset deformation threshold, the geometric parameters, and the preset geometric threshold; an anomaly classification module for determining an abnormal time sequence parameter of the frame image based on the frame image, adjacent frame images, the comprehensive deformation index of the frame image, and the comprehensive deformation index of adjacent frame images when the frame image is an abnormal molten pool image, and classifying the frame image into a spatter anomaly image or a molten pool instability image based on the abnormal time sequence parameter; and an early warning module for outputting an early warning signal if the frame image is a molten pool instability image.

[0015] This application proposes a method and system for real-time monitoring of the molten pool morphology during the welding process of a crane boom. By acquiring multiple consecutive frames of images of the molten pool region during the welding process, extracting the geometric parameters of the molten pool region in each frame, and determining a comprehensive deformation index based on preset standard molten pool parameters, the overall morphology of the molten pool can be quantitatively characterized. Based on this, for cases where the molten pool image is determined to be abnormal, this application combines the comprehensive deformation index and image difference information of the current frame image and its adjacent frames to determine the abnormal time sequence parameters. This can effectively distinguish between instantaneous distortion caused by spatter and continuous deformation caused by the instability of the molten pool itself. Finally, based on the classification results, only warning signals are output for images of molten pool instability. This solves the technical problem in existing molten pool state detection where it is difficult to distinguish between spatter interference and structural instability in the image, leading to frequent false alarms and low detection accuracy. Attached Figure Description

[0016] Figure 1 One of the flowcharts for a real-time monitoring method of the molten pool morphology during the welding process of a crane boom provided in an embodiment of this application; Figure 2 This is an example diagram of a binary image provided in an embodiment of this application; Figure 3 A second flowchart of a method for real-time monitoring of the molten pool morphology during the welding process of a crane boom provided in an embodiment of this application; Figure 4 This is a structural block diagram of a real-time monitoring system for the molten pool morphology during the welding process of a crane boom provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a welding process monitoring controller provided in one embodiment of this application.

[0017] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0019] In the welding of large structures such as heavy machinery booms, heat sources such as electric arcs and lasers continuously heat the base metal, melting it to form a molten pool. The morphology of the molten pool reflects the current welding state, and to ensure welding quality, the molten pool state needs to be monitored in real time. Existing technologies typically employ edge detection and feature extraction algorithms. By extracting geometric parameters from the edge contours of a single frame of the molten pool image and comparing them with a preset standard shape, they determine whether the molten pool morphology is abnormal, thereby assessing the stability of the welding process.

[0020] However, in actual welding processes, the surface of the molten pool is easily affected by heat source impact and protective gas flow, resulting in liquid splashing. The splashed metal droplets can cause uneven areas such as bulges or depressions at the edge of the molten pool. When the contour abnormalities caused by splashing are similar to the edge deformations caused by the instability of the molten pool structure itself in terms of image representation, existing edge detection and feature extraction methods are very likely to misjudge splashing as molten pool structural instability, thereby triggering frequent false alarms and reducing the accuracy of molten pool condition detection.

[0021] Furthermore, since the edge deformation is relatively slow and small in the early stage of the instability of the molten pool structure, existing edge detection and feature extraction methods tend to miss it due to spatter interference, making it difficult to repair the defect after solidification.

[0022] To address the aforementioned issues, this application provides a method and system for real-time monitoring of the molten pool morphology during the welding process of a crane boom. The solution described in this application will be detailed below.

[0023] To facilitate understanding, this application first introduces a specific scenario of the real-time monitoring method for the molten pool morphology during the welding process of a crane boom.

[0024] In the crane boom welding station, an industrial camera is fixedly installed in front of the welding torch, with its lens aimed at the welding center point, i.e., the area where the molten pool is located. The lens is equipped with a sealed, light-shielding protective shell to block the intense light from the electric arc generated during welding and to isolate it from high-temperature welding slag and fumes.

[0025] The industrial camera is fixed to the welding carriage or welding robot and can move synchronously with the welding heat source. During the welding process, the industrial camera continuously collects multiple frames of images of the molten pool area and transmits the collected images to the controller in real time. The controller then executes the real-time monitoring method of molten pool morphology provided in the embodiments of this application.

[0026] Example 1 Figure 1 This is one of the flowcharts for a real-time monitoring method of the molten pool morphology during the welding process of a crane boom provided in an embodiment of this application. The method can be executed by a controller, which can specifically be a welding process monitoring controller.

[0027] Reference Figure 1 The real-time monitoring method for the molten pool morphology during the welding process of a crane boom may include the following steps: S11. Acquire multiple consecutive frames of images of the molten pool area during crane arm welding.

[0028] In this embodiment, an industrial camera is fixedly mounted in front of the welding torch and moves synchronously with the welding carriage or welding robot. During the welding process, the industrial camera continuously acquires multiple frames of images of the molten pool area and transmits the acquired images to the controller in real time, thereby obtaining multiple frames of images of the molten pool area during the welding of the crane arm.

[0029] S12. For any frame image in the multi-frame image, extract the geometric parameters of the molten pool region in the frame image, and determine the comprehensive deformation index of the frame image based on the geometric parameters and the preset standard molten pool parameters. Also, determine whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, the preset deformation threshold, the geometric parameters and the preset geometric threshold.

[0030] Among them, the geometric parameters of the molten pool region refer to the morphological quantitative features of the molten pool region extracted from the frame image, which are used to characterize the geometric shape and grayscale distribution of the molten pool; the preset standard molten pool parameters refer to the area size of the molten pool region under normal welding conditions, which can be used as a reference benchmark for whether the molten pool shape has deviated; the comprehensive deformation index refers to the evaluation value constructed based on the geometric parameters and the preset standard molten pool parameters, which can be used to evaluate the overall deformation degree of the molten pool; the preset deformation threshold refers to the critical value determined according to the distribution of the comprehensive deformation index of the molten pool under normal welding conditions, which can be used to distinguish whether the molten pool shape is normal.

[0031] Before executing S12, this embodiment can also perform preprocessing operations on the acquired multi-frame images.

[0032] Specifically, any frame from all the frame images is taken as the target frame image. This embodiment uses the target frame image as an example for explanation. The target frame image is cropped so that the molten pool region is located in the center of the cropped image, and the cropped image is then subjected to Gaussian filtering. It can be understood that Gaussian filtering can remove noise caused by smoke and arc stray light.

[0033] Furthermore, local adaptive contrast enhancement is performed on the filtered image. The image is then converted into a binary image based on adaptive threshold segmentation, where white represents the molten pool region (foreground region) and black represents the irrelevant background. Based on this binary image, the grayscale image corresponding to the molten pool foreground region is extracted from the grayscale image corresponding to the original target frame image. In both the binary and grayscale images, the molten pool region is located at the image center. This contrast enhancement can balance the brightness difference in the image, reducing the degree of localized whitening and detail loss caused by the strong arc light.

[0034] Please see Figure 2 , Figure 2 Here is an example of a binary image. Figure 2 The white area represents the molten pool region, i.e., the foreground region, while the black area represents the irrelevant background.

[0035] It should be noted that in the subsequent steps of this embodiment, the frame image refers to the pre-processed binary image or grayscale image.

[0036] After preprocessing, the geometric parameters of the molten pool region are extracted from the binary image and grayscale image.

[0037] As an optional implementation, the geometric parameters include the molten pool area, aspect ratio, contour smoothness, center of gravity offset, and grayscale uniformity.

[0038] Among them, the molten pool area refers to the total number of pixels in the molten pool region in the image; the aspect ratio refers to the ratio of the length to the width of the minimum bounding rectangle of the molten pool; the contour smoothness can be used to characterize the smoothness of the molten pool edge; the centroid offset can be used to characterize the degree of deviation of the centroid of the molten pool region from the reference center point; and the gray level uniformity can be used to characterize the uniformity of the gray level distribution on the molten pool surface.

[0039] It should be noted that during the welding process of the crane boom, the area of ​​a normal molten pool fluctuates stably within a fixed range. An abnormally shaped molten pool occurs because excessively high temperatures cause the welding wire to melt too quickly, resulting in an excessively large molten pool area. A normal molten pool is elliptical in shape, with an aspect ratio between 1.5 and 2.0. When the welding temperature is too high or too low, the molten pool becomes non-elliptical, and its aspect ratio deviates from the normal range. The edge contour of a normal molten pool is smooth, but when impurities are present in the molten pool, the liquid may spread evenly along the edge, resulting in an uneven edge. The center of gravity of a normal molten pool is close to the reference center point. In abnormal conditions, a shift in the center of gravity can cause the molten pool to flow off-center, affecting the difference in penetration depth on both sides of the weld. A normal molten pool has a uniform grayscale distribution and smooth transitions between light and dark areas. In abnormal conditions, impurities or pores inside the molten pool can cause drastic fluctuations in the grayscale values ​​of the corresponding pixels in the molten pool area.

[0040] It is understood that in the embodiments of this application, the molten pool area, aspect ratio, contour smoothness, and centroid offset are all obtained through binary images, and the grayscale uniformity is obtained through grayscale images.

[0041] It should be noted that a single geometric parameter can only reflect the morphological characteristics of the molten pool from one dimension, and it is difficult to comprehensively characterize the overall deformation state of the molten pool.

[0042] In this embodiment, a comprehensive deformation index of the frame image is determined based on geometric parameters and preset standard melt pool parameters, which is used to evaluate the overall deformation degree of the melt pool.

[0043] Specifically, taking the target frame image as an example, the total number of pixels in the molten pool region of the target frame image is first obtained. That is, the number of pixels with a value of 1 (molten pool region) in the binary image, and the number of pixels with a value of 1 (molten pool region) in the standard binary image. That is, to calculate the area of ​​the pre-defined standard molten pool. and The difference. Next, the aspect ratio of the molten pool region in the target frame image is obtained. ,in, Indicates the length of the molten pool region. This represents the width of the molten pool region. If the aspect ratio exceeds the preset standard aspect ratio in the standard binary image, it indicates that the current molten pool region has a long and narrow shape, possibly due to excessively high temperature. If the aspect ratio is less than the preset standard aspect ratio in the standard binary image, it indicates that the current temperature is too low, resulting in a smaller overall molten pool area. and Determine the overall shape changes of the molten pool.

[0044] For example, it can be used and The greater the difference between the ratio and 1, the more it indicates that the area of ​​the molten pool has deformed. and It can be and The product of.

[0045] Understandable It can be used to measure the degree of change in the overall shape of the molten pool. The larger the value, the larger the molten pool area and the greater the aspect ratio, which deviates from the standard value. This may be due to excessive heating, which causes the molten pool to become narrow.

[0046] Furthermore, based on the centroid offset of the molten pool in the target frame image... Gray scale variation value and circularity of the molten pool area Determine the internal stability of the molten pool.

[0047] In this embodiment, the contour of the molten pool region in the target frame image is extracted, the contour center coordinates of the molten pool region are determined based on the extracted contour fitting, and then the Euclidean distance between the contour center coordinates and the geometric center coordinates of the target frame image is calculated. The calculated Euclidean distance is used as the centroid offset.

[0048] It is understandable that when there are areas with obvious gray changes inside the molten pool, it indicates that impurities have been mixed into the molten pool. The closer the roundness of the molten pool is to 0, the lower the smoothness of the edge of the molten pool and the greater the surface undulation. The closer the roundness is to 1, the smoother the outline.

[0049] For example, the grayscale change value can be based on the pixel value with the largest grayscale value in the target frame image. That is, the largest pixel value in the grayscale image and the smallest pixel value in the target frame image. That is, the minimum pixel value in the grayscale image and the average grayscale value of the target frame image. The mean value of all pixels in a grayscale image is obtained. The internal stability of the molten pool can be determined by the product of the centroid offset, grayscale change value, and the circularity of the molten pool region.

[0050] It should be noted that, It can be used to measure the degree of change in the local shape of the molten pool. The larger the value, the higher the degree of local deformation. The molten pool may contain impurities, have uneven edges, or have a shifted center of gravity.

[0051] For ease of description, the target frame image is used as an example. In this embodiment, the target frame image is referred to as... Let be the global deformation parameter of the target frame image, denoted as These are the local deformation parameters of the target frame image.

[0052] It is understood that the comprehensive deformation index includes global deformation parameters and local deformation parameters. For example, the product of the global deformation parameters and the local deformation parameters can be used as the comprehensive deformation index.

[0053] It should be noted that a standard binary image refers to a binary image obtained after preprocessing of multiple consecutive frames of images of the molten pool area acquired from a high-quality welding process. A high-quality welding process refers to a welding process in which the welding quality has passed inspection.

[0054] In this embodiment of the application, the standard binary image and the preset standard melt pool parameters obtained by using the standard binary image are pre-stored in the controller. The preset standard melt pool parameters include the preset standard melt pool area, the preset standard aspect ratio, the preset standard contour smoothness, the preset standard centroid offset, and the preset standard grayscale uniformity.

[0055] In this embodiment, after calculating the comprehensive deformation index of each frame image, the system determines whether the molten pool region exhibits any abnormalities in each frame image based on the comprehensive deformation index.

[0056] Specifically, based on multiple consecutive frames of images of the molten pool region pre-acquired from a high-quality welding process, the comprehensive deformation index corresponding to each frame is calculated to establish a normal sample dataset. The comprehensive deformation indices in the normal sample dataset are sorted in descending order, and the last value of the top 95% is selected as the preset deformation threshold. Simultaneously, the global deformation parameters corresponding to each frame are calculated. After sorting the global deformation parameters in descending order, the last digit of the first 95% is selected as the preset global parameter. Calculate the local deformation parameters corresponding to each frame of the image, sort the local deformation parameters in descending order, and select the last value of the first 95% as the preset local parameter. .

[0057] It is understandable that the preset global parameters consist of the preset standard melt pool area and the preset standard aspect ratio, while the preset local parameters consist of the preset standard contour smoothness, the preset standard centroid offset, and the preset standard grayscale uniformity.

[0058] When the target frame image simultaneously satisfies the following conditions: the comprehensive deformation index is less than the preset deformation threshold, and the global deformation parameter is less than the preset global parameter... And the local deformation parameter is less than the preset local parameter. If the target frame image is normal, it is determined to be a normal molten pool image; otherwise, it is determined to be an abnormal molten pool image. In other words, if any one of the comprehensive deformation index, global deformation parameter, and local deformation parameter of a certain frame image exceeds the corresponding preset threshold, the molten pool region is considered to have an abnormal shape at the corresponding time point of that frame image.

[0059] S13. When the frame image is an abnormal molten pool image, the abnormal timing parameters of the frame image are determined based on the frame image, the adjacent frame images of the frame image, the comprehensive deformation index of the frame image, and the comprehensive deformation index of the adjacent frame images of the frame image. Based on the abnormal timing parameters, the frame image is divided into a splashing abnormal image or a molten pool instability image.

[0060] In this context, adjacent frames refer to the preceding and following frames that are temporally adjacent to the current frame. Abnormal temporal parameters are comprehensive evaluation values ​​constructed based on the differences in image grayscale and overall deformation indices between the abnormal frame and its adjacent frames. These parameters can be used to quantify the degree of morphological change in the molten pool region of each frame relative to its preceding and following frames. A spatter anomalous image refers to an image where the edge of the molten pool region is distorted due to liquid spatter during welding; a molten pool instability image is an image of the molten pool itself becoming structurally unstable.

[0061] It should be noted that during the welding process, liquid spatter may occur due to contamination of the base material surface or moisture in the welding wire. Spatter appears as isolated, bright, raised areas in the image. When it lands on the edge of the molten pool, it may cause localized bulging at the edge. However, these bulges are short-lived, scattered, and do not affect the grayscale changes within the molten pool area. Abnormal areas caused by the instability of the molten pool itself will be distributed in patches and extend inwards, causing the overall outline of the molten pool area to continuously deteriorate.

[0062] In the specific implementation process, taking the target frame image as an example, the degree of deformation deviation of the target frame image can be determined first based on the comprehensive deformation index of the target frame image and the comprehensive deformation index of the adjacent frame images of the target frame image. Then, the joint judgment coefficient of the target frame image can be determined based on the target frame image and the adjacent frame images of the target frame image. Finally, the abnormal time series parameters of the target frame image can be determined based on the degree of deformation deviation and the joint judgment coefficient.

[0063] Specifically, the persistence similarity and abnormal region concentration of the target frame image can be determined based on the pixel values ​​of each pixel in the target frame image and the pixel values ​​of each pixel in the adjacent frame images of the target frame image; the joint determination coefficient of the target frame image can be determined based on the persistence similarity and abnormal region concentration.

[0064] Specifically, taking the target frame image as an example, if the target frame image is an abnormal melt pool image, the target frame image is denoted as the first frame in a multi-frame image. Frame image.

[0065] In this embodiment, the number of overlapping pixels in the molten pool region between the target frame image and its previous frame image is calculated respectively. And the number of overlapping pixels in the molten pool region between the target frame image and its subsequent frame image. ;Pick and The average value is used as the sustained similarity of the target frame image.

[0066] It is understandable that the greater the sustained similarity, the more the target frame image overlaps with the melt pool region of its neighboring frame images, and the smoother the morphological changes.

[0067] Furthermore, the distance from each pixel in the target frame image to the centroid of the melt pool region is calculated and normalized. Then, the mean and variance of the normalized distances are calculated, and the ratio of the mean to the variance is taken as the concentration of outlier regions in the target frame image. It should be noted that each pixel refers to a pixel with a value of 1 (white pixel) in the binary image.

[0068] It is understandable that the greater the concentration of abnormal areas, the more concentrated the abnormal areas are at the edge of the molten pool or in local areas.

[0069] Specifically, taking the target frame image as an example, let's denote the target frame image as the first of a multi-frame image. Frame images, firstly based on the comprehensive deformation index of the target frame image. The degree of deformation deviation of the target frame image is determined by the difference between the comprehensive deformation index of the target frame image and the comprehensive deformation index of the adjacent frame images. For example, the comprehensive deformation index of the target frame image can be the comprehensive deformation index of the previous frame image in time sequence. In addition, in terms of timing, the comprehensive deformation index of the next frame of the target frame image. , and , The differences between them can be utilized Perform the calculation.

[0070] Furthermore, it is possible to base the analysis on the sustained similarity of the target frame images. and concentration of abnormal regions Determine the joint decision coefficients of the target frame image; for example, the joint decision coefficients can be expressed as: .

[0071] Understandably, a greater degree of deformation deviation indicates that the deformation of the molten pool region in the target frame image is significantly higher or lower than that in adjacent frames, suggesting that the anomaly in the molten pool region corresponding to the target frame image may be sudden and likely caused by spatter. When the deformation deviation approaches 0, it indicates that the deformation of the molten pool region in the target frame image is consistent with that in adjacent frames, suggesting that the anomaly is gradual and continuous, and that the molten pool itself is more likely to be unstable. A smaller joint determination coefficient indicates a high degree of sustained image similarity between the target frame image and adjacent frames, and a low concentration of the anomaly region, suggesting that the current anomaly is more likely to be caused by the instability of the molten pool structure itself. A larger joint determination coefficient indicates a low degree of sustained image similarity between the target frame image and adjacent frames, and a high concentration of the anomaly region, suggesting that the current anomaly is more likely to be caused by transient interference from spatter.

[0072] In this embodiment, after calculating the abnormal timing parameters of each abnormal molten pool image, the abnormal molten pool images are divided into splashing abnormal images or molten pool instability images based on the abnormal timing parameters.

[0073] Specifically, in this embodiment, multiple consecutive frames of images of the molten pool area acquired in advance from the high-quality welding process are used as sample frames. Abnormal timing parameters of each sample frame are calculated, and the abnormal timing parameters are arranged in ascending order to form a sequence. The average value of the last 10% of the abnormal timing parameters in the sequence is selected as the abnormal timing threshold of the stable molten pool.

[0074] Taking the target frame image as an example, when the value of the abnormal timing parameter of the target frame image is greater than the abnormal timing threshold, the abnormality of the target frame image is determined to be caused by splashing, and the target frame image is marked as a splashing abnormal image; when the value of the abnormal timing parameter of the target frame image is less than or equal to the abnormal timing threshold, the abnormality of the target frame image is determined to be caused by the instability of the molten pool structure itself, and the target frame image is marked as a molten pool instability image.

[0075] S14. If the frame image is a melt pool instability image, output a warning signal.

[0076] In practice, when the current frame image is determined to be an image of molten pool instability, the controller outputs a warning signal. Upon receiving the warning signal, on-site operators can take timely adjustment measures to prevent the occurrence or expansion of welding defects.

[0077] It is understood that the determination of abnormal timing parameters in the embodiments of this application requires a comprehensive deformation index based on the previous and next frames of the target frame image, that is, based on the first... Taking a frame image as an example, when the controller processes the first frame... Only when the image is in frame can the first frame be calculated. Abnormal timing parameters of the frame image, and determine the first frame based on these parameters. Is the frame image a picture of melt pool instability? Therefore, for the first frame... The warning signal for the frame image is actually in the first frame. It is triggered when a frame image arrives.

[0078] Example 2 It should be noted that in the above embodiments, by combining the comprehensive deformation index with the inter-frame difference method, it is possible to effectively distinguish between spatter interference and the instability of the molten pool structure itself, thus solving the problem that the prior art easily misjudges spatter as molten pool structural instability, thereby triggering frequent false alarms. However, the above embodiments can only determine the current state of the molten pool and cannot effectively predict the subsequent changes in the molten pool's shape, making it difficult to avoid potential welding defects in advance.

[0079] Based on this, this embodiment, building upon the above embodiments, analyzes the deformation rate of the molten pool instability image and the morphological changes of the molten pool image within the historical window to predict the deformation trend of the molten pool region, thereby achieving prediction and graded early warning of the degree of molten pool anomaly.

[0080] Based on the above embodiments, Figure 3 This is the second flowchart of a method for real-time monitoring of the molten pool morphology during the welding process of a crane boom provided in an embodiment of this application. Figure 3 Based on Figure 1 A preferred embodiment of the real-time monitoring method for the molten pool morphology during the welding process of the crane boom is described below. Figure 3 The real-time monitoring method for the molten pool morphology during the welding process of a crane boom may include the following steps: S21. Acquire multiple consecutive frames of images of the molten pool area during crane boom welding; S22. For any frame image in the multi-frame image, extract the geometric parameters of the molten pool region in the frame image, and determine the comprehensive deformation index of the frame image based on the geometric parameters and the preset standard molten pool parameters. Also, determine whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, the preset deformation threshold, the geometric parameters and the preset geometric threshold. S23. When the frame image is an abnormal molten pool image, the abnormal timing parameters of the frame image are determined based on the frame image, the adjacent frame images of the frame image, the comprehensive deformation index of the frame image, and the comprehensive deformation index of the adjacent frame images of the frame image. Based on the abnormal timing parameters, the frame image is divided into a splashing abnormal image or a molten pool instability image. S24. If the frame image is a melt pool instability image, the melt pool morphology prediction deviation of the melt pool region is determined based on the comprehensive deformation index of the frame image, the comprehensive deformation index of each frame image within the preset time window before the frame image, and the comprehensive deformation index of the preset standard image. S25. Determine the degree of anomaly in the molten pool region based on the deviation predicted by the molten pool morphology; S26. Determine the level of the warning signal based on the degree of abnormality and output the warning signal of that level.

[0081] Here, "before this frame image" refers to the frames that precede this frame image in time sequence. The melt pool morphology prediction deviation can be used to predict future trends in the melt pool region. Anomaly severity refers to a quantitative value determined based on the melt pool morphology prediction deviation, which characterizes the severity of the current melt pool instability.

[0082] It is understood that the specific implementation of steps S21 to S23 in this embodiment is consistent with the relevant description of steps S11 to S13 in the aforementioned embodiment one, and will not be repeated here. This embodiment will directly describe the details starting from step S24.

[0083] In the specific implementation process, the deformation rate of the molten pool region can be determined based on the comprehensive deformation index of the frame image and the comprehensive deformation index of the adjacent frame images; the pre-anomaly degree of the molten pool region can be determined based on the comprehensive deformation index of each frame image within a preset time window before the frame image; then the standard deformation rate and standard pre-anomaly degree can be determined based on the comprehensive deformation index of the preset standard image; finally, the molten pool morphology prediction deviation of the molten pool region can be determined based on the deformation rate, pre-anomaly degree, standard deformation rate, and standard pre-anomaly degree.

[0084] Specifically, taking the target frame image as an example, if the target frame image is a melt pool instability image, the target frame image is denoted as the first of the multiple frames. The deformation rate of a target frame image can be calculated by dividing the difference between the comprehensive deformation index of the target frame image and the comprehensive deformation index of the preceding frame image in time sequence by the time interval between the two adjacent frames. The standard deformation rate can also be calculated by dividing the difference in the comprehensive deformation index corresponding to adjacent sample frames by the time interval between two adjacent frames. .

[0085] Furthermore, the deformation rate of the target frame image can be... With standard deformation rate By comparing and determining the rate of degradation of the target frame image, for example, this can be achieved by calculating... and The ratio between As a measure of the rate of degradation of the target frame image. This is understandable. The closer the value is to 1, the closer the rate of change of the molten pool region at the corresponding moment in the target frame image is to the normal rate of change, and the molten pool is evolving steadily. A value less than 1 indicates that the molten pool is changing at a slow rate. A value greater than 1 indicates that the molten pool is deteriorating at a relatively high rate.

[0086] Specifically, this embodiment sets a preset time window. For 10 frames, it can be derived from the previous frame image of the target frame. Frame image (i.e., the first) Frame to the The pre-anomaly degree of the target frame image is obtained by calculating the mean of the comprehensive deformation index corresponding to each frame. It can also be derived from the sample frames preceding the target frame image at the corresponding time. The standard pre-anomaly degree is obtained by calculating the mean of the comprehensive deformation index corresponding to each sample frame. .

[0087] Furthermore, the degree of anomaly and the comprehensive deformation index of the target frame image can be compared with the standard degree of anomaly to determine the degree of change in the abnormal region of the target frame image. For example, the standard degree of anomaly can be calculated. Differences in pre-existing anomaly degree and overall deformation index between the target frame image and the target frame image The ratio between To measure the degree of change in abnormal regions of the target frame image, and to avoid a denominator of 0, the ratio is... Become , The larger the value, the less the shape of the molten pool region changes compared to historical times at the corresponding moment in the target frame image, indicating that the molten pool maintains a relatively stable shape.

[0088] Furthermore, the degree of degradation rate of the target frame image Degree of change in abnormal areas Determine the predicted deviation of the molten pool shape at the corresponding time in the target frame image. For example, the product of the rate of deterioration and the degree of change in the anomalous region can be used as the deviation from the predicted melt pool morphology. Therefore, The larger the value, the more severe the instability of the molten pool at the corresponding moment in the target frame image, and the higher the possibility of continued deterioration in the future.

[0089] In the specific implementation process, it is also necessary to convert the predicted deviation of the molten pool shape into decision information that can be directly used by on-site operators.

[0090] Therefore, this embodiment establishes a molten pool visualization monitoring and multi-level anomaly early warning system to acquire real-time images of the molten pool during the welding process and intuitively display the real-time molten pool outline on a visualization interface. Simultaneously, a graded early warning mechanism is set up based on the actual working conditions of the crane boom welding. The severity of the molten pool area is determined according to the predicted deviation of the molten pool shape, with severity categorized as minor or severe anomalies. For anomalies caused by spatter, only the data is saved, without interfering with normal operations.

[0091] Specifically, in this embodiment, the comprehensive deformation index, abnormal timing parameters, and melt pool morphology prediction deviation obtained from the above steps are displayed on a visualization interface. The melt pool instability images are sorted from largest to smallest according to the comprehensive deformation index, and the 50th percentile position is taken as the first grade threshold. When the comprehensive deformation index is greater than the first grade threshold, it is judged as a severe anomaly, the page is displayed in red and an alarm is issued; when the comprehensive deformation index is less than or equal to the first grade threshold, it is judged as a slight anomaly, and in this case, the magnitude of the melt pool morphology prediction deviation needs to be combined for a comprehensive judgment.

[0092] In this embodiment, the average value of the predicted deviation of the melt pool shape corresponding to all sample frames is used as the standard value. When the predicted deviation of the melt pool shape is greater than the standard value, it indicates that the slight anomaly at this time may evolve into a severe anomaly in the future, and an orange warning is given.

[0093] It should be noted that after receiving the early warning signal, on-site personnel can adjust the process parameters in a timely manner based on the warning information to intervene in the occurrence of welding defects in advance.

[0094] Based on the above embodiments, Figure 4 This is a structural block diagram of a real-time monitoring system for the molten pool morphology during the welding process of a crane boom according to one embodiment of this application. Figure 4 As shown, the real-time monitoring system 300 for the molten pool morphology during the welding process of the crane boom may include an image acquisition module 310, a preliminary anomaly judgment module 320, an anomaly classification module 330, and an early warning module 340.

[0095] The image acquisition module 310 is used to acquire multiple consecutive frames of images of the molten pool area during crane boom welding; the preliminary anomaly judgment module 320 is used to extract the geometric parameters of the molten pool area in any frame of the multiple frames, and determine the comprehensive deformation index of the frame image based on the geometric parameters and preset standard molten pool parameters, and determine whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, preset deformation threshold, geometric parameters and preset geometric threshold; the anomaly classification module 330 is used to determine the abnormal time sequence parameters of the frame image based on the frame image, the adjacent frame images of the frame image, the comprehensive deformation index of the frame image, and the comprehensive deformation index of the adjacent frame images of the frame image when the frame image is an abnormal molten pool image, and classify the frame image into a spattering abnormal image or a molten pool instability image based on the abnormal time sequence parameters; the early warning module 340 is used to output an early warning signal if the frame image is a molten pool instability image.

[0096] It should be noted that for details not disclosed in the real-time monitoring system for the molten pool morphology of the crane boom welding process in this embodiment, please refer to the details disclosed in the embodiment of the real-time monitoring method for the molten pool morphology of the crane boom welding process in this specification, which will not be repeated here.

[0097] Based on the above embodiments, Figure 5 This is a schematic diagram of the structure of a welding process monitoring controller according to one embodiment of this application, as shown below. Figure 5 As shown, the welding process monitoring controller may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logic instructions in the memory 430 to execute a real-time monitoring method for the molten pool morphology during the welding process of a crane boom. The method includes: acquiring multiple consecutive frames of images of the molten pool region during crane boom welding; for any frame of the multiple frames, extracting the geometric parameters of the molten pool region in the frame image, and determining the comprehensive deformation index of the frame image based on the geometric parameters and preset standard molten pool parameters; and determining whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, a preset deformation threshold, the geometric parameters, and the preset geometric threshold; if the frame image is an abnormal molten pool image, determining the abnormal timing parameters of the frame image based on the frame image, adjacent frame images of the frame image, the comprehensive deformation index of the frame image, and the comprehensive deformation index of adjacent frame images of the frame image, and classifying the frame image into a spatter abnormal image or a molten pool instability image based on the abnormal timing parameters; and outputting a warning signal if the frame image is a molten pool instability image.

[0098] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for real-time monitoring of the molten pool morphology during the welding process of a crane boom, characterized in that, The method includes: Acquire consecutive multi-frame images of the molten pool area during crane boom welding; For any frame of the multi-frame images, the geometric parameters of the molten pool region in the frame image are extracted, and the comprehensive deformation index of the frame image is determined based on the geometric parameters and the preset standard molten pool parameters. Furthermore, the frame image is determined to be an abnormal molten pool image based on the comprehensive deformation index, the preset deformation threshold, the geometric parameters, and the preset geometric threshold. In the case where the frame image is an abnormal molten pool image, the abnormal timing parameters of the frame image are determined based on the frame image, the adjacent frame images of the frame image, the comprehensive deformation index of the frame image, and the comprehensive deformation index of the adjacent frame images of the frame image. Based on the abnormal timing parameters, the frame image is divided into a splashing abnormal image or a molten pool instability image. If the frame image is a melt pool instability image, an early warning signal is output.

2. The method as described in claim 1, characterized in that, The warning signal includes multiple levels. If the frame image is a melt pool instability image, the output warning signal includes: Based on the comprehensive deformation index of the frame image, the comprehensive deformation index of each frame image within a preset time window before the frame image, and the comprehensive deformation index of the preset standard image, the deviation of the molten pool morphology prediction in the molten pool region is determined. The degree of anomaly in the molten pool region is determined based on the predicted deviation of the molten pool morphology. The level of the warning signal is determined based on the degree of anomaly, and the warning signal of the specified level is output.

3. The method as described in claim 1, characterized in that, The determination of abnormal temporal parameters of the frame image based on the frame image, adjacent frame images, the comprehensive deformation index of the frame image, and the comprehensive deformation index of adjacent frame images includes: The degree of deformation deviation of the frame image is determined based on the comprehensive deformation index of the frame image and the comprehensive deformation index of the adjacent frame images. The joint determination coefficient of the frame image is determined based on the frame image and its adjacent frame images; The abnormal timing parameters of the frame image are determined based on the degree of deformation deviation and the joint determination coefficient.

4. The method as described in claim 3, characterized in that, The determination of the joint determination coefficient of the frame image based on the frame image and its neighboring frame images includes: The persistence similarity and abnormal region concentration of the frame image are determined based on the pixel values ​​of each pixel in the frame image and the pixel values ​​of each pixel in the adjacent frame images. The joint determination coefficient of the frame image is determined based on the sustained similarity and the concentration of abnormal regions.

5. The method as described in claim 2, characterized in that, The determination of the melt pool morphology prediction deviation of the melt pool region based on the comprehensive deformation index of the frame image, the comprehensive deformation index of each frame image within a preset time window preceding the frame image, and the comprehensive deformation index of a preset standard image includes: The deformation rate of the molten pool region is determined based on the comprehensive deformation index of the frame image and the comprehensive deformation index of the adjacent frame images of the frame image. The degree of pre-existing anomaly in the molten pool region is determined based on the comprehensive deformation index of each frame image within a preset time window prior to the frame image. The standard deformation rate and standard pre-anomaly degree are determined based on the comprehensive deformation index of the preset standard image; The predicted deviation of the molten pool morphology in the molten pool region is determined based on the deformation rate, the degree of preceding anomaly, the standard deformation rate, and the degree of standard preceding anomaly.

6. The method as described in claim 1, characterized in that, The geometric parameters include: molten pool area, aspect ratio, contour smoothness, center of gravity offset, and grayscale uniformity.

7. The method as described in claim 6, characterized in that, The preset geometric threshold includes preset global parameters and preset local parameters. The preset global parameters consist of a preset standard melt pool area and a preset standard aspect ratio. The preset local parameters consist of a preset standard contour smoothness, a preset standard centroid offset, and a preset standard grayscale uniformity.

8. The method as described in claim 1, characterized in that, Prior to extracting multiple geometric parameters of the molten pool region in the frame image, the method further includes: The frame image is cropped to obtain a cropped image, in which the melt pool region is located at the center of the image. A binary image and a grayscale image are obtained based on the cropped image.

9. The method as described in claim 8, characterized in that, The process of obtaining a binary image and a grayscale image based on the cropped image includes: The cropped image is binarized to obtain a binary image; A grayscale image is obtained based on the binary image and the frame image, wherein the melt pool region in both the binary image and the grayscale image is located at the center of the image.

10. A real-time monitoring system for the molten pool morphology during the welding process of a crane boom, characterized in that, include: The image acquisition module is used to acquire multiple consecutive frames of images of the molten pool area during crane boom welding; The preliminary anomaly judgment module is used to extract the geometric parameters of the molten pool region in any frame of the multi-frame images, and determine the comprehensive deformation index of the frame image based on the geometric parameters and the preset standard molten pool parameters, and determine whether the frame image is an abnormal molten pool image based on the comprehensive deformation index, the preset deformation threshold, the geometric parameters and the preset geometric threshold. An anomaly segmentation module is used to determine the abnormal timing parameters of the frame image based on the frame image, the adjacent frame images of the frame image, the comprehensive deformation index of the frame image, and the comprehensive deformation index of the adjacent frame images of the frame image when the frame image is an abnormal molten pool image, and to segment the frame image into a splashing abnormal image or a molten pool instability image based on the abnormal timing parameters. The early warning module is used to output an early warning signal if the frame image is a melt pool instability image.

Images