Visual detection method for arc welding penetration

Abstract

The application provides an arc welding penetration visual detection method, and relates to the technical field of visual detection in a welding process. The method comprises the following steps: based on an image sequence collected from an arc molten pool coupling area in an arc welding process, a molten pool mask of a molten pool area in each image is extracted frame by frame; based on the molten pool mask, a molten pool image feature of the molten pool area in each image is extracted, wherein the molten pool image feature comprises a molten pool projection area; according to an adaptive detection threshold, an area mutation point in a time sequence of the molten pool projection area is detected, and a time corresponding to the area mutation point is determined as a molten drop impact triggering time; the molten drop impact triggering time is taken as a starting point to determine an analysis window; a molten pool image feature signal segment corresponding to a time period of the analysis window is intercepted from the molten pool image feature time sequence; and according to the molten pool image feature signal segment, a penetration in the arc welding process corresponding to the analysis window is calculated.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology for welding processes, and more specifically, to a method for visual inspection of the penetration depth of arc welding. Background Technology

[0002] Weld penetration depth is one of the key indicators for measuring weld quality. Insufficient penetration depth will lead to a decrease in the load-bearing strength of the weld, while excessive penetration depth may cause burn-through defects. In the process of pulsed metal-inert-gas (MIG) welding, real-time online monitoring of penetration depth is of great significance for ensuring weld quality.

[0003] Currently, commonly used weld penetration testing methods can be divided into two categories: contact and non-contact. Contact testing methods require the installation of sensors on the weldment, which is complex to operate and the sensors are easily damaged by high temperatures, making them unsuitable for real-time online applications in industrial production. Among non-contact methods, ultrasonic testing and X-ray testing both suffer from high equipment costs and difficulties in on-site deployment. Summary of the Invention

[0004] In view of this, the present invention provides a visual inspection method for arc welding penetration depth.

[0005] One aspect of the present invention provides a visual detection method for arc welding penetration depth, comprising: extracting a molten pool mask in each image frame by frame from an image sequence acquired targeting the arc-molten pool coupling region during arc welding, wherein the arc-molten pool coupling region includes an arc column region and a molten pool region; extracting molten pool image features in each image based on the molten pool mask, wherein the molten pool image features include the molten pool projected area, which is determined based on the total count of pixels with a pixel value of 1 in the molten pool mask; arranging the molten pool image features of each image according to the acquisition time sequence of the image sequence to form a molten pool image feature time sequence, wherein the molten pool image feature time sequence includes a molten pool projected area time sequence; detecting area abrupt change points in the molten pool projected area time sequence according to an adaptive detection threshold, and determining the time corresponding to the area abrupt change point as the droplet impact trigger time; determining an analysis window starting from the droplet impact trigger time; extracting molten pool image feature signal segments corresponding to the time period of the analysis window from the molten pool image feature time sequence; and calculating the penetration depth corresponding to the analysis window during arc welding based on the molten pool image feature signal segments.

[0006] According to the above-described technical solution of the present invention, the analysis window is divided based on the moment of impact of the molten droplet detected in the image. This is a purely vision-driven adaptive windowing method that does not require access to the welding machine's electrical signals. Compared to methods that divide the analysis window based on the phase of the electrical pulse, it reduces the impact of the time deviation between the electrical pulse phase and the actual impact moment of the molten droplet on the positioning accuracy of the analysis window. It also eliminates the timing misalignment problem caused by the difference in the flight time of the molten droplet after detaching from the welding wire, enabling the analysis window to be accurately positioned based on the actual impact moment of the molten droplet. This effectively improves the estimation accuracy of the oscillation frequency, and eliminates the need for manually setting a fixed threshold or relying on external synchronization signals, thus exhibiting strong adaptability to operating conditions and practical engineering applications. Attached Figure Description

[0007] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0008] Figure 1A An exemplary system architecture for applying a visual inspection method for arc welding penetration depth according to an embodiment of the present invention is shown;

[0009] Figure 1B A schematic diagram of an acquisition system for acquiring image sequences of the arc-molten pool coupling region during arc welding is shown according to an embodiment of the present invention.

[0010] Figure 1C A schematic diagram of an image sequence acquired using a high-speed camera during pulsed MIG welding, according to an embodiment of the present invention, is shown.

[0011] Figure 2 A flowchart of a visual inspection method for arc welding penetration depth according to an embodiment of the present invention is shown;

[0012] Figure 3 A schematic diagram illustrating the impact-driven adaptive windowing principle according to an embodiment of the present invention is shown;

[0013] Figure 4 A schematic diagram illustrating the signal-to-noise ratio adaptive weighted frequency estimation principle according to an embodiment of the present invention is shown;

[0014] Figure 5 A block diagram of an arc welding penetration visual inspection device according to an embodiment of the present invention is shown;

[0015] Figure 6 A block diagram of an electronic device suitable for implementing a visual inspection method for arc welding penetration depth according to an embodiment of the present invention is shown. Detailed Implementation

[0016] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0019] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0020] The weld depth detection method based on molten pool oscillation is a non-contact method. Its physical basis is that the molten pool will oscillate freely after being subjected to external excitation, and the oscillation frequency is directly related to the geometry of the molten pool; the greater the weld depth, the lower the oscillation frequency. However, the arc voltage signal is greatly affected by electrical noise from the welding equipment, and changes in the curvature of the molten pool surface can cause the voltage signal to lose its oscillation frequency characteristics, limiting the detection accuracy of this method.

[0021] In actual welding production sites, high-speed cameras are the most commonly used sensing devices for visual inspection of the welding process. Real-time estimation of weld penetration relying solely on high-speed camera images has broader engineering applicability. Furthermore, the weld penetration estimation method based on the physical frequency of molten pool oscillation directly correlates with the physical mechanism of molten pool oscillation, providing clear physical meaning and facilitating a physical explanation of weld penetration variations in welding process research and quality traceability. Therefore, providing a physically interpretable real-time weld penetration estimation method based solely on high-speed camera image sequences and extracted from molten pool oscillation frequencies is of great significance for expanding the applicability of online weld penetration monitoring.

[0022] In related technologies, the method for extracting the oscillation frequency of the molten pool based on high-speed camera images usually uses the phase of the electrical pulse output by the welding machine as the basis for dividing the analysis window, and performs segmented frequency analysis on the feature time series of the molten pool image.

[0023] However, in the process of realizing the concept of this invention, the inventors discovered that there is a time deviation between the electrical pulse phase boundary and the actual moment when the molten droplet impacts the molten pool. After the molten droplet leaves the welding wire during the peak current stage, it needs to travel for a period of time before impacting the molten pool. Therefore, the analysis window divided by the electrical pulse phase cannot accurately include the complete oscillation decay process after a single impact excitation, which affects the estimation accuracy of the oscillation frequency.

[0024] In view of this, embodiments of the present invention provide a visual inspection method for arc welding weld penetration, applicable to arc welding processes where the molten pool oscillates due to periodic excitation. Since the present invention uses droplet impact as the excitation event, it is applicable to arc welding processes involving droplet transition, including but not limited to: pulsed MIG welding, pulsed Metal-Active-Gas (MAG) welding, etc. It can be implemented as long as the following four conditions are met simultaneously: a periodic excitation event causes the molten pool to oscillate, such as droplet impact; the excitation event has detectable abrupt changes in the image, such as a spike in the first-order area difference; the molten pool oscillation frequency is physically related to the molten pool size, and the molten pool surface is visible in the image; the molten pool oscillation frequency and the repetition frequency of the excitation event meet the oscillation cycle requirements of the analysis window. When the molten pool oscillation frequency is higher than a preset multiple of the excitation repetition frequency, the analysis window can be taken within a single excitation interval; when the two are close or the molten pool oscillation frequency is lower than the excitation repetition frequency, an analysis window spanning multiple excitation intervals is used, as described later, or signal segments from multiple excitation intervals are aligned and superimposed.

[0025] The main implementation method includes: based on the image sequence acquired during the arc welding process targeting the coupled region of the arc-molten pool, extracting the molten pool mask of the molten pool area frame by frame in each image, wherein the coupled region of the arc-molten pool includes the arc column area and the molten pool area; based on the molten pool mask, extracting the molten pool image features of the molten pool area in each image, wherein the molten pool image features include the molten pool projected area, which is determined according to the total count of pixels with a pixel value of 1 in the molten pool mask; arranging the molten pool image features of each image according to the acquisition time sequence of the image sequence to form a molten pool image feature time series, wherein the molten pool image feature time series includes the molten pool projected area time series; detecting area abrupt change points in the molten pool projected area time series according to an adaptive detection threshold, and determining the time corresponding to the area abrupt change point as the droplet impact trigger time; determining the analysis window with the droplet impact trigger time as the starting point; extracting the molten pool image feature signal segment corresponding to the time period of the analysis window from the molten pool image feature time series; and calculating the penetration depth corresponding to the analysis window during the arc welding process based on the molten pool image feature signal segment.

[0026] Figure 1A An exemplary system architecture for applying a visual inspection method for arc welding penetration depth according to an embodiment of the present invention is shown. It should be noted that... Figure 1A The examples shown are merely examples of system architectures that can be applied to embodiments of the present invention, in order to help those skilled in the art understand the technical content of the present invention, but do not mean that embodiments of the present invention cannot be used in other devices, systems, environments or scenarios.

[0027] like Figure 1A As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0028] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0029] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0030] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0031] It should be noted that the arc welding penetration visual inspection method provided in this embodiment of the invention can generally be executed by server 105. Correspondingly, the arc welding penetration visual inspection device provided in this embodiment of the invention can generally be located in server 105. The arc welding penetration visual inspection method provided in this embodiment of the invention can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the arc welding penetration visual inspection device provided in this embodiment of the invention can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Alternatively, the arc welding penetration visual inspection method provided in this embodiment of the invention can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the arc welding penetration visual inspection device provided in the embodiments of the present invention can also be installed in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0032] For example, the image sequence acquired for the coupled region of the arc weld pool during arc welding can be originally stored in any one of the first terminal device 101, the second terminal device 102, or the third terminal device 103 (e.g., the first terminal device 101, but not limited thereto), or stored on an external storage device and can be imported into the first terminal device 101. Then, the first terminal device 101 can locally execute the arc welding penetration depth visual detection method provided in this embodiment of the invention, or send the image sequence to other terminal devices, servers, or server clusters, and have other terminal devices, servers, or server clusters that receive the image sequence execute the arc welding penetration depth visual detection method provided in this embodiment of the invention.

[0033] It should be understood that Figure 1A The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0034] According to an embodiment of the present invention, the image sequence acquired for the coupled region of the arc pool during the arc welding process can be acquired using a high-speed camera with a frame rate greater than or equal to 5000 frames per second, with the natural arc light of the welding arc as the light source.

[0035] Figure 1B A schematic diagram of an acquisition system for acquiring image sequences of the arc pool coupling region during arc welding is shown according to an embodiment of the present invention.

[0036] like Figure 1B As shown, the application is based on the pulsed MIG welding process of steel. The acquisition device is a high-speed camera 130 with a frame rate of not less than 5000 frames per second (fps), which is mounted sideways next to the welding torch 111. The natural arc light of the welding arc is used as the image acquisition light source, and no additional lighting equipment is required.

[0037] Specifically, the welding torch 111 is mounted on the camera bracket 110. A welding wire 113 extends from the nozzle 112 at the end of the welding torch 111, forming a molten droplet 114 at the end of the welding wire. An electric arc (arc column area) 121 is generated above the base material (workpiece) 120, and a molten pool area 122 is formed on the surface of the base material (workpiece) 120. The high-speed camera 130, through the lens 131, is positioned at an angle to the horizontal direction. The optical axis 132 of the angle faces the welding area, the welding direction is perpendicular to the paper and inwards, and D is the depth of the molten pool. The high-speed camera 130 is used to acquire visual sensing images of the welding process, mainly during the arc welding process, to acquire image sequences of the arc-molten pool coupling area, including the arc (arc column area) 121 and the molten pool area 122.

[0038] Figure 1CA schematic diagram of an image sequence acquired using a high-speed camera during pulsed MIG welding, according to an embodiment of the present invention, is shown.

[0039] like Figure 1C The image shown represents three phases captured by a high-speed camera: image 141 of the droplet flight phase, image 142 of the droplet impact phase, and image 143 of the base phase phase. It can be seen that each image exhibits a distinct vertical layered structure: the upper part is the arc-shaped column region, characterized by a cone-shaped luminous area extending downwards from the tip of the welding wire; the lower part is the molten pool region, characterized by the surface of the liquid metal pool and its ripple texture; a brightness transition zone exists between the two regions.

[0040] As shown in image 141 of the droplet flight period and image 142 of the droplet impact period, the droplet forms and falls off at the tip of the welding wire during the peak pulse current stage, and flies through the arc column area to the molten pool area at a high speed.

[0041] As shown in the base phase period image 143, during the base current phase, the brightness of the arc column decreases significantly and the conical region shrinks.

[0042] The image sequences processed in the following examples are all related to Figure 1C The images shown are similar to a sequence of images. It should be noted that when acquiring images, the row direction of the image sensor in the high-speed camera can be set to be parallel to the upper surface of the base material, so that the vertical direction of the image is consistent with the projection direction of the welding torch axis in the imaging plane. This ensures that the arc column area is located in the upper part of the image and the molten pool area is located in the lower part of the image.

[0043] The specific implementation of this solution will be described below with reference to this type of image sequence.

[0044] Figure 2 A flowchart of a visual inspection method for arc welding penetration depth according to an embodiment of the present invention is shown.

[0045] like Figure 2 As shown, the method includes operations S201 to S207.

[0046] In operation S201, based on the image sequence acquired for the arc welding process of the arc molten pool coupling region, the molten pool mask of the molten pool region in each image is extracted frame by frame. The arc molten pool coupling region includes the arc column region and the molten pool region.

[0047] For example, image sequences captured by a high-speed camera can be divided into regions frame by frame: by detecting the locations of vertical brightness abrupt changes in each frame, the dynamic boundary between the arc-shaped region and the molten pool region can be determined, thus dividing each image into two regions of interest: the arc-shaped region and the molten pool region. (See also...) Figure 1CAs shown, the dynamic boundary lines of each image are adaptively updated frame by frame as the arc light intensity changes caused by the pulse phase.

[0048] Then, the molten pool region is cut out separately, and arc scattering background suppression processing is performed on the molten pool region to remove background interference caused by arc scattering. Finally, a clean molten pool mask can be extracted through adaptive threshold segmentation for subsequent feature extraction.

[0049] In operation S202, based on the molten pool mask, the molten pool image features of the molten pool region in each image are extracted. The molten pool image features include the molten pool projection area, which is determined based on the total count of pixels with a pixel value of 1 in the molten pool mask.

[0050] According to embodiments of the present invention, the features of the molten pool image may further include at least one of the following: centroid vertical displacement, mean curvature of the upper edge, average brightness of the central region, horizontal width, etc., and may not be limited to these.

[0051] For example, based on the molten pool mask, five dimensions of molten pool image features can be extracted synchronously frame by frame, including the molten pool projection area, centroid vertical displacement, upper edge curvature mean, central region average brightness, and horizontal width. Arranging these features according to the acquisition time sequence can form a five-dimensional feature time series.

[0052] When the lower edge of the molten pool exceeds the image range, the vertical position of the upper edge of the molten pool can be used to replace the projected area of ​​the molten pool.

[0053] In operation S203, the molten pool image features of each image are arranged according to the acquisition time sequence of the image sequence to form a molten pool image feature time series, wherein the molten pool image feature time series includes the molten pool projected area time series.

[0054] In operation S204, based on the adaptive detection threshold, the abrupt change point in the area of ​​the molten pool projection area time series is detected, and the time corresponding to the abrupt change point is determined as the droplet impact trigger time.

[0055] For example, by calculating the first-order difference frame by frame on the time series of the molten pool's projected area, the difference in area between adjacent frames is reflected, transforming "how large the area is" into "how fast the area changes." Furthermore, by employing a backward sliding window adaptive threshold to detect abrupt changes in area, the moment of these abrupt changes can be determined as the trigger moment for the molten droplet impact. The determination of the trigger moment is entirely based on image visual information and does not depend on the electrical signals of the welding machine.

[0056] In operation S205, the analysis window is determined starting from the moment of droplet impact triggering.

[0057] According to an embodiment of the present invention, the number of frames in the analysis window is determined based on the molten pool oscillation period, taking a number of frames corresponding to a number of molten pool oscillation periods not less than a preset number of periods to ensure that the signal segment contains a sufficient number of oscillation periods for frequency estimation. Since the molten pool oscillation period itself is the output of this method, to break the interdependence between the number of frames and the oscillation period, the number of frames in the first analysis window is taken as a preset initial number of frames. From the second analysis window onwards, the molten pool oscillation period is determined based on the estimated oscillation frequency of the previous analysis window, and the number of frames in the current analysis window is obtained by multiplying the preset number of periods by the oscillation period and rounding it down using the image acquisition frame rate. This method shares the same estimated oscillation frequency with the frequency continuity constraint described later, without the need to introduce additional parameters.

[0058] In operation S206, the characteristic signal segment of the molten pool image is extracted from the time series of molten pool image features, corresponding to the time period of the analysis window.

[0059] In operation S207, the penetration depth corresponding to the analysis window is calculated based on the characteristic signal segments of the molten pool image during the arc welding process.

[0060] For example, the oscillation frequency can be estimated for the multidimensional feature time series in each analysis window. Then, the frequency estimates corresponding to the multidimensional feature time series are weighted and fused to obtain a final oscillation frequency. The melting depth can be obtained by converting the oscillation frequency.

[0061] Through the above embodiments of the present invention, the analysis window is divided based on the moment of impact of the molten droplet detected in the image. This is a purely vision-driven adaptive windowing method that does not require access to the electrical signal of the welding machine. Compared with the method of dividing the analysis window based on the phase of the electrical pulse, it reduces the impact of the time deviation between the phase of the electrical pulse and the actual impact moment of the molten droplet on the positioning accuracy of the analysis window, eliminates the timing misalignment problem caused by the difference in the flight time of the molten droplet after it leaves the welding wire, and enables the analysis window to be accurately positioned based on the moment of impact of the molten droplet. This effectively improves the estimation accuracy of the oscillation frequency. At the same time, it does not require manual setting of a fixed threshold or reliance on an external synchronization signal, and has strong adaptability to working conditions and practicality in engineering.

[0062] The following describes specific embodiments. Figure 2 The method shown will be further explained.

[0063] According to an embodiment of the present invention, corresponding to the above operation S201, the dynamic boundary line used to divide the arc column area and the molten pool area in each image can be detected frame by frame first. Specifically, this includes: extracting the vertical brightness profile line along the direction of the center column of each image frame by frame; performing second-order difference calculation on the vertical brightness profile line of each image to obtain the second-order difference curve corresponding to each image; searching for the maximum positive value point of the corresponding second-order difference curve in the upper half region of each image; and determining the dynamic boundary line according to the row coordinates corresponding to the maximum positive value point.

[0064] For example, suppose the image height is H pixels and the width is... Pixels, the image coordinate system origin is located at the top left corner, x is the column coordinate (increasing to the right), and y is the row coordinate (increasing downwards). Take the center column of the image. Extract the grayscale values ​​of the pixels in the column row by row from top to bottom to form a vertical brightness profile. Perform second-order difference operations on the profile. In the upper half of the image height search The point with the largest positive value will correspond to the row coordinates. Defined as the dynamic boundary between the arc column region and the molten pool region. ,in, This indicates finding the second-order difference value of the upper half of the vertical brightness profile. The y that reaches its maximum value U The value of .

[0065] The dynamic boundary line updates automatically frame-by-frame according to the change in arc intensity between the pulse peak phase and the base phase, eliminating the need for manually setting a fixed threshold. Limiting the search range to the upper half of the image eliminates interfering extreme points caused by textures within the melt pool region.

[0066] It should be noted that the brightness transition edge at the tip of the welding wire is steeper than that at the boundary line and is also located in the upper half of the image. To avoid it being misjudged as the boundary line, the search range can be further limited to the row below the welding wire tip and the search can be performed in conjunction with the neighborhood of the boundary line coordinates of the previous frame. In addition, the vertical brightness profile can be smoothed and then a second-order difference can be performed to suppress noise interference.

[0067] In some embodiments, this additional limitation can further eliminate interfering extreme points generated by the apex of the arc column and the flying droplets.

[0068] Through the above embodiments of the present invention, by extracting the vertical brightness profile of the center column of the image frame by frame and performing second-order difference calculation, and utilizing the physical characteristic that the brightness gradient is the largest at the junction of the arc column area and the molten pool area, the maximum positive value point of the second-order difference curve is searched in the upper part of the region to determine the dynamic boundary line. This achieves fully automatic, frame-by-frame adaptive detection of the boundary between the arc column area and the molten pool area without the need for manual setting of a fixed threshold. It can update the boundary line position in real time following the change of arc light intensity during the welding process, effectively eliminating the internal texture interference of the molten pool area, and providing a reliable basis for accurate windowing based on the droplet impact time.

[0069] According to an embodiment of the present invention, corresponding to the above operation S201, after obtaining the dynamic boundary line used to divide the arc column region and the molten pool region in each image frame by frame, the following operations can be performed: according to the dynamic boundary line, each image is segmented to obtain the original image of the molten pool region; Gaussian filtering is performed on the original image of the molten pool region to obtain the low-frequency background component; the product of the low-frequency background component and a preset coefficient is subtracted from the original image of the molten pool region to obtain the enhanced image of the molten pool region; according to the adaptive segmentation threshold, the enhanced image of the molten pool region is binarized to obtain a binarized image; the connected region with the largest area in the binarized image is extracted as the molten pool mask.

[0070] For example, after detecting the dynamic boundary line in each image frame by frame, arc scattering background suppression can be performed on the original image of the molten pool region below the dynamic boundary line in each image. Specifically, this includes: using 30% of the molten pool region height as the standard deviation for the original image of the molten pool region. Gaussian filtering is performed to obtain the low-frequency background component. This leads to the acquisition of enhanced images of the molten pool region. Where α is the background subtraction coefficient, which can be, for example, 0.5 to 0.7. In this embodiment, it is, for example, α. The enhanced image of the molten pool region is binarized using the Otsu's maximum inter-class variance (MOV) adaptive thresholding method, and the connected regions with the largest areas are extracted as the molten pool mask. In this context, 1 represents a molten pool pixel and 0 represents a non-molten pool pixel.

[0071] Through the above embodiments of the present invention, the original image of the molten pool area is accurately segmented by dynamic boundary lines, and the background is enhanced by extracting low-frequency background components using Gaussian filtering, which effectively suppresses the non-uniform background interference caused by arc scattering and highlights the foreground information of the molten pool area. In addition, by combining adaptive threshold binarization segmentation and maximum connected component extraction, the complete molten pool mask can be accurately separated from the complex welding scene, eliminating the interference of isolated noise points caused by spatter, smoke and arc reflection, and providing a highly reliable regional positioning basis for the stable extraction of subsequent molten pool oscillation feature parameters.

[0072] In the process of realizing the concept of this invention, the inventors also found that related methods usually only use single image features such as the molten pool area for frequency analysis, without considering the problem that the arc intensity changes significantly with the peak phase and the base phase during pulsed MIG welding, resulting in significant differences in the signal-to-noise ratio (SNR) of different image features, and thus the frequency estimation is not robust enough.

[0073] According to embodiments of the present invention, the molten pool image features may include multi-dimensional image features such as the molten pool projected area, the centroid vertical displacement, the mean curvature of the upper edge, the average brightness of the central region, and the horizontal width.

[0074] Based on the molten pool mask M(x,y) output frame by frame in operation S201, multidimensional image features can be automatically calculated. All features do not require manual annotation, are obtained entirely from the mask M(x,y) through arithmetic operations, and can be arranged according to the acquisition time sequence to form a multidimensional feature time series.

[0075] For example, the projected area of ​​the molten pool This refers to the total count of pixels with a value of 1 in the molten pool mask. The projected area of ​​the molten pool has a high signal-to-noise ratio under most operating conditions and is relatively sensitive to oscillations.

[0076] According to an embodiment of the present invention, the vertical displacement of the centroid can be determined based on the offset of the centroid row coordinates of the molten pool mask relative to the reference position, which characterizes the overall rising and falling trend of the molten pool surface.

[0077] For example, the image moment method can be used to automatically calculate the centroid ordinate of the molten pool mask. By combining multiple centroid ordinate sequences corresponding to the image sequence, the vertical displacement of the centroid can be calculated. Specifically, the zeroth moment is defined. and first moment .in This equals the total number of pixels (with a pixel value of 1) within the molten pool mask, i.e., the projected area S of the molten pool. The centroid's ordinate is... To stabilize the welding stage (the first stage after arc ignition) Frame to the (frame) Time average As a reference, the vertical displacement of the centroid is defined as . This indicates that the molten pool has sunk as a whole, resulting in a depression in the liquid surface (a trend of increasing melting depth). This indicates that the molten pool as a whole floats upwards, corresponding to a bulge on the liquid surface. This feature is most sensitive to the oscillation component in the direction of melt depth.

[0078] According to an embodiment of the present invention, the average curvature of the upper edge can be determined based on the average value of the signed discrete curvature at each pixel point on the upper edge of the molten pool mask, which characterizes the overall curvature degree and curvature direction of the upper edge contour of the molten pool mask. The discrete curvature at the upward protrusion of the upper edge contour takes a positive value, and the discrete curvature at the downward concavity takes a negative value.

[0079] For example, extract the ordered sequence of upper edge boundary points from the molten pool mask M(x,y). ,in, Monotonically increasing by column index Let be the row coordinate of the smallest row with a mask value of 1 in column j, and N be the total number of extracted ordered boundary points on the upper edge. Define a first-order central difference for each point in the ordered boundary point sequence on the upper edge. and second-order difference components The discrete curvature at point j is Take the mean curvature of all internal boundary points (j=2,...,N-1) as the mean curvature of the upper edge. The mean curvature K of the upper edge can reflect the overall curvature degree and direction of the upper surface boundary of the molten pool. When the molten pool surface oscillates, the surface periodically bulges and sags in the vertical direction, and the upper edge profile alternates periodically between upward convexity and downward concavity. The mean curvature K with a sign changes periodically between positive and negative, and its oscillation fundamental frequency is consistent with the natural oscillation frequency of the molten pool.

[0080] According to an embodiment of the present invention, the average brightness of the central region is determined based on the average gray level of each pixel within the inscribed elliptical region of the molten pool mask, characterizing the average level of image gray level within the central region of the molten pool mask.

[0081] For example, defining an inscribed elliptical region within the molten pool mask. (The major and minor axes are, for example, 50% of the major and minor axes of the molten pool bounding box), the original grayscale image within this inscribed elliptical region. Take pixel average The average brightness I of the central region can then be obtained, where, Represents the original grayscale image The pixel value at the (x,y) coordinate. The average brightness I of the central region is based on photometric principles. When the molten pool surface bulges, the normal vector points towards the camera, enhancing reflection and increasing brightness; when the surface sags, the brightness decreases. The time series of the average brightness I of the central region also contains modulation components synchronized with the pulse phase, which need to be suppressed during frequency estimation.

[0082] According to an embodiment of the present invention, the horizontal width is determined based on the difference between the maximum and minimum column coordinates of the molten pool mask in the row where the centroid row coordinates are located, and characterizes the lateral span of the molten pool.

[0083] For example, in the centroid ordinate Corresponding row Above, extract the column coordinates with a mask value of 1. and minimum column coordinates Calculate the horizontal span The horizontal width W can then be obtained. The horizontal width W can reflect the lateral oscillation component of the molten pool.

[0084] The above-mentioned multidimensional image features are all lateral projections of the molten pool. Their absolute values ​​are affected by the shooting angle, but there is a definite correspondence between the temporal oscillation components of each feature and the inherent oscillation frequency of the molten pool, which is not affected by the shooting angle. Therefore, the frequency extraction results are valid.

[0085] Through the above embodiments of the present invention, the introduction of multi-dimensional image features overcomes the problem of frequency estimation inaccuracy caused by signal-to-noise ratio fluctuations when the intensity of a single feature changes significantly. Different features have different sensitivities to arc interference. When the signal-to-noise ratio of a certain feature decreases due to strong arc interference, the other features can still provide effective oscillation information. Information redundancy and complementarity are formed among the features, which together ensure the robustness and continuity of frequency estimation. Thus, reliable molten pool oscillation information can be stably obtained throughout the entire cycle in which the pulse peak and base phase alternate.

[0086] According to an embodiment of the present invention, in response to detecting that the lowest non-zero row of the molten pool mask reaches the bottom boundary of the image, it is determined that the lower edge of the molten pool exceeds the image range, and the time series of the molten pool projected area is replaced with the time series of the average height of the upper edge of the molten pool. The average height of the upper edge of the molten pool is determined based on the average value of the row coordinates of each pixel at the upper edge of the molten pool.

[0087] For example, when the lowest non-zero row of the molten pool mask M(x,y) is detected to reach the bottom boundary of the image, it is determined that the lower edge of the molten pool exceeds the image range. In this case, the time series of the vertical position of the upper edge of the molten pool can be used instead of the projected area S of the molten pool for frequency estimation. At the same time, the trigger time of the droplet impact is determined by the time series of the average height of the upper edge of the molten pool. The vertical displacement of the centroid and the horizontal width are also affected by truncation due to their dependence on the integrity of the molten pool mask. Their signal-to-noise ratio is reduced and will be automatically downweighted or excluded by the aforementioned adaptive weighting mechanism; the extraction methods of the mean curvature of the upper edge and the average brightness of the central region remain unchanged.

[0088] The time series of the vertical position of the upper edge of the molten pool can be calculated as follows: given the row coordinates of the upper edge in column j. If the upper edge of the molten pool spans N columns in a single image frame, then N images can be obtained from that single image frame. N Arranging them in time sequence forms a curve. Considering the requirement that the time series at the vertical position of the upper edge of the melt pool can only contribute one number per frame in order to perform frequency analysis by arranging them frame by frame. This can be achieved by taking N values ​​from a single frame image. Compress it into a scalar and calculate its arithmetic mean. For example, extract all the values ​​from the t-th frame image. (j=1…N), the arithmetic mean can be calculated. The combined results from multiple frames This can be used as a time series of the vertical position of the upper edge of the molten pool. Among them, This represents the average height of the upper edge of the molten pool in frame t.

[0089] The average height of the upper edge of the molten pool increases when the liquid surface is concave (the row coordinate increases downwards) and decreases when it is convex. Therefore, the time series of the vertical position of the upper edge of the molten pool exhibits periodic fluctuations. Since its period is consistent with the time series of the projected area of ​​the molten pool, it can be used to replace the projected area of ​​the molten pool for frequency analysis.

[0090] The above embodiments of the present invention effectively solve the problem of the failure of projected area features when the molten pool is partially out of the image field of view due to dynamic changes in the welding process, ensuring the continuity of feature extraction and reducing feature interruption caused by limited field of view.

[0091] According to an embodiment of the present invention, the above operation S203 may include: performing first-order difference calculation on the time series of the molten pool projected area frame by frame to obtain the amplitude sequence of the rate of change of the molten pool projected area, that is, the absolute value sequence of the difference between the molten pool projected areas of two adjacent frames; taking the current time as the endpoint, taking a backward window of a preset number of frames, and calculating an adaptive detection threshold based on the mean and standard deviation of the amplitude of the rate of change of the molten pool projected area within the backward window, as well as a preset standard deviation coefficient; determining the time corresponding to the amplitude of the rate of change of the molten pool projected area that is greater than the adaptive detection threshold as a candidate time for triggering molten droplet impact; applying a minimum interval constraint to the candidate time for triggering molten droplet impact, and determining the candidate time for triggering molten droplet impact that satisfies the minimum interval constraint as the molten droplet impact trigger time. Wherein, when the lowest non-zero row of the molten pool mask is detected to reach the bottom boundary of the image, and it is determined that the lower edge of the molten pool exceeds the image range, and the time series of the molten pool projected area is replaced with the time series of the average height of the upper edge of the molten pool, the preset number of frames and the standard deviation coefficient in the adaptive detection threshold are re-determined according to the value range of the average height of the upper edge of the molten pool.

[0092] Figure 3 A schematic diagram illustrating the impact-driven adaptive windowing principle according to an embodiment of the present invention is shown. Figure 3 The time coordinates and segment lengths shown are only used to illustrate the principle relationship between impact detection and windowing, and do not represent the specific numerical proportions in actual implementation.

[0093] For example, firstly, the frame-by-frame first-order difference is calculated for the time series S(t) of the projected area of ​​the molten pool. S(t) represents the projected area of ​​the molten pool region in the t-th frame image, and S(t-1) represents the projected area of ​​the molten pool region in the (t-1)-th frame image. For example... Figure 3 As shown in the lower panel 310, the vertical axis The surface area of ​​the molten pool remains at a low level most of the time, reflecting normal oscillatory changes in the molten pool surface; however, at the moment the molten droplet impacts the molten pool, the area of ​​the molten pool undergoes a drastic change. A significant spike appears, which is much higher than the differential amplitude during the normal oscillation period.

[0094] A backward sliding window adaptive threshold is used to detect the aforementioned spikes. Taking the current time t (e.g., corresponding to the t-th frame) as the endpoint, a backward window of length L frames is taken, and the threshold within the window is calculated. mean and standard deviation , This is an intermediate parameter. Set the adaptive detection threshold to... c is the standard deviation coefficient; for example, L=18 and c=2.5. When When the time is determined, the corresponding moment will be identified as a candidate moment for impact triggering.

[0095] To prevent consecutive threshold exceedances caused by a single impact, a minimum interval constraint can be applied to adjacent trigger times. .in, This indicates the k-th impact time detected. This indicates the (k+1)th time of the detected impact. It should be no less than 60% of the theoretical minimum time interval between adjacent droplet impacts. For example, at a pulse frequency of 60 Hz, the theoretical time interval between adjacent droplet impacts is approximately 16.7 ms. Approximately 10ms is acceptable.

[0096] The dynamic boundary line shifts periodically with the pulse phase, causing the upper part of the molten pool mask to be periodically truncated or released. This introduces a step component synchronized with the pulse phase, rather than originating from the molten pool oscillation, into the time series of the molten pool projected area. This component also manifests as a spike in the first-order difference. Therefore, the candidate moment is confirmed as the molten droplet impact trigger moment only if the first-order difference of the vertical displacement of the centroid at the droplet impact triggering candidate moment is positive and greater than its adaptive detection threshold, and the amplitude of the first-order difference of the average brightness of the arc column region is less than the preset brightness step threshold. Otherwise, it is judged as a false trigger and discarded.

[0097] Through the above embodiments of the present invention, the adaptive detection threshold is calculated using a backward window, relying only on historical data prior to the current moment and unaffected by future impact events. This reduces the problem of the central window artificially inflating the threshold near the impact moment due to the influence of large differential values ​​introduced after an impact on the threshold calculation before the impact, thus ensuring the adaptability of the threshold calculation and the reliability of impact detection during highly dynamic welding processes. It also ensures the online real-time feasibility of the algorithm in actual welding monitoring. Furthermore, applying a minimum interval constraint helps to eliminate continuous threshold exceedances caused by a single impact, further improving the accuracy of the detection results.

[0098] According to an embodiment of the present invention, the above operation S204 may include: determining the number of window frames of the analysis window, wherein the number of window frames is greater than or equal to the number of frames corresponding to a preset number of molten pool oscillation cycles; for the first analysis window, a preset initial number of frames is taken; for the remaining analysis windows, the number of frames is determined based on the estimated oscillation frequency of the previous analysis window; in response to detecting that the number of frames between the current droplet impact triggering time and the next adjacent droplet impact triggering time is greater than or equal to the sum of a preset number of transient exclusion frames and the number of window frames, starting from the current droplet impact triggering time, skipping the number of transient exclusion frames, and intercepting the signal of the number of window frames. The segment is used as the analysis window; in response to the detection that the number of frames between the current droplet impact trigger time and the next adjacent droplet impact trigger time is less than the sum of the number of transient exclusion frames and the number of window frames, the signal segment spanning the window frame number is extracted from the current droplet impact trigger time as the starting point, skipping the number of transient exclusion frames, and crossing multiple adjacent droplet impact trigger times as the analysis window; or, the effective signal segments corresponding to each of the consecutively preset number of droplet impact trigger times, after skipping the number of transient exclusion frames, are aligned according to the droplet impact trigger times corresponding to each effective signal segment and then averaged point by point to obtain the superimposed signal segment as the analysis window.

[0099] For example, it can be defined as the k-th impact moment detected. The analysis window starting from point k (hereinafter referred to as the kth analysis window) is This expression applies when the interval between adjacent impacts can accommodate the sum of the number of transient exclusion frames and the number of window frames; otherwise, the analysis window is determined according to Method 1 or Method 2 described later. To exclude frames in the transient state, for example, 6 to 8 frames can be selected (corresponding to 1.2 to 1.6 ms). The number of window frames is determined according to the oscillation period of the molten pool as described above. The preset number of periods can be, for example, 3 to 5.

[0100] When the interval between two adjacent impacts is insufficient to accommodate the sum of the transient exclusion frames and the window frame count, for example, when the pulse frequency is 60Hz and the interval between adjacent impacts is approximately 83 frames, ... Approximately 178 frames. The analysis window is determined using one of the following two methods.

[0101] Method 1: Starting from the current moment of droplet impact triggering, skip the transient exclusion frames and continuously capture... A frame allows the signal segment to span one or more subsequent droplet impact trigger times. Since the window contains multiple impact excitations, the transient segments adjacent to each impact trigger time in the signal segment can be interpolated or weighted and suppressed before frequency estimation.

[0102] Method 2: Take M consecutive droplet impact trigger times (M is the preset number of superpositions, for example, 3 to 5), take each droplet impact trigger time as the starting point, skip the number of transient exclusion frames, and extract the same number of valid signal segments. Align the M valid signal segments according to their corresponding droplet impact trigger times and calculate the average point by point to obtain the superimposed signal segment, which is used as the analysis window.

[0103] Method 2 is valid only if the melting depth remains approximately constant during the superposition period. Therefore, the fluctuation of the M impact intervals involved in the superposition is examined: when the fluctuation of adjacent impact intervals exceeds a preset proportion, the process state is determined to have changed, and the process reverts to Method 1 or uses only the signal segment of a single interval for estimation.

[0104] Based on the above embodiments, the determined analysis window is as follows: Figure 3 As shown in the top panel 320: the first region 321 and the third region 323 are transient exclusion segments, and the second region 322 and the fourth region 324 are the corresponding valid analysis windows.

[0105] Through the above embodiments of the present invention, by skipping the number of transient exclusion frames and extracting a signal segment with a window frame number determined by the molten pool oscillation period as the analysis window, the interference of the nonlinear transient process at the moment of impact is effectively reduced, while ensuring that the signal segment contains a sufficient number of oscillation periods to support frequency estimation. Simultaneously, when the adjacent impact interval is insufficient to accommodate the sum of the number of transient exclusion frames and the window frame number, the problem of temporal conflict between the required length of the analysis window and the repeated interval of molten droplet impact is solved by extracting signal segments across multiple impact intervals or by aligning and superimposing signal segments from multiple impact intervals according to their respective trigger times. This makes the method applicable even when the molten pool oscillation frequency is close to or lower than the pulse frequency. Furthermore, since the flight time of the molten droplet after detaching from the welding wire fluctuates randomly, when aligning and superimposing signal segments after multiple excitations based on the electrical pulse phase, the signal segments are decoherent, and the oscillation components cancel each other out during superposition. However, by using the actual impact time of the molten droplet detected in the image as the reference, coherent superposition can be achieved, thereby improving the signal-to-noise ratio of the signal segment without increasing the length of a single signal segment.

[0106] According to an embodiment of the present invention, the above operation S206 may include: calculating the frequency estimate and real-time logarithmic signal-to-noise ratio of the corresponding category of molten pool image features within the analysis window based on various molten pool image feature signal segments; converting the real-time logarithmic signal-to-noise ratio into a real-time linear signal-to-noise ratio; calculating the normalized dynamic weights of various molten pool image features based on the real-time linear signal-to-noise ratios of multiple molten pool image features that satisfy a preset threshold, wherein satisfying the preset threshold indicates that the real-time logarithmic signal-to-noise ratio of the molten pool image features is greater than or equal to the preset threshold value; weighting and fusing the frequency estimates of multiple molten pool image features that satisfy the preset threshold value based on the normalized dynamic weights of various molten pool image features to obtain the oscillation frequency estimate of the analysis window; and calculating the weld depth of the analysis window based on the oscillation frequency estimate and calibration coefficients related to welding materials and process parameters.

[0107] For example, corresponding to each analysis window It is possible to extract the molten pool image feature signal segment corresponding to the time period from the multi-dimensional feature time series output based on operation S202 and its embodiments. Let the i-th dimension feature time series be denoted as... Then the i-th dimension of the molten pool image features in the analysis window The signal segment within is defined as .in, The signal segment corresponding to the projected area S of the molten pool. Corresponding vertical displacement of the centroid The signal segment, The signal segment corresponding to the mean curvature K at the upper edge, The signal segment corresponding to the average brightness I in the central region. The signal segment corresponding to a horizontal width W. For each signal segment... Subsequent oscillation frequency estimations are performed separately, thereby incorporating the multidimensional feature time series output based on operation S202 and its embodiments, as well as the analysis windows defined based on operations S203 to S204 and their embodiments, into the processing.

[0108] For each dimension of the melt pool image feature signal segment in each analysis window The oscillation frequency can be independently calculated using three methods: short-time Fourier transform, continuous wavelet transform, and empirical mode decomposition. The median of these three methods is then taken as the frequency estimate of the i-th dimension molten pool image feature within this analysis window. .

[0109] Before performing the above frequency estimation, the mean and linear trend of each signal segment are first removed, and a window function is applied to suppress spectral leakage caused by the truncation at both ends of the signal segment; when the pulse frequency falls within the frequency search range, notch filtering is also performed on the pulse frequency and the frequency points where its harmonics are located.

[0110] Furthermore, when the number of oscillation periods contained in the analysis window is small (e.g., less than 3), the short-time Fourier transform degenerates into a single Fourier transform of the entire signal segment, the empirical mode decomposition cannot complete the screening due to insufficient number of extrema, and the influence cone of the continuous wavelet transform on the target frequency scale will also cover the entire signal segment. The estimation performance of all three methods decreases significantly. In this case, a parametric frequency estimation method can be used instead. The frequency estimation accuracy of this type of method is determined by the signal-to-noise ratio rather than the signal segment length. It can obtain a frequency resolution higher than the reciprocal of the signal segment length on shorter signal segments, directly corresponds to the physical model of the free decay oscillation of the molten pool after impact excitation, and can simultaneously provide the oscillation decay coefficient.

[0111] Considering that the frequency of capillary oscillations in the molten pool is determined by the size of the molten pool and the surface tension, and the magnitude is fixed, spectral peaks outside this range cannot be molten pool oscillations. A frequency search range can be defined for each analysis window.

[0112] Based on the physically possible range, the frequency search range for the first analysis window can be set to 10Hz to 400Hz. When using a non-parametric frequency estimation method, the lower limit of the frequency search range should not be lower than the lowest frequency that the current analysis window can resolve. When the lower limit of the required search range is lower than this value, the number of window frames should be increased accordingly, or a parametric frequency estimation method should be used. Components below 10Hz include slow area drift caused by thermal accumulation and mechanical movement of the welding torch; components above 400Hz include high-frequency interference such as image noise, spatter, and pixel jitter at mask edges. Based on this setting, physically unreasonable candidates can be eliminated first to prevent the algorithm from mistaking noise peaks for the dominant oscillation frequency.

[0113] Considering that the oscillation frequency of the molten pool is directly related to the melt depth, and that the melt depth cannot change abruptly within an analysis window timescale, and that the interval between two adjacent droplet impacts is only a few milliseconds to tens of milliseconds, the changes in the thermal state of the base material and the size of the molten pool are gradual. Therefore, the true frequency of the later analysis window must fall near the estimated value of the previous analysis window. Thus, starting from the second analysis window, the frequency can be further narrowed using continuity, for example, by using the estimated value from the previous window. Centered on, in Search within the range. When the fused signal-to-noise ratio of three consecutive analysis windows is lower than the preset threshold, resume full-range search.

[0114] After obtaining the frequency estimates of each dimension of the molten pool image features, the real-time signal-to-noise ratio (SNR) of each dimension of the molten pool image feature can be estimated within the analysis window. Specifically, for the i-th dimension of the molten pool image feature, its frequency estimate can be used to estimate the real-time signal-to-noise ratio (SNR) within the analysis window. Nearby signal frequency band [ - , + ],in The noise frequency band should be no less than twice the frequency resolution of the current analysis window. The noise band should be the portion within the frequency search range but outside the signal band, not the entire frequency band, to avoid the signal-to-noise ratio values ​​becoming incomparable due to the uncertainty of the upper bound of the noise band. Calculate the average power of the i-th dimension molten pool image feature within the corresponding signal band. Average power within the corresponding noise frequency band ratio This allows us to obtain the real-time logarithmic signal-to-noise ratio of the i-th dimension molten pool image features. .

[0115] When calculating the normalized dynamic weights, the real-time logarithmic signal-to-noise ratio of the i-th dimension of the melt pool image features is first used. From logarithm Convert to linear value The real-time linear signal-to-noise ratio of the i-th dimension molten pool image features is obtained. .

[0116] When a certain dimension of the molten pool image features Below the preset threshold When the threshold value is 8dB, the features of that dimension of the molten pool image are excluded from the fusion calculation. For the remaining features that meet the threshold, normalized dynamic weights are calculated. ,in, Here, represents the normalized dynamic weights of the i-th dimension of the molten pool image features, where i indicates which dimension's weight is currently being calculated, v is used to iterate through all valid dimensions and sum, and valid indicates that the current analysis window satisfies the condition. The set of feature dimensions.

[0117] When the real-time logarithmic signal-to-noise ratio of all categories of melt pool image features within this analysis window is lower than the preset threshold... When the valid feature dimension set is empty, the denominator of the normalized dynamic weights is zero. At this time, the estimated oscillation frequency and melt depth of this analysis window are not output; instead, an invalid flag is output, and the melt depth estimate from the previous analysis window is used. When a preset number of invalid analysis windows appear consecutively, the full-range frequency search is resumed, and initialization is performed again based on the first analysis window.

[0118] Using linear values ​​instead of logarithmic (dB) values ​​to calculate weights can truly reflect the absolute differences in the power of each feature signal. This allows high signal-to-noise ratio (SNR) features to obtain dominant weights that match their data quality during fusion. It also reduces the problem that low SNR features still occupy an unreasonable fusion weight due to the logarithmic compression of the weight difference between high and low SNR features, thus effectively improving the accuracy and reliability of multi-feature fusion.

[0119] Based on the normalized dynamic weights of the multidimensional molten pool image features, the frequency estimates of the multidimensional molten pool image features are weighted and fused to obtain the estimated oscillation frequency of this analysis window. .

[0120] Figure 4 A schematic diagram illustrating the signal-to-noise ratio adaptive weighted frequency estimation principle according to an embodiment of the present invention is shown.

[0121] like Figure 4 As shown, the estimation error of feature frequencies with low SNR is large and the estimated value deviates far from the true frequency. For example, the average brightness I in the central region when the arc interference is strong, the signal-to-noise ratio drops significantly (SNR is only 8dB) when the arc interference is strong. By automatically reducing its fusion weight (weight is 1%) through an adaptive weighting mechanism, its contribution is reduced, and the robustness of the fusion result to the failure of a single feature due to interference is significantly improved, and it is closer to the true oscillation frequency.

[0122] When only one analysis window exists in the image sequence, the estimated oscillation frequency will be... Substituting the values ​​into the physical constraint regression model, the estimated melt depth can be calculated. , where C and As a calibration coefficient related to welding materials and process parameters, this estimated weld depth can be directly used as the weld depth for this analysis window. This model is based on the capillary oscillation theory of molten pool surface tension, where the oscillation frequency is related to the characteristic dimensions of the molten pool's free surface. The power is inversely proportional. Since there is a monotonic correspondence between the characteristic dimensions of the free surface of the molten pool and the depth of penetration under given material and process parameters, this relationship can be included in the calibration coefficients and directly characterized by the depth of penetration. C and This is an empirical calibration coefficient, the value of which can be determined by fitting multiple sets of welding experimental data with known penetration depth using the least squares method. It is only valid within the range of nominal material and process parameters corresponding to the calibration. For example, for pulsed MIG welding of steel, C=250... =1.5.

[0123] Through the above embodiments of the present invention, by calculating the real-time signal-to-noise ratio of each dimension of the molten pool image features and converting them into linear values, and using the linear signal-to-noise ratio as a weight to perform weighted fusion of the frequency estimates of multidimensional features that meet the signal-to-noise ratio threshold, high-quality features occupy the dominant weight in the fusion, while low-quality features are automatically suppressed or excluded. This overcomes the problems of significant differences in the signal-to-noise ratio of different features caused by the periodic change of arc intensity, and the tendency of fixed-weight fusion or equal-weight fusion to be dominated by low-quality features and produce deviations. It ensures that accurate and reliable oscillation frequency estimates can be obtained throughout the entire cycle of alternating phase changes between the pulse peak and the base value, thereby providing a stable and reliable input for molten depth calculation.

[0124] According to an embodiment of the present invention, when an image sequence corresponds to multiple analysis windows, the above-mentioned calculation of the melt depth of the analysis window based on the oscillation frequency estimate and the calibration coefficient related to the welding material and process parameters can be expressed as follows: obtaining multiple melt depth estimates corresponding one-to-one with the multiple analysis windows; calculating the smoothed melt depth estimate of the current analysis window by weighted averaging the melt depth estimates of the second preset number of consecutive analysis windows centered on the current analysis window, wherein the smoothing weight is obtained by weighted summing of the real-time linear signal-to-noise ratio of the multiple types of molten pool image features based on the normalized dynamic weights of the multiple types of molten pool image features that meet the preset threshold values ​​in the corresponding analysis window; and determining the smoothed melt depth estimate as the melt depth of the current analysis window.

[0125] For example, each analysis window obtained based on operation S204 and its embodiments is calculated using the aforementioned method to obtain a melt depth estimate, and the melt depth estimate of the k-th analysis window is denoted as... The melt depth estimates from multiple analysis windows are weighted by a moving average with signal-to-noise ratio as the weight, and the final melt depth estimate sequence is output. Each final melt depth estimate in the final melt depth estimate sequence can be identified as the melt depth corresponding to the respective analysis window.

[0126] Considering that the data in a single analysis window may be affected by occasional interference such as spatter and strong arc light, the calculated results may occasionally fluctuate. To obtain a stable and reliable output, the melt depth estimates of several adjacent analysis windows can be averaged, and the windows with better data quality should have a larger weight in the average. The basis for measuring data quality is the signal-to-noise ratio (SNR) calculated in the previous embodiment: if the SNR of the multidimensional melt pool image features of a certain window is generally high, it indicates that the image signal of this segment is clean and the result is reliable; otherwise, it indicates that it has been interfered with. Based on this, the smoothing weight of the k-th analysis window is defined. The real-time linear signal-to-noise ratio of each effective dimension within the analysis window is the weighted average of its normalized dynamic weights. The larger the value, the better the data quality of the analysis window.

[0127] by As the weight, take the values ​​centered on the k-th analysis window and the values ​​before and after it. Average of windows ,in, This is the smoothed melt depth estimate for the k-th analysis window. This is the estimated melt depth for the p-th analysis window. Let p be the smoothing weight of the p-th analysis window. To smooth the window length, for example, using 3 to 7 analysis windows, and in this embodiment using 5, a center-smoothing window is introduced after the current analysis window. The output delay of each analysis window, calculated at a pulse frequency of 60Hz, is approximately 33ms. When using the aforementioned alignment and overlay method to determine the analysis window, the M-1 impact intervals required for overlay should also be taken into account. Calculated at a pulse frequency of 60Hz and M=5, the total output delay is approximately 100ms, which is much smaller than the response time scale of welding process adjustments and does not affect the real-time performance of weld penetration monitoring. Based on this, the results retain the true trend of weld penetration variation during the welding process while suppressing jumps caused by occasional interference. The output delay of each analysis window... Arranged chronologically, these sequences form the final weld depth estimation sequence, which is used for real-time monitoring of the welding process.

[0128] Through the above embodiments of the present invention, by weighted averaging of the melt depth estimates of multiple consecutive analysis windows, and using the linear signal-to-noise ratio weighted summation of multiple types of molten pool image features within each window as the smoothing weight, the windows with high signal-to-noise ratios occupy a larger proportion in the smoothing results. Thus, while preserving the true trend of melt depth changes with the welding process, it effectively suppresses the jumps in single-window estimates caused by occasional interferences such as spatter and strong arc light, thereby improving the stability and reliability of melt depth monitoring output.

[0129] Based on the above embodiments of the present invention, a method is realized that uses a high-speed camera to acquire image sequences, extracts the oscillation frequency of the molten pool through image processing and signal analysis methods, and estimates the arc welding penetration depth. This method can effectively alleviate the problems of analysis window division depending on electrical pulse phase and insufficient robustness of frequency estimation based on single image features in related methods.

[0130] Figure 5 A block diagram of a visual inspection device for arc welding penetration depth according to an embodiment of the present invention is shown.

[0131] like Figure 5 As shown, the arc welding penetration visual inspection device 500 includes a molten pool mask extraction module 510, a molten pool image feature extraction module 520, a timing arrangement module 530, a droplet impact trigger time determination module 540, an analysis window determination module 550, a molten pool image feature signal segment extraction module 560, and a penetration depth calculation module 570.

[0132] The molten pool mask extraction module 510 is used to extract the molten pool mask of the molten pool area in each image frame by frame based on the image sequence acquired for the arc molten pool coupling area during the arc welding process. The arc molten pool coupling area includes the arc column area and the molten pool area.

[0133] The molten pool image feature extraction module 520 is used to extract molten pool image features of the molten pool region in each image based on the molten pool mask. The molten pool image features include the molten pool projection area, which is determined based on the total number of pixels with a pixel value of 1 in the molten pool mask.

[0134] The timing arrangement module 530 is used to arrange the melt pool image features of each image according to the acquisition timing of the image sequence to form a melt pool image feature time series, wherein the melt pool image feature time series includes the melt pool projected area time series.

[0135] The droplet impact triggering time determination module 540 is used to detect the area change point in the time series of the projected area of ​​the molten pool according to the adaptive detection threshold, and determine the time corresponding to the area change point as the droplet impact triggering time.

[0136] The analysis window determination module 550 is used to determine the analysis window starting from the moment of droplet impact triggering.

[0137] The molten pool image feature signal segment extraction module 560 is used to extract the molten pool image feature signal segment corresponding to the analysis window from the molten pool image feature time series.

[0138] The penetration depth calculation module 570 is used to calculate the penetration depth corresponding to the analysis window during the arc welding process based on the characteristic signal segments of the molten pool image.

[0139] According to an embodiment of the present invention, the molten pool mask extraction module includes a dynamic boundary detection unit, a molten pool region original image segmentation unit, a Gaussian filtering unit, a molten pool region enhanced image acquisition unit, a binarization segmentation unit, and a molten pool mask extraction unit.

[0140] The dynamic boundary detection unit is used to detect the dynamic boundary lines used to divide the arc column region and the molten pool region in each image frame by frame.

[0141] The original image segmentation unit for the molten pool region is used to segment each image according to the dynamic boundary line to obtain the original image of the molten pool region.

[0142] The Gaussian filtering unit is used to perform Gaussian filtering on the original image of the molten pool area to obtain the low-frequency background component.

[0143] The molten pool region enhancement image acquisition unit is used to subtract the product of the low-frequency background component and a preset coefficient from the original image of the molten pool region to obtain the molten pool region enhancement image.

[0144] The binarization segmentation unit is used to perform binarization segmentation on the enhanced image of the molten pool region according to the adaptive segmentation threshold, so as to obtain a binarized image.

[0145] The melt pool mask extraction unit is used to extract the largest connected region in the binarized image as the melt pool mask.

[0146] According to embodiments of the present invention, each image includes at least an arc-shaped region located at the top of the image and a molten pool region located below the arc-shaped region. The dynamic boundary detection unit includes a vertical brightness profile extraction subunit, a second-order difference calculation subunit, a maximum positive value point search subunit, and a dynamic boundary determination subunit.

[0147] The vertical brightness profile extraction subunit is used to extract the vertical brightness profile along the central column direction of each image frame by frame.

[0148] The second-order difference calculation subunit is used to perform second-order difference calculation on the vertical brightness profile of each image to obtain the second-order difference curve corresponding to each image.

[0149] The maximum positive value search subunit is used to search for the maximum positive value of the corresponding second-order difference curve in the upper half region of each image.

[0150] The dynamic boundary line determines the sub-unit and is used to determine the dynamic boundary line based on the row coordinates corresponding to the maximum positive value point.

[0151] According to an embodiment of the present invention, the droplet impact triggering time determination module includes a first-order differential calculation unit, an adaptive detection threshold calculation unit, a droplet impact triggering candidate time determination unit, and a droplet impact triggering time determination unit.

[0152] The first-order difference calculation unit is used to perform first-order difference calculation on the time series of the projected area of ​​the molten pool frame by frame to obtain the amplitude sequence of the rate of change of the projected area of ​​the molten pool, wherein the amplitude of the rate of change of the projected area of ​​the molten pool is the absolute value of the difference between the projected areas of the molten pool in two adjacent frames.

[0153] The adaptive detection threshold calculation unit is used to take the current time as the end point, take a backward window with a length of preset number of frames, and calculate the adaptive detection threshold based on the mean and standard deviation of the amplitude of the rate of change of the projected area of ​​the melt pool within the backward window, as well as the preset standard deviation coefficient.

[0154] The droplet impact triggering candidate time determination unit is used to determine the time corresponding to the amplitude of the rate of change of the projected area of ​​the molten pool that is greater than the adaptive detection threshold as the droplet impact triggering candidate time.

[0155] The droplet impact triggering time determination unit is used to apply a minimum interval constraint to the candidate droplet impact triggering times and determine the candidate droplet impact triggering times that satisfy the minimum interval constraint as the droplet impact triggering times.

[0156] According to an embodiment of the present invention, the analysis window determination module includes a window frame number determination unit, an analysis window truncation unit, and a cross-interval processing unit.

[0157] The window frame number determination unit is used to determine the number of window frames for the analysis window. The number of window frames is greater than or equal to the number of frames corresponding to a preset number of melt pool oscillation cycles. For the first analysis window, the preset initial frame number is taken, and for the remaining analysis windows, the number of frames is determined based on the estimated value of the oscillation main frequency of the previous analysis window.

[0158] The analysis window capture unit is used to capture a signal segment of the window frame number as the analysis window when the number of frames between the detected droplet impact trigger time and the next adjacent droplet impact trigger time is greater than or equal to the sum of the preset transient exclusion frame number and the window frame number. Starting from the droplet impact trigger time, the unit skips the transient exclusion frame number and captures the signal segment of the window frame number as the analysis window.

[0159] The cross-interval processing unit is used to respond to the detection that the number of frames between the droplet impact trigger moment and the next adjacent droplet impact trigger moment is less than the sum of the number of transient exclusion frames and the number of window frames. Starting from the droplet impact trigger moment, it skips the number of transient exclusion frames and captures a signal segment spanning multiple adjacent droplet impact trigger moments as the analysis window. Alternatively, it can align the effective signal segments corresponding to each of the consecutively preset number of droplet impact trigger moments after skipping the number of transient exclusion frames according to the droplet impact trigger moment corresponding to each effective signal segment, and then calculate the average point by point to obtain the superimposed signal segment as the analysis window.

[0160] According to an embodiment of the present invention, the melting depth calculation module includes a frequency signal-to-noise ratio calculation unit, a signal-to-noise ratio conversion unit, a normalized dynamic weight calculation unit, a weighted fusion unit, and a melting depth calculation unit.

[0161] The frequency signal-to-noise ratio calculation unit is used to calculate the frequency estimate and real-time logarithmic signal-to-noise ratio of the corresponding category of molten pool image features within the analysis window based on the characteristic signal segments of various types of molten pool images.

[0162] The signal-to-noise ratio conversion unit is used to convert the real-time logarithmic signal-to-noise ratio into the real-time linear signal-to-noise ratio.

[0163] The normalized dynamic weight calculation unit is used to calculate the normalized dynamic weight of each type of molten pool image feature based on the real-time linear signal-to-noise ratio of the multi-type molten pool image features that meet the preset threshold value. The real-time logarithmic signal-to-noise ratio of the molten pool image features that meet the preset threshold value is greater than or equal to the preset threshold value.

[0164] The weighted fusion unit is used to perform weighted fusion of the frequency estimates of multiple types of melt pool image features that meet the preset threshold values ​​according to the normalized dynamic weights of various melt pool image features, so as to obtain the oscillation main frequency estimate of the analysis window.

[0165] The penetration depth calculation unit is used to calculate the penetration depth of the analysis window based on the estimated oscillation frequency and calibration coefficients related to welding materials and process parameters.

[0166] According to an embodiment of the present invention, the image sequence corresponds to multiple analysis windows. The melt depth calculation unit includes multiple melt depth estimation subunits, melt depth estimation smoothing subunits, and melt depth determination subunits.

[0167] The Multiple Melt Depth Estimate Acquisition Sub-unit is used to acquire multiple melt depth estimates that correspond one-to-one with multiple analysis windows.

[0168] The melt depth estimation smoothing subunit is used to calculate the smoothed melt depth estimate of the current analysis window by weighted averaging the melt depth estimates of the second preset number of consecutive analysis windows centered on the current analysis window, based on the smoothing weights of the respective smoothing weights of the second preset number of consecutive analysis windows. The smoothing weights are obtained by weighted summing of the real-time linear signal-to-noise ratios of the multiple types of melt pool image features that meet preset threshold values ​​in the corresponding analysis windows.

[0169] The melt depth determination sub-unit is used to determine the smoothed melt depth estimate as the melt depth of the current analysis window.

[0170] Any one or more of the modules, units, and subunits, or at least part of the functions of any one or more of them, according to embodiments of the present invention, can be implemented in one module. Any one or more of the modules, units, and subunits according to embodiments of the present invention can be implemented by dividing them into multiple modules. Any one or more of the modules, units, and subunits according to embodiments of the present invention can be at least partially implemented as hardware circuits, such as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or implemented by hardware or firmware through any other reasonable means of integrating or packaging circuits, or implemented in any one of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, one or more of the modules, units, and subunits according to embodiments of the present invention can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0171] For example, any multiple modules among the following can be implemented in one module / unit / subunit: molten pool mask extraction module 510, molten pool image feature extraction module 520, timing arrangement module 530, droplet impact trigger time determination module 540, analysis window determination module 550, molten pool image feature signal segment extraction module 560, and melt depth calculation module 570. Alternatively, any one of these modules / units / subunits can be split into multiple modules / units / subunits. Or, at least some of the functions of one or more of these modules / units / subunits can be combined with at least some of the functions of other modules / units / subunits and implemented in one module / unit / subunit. According to embodiments of the present invention, at least one of the following modules can be implemented, at least partially, as hardware circuits: molten pool mask extraction module 510, molten pool image feature extraction module 520, timing arrangement module 530, droplet impact trigger time determination module 540, analysis window determination module 550, molten pool image feature signal segment extraction module 560, and melt depth calculation module 570. These can be implemented, for example, as field-programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems-on-a-chip, systems-on-a-substrate, systems-on-package, application-specific integrated circuits (ASICs), or as hardware or firmware through any other reasonable means of integrating or packaging circuits. Alternatively, they can be implemented in any one of the three methods of software, hardware, and firmware, or in a suitable combination of any of these methods. Alternatively, at least one of the following modules can be implemented as a computer program module: molten pool mask extraction module 510, molten pool image feature extraction module 520, timing arrangement module 530, droplet impact trigger time determination module 540, analysis window determination module 550, molten pool image feature signal segment extraction module 560, and melt depth calculation module 570. When the computer program module is run, it can perform the corresponding function.

[0172] Figure 6 A block diagram of an electronic device suitable for implementing a visual inspection method for arc welding penetration depth according to an embodiment of the present invention is shown. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0173] like Figure 6As shown, an electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0174] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0175] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0176] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by processor 601, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0177] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the arc welding penetration visual inspection method according to embodiments of the present invention.

[0178] According to embodiments of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0179] For example, according to embodiments of the present invention, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0180] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of the present invention. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the arc welding penetration visual inspection method provided in the embodiments of the present invention.

[0181] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this embodiment of the invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0182] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0183] According to embodiments of the present invention, program code for executing the computer programs provided in the embodiments of the present invention can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0184] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of the present invention can be combined and / or combined in various ways, even if such combinations or combinations are not expressly stated in the present invention. In particular, the features described in the various embodiments and / or claims of this invention can be combined and / or combined in various ways without departing from the spirit and teachings of this invention. All such combinations and / or combinations fall within the scope of this invention.

[0185] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of the invention is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. A visual inspection method for arc welding penetration depth, characterized in that, The method includes: Based on the image sequence acquired for the arc-molten pool coupling region during the arc welding process, the molten pool mask of the molten pool region in each image is extracted frame by frame. The arc-molten pool coupling region includes the arc column region and the molten pool region. Based on the molten pool mask, molten pool image features of the molten pool region in each image are extracted, wherein the molten pool image features include the molten pool projection area, which is determined based on the total count of pixels with a pixel value of 1 in the molten pool mask; Arrange the molten pool image features of each image according to the acquisition time sequence of the image sequence to form a molten pool image feature time series, wherein the molten pool image feature time series includes the molten pool projected area time series. Based on the adaptive detection threshold, the abrupt change point of the area in the time series of the projected area of ​​the molten pool is detected, and the time corresponding to the abrupt change point is determined as the trigger time of the droplet impact. The analysis window is determined starting from the moment when the molten droplet impacts are triggered. Extract the molten pool image feature signal segment from the time series of the molten pool image features, corresponding to the time period of the analysis window; Based on the feature signal segments of the molten pool image, the penetration depth corresponding to the analysis window during the arc welding process is calculated.

2. The method according to claim 1, characterized in that, The step of extracting the molten pool mask from each image frame by frame, based on the image sequence acquired during the arc welding process targeting the coupled region of the molten pool, includes: Detect the dynamic boundary lines used to divide the arc column region and the molten pool region in each image frame by frame; Based on the dynamic boundary line, the original image of the molten pool region is obtained by segmenting each image. Gaussian filtering is applied to the original image of the molten pool region to obtain the low-frequency background component; The product of the low-frequency background component and a preset coefficient is subtracted from the original image of the molten pool region to obtain the enhanced image of the molten pool region. The enhanced image of the molten pool region is binarized according to the adaptive segmentation threshold to obtain a binarized image; The largest connected region in the binarized image is extracted and used as the molten pool mask.

3. The method according to claim 2, characterized in that, Each image includes at least an arc-shaped region at the top of the image and a molten pool region below the arc-shaped region; the frame-by-frame detection of the dynamic boundary line used to divide the arc-shaped region and the molten pool region in each image includes: Extract the vertical brightness profile along the central column direction of each image frame by frame; The second-order difference is calculated for the vertical brightness profile of each image to obtain the second-order difference curve corresponding to each image; In the upper half of each image, search for the maximum positive value point of the corresponding second-order difference curve; The dynamic boundary line is determined based on the row coordinates corresponding to the maximum positive value point.

4. The method according to claim 1, characterized in that, The molten pool image features also include at least one of the following: centroid vertical displacement, mean curvature of the upper edge, average brightness of the central region, and horizontal width; The vertical displacement of the centroid is determined based on the offset of the centroid row coordinates of the molten pool mask relative to the reference position, and characterizes the overall rising and falling trend of the molten pool surface. The mean curvature of the upper edge is determined based on the average value of the signed discrete curvature at each pixel point on the upper edge of the molten pool mask, which characterizes the overall curvature degree and curvature direction of the upper edge contour of the molten pool mask. The discrete curvature at the upward protrusion of the upper edge contour takes a positive value, and the discrete curvature at the downward concavity takes a negative value. The average brightness of the central region is determined based on the average gray level of each pixel within the inscribed elliptical region of the molten pool mask, representing the average level of image gray level within the central region of the molten pool mask. The horizontal width is determined based on the difference between the maximum and minimum column coordinates of the molten pool mask in the row of the centroid row coordinates, and represents the lateral span of the molten pool.

5. The method according to claim 1, characterized in that, The step of detecting abrupt changes in the projected area of ​​the molten pool in the time series based on an adaptive detection threshold, and determining the time corresponding to the abrupt change as the trigger time for droplet impact, includes: The first-order difference calculation is performed frame by frame on the time series of the projected area of ​​the molten pool to obtain the amplitude sequence of the rate of change of the projected area of ​​the molten pool, wherein the amplitude of the rate of change of the projected area of ​​the molten pool is the absolute value of the difference between the projected areas of the molten pool in two adjacent frames. Taking the current time as the end point, take a backward window with a length of a preset number of frames, and calculate the adaptive detection threshold based on the mean and standard deviation of the amplitude of the rate of change of the projected area of ​​the melt pool within the backward window, as well as the preset standard deviation coefficient. The moment corresponding to the amplitude of the rate of change of the projected area of ​​the molten pool that is greater than the adaptive detection threshold is determined as the candidate moment for triggering droplet impact. A minimum interval constraint is applied to the candidate times for triggering droplet impact, and the candidate times for triggering droplet impact that satisfy the minimum interval constraint are determined as the trigger times for triggering droplet impact.

6. The method according to claim 1 or 5, characterized in that, In response to the detection that the lowest non-zero row of the molten pool mask reaches the bottom boundary of the image, it is determined that the lower edge of the molten pool exceeds the image range. The time series of the molten pool projected area is replaced with the time series of the average height of the upper edge of the molten pool. The average height of the upper edge of the molten pool is determined based on the average value of the row coordinates of each pixel on the upper edge of the molten pool. After the replacement, the determination of the droplet impact trigger time is based on the time series of the average height of the upper edge of the molten pool, and the preset number of frames and the standard deviation coefficient in the adaptive detection threshold are re-determined according to the value range of the average height of the upper edge of the molten pool.

7. The method according to claim 1, characterized in that, The determination of the analysis window, starting from the moment of droplet impact, includes: The number of window frames for the analysis window is determined, wherein the number of window frames is greater than or equal to the number of frames corresponding to a preset number of melt pool oscillation cycles; for the first analysis window, the number of window frames is taken as a preset initial number of frames; for the remaining analysis windows, the number of window frames is determined based on the estimated value of the oscillation main frequency of the previous analysis window. In response to detecting that the number of frames between the trigger time of the molten droplet impact and the next adjacent trigger time of the molten droplet impact is greater than or equal to the sum of the preset number of transient exclusion frames and the number of window frames, starting from the trigger time of the molten droplet impact, after skipping the number of transient exclusion frames, the signal segment of the number of window frames is extracted as the analysis window; In response to the detection that the number of frames between the droplet impact trigger moment and the next adjacent droplet impact trigger moment is less than the sum of the number of transient exclusion frames and the number of window frames, starting from the droplet impact trigger moment, after skipping the number of transient exclusion frames, the signal segment of the number of window frames is intercepted across multiple adjacent droplet impact trigger moments as the analysis window. Alternatively, the effective signal segments corresponding to each of the consecutively preset superimposed droplet impact trigger moments after skipping the number of transient exclusion frames are aligned according to the droplet impact trigger moments corresponding to each effective signal segment, and then the average is calculated point by point to obtain the superimposed signal segment as the analysis window.

8. The method according to claim 4, characterized in that, The step of calculating the weld penetration depth corresponding to the analysis window during the arc welding process based on the feature signal segments of the molten pool image includes: Based on the characteristic signal segments of various types of molten pool images, calculate the frequency estimate and real-time logarithmic signal-to-noise ratio of the corresponding type of molten pool image features within the analysis window; The real-time logarithmic signal-to-noise ratio is converted into a real-time linear signal-to-noise ratio; Based on the real-time linear signal-to-noise ratio of multiple types of molten pool image features that meet a preset threshold, the normalized dynamic weights of each type of molten pool image feature are calculated, wherein the real-time logarithmic signal-to-noise ratio of the molten pool image features that meet the preset threshold is greater than or equal to the preset threshold. Based on the normalized dynamic weights of various molten pool image features, the frequency estimates of multiple molten pool image features that meet the preset threshold are weighted and fused to obtain the oscillation main frequency estimate of the analysis window. The penetration depth of the analysis window is calculated based on the estimated oscillation frequency and the calibration coefficients related to the welding materials and process parameters.

9. The method according to claim 8, characterized in that, The image sequence corresponds to multiple analysis windows; calculating the weld depth of each analysis window based on the estimated oscillation frequency and calibration coefficients related to welding materials and process parameters includes: Obtain multiple melt depth estimates that correspond one-to-one with the multiple analysis windows; Based on the smoothing weights of the second preset number of consecutive analysis windows centered on the current analysis window, the melt depth estimates of the second preset number of consecutive analysis windows are weighted and averaged to obtain the smoothed melt depth estimate of the current analysis window. The smoothing weights are obtained by weighting and summing the real-time linear signal-to-noise ratios of the multiple types of melt pool image features that meet preset threshold values ​​in the corresponding analysis windows. The smoothed melt depth estimate is determined as the melt depth of the current analysis window.

10. The method according to claim 1, characterized in that, The image sequence was acquired using the natural arc light of the welding arc as the light source and a high-speed camera with a frame rate of 5000 frames per second or higher.

Images