Real-time welding quality control method, control device and welding system

Abstract

The application provides a real-time welding quality control method, a control device and a welding system. The method comprises the following steps: acquiring an image of a welding area of real-time welding to obtain a welding image; extracting welding features from the welding image, wherein the welding features comprise a welding fusion line included angle, a welding torch welding wire included angle, a welding wire dry elongation, a picture offset angle and a welding wire swing center coincidence degree; and issuing an alarm information in the case that the welding features are not within the corresponding predetermined range, wherein the alarm information is used to remind possible welding defects, so as to assist a welder in adjusting welding process parameters in real time, improve a welding environment of the welder, reduce the generation of welding defects, improve welding quality, and solve the problem that the welding quality is guaranteed by relying on manual experience in the prior art without manual welding or close observation of welding by a welding machine.

Description

Technical Field

[0001] This invention relates to the field of welding technology, and more specifically, to a method, control device, computer-readable storage medium, and welding system for real-time welding quality control. Background Technology

[0002] Existing real-time quality control of outdoor structural component welding is mostly based on manual welding or close observation of welding through welding machines. Welders rely on their own experience to judge the dwell time of the welding wire, the size of the oscillation amplitude, and the degree of coincidence between the center line of the oscillation position and the center line of the bevel by the size of the weld bevel, the gap between the welding plates, and the melting state of the molten pool and weld bead, thereby controlling the generation of welding defects in real time.

[0003] Existing real-time welding quality control methods mostly rely on welders judging information during the welding process through goggles. They adjust welding parameters and the relative position of the welding wire and weld bead by visual observation. The quality of the weld formation depends on the welder's skill level. Moreover, long welding times not only affect the welder's health, but also reduce the accuracy of welding defect identification as the welder's condition deteriorates. In addition, while existing human-computer interaction screens can display images of the molten pool, the images are black and white, and the outlines of the welding wire and molten pool are not obvious, making it difficult to observe welding defects, especially in outdoor operations and under strong light. Therefore, it is of great significance to find a method that can replace close-range observation by personnel and provide early warning of areas where welding defects may occur. Summary of the Invention

[0004] The main objective of this application is to provide a method, control device, computer-readable storage medium, and welding system for real-time welding quality control, so as to at least solve the problem of relying on human experience to ensure welding quality in the prior art.

[0005] To achieve the above objectives, according to one aspect of this application, a method for controlling real-time welding quality is provided, comprising: acquiring an image of the welding area of ​​a real-time welding area to obtain a welding image; extracting welding features from the welding image, the welding features including a weld fusion line angle, a welding torch and welding wire angle, welding wire extension, an image offset angle, and a welding wire oscillation center coincidence, wherein the weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding torch and welding wire angle is the angle between the welding torch and the welding wire, the image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the welding wire oscillation center coincidence is the coincidence of the center line of the welding wire in the welding image with the center line of the corresponding position of the welding path; and issuing an alarm message when the welding features are not within the corresponding predetermined range, the alarm message being used to remind of possible welding defects.

[0006] Optionally, the weld fusion line angle includes a first weld fusion line angle and a second weld fusion line angle. Extracting welding features from the welding image includes: extracting the molten pool contour line and two first edge lines from the welding image, wherein the molten pool contour line is the edge contour line of the molten pool formed by welding, and the two first edge lines are respectively located at both ends of the bevel; detecting the angle between the molten pool contour line and one of the first edge lines to obtain the first weld fusion line angle; and detecting the angle between the molten pool contour line and the other first edge line to obtain the second weld fusion line angle.

[0007] Optionally, the alarm information includes a first alarm information and a second alarm information. When the welding feature is not within the corresponding predetermined range, an alarm information is issued, including: issuing a first alarm information when the angle between the first weld fusion line and / or the angle between the second weld fusion line is less than or equal to a first angle threshold, wherein the first alarm information indicates a possible defect of excessive weld height; issuing a second alarm information when the angle between the first weld fusion line and / or the angle between the second weld fusion line is greater than or equal to a second angle threshold, wherein the second alarm information indicates a possible defect of insufficient weld material filling, wherein the second angle threshold is greater than the first angle threshold; and not issuing the first alarm information and the second alarm information when both the angle between the first weld fusion line and the angle between the second weld fusion line are greater than the first angle threshold and less than the second angle threshold.

[0008] Optionally, extracting welding features from the welding image further includes: extracting the outlines of the welding torch and the welding wire from the welding image, and determining the welding torch-welding wire angle based on the outlines of the welding torch and the welding wire; and identifying the wire extension length based on the welding image.

[0009] Optionally, the alarm information includes a third alarm information and a fourth alarm information. When the welding feature is outside the corresponding predetermined range, an alarm information is issued. The alarm information further includes: calculating the absolute value of the difference between the welding torch wire angle and the predetermined angle to obtain a deviation angle; and issuing the third alarm information when the deviation angle is greater than a deviation angle threshold. The third alarm information indicates a welding defect that may be caused by a deviation in the welding torch wire angle. The fourth alarm information includes: calculating the absolute value of the difference between the welding wire extension and the predetermined welding wire extension to obtain a deviation length; and issuing the fourth alarm information when the deviation angle is greater than a deviation angle threshold. The fourth alarm information indicates a welding defect that may be caused by an unreasonable amount of welding wire melting.

[0010] Optionally, extracting welding features from the welding image further includes: identifying two second edge lines from the welding image, the two second edge lines being located at opposite ends of the bevel; detecting the angles between the two second edge lines and the vertical direction of the welding image to obtain two reference angles; calculating the absolute values ​​of the differences between the two reference angles and corresponding predetermined reference angles to obtain two image offset angles, the predetermined reference angles being the reference angles corresponding to the welding image taken before welding.

[0011] Optionally, the alarm information includes a fifth alarm information, which is issued when the welding feature is not within the corresponding predetermined range. It also includes issuing a fifth alarm information when the average of the two image offset angles is greater than the image offset angle threshold. The fifth alarm information is to remind that the shooting angle of the welding image has shifted.

[0012] Optionally, extracting welding features from the welding image further includes: identifying the centerline of the welding wire from the welding image and calculating the distance between the centerline of the welding wire in the welding image and the centerline of the corresponding position of the welding path to obtain the overlap of the welding wire swing center; the alarm information includes a sixth alarm information, which issues an alarm information when the welding feature is not within the corresponding predetermined range, and further includes: issuing a sixth alarm information when the overlap of the welding wire swing center is greater than a distance threshold, wherein the sixth alarm information is to remind the welding wire to deviate from the welding path.

[0013] Optionally, after acquiring an image of the welding area in real-time welding and obtaining the welding image, the method further includes: performing three-dimensional reconstruction based on the welding image to obtain the weld width and weld depth of the molten pool; inputting the weld width, the weld depth, and the corresponding welding process parameters into a detection model to obtain a detection result, wherein the detection result represents the risk of welding defects and the risk of no welding defects, the welding process parameters include at least the welding voltage, and the detection model determines whether welding defects occur under historical weld width, historical weld depth, corresponding historical welding process parameters, and corresponding working conditions; and issuing a seventh alarm message based on the detection result, wherein the seventh alarm message is used to prompt adjustment of the welding process parameters.

[0014] According to another aspect of this application, a real-time welding quality control device is provided, comprising: an acquisition unit for acquiring an image of the welding area in real-time welding to obtain a welding image; a processing unit for extracting welding features from the welding image, the welding features including a weld fusion line angle, a welding torch and welding wire angle, welding wire extension, an image offset angle, and a welding wire oscillation center coincidence, wherein the weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding torch and welding wire angle is the angle between the welding torch and the welding wire, the image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the welding wire oscillation center coincidence is the coincidence of the center line of the welding wire in the welding image with the center line of the corresponding position of the welding path; and a first sending unit for issuing an alarm message when the welding features are not within the corresponding predetermined range, the alarm message being used to remind of possible welding defects.

[0015] According to another aspect of this application, a display terminal is provided, characterized in that the display terminal includes a real-time welding quality control device, the control device further including: a display unit, used at least for real-time display of the weld fusion line angle, the welding torch wire angle, the wire extension length, the screen offset angle, the overlap of the wire swing center, and alarm information.

[0016] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0017] According to another aspect of this application, a welding system is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any one of the methods described.

[0018] Applying the technical solution of this application, in the above-mentioned real-time welding quality control method, firstly, an image of the welding area of ​​the real-time welding is acquired to obtain a welding image; then, welding features are extracted based on the welding image, including the weld fusion line angle, the welding torch and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire oscillation center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding torch and welding wire angle is the angle between the welding torch and the welding wire, the image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the overlap of the welding wire oscillation center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image; subsequently, if the above welding features are not within the corresponding predetermined range, an alarm message is issued to remind that a welding defect may occur. This method acquires images of the welding area in real time to obtain welding images, and then extracts welding features from the welding images to determine whether welding defects may occur. If so, an alarm is issued, thereby assisting the welder in adjusting the welding process parameters in real time, improving the welder's working environment, reducing the occurrence of welding defects, and improving welding quality. It eliminates the need for manual welding or close observation of the welding through the welding machine, solving the problem of relying on human experience to ensure welding quality in existing technologies. Attached Figure Description

[0019] Figure 1 A hardware structure block diagram of a mobile terminal for performing a real-time welding quality control method according to an embodiment of this application is shown.

[0020] Figure 2 A flowchart illustrating a real-time welding quality control method according to an embodiment of this application is shown.

[0021] Figure 3 A schematic diagram of a bottom curve of a molten pool profile provided according to an embodiment of this application is shown;

[0022] Figure 4 A flowchart illustrating a tracking algorithm provided according to an embodiment of this application is shown;

[0023] Figure 5 A schematic diagram of a welding wire welding gun angle provided according to an embodiment of this application is shown;

[0024] Figure 6 A schematic diagram of a reference angle provided according to an embodiment of this application is shown;

[0025] Figure 7 A schematic diagram of a welding wire oscillation trajectory provided according to an embodiment of this application is shown;

[0026] Figure 8 A structural block diagram of a real-time welding quality control device provided according to an embodiment of this application is shown.

[0027] The above figures include the following reference numerals:

[0028] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0029] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] As described in the background section, existing technologies rely on human experience to ensure welding quality. To address this issue, embodiments of this application provide a method, control device, computer-readable storage medium, and welding system for real-time welding quality control.

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0034] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a real-time welding quality control method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the device information display method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] This embodiment provides a method for controlling the real-time welding quality of a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] Figure 2 This is a flowchart of a real-time welding quality control method according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0038] Step S201: Obtain an image of the welding area in real-time welding to obtain a welding image;

[0039] Specifically, a crawler tracks the welding area to acquire welding images in real time, and the molten pool camera on the crawler sends the captured welding images to the human-computer interaction screen for display.

[0040] Step S202: Extract welding features from the welding image. The welding features include the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire swing center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding gun and welding wire angle is the angle between the welding gun and the welding wire. The image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The overlap of the welding wire swing center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image.

[0041] Specifically, multiple welding features are extracted from the above welding images. Each welding feature corresponds to one or more welding defects, and it is possible to determine whether welding defects may occur based on these welding features.

[0042] Step S203: If the above-mentioned welding features are not within the corresponding predetermined range, an alarm message is issued. The alarm message is used to remind that a welding defect may occur.

[0043] Specifically, by extracting features, it can determine whether welding defects are occurring in the current welding process, whether defects are about to occur, and the type of defects that are occurring. Through the communication harness settings between the remote control and the host computer, it can provide real-time warnings and adjust welding process parameters to reduce the occurrence of welding defects and improve weld quality.

[0044] In the above-mentioned real-time welding quality control method, firstly, an image of the welding area of ​​the real-time welding is acquired to obtain a welding image; then, welding features are extracted from the welding image, including the weld fusion line angle, the welding torch and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire oscillation center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding torch and welding wire angle is the angle between the welding torch and the welding wire, the image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the overlap of the welding wire oscillation center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image; then, if the above welding features are not within the corresponding predetermined range, an alarm message is issued to remind that a welding defect may occur. This method acquires images of the welding area in real time to obtain welding images, and then extracts welding features from the welding images to determine whether welding defects may occur. If so, an alarm is issued, thereby assisting the welder in adjusting the welding process parameters in real time, improving the welder's working environment, reducing the occurrence of welding defects, and improving welding quality. It eliminates the need for manual welding or close observation of the welding through the welding machine, solving the problem of relying on human experience to ensure welding quality in existing technologies.

[0045] To obtain welding features that facilitate defect identification, in one optional embodiment, the weld fusion line angle includes a first weld fusion line angle and a second weld fusion line angle, and step S202 includes:

[0046] Step S2021: Extract the molten pool outline and two first edge lines from the above welding image. The molten pool outline is the edge outline of the molten pool formed by welding, and the two first edge lines are located at both ends of the above bevel.

[0047] Step S2022: Detect the angle between the molten pool outline and one of the first edge lines to obtain the first weld fusion line angle; detect the angle between the molten pool outline and another of the first edge lines to obtain the second weld fusion line angle.

[0048] Specifically, based on the above image denoising and image enhancement, Hough linear transform is used to detect the edge points on both sides of the bevel, identify outliers, and perform linear fitting on the discrete points to obtain the first welding fusion line angle and the second welding fusion line angle.

[0049] To ensure the accuracy of the early warning, in one optional implementation, the alarm information includes first alarm information and second alarm information, and step S203 includes:

[0050] Step S2031: If the included angle of the first weld fusion line and / or the included angle of the second weld fusion line are less than or equal to the first included angle threshold, a first alarm message is issued, wherein the first alarm message indicates that a defect of excessive excess height may occur.

[0051] Step S2032: If the angle between the first weld fusion line and / or the second weld fusion line is greater than or equal to the second angle threshold, a second alarm message is issued. The second alarm message indicates that a defect of insufficient welding material filling may occur. The second angle threshold is greater than the first angle threshold.

[0052] Step S2033: If the included angle of the first weld fusion line and the included angle of the second weld fusion line are both greater than the first included angle threshold and less than the second included angle threshold, then the first alarm message and the second alarm message shall not be issued.

[0053] Specifically, the characteristic parameters of the weld pool profile and the characteristic parameters of the groove weld bead are extracted, and mathematical and geometric analysis is performed, such as... Figure 3 As shown, a "flat" or "concave" bottom curve of the molten pool profile is preferable. α and β are the angles of the first and second weld fusion lines, respectively. Specifically, thresholds θ1 and θ2 for the first and second angles are set to determine the boundaries of welding defects. When θ2 > θ1, and α, β < θ1, a first alarm is issued, indicating excessive weld reinforcement and a groove defect between the weld and the unmelted area of ​​the base material. When θ1 < α, β < θ2, the welding is normal, with the weld reinforcement controlled between 1mm and 2mm. When π > α, β > θ2, a second alarm is issued, indicating excessive dwell time on both sides and excessively fast oscillation speed in the middle, resulting in insufficient filling of the weld material in the middle. Furthermore, a mathematical model F1(α, β) can be established to evaluate the types of defects that may arise from the real-time angles of the two features.

[0054] To obtain welding features that facilitate defect identification, in an optional embodiment, step S202 further includes:

[0055] Step S2023: Extract the outlines of the welding gun and the welding wire from the above welding image, and determine the welding gun-welding wire angle based on the outlines of the welding gun and the welding wire.

[0056] Step S2024: Identify the dry extension of the welding wire based on the above welding image.

[0057] Specifically, the angle between the welding torch and the welding wire can determine whether the welding torch is affected by external forces (such as cable jamming causing torch angle deviation) during the welding process, resulting in an angle that affects welding quality. The wire extension affects the welding voltage and arc stiffness during the welding process. Identifying the difference between the actual and preset wire extension during the welding process allows for adjustment of the welding voltage, reducing welding defects. The angle between the welding torch and the welding wire is also evaluated. Based on target recognition and target tracking algorithms, the position of the welding torch and welding wire, as well as the wire extension length, are extracted in real time, improving tracking accuracy and robustness compared to traditional tracking algorithms. Figure 4 As shown, the algorithm steps are as follows: Step 1 Initialization: Initialize the recognition algorithm and the tracking algorithm; Step 2 Algorithm Tracking: Use the tracking algorithm to track the target, and use the indicator value to determine whether there is a deviation or loss of the target; 1) Deviation: If value ≥ β, where β is the deviation threshold, then the target tracking is considered to have deviated. After determining that the target has deviated, the target bounding box of the recognition algorithm is assigned to the tracking algorithm to re-track. 2) No Deviation: If value ≥ β, then the target is considered to be tracked, and the tracking algorithm continues to track. 3) Lost Target: If value = 0, the tracking algorithm has lost the target. The tracking algorithm is re-initialized using the recognition algorithm to achieve target tracking.

[0058] To ensure the accuracy of the warning, in an optional implementation, the alarm information includes a third alarm information and a fourth alarm information, and step S203 further includes:

[0059] Step S2034: Calculate the absolute value of the difference between the welding gun wire angle and the predetermined angle to obtain the deviation angle. If the deviation angle is greater than the deviation angle threshold, issue the third alarm information. The third alarm information indicates a welding defect that may be caused by the welding gun wire angle deviation.

[0060] Step S2035: Calculate the absolute value of the difference between the above-mentioned wire extension and the predetermined wire extension to obtain the deviation length, and issue the above-mentioned fourth alarm information when the above-mentioned offset angle is greater than the offset angle threshold. The above-mentioned fourth alarm information is a welding defect that may be caused by unreasonable wire melting amount.

[0061] Specifically, the angle between the welding torch and the welding wire is approximately 90° during normal operation. When an external force is applied to the welding torch, the angle between the torch and the welding wire will shift to a certain extent, such as... Figure 5As shown, θ = |α-90°|. When the offset angle θ > δ (the set threshold), the above third alarm message is issued. The angle between the welding torch and the welding wire has affected the welding quality. It is necessary to stop the machine to check the cause of the angle offset and correct the angle of the welding torch and the welding wire. In addition, the wire extension needs to be identified in real time. The wire extension affects the pass rate of the weld quality. The length of the wire extension is mainly affected by factors such as welding conditions, wire type, and welding type. During the welding process, if the wire extension is too long or too short, welding defects will occur. If the wire extension is too long, it is easy to thread the wire during the root pass stage. During the filling and cover passes stages, the amount of wire melted will be too much, changing the reciprocal of the weld layer. If the welding wire extension is too short, insufficient melting will result in welding defects such as incomplete filling. By real-time identification of the welding wire extension during the welding process, it is possible to determine whether the change in the current welding wire extension is normal, and timely adjustments are made to the wire feeding speed of the wire feeder. The judgment standard is as follows: △l=|ll*|. When △l>k, the above-mentioned fourth alarm information is issued, and it is necessary to adjust the wire feeding speed of the wire feeder or the welding current and voltage parameters to ensure that the change in △l is controlled within the value of k. For example, the contour and extension of the welding wire during the welding process can be identified by a deep learning YOLO network, and the end information of the welding wire can be captured by drawing line segments. During the dynamic swing, the sweeping trajectory is drawn to realize the dynamic scanning of the welding wire back and forth.

[0062] To obtain welding features that facilitate defect identification, in an optional embodiment, step S202 further includes:

[0063] Step S2025: Identify two second edge lines based on the above welding image, wherein the two second edge lines are located at both ends of the bevel.

[0064] Step S2026: Detect the angles between the two second edge lines and the vertical direction of the welding image to obtain the two reference angles.

[0065] Step S2027: Calculate the absolute value of the difference between the two reference angles and the corresponding predetermined reference angle to obtain the two image offset angles. The predetermined reference angle is the reference angle corresponding to the welding image taken before welding.

[0066] Specifically, before the capping stage, the angle between the outer weld bead and the vertical line of the observed image can be used to determine whether the crawler has deviated during the welding process, allowing for timely correction of the vehicle body and ensuring welding quality. During the root pass stage, the inner weld bead can be used to determine if there is a step change in the weld bottom gap, recording the area of ​​change. By using the lag distance, corresponding welding parameters are matched when the crawler reaches the designated position, improving the level of intelligent welding. Based on the preprocessing of the molten pool image, the contrast between the inner and outer sides of the outer weld bead area is significant. Furthermore, the outer weld bead remains unchanged before the capping stage, while the inner weld bead changes considerably during the filling and capping stages. Therefore, the outer weld bead information is selected for crawler deviation identification. Before completing the capping weld, at least one side of the outer weld bead feature information can be guaranteed. Therefore, the angle between the slope of the outer weld bead and the vertical line segment of the camera image can be used to determine whether the crawler has deviated during the welding process, and a certain threshold θ is set.

[0067] To ensure the accuracy of the early warning, in an optional implementation, the alarm information includes a fifth alarm information, and step S203 further includes:

[0068] Step S2036: If the average of the two aforementioned image offset angles is greater than the image offset angle threshold, a fifth alarm message is issued. The fifth alarm message is to remind that the shooting angle of the aforementioned welding image has shifted.

[0069] Specifically, before the covering stage, such as Figure 6 As shown, let α1 and α2 be the angles between the weld bead outside the groove and the vertical line of the screen when the molten pool observation screen is aligned before the capping stage, i.e., the predetermined reference angles. The real-time calculated reference angles on both sides are α1* and α2*, then we have When Δα > θ, an early warning message is generated. At this time, during the welding process, the crawler will offset the cover layer during the first weld pass, covering one side of the weld bead outside the groove. Therefore, the angle calculation at this time only uses the change in angle on one side, and the offset evaluation standard is the same as above. In addition, since there are nail points and groove bottom gap step points during the root pass stage of welding, the welding process can be warned by identifying the gap change at the bottom of the groove, and the current position of the change is recorded. By using the hysteresis distance, the parameters are adjusted after the crawler reaches the designated position to ensure the predictability of the welding process and improve the welding quality. Since the ceramic deposit in the root pass stage has reflective properties and appears as a continuous area in the molten pool image, the image is binarized, and the backing contour is extracted using the region growing method, as shown below. The degree of change in the bottom gap is calculated and displayed in real time. When the crawler crawls to the designated position through the hysteresis distance, the corresponding current, voltage, and process parameters are matched to complete the real-time early warning of the welding process.

[0070] To ensure the accuracy of the early warning, in an optional implementation, step S202 further includes:

[0071] Step S2028: Identify the center line of the welding wire based on the above welding image, and calculate the distance between the center line of the welding wire in the above welding image and the center line of the corresponding position of the welding path to obtain the overlap of the welding wire swing center.

[0072] The above alarm information includes the sixth alarm information, and step S203 also includes:

[0073] Step S2037: If the overlap of the welding wire swing center is greater than the distance threshold, a sixth alarm message is issued. The sixth alarm message is to remind the welding wire that it has deviated from the welding path.

[0074] Specifically, the degree of overlap between the center line of the welding wire oscillation trajectory and the laser line extracted from the laser inflection point determines whether the welding wire oscillation is following the preset trajectory. The degree of overlap between the coordinates of the welding wire's dwell position and the coordinates of the inflection point determines the weld formation, ensuring welding quality. Information from the previous layer's weld pool image scan guides the start of the next weld and the correction trajectory of the crawler during the welding process, adjusting the welding wire oscillation amplitude and the degree of overlap between the oscillation center and the current weld bead, improving the automated welding capability of the pre-scan. Figure 7 As shown, each inflection point of each layer and pass acquired by the laser module is extracted, i.e., feature points. The center line of the inflection point of the current weld pass, i.e., the center line of the bottom feature point, is calculated. The oscillation trajectory of the welding wire is generated by recognizing the welding wire outline in the molten pool image, and the center line of the welding wire oscillation amplitude is calculated. The information features acquired by the molten pool camera are rich, and the weld formation of the previous layer can be clearly observed in the current welding state. Welders can observe the welding situation in real time through the visualized molten pool image, assist in the adjustment of process parameters, and improve the weld quality.

[0075] To ensure the accuracy of the early warning, in one optional implementation, after acquiring an image of the welding area in real-time welding and obtaining the welding image, the above method further includes:

[0076] Step S301: Perform three-dimensional reconstruction based on the above welding image to obtain the weld width and weld depth of the weld pool;

[0077] Step S302: Input the above-mentioned weld width, above-mentioned weld depth and corresponding welding process parameters into the detection model to obtain the detection results. The above-mentioned detection results are the risk of having welding defects and the risk of not having welding defects. The above-mentioned welding process parameters include at least the welding voltage. The above-mentioned detection model is based on whether welding defects occur under the historical weld width, historical weld depth, corresponding historical welding process parameters and corresponding working conditions.

[0078] Step S303: Based on the above detection results, issue a seventh alarm message. The seventh alarm message is used to prompt adjustment of the above welding process parameters.

[0079] Specifically, the molten pool area is reconstructed in three dimensions using the installation position parameters and calibration parameters of the molten pool camera, and the molten pool depth is reproduced in real time. Based on the molten depth and width, the welding voltage and process parameters during the welding process are combined with the weld plate thickness and groove size to determine whether welding defects are occurring in the current welding state. This allows for real-time adjustment of welding parameters to reduce welding defects and improve weld quality. By extracting the molten pool contour and reconstructing it in three dimensions, parameters such as the molten pool width and depth during real-time welding are obtained. Correlation analysis is performed in conjunction with parameters such as welding current and voltage to obtain a correlation model and perform defect classification and prediction. Based on the early warning information, the adjustment of welding current and voltage parameters and process parameters is guided.

[0080] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0081] This application also provides a real-time welding quality control device. It should be noted that this device can be used to execute the real-time welding quality control method provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0082] The following describes the real-time welding quality control device provided in the embodiments of this application.

[0083] Figure 8 This is a structural block diagram of a real-time welding quality control device according to an embodiment of this application. Figure 8 As shown, the device includes:

[0084] The acquisition unit 10 is used to acquire an image of the welding area in real-time welding to obtain a welding image;

[0085] Specifically, a crawler tracks the welding area to acquire welding images in real time, and the molten pool camera on the crawler sends the captured welding images to the human-computer interaction screen for display.

[0086] The processing unit 20 is used to extract welding features based on the welding image. The welding features include the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the welding wire oscillation center coincidence. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding gun and welding wire angle is the angle between the welding gun and the welding wire. The image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The welding wire oscillation center coincidence is the coincidence of the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image.

[0087] Specifically, multiple welding features are extracted from the above welding images. Each welding feature corresponds to one or more welding defects, and it is possible to determine whether welding defects may occur based on these welding features.

[0088] The first sending unit 30 is used to issue an alarm message when the welding feature is not within the corresponding predetermined range. The alarm message is used to remind that a welding defect may occur.

[0089] Specifically, by extracting features, it can determine whether welding defects are occurring in the current welding process, whether defects are about to occur, and the type of defects that are occurring. Through the communication harness settings between the remote control and the host computer, it can provide real-time warnings and adjust welding process parameters to reduce the occurrence of welding defects and improve weld quality.

[0090] In the aforementioned real-time welding quality control device, the acquisition unit acquires an image of the welding area in real-time welding to obtain a welding image; the processing unit extracts welding features based on the welding image, including the weld fusion line angle, the welding torch and wire angle, the wire extension, the image offset angle, and the overlap of the wire oscillation center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding torch and wire angle is the angle between the welding torch and the wire, the image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the overlap of the wire oscillation center is the overlap between the center line of the wire and the center line of the corresponding position of the welding path in the welding image; the first sending unit issues an alarm message when the welding features are not within the corresponding predetermined range, the alarm message being used to remind of possible welding defects. This device acquires images of the welding area in real time to obtain welding images, and then extracts welding features from the welding images to determine whether welding defects may occur. If so, it issues an alarm message, thereby assisting welders in adjusting welding process parameters in real time, improving the working environment of welders, reducing the occurrence of welding defects, and improving welding quality. It eliminates the need for manual welding or close observation of welding through welding machines, solving the problem of relying on human experience to ensure welding quality in existing technologies.

[0091] To obtain welding features that facilitate defect identification, in one optional embodiment, the weld fusion line angle includes a first weld fusion line angle and a second weld fusion line angle, and the processing unit includes:

[0092] The first processing module is used to extract the molten pool outline and two first edge lines from the above welding image. The molten pool outline is the edge outline of the molten pool formed by welding, and the two first edge lines are located at the two ends of the above bevel.

[0093] The second processing module is used to detect the angle between the molten pool outline and one of the first edge lines to obtain the angle between the first weld fusion line and the angle between the molten pool outline and another of the first edge lines to obtain the angle between the second weld fusion line.

[0094] Specifically, based on the above image denoising and image enhancement, Hough linear transform is used to detect the edge points on both sides of the bevel, identify outliers, and perform linear fitting on the discrete points to obtain the first welding fusion line angle and the second welding fusion line angle.

[0095] To ensure the accuracy of the early warning, in one optional implementation, the alarm information includes first alarm information and second alarm information, and the first sending unit includes:

[0096] The first sending module is used to send a first alarm message when the included angle of the first weld fusion line and / or the included angle of the second weld fusion line is less than or equal to the first included angle threshold. The first alarm message indicates that a defect of excessive excess height may occur.

[0097] The second sending module is used to issue a second alarm message when the included angle of the first weld fusion line and / or the included angle of the second weld fusion line is greater than or equal to the second included angle threshold. The second alarm message indicates that a defect of insufficient filling of welding material may occur. The second included angle threshold is greater than the first included angle threshold.

[0098] The third sending module is used to not send the first alarm information and the second alarm information when both the included angle of the first weld fusion line and the included angle of the second weld fusion line are greater than the first included angle threshold and less than the second included angle threshold.

[0099] Specifically, the characteristic parameters of the weld pool profile and the characteristic parameters of the groove weld bead are extracted, and mathematical and geometric analysis is performed, such as... Figure 3 As shown, a "flat" or "concave" bottom curve of the molten pool profile is preferable. α and β are the angles of the first and second weld fusion lines, respectively. Specifically, thresholds θ1 and θ2 for the first and second angles are set to determine the boundaries of welding defects. When θ2 > θ1, and α, β < θ1, a first alarm is issued, indicating excessive weld reinforcement and a groove defect between the weld and the unmelted area of ​​the base material. When θ1 < α, β < θ2, the welding is normal, with the weld reinforcement controlled between 1mm and 2mm. When π > α, β > θ2, a second alarm is issued, indicating excessive dwell time on both sides and excessively fast oscillation speed in the middle, resulting in insufficient filling of the weld material in the middle. Furthermore, a mathematical model F1(α, β) can be established to evaluate the types of defects that may arise from the real-time angles of the two features.

[0100] To obtain welding features that facilitate defect identification, in one optional embodiment, the above-mentioned processing unit further includes:

[0101] The third processing module is used to extract the outlines of the welding gun and the welding wire from the above welding image, and to determine the welding gun-welding wire angle based on the outlines of the welding gun and the welding wire.

[0102] The fourth processing module is used to identify the dry extension of the welding wire based on the above welding image.

[0103] Specifically, the angle between the welding torch and the welding wire can determine whether the welding torch is affected by external forces (such as cable jamming causing torch angle deviation) during the welding process, resulting in an angle that affects welding quality. The wire extension affects the welding voltage and arc stiffness during the welding process. Identifying the difference between the actual and preset wire extension during the welding process allows for adjustment of the welding voltage, reducing welding defects. The angle between the welding torch and the welding wire is also evaluated. Based on target recognition and target tracking algorithms, the position of the welding torch and welding wire, as well as the wire extension length, are extracted in real time, improving tracking accuracy and robustness compared to traditional tracking algorithms. Figure 4 As shown, the algorithm steps are as follows: Step 1 Initialization: Initialize the recognition algorithm and the tracking algorithm; Step 2 Algorithm Tracking: Use the tracking algorithm to track the target and use the indicator value to determine whether there is a deviation or loss of the target; 1) Deviation: If value ≥ β, where β is the deviation threshold, then the target tracking is considered to have deviated. After determining that the target has deviated, the target bounding box of the recognition algorithm is assigned to the tracking algorithm to re-track. 2) No Deviation: If value ≥ β, then the target is considered to be tracked, and the tracking algorithm continues to track. 3) Loss of Target: If value = 0, the tracking algorithm loses the target. The target tracking is achieved by re-initializing the tracking algorithm through the recognition algorithm. For example, the contour and extension of the welding wire during the welding process are identified by the deep learning YOLO network, and the end information of the welding wire is captured by drawing line segments. The sweeping trajectory is drawn during the dynamic swing, realizing the dynamic scanning of the welding wire back and forth.

[0104] To ensure the accuracy of the early warning, in one optional implementation, the alarm information includes a third alarm information and a fourth alarm information, and the first sending unit further includes:

[0105] The fourth sending module is used to calculate the absolute value of the difference between the welding gun wire angle and the predetermined angle to obtain the deviation angle, and to issue the third alarm information when the deviation angle is greater than the deviation angle threshold. The third alarm information is a welding defect that may be caused by the deviation of the welding gun wire angle.

[0106] The fifth sending module is used to calculate the absolute value of the difference between the above-mentioned wire extension and the predetermined wire extension to obtain the deviation length, and to issue the above-mentioned fourth alarm information when the above-mentioned offset angle is greater than the offset angle threshold. The above-mentioned fourth alarm information is a welding defect that may be caused by unreasonable wire melting amount.

[0107] Specifically, the angle between the welding torch and the welding wire is approximately 90° during normal operation. When an external force is applied to the welding torch, the angle between the torch and the welding wire will shift to a certain extent, such as... Figure 5As shown, θ = |α-90°|. When the offset angle θ > δ (the set threshold), the above third alarm message is issued. The angle between the welding torch and the welding wire has affected the welding quality. It is necessary to stop the machine to check the cause of the angle offset and correct the angle of the welding torch and the welding wire. In addition, the wire extension needs to be identified in real time. The wire extension affects the pass rate of the weld quality. The length of the wire extension is mainly affected by factors such as welding conditions, wire type, and welding type. During the welding process, if the wire extension is too long or too short, welding defects will occur. If the wire extension is too long, it is easy to thread the wire during the root pass stage. During the filling and cover passes stages, the amount of wire melted will be too much, changing the reciprocal of the weld layer. If the welding wire extension is too short, insufficient melting will result in welding defects such as incomplete filling. By real-time identification of the welding wire extension during the welding process, it is possible to determine whether the change in the current welding wire extension is normal, and timely adjustment of the wire feeder speed is made. The judgment standard is as follows: △l=|ll*|. When △l>k, the above-mentioned fourth alarm information is issued, and it is necessary to adjust the wire feeder speed or welding current and voltage parameters to ensure that the change in △l is controlled within the value of k.

[0108] To obtain welding features that facilitate defect identification, in one optional embodiment, the above-mentioned processing unit further includes:

[0109] The fifth processing module is used to identify two second edge lines based on the above welding image, wherein the two second edge lines are respectively located at both ends of the bevel;

[0110] The sixth processing module is used to detect the angles between the two second edge lines and the vertical direction of the welding image to obtain the two reference angles.

[0111] The seventh processing module is used to calculate the absolute value of the difference between the two aforementioned reference angles and the corresponding predetermined reference angle to obtain the two aforementioned image offset angles. The aforementioned predetermined reference angle is the reference angle corresponding to the aforementioned welding image taken before welding.

[0112] Specifically, before the capping stage, the angle between the outer weld bead and the vertical line of the observed image can be used to determine whether the crawler has deviated during the welding process, allowing for timely correction of the vehicle body and ensuring welding quality. During the root pass stage, the inner weld bead can be used to determine if there is a step change in the weld bottom gap, recording the area of ​​change. By using the lag distance, corresponding welding parameters are matched when the crawler reaches the designated position, improving the level of intelligent welding. Based on the preprocessing of the molten pool image, the contrast between the inner and outer sides of the outer weld bead area is significant. Furthermore, the outer weld bead remains unchanged before the capping stage, while the inner weld bead changes considerably during the filling and capping stages. Therefore, the outer weld bead information is selected for crawler deviation identification. Before completing the capping weld, at least one side of the outer weld bead feature information can be guaranteed. Therefore, the angle between the slope of the outer weld bead and the vertical line segment of the camera image can be used to determine whether the crawler has deviated during the welding process, and a certain threshold θ is set.

[0113] To ensure the accuracy of the early warning, in one optional implementation, the alarm information includes a fifth alarm information, and the first sending unit further includes:

[0114] The sixth sending module is used to issue a fifth alarm message when the average of the two aforementioned image offset angles is greater than the image offset angle threshold. The fifth alarm message is to remind that the shooting angle of the aforementioned welding image has shifted.

[0115] Specifically, before the covering stage, such as Figure 6 As shown, let α1 and α2 be the angles between the weld bead outside the groove and the vertical line of the screen when the molten pool observation screen is aligned before the capping stage, i.e., the predetermined reference angles. The real-time calculated reference angles on both sides are α1* and α2*, then we have When Δα > θ, an early warning message is generated. At this time, during the welding process, the crawler will offset the cover layer during the first weld pass, covering one side of the weld bead outside the groove. Therefore, the angle calculation at this time only uses the change in angle on one side, and the offset evaluation standard is the same as above. In addition, since there are nail points and groove bottom gap step points during the root pass stage of welding, the welding process can be warned by identifying the gap change at the bottom of the groove, and the current position of the change is recorded. By using the hysteresis distance, the parameters are adjusted after the crawler reaches the designated position to ensure the predictability of the welding process and improve the welding quality. Since the ceramic deposit in the root pass stage has reflective properties and appears as a continuous area in the molten pool image, the image is binarized, and the backing contour is extracted using the region growing method, as shown below. The degree of change in the bottom gap is calculated and displayed in real time. When the crawler crawls to the designated position through the hysteresis distance, the corresponding current, voltage, and process parameters are matched to complete the real-time early warning of the welding process.

[0116] To ensure the accuracy of the early warning, in one optional implementation, the processing unit further includes:

[0117] The eighth processing module is used to identify the center line of the welding wire based on the above welding image, and calculate the distance between the center line of the welding wire in the above welding image and the center line of the corresponding position of the welding path, so as to obtain the overlap of the welding wire swing center.

[0118] The aforementioned alarm information includes the sixth alarm information, and the aforementioned first sending unit further includes:

[0119] The seventh sending module is used to issue a sixth alarm message when the overlap of the welding wire swing center is greater than the distance threshold. The sixth alarm message is to remind the welding wire that it has deviated from the welding path.

[0120] Specifically, the degree of overlap between the center line of the welding wire oscillation trajectory and the laser line extracted from the laser inflection point determines whether the welding wire oscillation is following the preset trajectory. The degree of overlap between the coordinates of the welding wire's dwell position and the coordinates of the inflection point determines the weld formation, ensuring welding quality. Information from the previous layer's weld pool image scan guides the start of the next weld and the correction trajectory of the crawler during the welding process, adjusting the welding wire oscillation amplitude and the degree of overlap between the oscillation center and the current weld bead, improving the automated welding capability of the pre-scan. Figure 7 As shown, each inflection point of each layer and pass acquired by the laser module is extracted, i.e., feature points. The center line of the inflection point of the current weld pass, i.e., the center line of the bottom feature point, is calculated. The oscillation trajectory of the welding wire is generated by recognizing the welding wire outline in the molten pool image, and the center line of the welding wire oscillation amplitude is calculated. The information features acquired by the molten pool camera are rich, and the weld formation of the previous layer can be clearly observed in the current welding state. Welders can observe the welding situation in real time through the visualized molten pool image, assist in the adjustment of process parameters, and improve the weld quality.

[0121] To ensure the accuracy of the early warning, in one optional embodiment, the above-mentioned device further includes:

[0122] The reconstruction unit is used to acquire an image of the welding area in real-time welding, and after obtaining the welding image, to perform three-dimensional reconstruction based on the welding image to obtain the weld width and weld depth of the molten pool.

[0123] The detection unit is used to input the above-mentioned weld width, above-mentioned weld depth and corresponding welding process parameters into the detection model to obtain the detection result. The detection result is the risk of having welding defects and the risk of not having welding defects. The above-mentioned welding process parameters include at least the welding voltage. The above-mentioned detection model is used to determine whether welding defects occur under the historical weld width, historical weld depth, corresponding historical welding process parameters and corresponding working conditions.

[0124] The second sending unit is used to issue a seventh alarm message based on the above detection results. The seventh alarm message is used to prompt the adjustment of the above welding process parameters.

[0125] Specifically, the molten pool area is reconstructed in three dimensions using the installation position parameters and calibration parameters of the molten pool camera, and the molten pool depth is reproduced in real time. Based on the molten depth and width, the welding voltage and process parameters during the welding process are combined with the weld plate thickness and groove size to determine whether welding defects are occurring in the current welding state. This allows for real-time adjustment of welding parameters to reduce welding defects and improve weld quality. By extracting the molten pool contour and reconstructing it in three dimensions, parameters such as the molten pool width and depth during real-time welding are obtained. Correlation analysis is performed in conjunction with parameters such as welding current and voltage to obtain a correlation model and perform defect classification and prediction. Based on the early warning information, the adjustment of welding current and voltage parameters and process parameters is guided.

[0126] The aforementioned real-time welding quality control device includes a processor and a memory. The acquisition unit, processing unit, and first sending unit are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0127] The processor contains a core, which retrieves the corresponding program unit from memory. One or more cores can be configured, and adjusting core parameters can address the problem of relying on human experience to ensure soldering quality in existing technologies.

[0128] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0129] According to another aspect of this application, a display terminal is provided, the display terminal including a real-time welding quality control device, the control device further including: a display unit, used at least for real-time display of the weld fusion line angle, the welding gun wire angle, the wire extension length, the screen offset angle, the overlap of the wire swing center and alarm information.

[0130] Specifically, the aforementioned display terminal can display real-time welding fusion line angle, welding torch and welding wire angle, welding wire extension, image offset angle, welding wire oscillation center overlap, and alarm information, allowing welders to monitor the welding process in real time without close observation, thus avoiding any impact on their health. For example, the fitted molten pool contour curve can be deployed on the display terminal interface using visualization tools such as QT and matplotlib, and the characteristic information of the molten pool image can be displayed in real time using signals and slots. Furthermore, a three-dimensional coordinate system can be established using the extracted molten pool 3D information, and the 3D model generated by the molten pool camera can be updated in real time on the coordinate system and displayed on the terminal, facilitating an intuitive understanding of the status and changes of various molten pool parameters. Additionally, image segmentation and Hough transform can be used to synchronize the angle changes and display dynamics to the display terminal.

[0131] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the real-time welding quality control method.

[0132] Specifically, methods for controlling real-time welding quality include:

[0133] Step S201: Obtain an image of the welding area in real-time welding to obtain a welding image;

[0134] Specifically, a crawler tracks the welding area to acquire welding images in real time, and the molten pool camera on the crawler sends the captured welding images to the human-computer interaction screen for display.

[0135] Step S202: Extract welding features from the welding image. The welding features include the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire swing center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding gun and welding wire angle is the angle between the welding gun and the welding wire. The image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The overlap of the welding wire swing center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image.

[0136] Specifically, multiple welding features are extracted from the above welding images. Each welding feature corresponds to one or more welding defects, and it is possible to determine whether welding defects may occur based on these welding features.

[0137] Step S203: If the above-mentioned welding features are not within the corresponding predetermined range, an alarm message is issued. The alarm message is used to remind that a welding defect may occur.

[0138] Specifically, by extracting features, it can determine whether welding defects are occurring in the current welding process, whether defects are about to occur, and the type of defects that are occurring. Through the communication harness settings between the remote control and the host computer, it can provide real-time warnings and adjust welding process parameters to reduce the occurrence of welding defects and improve weld quality.

[0139] This invention provides a processor for running a program, wherein the program executes the real-time welding quality control method.

[0140] Specifically, methods for controlling real-time welding quality include:

[0141] Step S201: Obtain an image of the welding area in real-time welding to obtain a welding image;

[0142] Specifically, a crawler tracks the welding area to acquire welding images in real time, and the molten pool camera on the crawler sends the captured welding images to the human-computer interaction screen for display.

[0143] Step S202: Extract welding features from the welding image. The welding features include the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire swing center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding gun and welding wire angle is the angle between the welding gun and the welding wire. The image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The overlap of the welding wire swing center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image.

[0144] Specifically, multiple welding features are extracted from the above welding images. Each welding feature corresponds to one or more welding defects, and it is possible to determine whether welding defects may occur based on these welding features.

[0145] Step S203: If the above-mentioned welding features are not within the corresponding predetermined range, an alarm message is issued. The alarm message is used to remind that a welding defect may occur.

[0146] Specifically, by extracting features, it can determine whether welding defects are occurring in the current welding process, whether defects are about to occur, and the type of defects that are occurring. Through the communication harness settings between the remote control and the host computer, it can provide real-time warnings and adjust welding process parameters to reduce the occurrence of welding defects and improve weld quality.

[0147] This invention provides a welding system, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0148] Step S201: Obtain an image of the welding area in real-time welding to obtain a welding image;

[0149] Specifically, a crawler tracks the welding area to acquire welding images in real time, and the molten pool camera on the crawler sends the captured welding images to the human-computer interaction screen for display.

[0150] Step S202: Extract welding features from the welding image. The welding features include the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire swing center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding gun and welding wire angle is the angle between the welding gun and the welding wire. The image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The overlap of the welding wire swing center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image.

[0151] Specifically, multiple welding features are extracted from the above welding images. Each welding feature corresponds to one or more welding defects, and it is possible to determine whether welding defects may occur based on these welding features.

[0152] Step S203: If the above-mentioned welding features are not within the corresponding predetermined range, an alarm message is issued. The alarm message is used to remind that a welding defect may occur.

[0153] Specifically, by extracting features, it can determine whether welding defects are occurring in the current welding process, whether defects are about to occur, and the type of defects that are occurring. Through the communication harness settings between the remote control and the host computer, it can provide real-time warnings and adjust welding process parameters to reduce the occurrence of welding defects and improve weld quality.

[0154] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0155] Step S201: Obtain an image of the welding area in real-time welding to obtain a welding image;

[0156] Specifically, a crawler tracks the welding area to acquire welding images in real time, and the molten pool camera on the crawler sends the captured welding images to the human-computer interaction screen for display.

[0157] Step S202: Extract welding features from the welding image. The welding features include the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire swing center. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding gun and welding wire angle is the angle between the welding gun and the welding wire. The image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The overlap of the welding wire swing center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image.

[0158] Specifically, multiple welding features are extracted from the above welding images. Each welding feature corresponds to one or more welding defects, and it is possible to determine whether welding defects may occur based on these welding features.

[0159] Step S203: If the above-mentioned welding features are not within the corresponding predetermined range, an alarm message is issued. The alarm message is used to remind that a welding defect may occur.

[0160] Specifically, by extracting features, it can determine whether welding defects are occurring in the current welding process, whether defects are about to occur, and the type of defects that are occurring. Through the communication harness settings between the remote control and the host computer, it can provide real-time warnings and adjust welding process parameters to reduce the occurrence of welding defects and improve weld quality.

[0161] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0162] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0166] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0167] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0168] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0169] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0170] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0171] 1) In the real-time welding quality control method of this application, firstly, an image of the welding area of ​​the real-time welding is acquired to obtain a welding image; then, welding features are extracted based on the welding image, including the welding fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the overlap of the welding wire swing center. The welding fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding gun and welding wire angle is the angle between the welding gun and the welding wire, the image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the overlap of the welding wire swing center is the overlap between the center line of the welding wire and the center line of the corresponding position of the welding path in the welding image; then, if the welding features are not within the corresponding predetermined range, an alarm message is issued to remind that a welding defect may occur. This method acquires images of the welding area in real time to obtain welding images, and then extracts welding features from the welding images to determine whether welding defects may occur. If so, an alarm is issued, thereby assisting the welder in adjusting the welding process parameters in real time, improving the welder's working environment, reducing the occurrence of welding defects, and improving welding quality. It eliminates the need for manual welding or close observation of the welding through the welding machine, solving the problem of relying on human experience to ensure welding quality in existing technologies.

[0172] 2) In the real-time welding quality control device of this application, the acquisition unit acquires an image of the welding area of ​​the real-time welding to obtain a welding image; the processing unit extracts welding features based on the welding image, the welding features including the weld fusion line angle, the welding gun and welding wire angle, the welding wire extension, the image offset angle, and the welding wire oscillation center coincidence. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, the first edge line is the boundary line between the bevel and the molten pool, the welding gun and welding wire angle is the angle between the welding gun and the welding wire, the image offset angle is the absolute value of the difference between the reference angle and the predetermined reference angle, the reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, the second edge line is the edge line on the bevel that does not contact the molten pool, and the welding wire oscillation center coincidence is the coincidence of the center line of the welding wire in the welding image with the center line of the corresponding position of the welding path; the first sending unit issues an alarm message when the welding features are not within the corresponding predetermined range, the alarm message is used to remind that welding defects may occur. This device acquires images of the welding area in real time to obtain welding images, and then extracts welding features from the welding images to determine whether welding defects may occur. If so, it issues an alarm message, thereby assisting welders in adjusting welding process parameters in real time, improving the working environment of welders, reducing the occurrence of welding defects, and improving welding quality. It eliminates the need for manual welding or close observation of welding through welding machines, solving the problem of relying on human experience to ensure welding quality in existing technologies.

[0173] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for controlling welding quality in real time, characterized in that, include: Acquire images of the welding area in real-time to obtain welding images; Welding features are extracted from the welding image, including the weld fusion line angle, the welding torch and welding wire angle, the welding wire extension length, the image offset angle, and the welding wire oscillation center coincidence. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel, where the first edge line is the boundary line between the bevel and the molten pool. The welding torch and welding wire angle is the angle between the welding torch and the welding wire. The image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel, where the second edge line is the edge line on the bevel that does not contact the molten pool. The welding wire oscillation center coincidence is the degree of coincidence between the center line of the welding wire oscillation trajectory in the welding image and the center line of the corresponding position of the welding path. If the welding feature is not within the corresponding predetermined range, an alarm message is issued to alert that a welding defect may have occurred.

2. The method according to claim 1, characterized in that, The weld fusion line angle includes a first weld fusion line angle and a second weld fusion line angle. Welding features are extracted from the weld image, including: Extract the molten pool outline and two first edge lines from the welding image. The molten pool outline is the edge outline of the molten pool formed by welding, and the two first edge lines are located at both ends of the bevel. The angle between the molten pool outline and one of the first edge lines is detected to obtain the first weld fusion line angle, and the angle between the molten pool outline and another of the first edge lines is detected to obtain the second weld fusion line angle.

3. The method according to claim 2, characterized in that, The alarm information includes a first alarm information and a second alarm information. An alarm information is issued when the welding feature is outside the corresponding predetermined range, including: If the angle between the first weld fusion line and / or the second weld fusion line is less than or equal to the first angle threshold, a first alarm message is issued, the first alarm message being that a defect of excessive excess height may occur. If the angle between the first weld fusion line and / or the second weld fusion line is greater than or equal to the second angle threshold, a second alarm message is issued. The second alarm message indicates that a defect of insufficient welding material filling may occur. The second angle threshold is greater than the first angle threshold. If both the included angle of the first weld fusion line and the included angle of the second weld fusion line are greater than the first included angle threshold and less than the second included angle threshold, the first alarm message and the second alarm message will not be issued.

4. The method according to claim 1, characterized in that, Extracting welding features from the welding image further includes: The contour lines of the welding gun and the welding wire are extracted from the welding image, and the angle between the welding gun and the welding wire is determined based on the contour lines of the welding gun and the welding wire. The wire extension is identified based on the welding image.

5. The method according to claim 4, characterized in that, The alarm information includes a third alarm information and a fourth alarm information. When the welding feature is outside the corresponding predetermined range, the alarm information is issued, and it also includes: The absolute value of the difference between the welding torch wire angle and the predetermined angle is calculated to obtain the deviation angle. If the deviation angle is greater than the deviation angle threshold, the third alarm information is issued. The third alarm information indicates a welding defect that may be caused by the deviation of the welding torch wire angle. The absolute value of the difference between the wire extension and the predetermined wire extension is calculated to obtain the deviation length. If the deviation length is greater than the deviation length threshold, the fourth alarm message is issued. The fourth alarm message indicates that welding defects may be caused by unreasonable wire melting amount.

6. The method according to claim 1, characterized in that, Extracting welding features from the welding image further includes: Two second edge lines are identified based on the welding image, and the two second edge lines are located at the two ends of the bevel, respectively; The two reference angles are obtained by detecting the angles between the two second edge lines and the vertical direction of the welding image; Calculate the absolute value of the difference between the two reference angles and the corresponding predetermined reference angle to obtain the two image offset angles. The predetermined reference angle is the reference angle corresponding to the welding image taken before welding.

7. The method according to claim 6, characterized in that, The alarm information includes a fifth alarm information, which issues an alarm when the welding feature is outside the corresponding predetermined range, and also includes: If the average of the two aforementioned image offset angles is greater than the image offset angle threshold, a fifth alarm message is issued, which serves as a reminder that the shooting angle of the welding image has shifted.

8. The method according to claim 1, characterized in that, Extracting welding features from the welding image further includes: The center line of the welding wire oscillation trajectory is identified based on the welding image, and the distance between the center line of the welding wire oscillation trajectory in the welding image and the center line of the corresponding position of the welding path is calculated to obtain the coincidence degree of the welding wire oscillation center. The alarm information includes a sixth alarm information, which issues an alarm when the welding feature is outside the corresponding predetermined range, and also includes: If the overlap of the welding wire oscillation center is greater than the distance threshold, a sixth alarm message is issued, which is to remind the welding wire that it has deviated from the welding path.

9. The method according to any one of claims 1 to 8, characterized in that, After acquiring an image of the welding area in real-time welding and obtaining the welding image, the method further includes: The width and depth of the molten pool are obtained by performing three-dimensional reconstruction based on the welding image. The weld width, the weld depth, and the corresponding welding process parameters are input into the detection model to obtain the detection results. The detection results represent the risk of having welding defects and the risk of not having welding defects. The welding process parameters include at least the welding voltage. A seventh alarm message is issued based on the test results, and the seventh alarm message is used to prompt adjustment of the welding process parameters.

10. A real-time welding quality control device, characterized in that, include: The acquisition unit is used to acquire images of the welding area in real-time welding to obtain welding images; The processing unit is used to extract welding features from the welding image. The welding features include the weld fusion line angle, the welding torch and welding wire angle, the welding wire extension length, the image offset angle, and the welding wire oscillation center coincidence. The weld fusion line angle is the angle between the outline of the molten pool and the first edge line of the bevel. The first edge line is the boundary line between the bevel and the molten pool. The welding torch and welding wire angle is the angle between the welding torch and the welding wire. The image offset angle is the absolute value of the difference between a reference angle and a predetermined reference angle. The reference angle is the angle between the vertical direction of the welding image and the second edge line of the bevel. The second edge line is the edge line on the bevel that does not contact the molten pool. The welding wire oscillation center coincidence is the coincidence between the center line of the welding wire oscillation trajectory in the welding image and the center line of the corresponding position of the welding path. The first sending unit is used to issue an alarm message when the welding feature is not within the corresponding predetermined range, the alarm message being used to remind that a welding defect may occur.

11. A display terminal, characterized in that, The display terminal includes the real-time welding quality control device as described in claim 10, and the control device further includes: The display unit is used to display, at least in real time, the welding fusion line angle, the welding gun and welding wire angle, the welding wire extension length, the screen offset angle, the overlap of the welding wire swing center, and alarm information.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 9.

13. A welding system, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 9.

Images