Weld pool monitoring system and method

Abstract

The application provides a welding molten pool monitoring system and method, the system comprises: an input module for obtaining a to-be-processed welding image according to a selected welding image input source; a processing module for selecting a matched welding information recognition model, inputting the to-be-processed welding image into the welding information recognition model, obtaining a welding molten pool recognition result, processing the welding molten pool recognition result by using a preset parameter algorithm to obtain a molten pool detection parameter; and an output module for pushing the molten pool detection parameter to a user. The application can obtain a to-be-processed welding image from a selected welding image input source, select a matched welding information recognition model for image recognition, process the image recognition result to obtain a molten pool detection parameter to monitor the welding effect, and has better applicability and deployment convenience.

Description

Technical Field

[0001] This application relates to the field of intelligent welding technology, specifically to a welding pool monitoring system and method. Background Technology

[0002] Existing welding quality monitoring and prediction technologies encompass various monitoring methods. These include laser welding monitoring methods based on online incremental learning, weld quality monitoring methods based on machine vision, welding quality prediction methods based on machine learning models, and welding quality monitoring based on multi-sensor data fusion. While these technologies have achieved monitoring and quality prediction of the welding process to some extent, they have significant limitations in terms of monitoring scope, data fusion capabilities, and adaptability to multiple welding processes. Specifically, existing technologies generally suffer from the following shortcomings:

[0003] (1) For example, the laser welding monitoring method based on online incremental learning relies too much on the single sensor signal of the welding process for monitoring welding quality. It lacks comprehensive acquisition and fusion processing of multimodal data, making it difficult to monitor and detect key visual information such as weld surface morphology, and thus failing to meet the complex requirements of various welding processes.

[0004] (2) For example, the weld quality monitoring method based on machine vision uses machine learning models to detect weld defects. The monitoring scope is limited to the image data of the weld surface. It fails to effectively obtain other key physical parameters in the welding process and relies on a single visual source. It lacks support for different welding processes. This limitation makes the existing method perform poorly in adapting to complex industrial environments, especially in the application of multiple data sources and heterogeneous process environments.

[0005] (3) For example, welding quality prediction methods based on machine learning models rely too heavily on training historical data models when using machine learning models to predict welding quality, making it difficult to maintain prediction accuracy in the ever-changing real-time welding process. In addition, this method lacks the ability to integrate and process different types of input sources, making it difficult to apply flexibly in multiple welding process scenarios.

[0006] (4) Although welding quality monitoring based on multi-sensor data fusion has emerged in the existing technology, the multi-source data it collects generally includes parameters such as amplitude, pressure, and power. It lacks real-time acquisition and fusion of visual information from different sources, and cannot support adaptive monitoring of various welding processes, resulting in a deficiency in the monitoring of welding area morphology. Summary of the Invention

[0007] To address the problems in the existing technology, the purpose of this application is to provide a welding pool monitoring system and method, which can acquire welding images to be processed from a selected welding image input source, select a matching welding information recognition model to perform image recognition, and process the image recognition results to obtain welding pool detection parameters to monitor the welding effect, thus having better applicability and ease of deployment.

[0008] This application provides a welding pool monitoring system, including:

[0009] The input module is used to acquire the welding image to be processed based on the selected welding image input source;

[0010] The processing module is used to select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, obtain the welding pool recognition result, process the welding pool recognition result with a preset parameter algorithm, and obtain the pool detection parameters.

[0011] The output module is used to push the molten pool detection parameters to the user.

[0012] In some embodiments, the input module is further configured to receive a user's selection of a welding image input source, the optional welding image input sources including a screen capture input source, a camera capture input source, and a network input source;

[0013] The input module obtains the welding image to be processed based on the selected welding image input source, including: the input module captures image frames from the selected welding image input source as the welding image to be processed and stores them in the inference queue.

[0014] In some embodiments, the processing module is used to receive inference requirement information and select a matching welding information recognition model based on the correspondence between the inference requirement information and the welding information recognition model.

[0015] In some embodiments, the weld pool identification result includes the calibrated target position and the detected target position in the weld image to be processed, wherein the calibrated target includes the welding wire and / or tungsten electrode, and the detected target includes the weld pool;

[0016] The processing module uses the following steps to process the weld pool identification results using a preset parameter algorithm:

[0017] The pixel size of the calibration target in the welding image to be processed is determined based on the location of the calibration target in the welding image to be processed;

[0018] Obtain the actual size of the calibration target, and calculate the conversion relationship from the pixel size in the welding image to the actual size based on the pixel size of the calibration target and the actual size;

[0019] The pixel size parameters of the target are calculated based on the target location. Based on the pixel size parameters and the transformation relationship, the actual size parameters of the target are calculated and used as the melt pool detection parameters.

[0020] In some embodiments, the detection target also includes the bevel, and the molten pool detection parameters include the actual size parameters of the molten pool and the actual distance parameters between the edge of the molten pool and the bevel.

[0021] In some embodiments, the processing module is further configured to, after obtaining the weld pool identification result, draw the calibration target contour and the detection target contour in the welding image to be processed according to the weld pool identification result, obtain the processed welding image, and output the processed welding image to the image output queue.

[0022] In some embodiments, the output module is further configured to: display the processed welding images in the image output queue in real time, and / or,

[0023] Save the welding image to be processed and the processed welding image as video files respectively, and store them in the specified output folder.

[0024] In some embodiments, an alarm module is also included, which is used to trigger an alarm operation when an alarm is required based on the molten pool detection parameters and a preset parameter value range.

[0025] In some embodiments, an adaptive control module is also included, which is used to determine the type of weld pool detection parameter that exceeds the preset parameter value range when it is determined that the welding parameters need to be adjusted based on the weld pool detection parameters and the preset parameter value range, and adaptively adjust the welding parameters according to the correspondence between the weld pool detection parameter type and the welding parameter adjustment method.

[0026] This application embodiment also provides a method for monitoring a weld pool, using a weld pool monitoring system, which includes the following steps:

[0027] Obtain the welding image to be processed based on the selected welding image input source;

[0028] Select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, and obtain the welding pool recognition result;

[0029] The weld pool identification results are processed using a preset parameter algorithm to obtain the weld pool detection parameters;

[0030] Push the molten pool detection parameters to the user.

[0031] The welding pool monitoring system and method provided in this application have the following advantages:

[0032] By employing this application, a welding image to be processed can be obtained from a selected welding image input source. The processing module selects a matching welding information recognition model for image recognition. Based on the image recognition results, molten pool detection parameters are obtained to monitor the welding effect. These parameters are then output to the user, allowing for timely monitoring of the current welding status. This application can provide different welding information recognition models for different welding processes, allowing users to change models according to process requirements, thus improving adaptability to various welding processes. Furthermore, it supports multiple image input methods, allowing users to select the most suitable input method based on the application scenario. Therefore, this method has better applicability and ease of deployment. Attached Figure Description

[0033] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0034] Figure 1 This is a structural block diagram of a welding pool monitoring system according to an embodiment of this application;

[0035] Figure 2 This is a flowchart of a welding pool monitoring method according to an embodiment of this application;

[0036] Figure 3 This is a detailed structural block diagram of a welding pool monitoring system according to an embodiment of this application;

[0037] Figure 4 This is a detailed flowchart of a welding pool monitoring method according to an embodiment of this application;

[0038] Figure 5 This is a flowchart illustrating an embodiment of the present application of capturing image frames via a camera;

[0039] Figure 6 This is a flowchart illustrating a network transmission capture and transmission of image frames according to an embodiment of this application;

[0040] Figure 7 This is a flowchart illustrating a network transmission capture and reception of image frames according to an embodiment of this application;

[0041] Figure 8 This is a flowchart of adaptive control performed after calculating the molten pool detection parameters according to an embodiment of this application. Detailed Implementation

[0042] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore repeated descriptions of them will be omitted.

[0043] like Figure 1 As shown, this application provides a welding pool monitoring system, including:

[0044] The input module M100 is used to acquire the welding image to be processed according to the selected welding image input source. In this embodiment, the welding image input source is welding video data, which can be acquired by a camera set on the welding equipment or welding production line. Image frames are captured from the video data and added to the inference queue as the welding image to be processed for recognition and analysis.

[0045] The processing module M200 is used to select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, obtain the welding pool recognition result, and process the welding pool recognition result using a preset parameter algorithm to obtain the molten pool detection parameters. Specifically, after selecting a specified model, the processing module M200 selects the welding image to be processed from the inference queue and inputs it into the welding information recognition model to obtain the welding pool recognition result output by the model.

[0046] The output module M300 is used to push the molten pool detection parameters to the user. This can be done by pushing the parameters to the user terminal in the form of a message, or by displaying them on a screen for the user to view.

[0047] By employing the welding pool monitoring system of this application, the input module M100 acquires the welding image to be processed from a selected welding image input source, and the processing module M200 selects a matching welding information recognition model for image recognition. Based on the image recognition results, the system processes the image to obtain weld pool detection parameters for monitoring the welding effect. Then, the output module M300 outputs the weld pool detection parameters to the user, allowing the user to promptly obtain information on the current welding status. This application can provide different welding information recognition models for different welding processes, allowing users to change models according to process requirements, thus improving adaptability to different welding processes. Furthermore, it supports multiple image input methods, allowing users to select the most suitable input method based on the application scenario. Therefore, this method has better applicability and ease of deployment.

[0048] like Figure 2 As shown, this application also provides a method for monitoring a weld pool, and the weld pool monitoring system includes the following steps:

[0049] S100: Obtain the welding image to be processed based on the selected welding image input source;

[0050] Specifically, image frames of welding video are captured from a specified welding image input source and stored in the inference queue as welding images to be processed;

[0051] S200: Select the matching welding information recognition model, input the welding image to be processed into the welding information recognition model, and obtain the welding pool recognition result;

[0052] Specifically, after selecting a specified model, the welding image to be processed is selected from the inference queue and input into the welding information recognition model to obtain the welding pool recognition result output by the model.

[0053] S300: The weld pool identification results are processed using a preset parameter algorithm to obtain weld pool detection parameters; the weld pool detection parameters are the results of real-time monitoring and detection during the welding monitoring process.

[0054] S400: Pushes the molten pool detection parameters to the user.

[0055] By employing the welding pool monitoring method of this application, in step S100, a welding image to be processed can be obtained from a selected welding image input source; in step S200, a matching welding information recognition model is selected for image recognition; in step S300, the image recognition results are processed to obtain welding pool detection parameters for monitoring the welding effect; and then in step S400, the welding pool detection parameters are output to the user, allowing the user to promptly obtain the current welding status. This application can provide different welding information recognition models for different welding processes, allowing users to change models according to process requirements, thus improving adaptability to different welding processes. Furthermore, it supports multiple different image input methods, allowing users to select the most suitable input method based on the application scenario. Therefore, this method has better applicability and ease of deployment.

[0056] The following is combined with Figure 3 and Figure 4 The implementation method of the welding pool monitoring system and method in this embodiment is described in detail. It is understood that the structural components and steps shown in the accompanying drawings and the following description are merely examples and are not intended to limit the scope of protection of this application.

[0057] like Figure 3 As shown, the input module M100 is also used to receive the user's selection of the welding image input source. Selectable welding image input sources include screen capture input, camera capture input, and network input. For example... Figure 4As shown, in step S100, the input module acquires the welding image to be processed based on the selected welding image input source. This includes: the input module selecting a capture area from the selected welding image input source, continuously capturing image frames as the welding image to be processed, storing them in the inference queue, and simultaneously storing the image frames in the original video queue for subsequent original video output. The image frames stored in the original video queue and the inference queue are the same, and are subsequently used for different processing. The input module M100 includes: a screen capture unit for acquiring the welding image to be processed from a screen capture input source; a camera capture unit for directly capturing the welding image to be processed from a camera; and a network transmission unit for receiving the welding image to be processed through a network input source.

[0058] The screen capture unit includes: a capture area selection subunit, where users create a transparent full-screen window using the Tkinter library to display selectable capture areas; a frame capture subunit, which uses the mss library to continuously capture the selected screen area, obtaining image frames and storing them in the raw video save queue and inference queue; a configuration management subunit, which allows users to dynamically modify video recording configuration information, including capture frame rate, window transparency, capture box color and width; and a resource management subunit, used to stop video recording, clean up resources, and release memory when monitoring ends. When capturing image frames using the screen capture unit, it first receives the user's selection of the screen area to be captured, continuously captures the selected area, obtains image frames, and stores them in the raw video save queue and inference queue.

[0059] The camera capture unit includes: a camera initialization subunit, used to call the camera system library, initialize the camera device, and obtain basic camera information and configuration information; a frame capture subunit, used to continuously acquire image frames from the camera and store them in the raw video storage queue and inference queue; a camera configuration module, used to configure camera parameters according to user operations, configurable camera parameters include, for example, analog gain, exposure time, contrast, gamma value, rotation angle, etc.; and a resource cleanup subunit, used to release camera resources and clear the queue when monitoring ends. Figure 5 This is a flowchart illustrating an embodiment of this application of capturing image frames using a camera, as shown below. Figure 5 As shown, when capturing image frames using the camera capture unit, the camera (camera device) parameters are first set, the camera is started, and image frames are continuously captured by the camera and placed into the inference queue and the original video queue.

[0060] The network transmission unit includes: an RTSP server subunit, used to start or stop the RTSP (Real-Time Streaming Protocol) server and enable or disable the function of transmitting image frames over the network; a capture area selection subunit, used to call the camera through the capture area selection subunit of the direct screen capture module or directly call the camera; a frame capture subunit, used to call the camera through the frame capture subunit of the direct screen capture module or directly call the camera, and the captured image frames are directly passed to the FFmpeg (an open-source multimedia processing tool) push streaming subunit; an FFmpeg push streaming subunit, used to send the image frames to the specified URL (Uniform Resource Locator) address via the RTSP protocol using FFmpeg as a video stream; and a resource cleanup subunit, used to stop capture, shut down the RTSP server, and clean up resources. Figure 6 This is a flowchart illustrating a network transmission capture and transmission of image frames according to an embodiment of this application. Figure 7 This is a flowchart illustrating a network transmission capture and reception of image frames according to an embodiment of this application. Figure 6 and Figure 7 As shown, capturing image frames using a network transmission unit involves two processes: sending at the sending end and receiving at the receiving end. Sending at the sending end includes: starting the RTSP server, inputting the transmission address and capture frame rate, determining the capture area, continuously capturing image frames, and starting FFmpeg to push the image frames to the specified URL address. Receiving at the receiving end includes: starting the receiving thread, attempting to establish a connection with the specified URL address, connecting to the specified address, obtaining the video stream including image frames from the specified address, decoding the video stream, and storing the decoded video frames into the inference queue and the original video queue.

[0061] like Figure 3As shown, the processing module M200 includes a frame selection unit, a model selection unit, an inference unit, a mask extraction unit, a calculation unit, and a contour drawing unit. The frame selection unit retrieves the welding images to be processed from the inference queue for subsequent processing. The model selection unit determines the selection of the welding information recognition model. Optional welding information recognition models include, for example, the PT model, the ONNX model, and the OpenVINO model. The PT model is a PT (PyTorch) model file directly trained using YOLOv8n-seg, offering advantages in rapid development and debugging. Its applicable scenarios include environments with strong GPU support, facilitating model training and testing. The ONNX model is a PT model converted to an ONNX (Open Neural Network Exchange) model file, possessing good cross-platform compatibility. Its applicable scenarios include deployment on various hardware devices, such as CPUs, GPUs, and mobile devices. The OpenVINO model is an ONNX model converted to an OpenVINO model file, optimized for Intel hardware, resulting in faster inference speeds. Its applicable scenarios include scenarios requiring high-performance real-time inference, especially in Intel hardware environments. The monitoring system pre-stores multiple pre-trained welding information recognition models. These models are machine learning models that are iteratively trained by collecting sample image data and labeling targets, enabling them to accurately identify the targets to be detected in the images.

[0062] Specifically, the model selection unit receives inference requirement information and selects a matching welding information recognition model based on the correspondence between the inference requirement information and the welding information recognition model. This inference requirement information can be user-inputted or automatically detected by the monitoring system. For example, the inference requirement information can be the welding process type (MIG, TIG, etc.). The monitoring system pre-stores the correspondence between welding information recognition model types for various welding process types, i.e., information on the model type selected for each welding process type. The model selection unit can receive the welding process type input by the user and match it with the corresponding model. The inference requirement information can also be hardware environment information. For example, when the user inputs or the monitoring system detects that the current GPU meets certain performance requirements, the PT model is selected; when the user inputs or the monitoring system detects that deployment on multiple hardware devices is required, the ONNX model is selected. Users can also directly select a model as needed; that is, the inference requirement information can directly be the welding information recognition model type selected by the user.

[0063] The welding information recognition models corresponding to various welding process types are deep learning models trained based on sample data for that specific welding process type. For example, for MIG welding, historical welding images from past MIG welding processes are collected as sample images. The locations of the molten pool, welding wire, and bevel are manually labeled in these sample images as tags. A deep learning model specific to MIG is then used for prediction. The loss function value is obtained based on the predicted molten pool, welding wire, and bevel locations and their corresponding labels in the sample images. This loss function value is then used to iteratively train the deep learning model, resulting in a trained deep learning model specific to MIG. This model can be distinguished from other models by assigning a number or name. Similarly, for TIG welding, historical welding images from past TIG welding processes can also be collected as sample images, and a similar method can be used to train a deep learning model specific to TIG welding. When constructing and training models, the same model type can be used for deep learning models of different welding process types, or different model types can be used depending on the specific welding monitoring requirements.

[0064] The inference unit inputs the image to be processed acquired by the frame selection unit into the specified model selected by the model selection unit to obtain the weld pool recognition result output by the model. The weld pool recognition result includes the calibrated target position and the detection target position in the weld image to be processed. The calibrated target includes the welding wire and / or tungsten electrode, and the detection target includes the weld pool. The calibrated target is used to calibrate the conversion relationship between pixel size and actual size in the weld image to be processed. Since the actual size of the welding wire and / or tungsten electrode is known, the conversion relationship can be calibrated by its pixel size in the image. The following explanation uses the welding wire as an example of the calibrated target. The mask extraction unit extracts the position mask information of the calibrated target and the detection target in the image from the weld pool recognition result output by the model, for use by the calculation unit and the contour drawing unit.

[0065] The calculation unit is used to process the weld pool identification results using a preset parameter algorithm in the following steps:

[0066] The pixel size of the calibration target in the welding image to be processed is determined based on the location of the calibration target in the welding image to be processed;

[0067] Obtain the actual size of the calibration target, and calculate the conversion relationship from the pixel size in the welding image to the actual size based on the pixel size of the calibration target and the actual size;

[0068] The pixel size parameters of the target are calculated based on the target location. Based on the pixel size parameters and the transformation relationship, the actual size parameters of the target are calculated and used as the melt pool detection parameters.

[0069] In this embodiment, the detection target also includes the bevel, and the molten pool detection parameters include the actual size parameters of the molten pool and the actual distance parameters between the edge of the molten pool and the bevel. The actual size parameters of the molten pool include, for example, the area, length, and width of the molten pool, and the actual distance parameters between the edge of the molten pool and the bevel include, for example, the distance from the upper and lower edges of the molten pool to the upper and lower bevels.

[0070] Taking the calibration target as including the welding wire and the inspection target as including the molten pool and bevel as an example, the calculation process of the calculation module is explained in detail as follows:

[0071] (1) Calculation of transformation relations:

[0072] The bounding box of the welding wire is obtained through model detection, and its pixel height in the vertical direction (i.e., corresponding to the diameter of the welding wire) is extracted.

[0073] Pixel size L of the welding wire pixels The calculation formula is: L pixels =y max -y min

[0074] Among them, y max and y min These are the pixel coordinates of the lower and upper edges of the wire bounding box, respectively.

[0075] Obtain the true size L of the welding wire real (Equal to the actual diameter of the welding wire);

[0076] The conversion ratio from pixel size in the welding image to actual size is calculated as follows:

[0077]

[0078] Where R is the pixel-to-millimeter conversion ratio, with the unit being millimeters per pixel.

[0079] (2) Calculation of molten pool area

[0080] The mask region of the molten pool is obtained through model recognition, and the polygonal contour coordinates of the molten pool are extracted.

[0081] The pixel area of ​​the melt pool is calculated using the polygon area calculation formula (Shoelace formula) as follows:

[0082]

[0083] Among them, (x i ,y i Let x be the coordinates of a vertex of the polygon, n be the number of vertices of the polygon, and x n+1 =x1,y n+1 =y1.

[0084] The pixel area of ​​the molten pool is converted to the actual physical area as follows:

[0085] A real =A pixels ×R 2

[0086] Among them, A real This represents the actual physical area of ​​the molten pool, expressed in millimeters squared.

[0087] (3) Calculation of the length and width of the molten pool

[0088] Model detection is used to obtain the bounding box of the melt pool, and its horizontal pixel length L is extracted. pool_pixels and vertical direction W pool_pixel The pixel width is calculated using the following formula:

[0089] L po_pixe =x max -x min

[0090] W pool_pixels =y max -y min

[0091] Where, x max and x min Let y be the pixel coordinates of the left and right edges of the molten pool bounding box. max and y min These are the pixel coordinates of the upper and lower edges of the molten pool bounding box.

[0092] The pixel length and width are converted to the actual physical length and width using the following formula:

[0093] L pool_real =L pool_pixels ×R

[0094] W pool_rea =W pool_pixels ×R

[0095] Among them, L pool_real and W pool_real These are the actual length and width of the molten pool, respectively, in millimeters.

[0096] (4) Calculation of the distance from the upper and lower edges of the molten pool to the upper and lower bevels

[0097] Based on the model recognition results, obtain the coordinates of key points in the molten pool and bevel. Calculate the pixel distance D from the upper edge of the molten pool to the bevel. to_pixels The pixel distance D from the lower edge of the molten pool to the bevel bottom_pixel as follows:

[0098] D top_pixels =ygroove_top -y pool_top

[0099] D bottom_pixel =y pool_bottom -y groove_bottom

[0100] Where, y groove_top and y groove_bottom Let y be the pixel coordinates of the upper and lower edges of the bevel. pool_top and y pool_bottom These are the pixel coordinates of the upper and lower edges of the molten pool.

[0101] The pixel distance is converted to the actual physical distance using the following formula:

[0102] D to_real =D top_pixels ×R

[0103] D bottom_re =D bottom_pixels ×R

[0104] Among them, D to_rea and D bottom_real These are the actual distances from the upper and lower edges of the molten pool to the upper and lower bevels, respectively, in millimeters.

[0105] The contour drawing unit is used to draw the calibration target contour and detection target contour in the welding image to be processed based on the weld pool identification result after obtaining the weld pool identification result (such as drawing the contours of the weld pool, welding wire, tungsten electrode, bevel, etc. in the welding image to be processed based on the extracted mask information), thereby obtaining the processed welding image, and outputting the processed welding image to the image output queue. Figure 4 As shown, in this embodiment, the image output queue includes a real-time display queue and an inference video storage queue, which are used by the subsequent output module.

[0106] like Figure 3 As shown, the output module M300 includes a real-time display unit, a video storage unit, and a parameter output unit. The parameter output unit outputs calculated key parameters such as the length, width, and area of ​​the weld pool, as well as the distances from the upper and lower edges of the weld pool to the upper and lower bevels. The real-time display unit displays the processed welding images from the image output queue in real time. Specifically, the real-time display unit acquires and displays image frames from the real-time display queue at a set frame rate, allowing the user to monitor the current welding status in real time via the display screen.

[0107] The video saving unit is used to save the welding images to be processed and the processed welding images as video files, and store them in a designated output folder. Specifically, the video saving unit retrieves image frames from the inference video saving queue and the original video saving queue, saves them as MP4 format video files, and then stores them in the designated output folder. The video saving unit can be further divided into: a configuration management subunit, a folder management subunit, a video writing subunit, a storage monitoring subunit, and a resource cleanup subunit. The configuration management subunit is used to set relevant configurations for the output videos, including the output folder path, output frame rate, maximum size of a single video (MB), and total video size limit (GB). The folder management subunit is used to create subfolders in the designated output folder according to the current date (year-month-day) and save the output videos. The video naming method is the timestamp of the first captured frame (year-month-day-hour-minute-second). The video writing subunit is used to use a video writer to write image frames to MP4 format video files at a set frame rate. The storage monitoring subunit is used to monitor the total size of individual video files and the output folder in real time to ensure the reasonable use of storage space. The resource cleanup subunit stops video writing, shuts down the video writer, clears the frame queue, and releases resources after monitoring stops. The video monitoring subunit monitors the total size of individual video files in real time, determining if the size of a single generated video file exceeds the preset maximum size for a single video. If so, it splits the video file into at least two new video files before saving the output video. The video monitoring subunit also monitors the total size of videos already stored in the output folder in real time. If the total size of stored videos exceeds the total video size limit, it changes the storage location. If no other storage location is available, it prompts the user to delete some videos, or deletes the oldest historical videos according to user-preset rules. Therefore, this welding pool monitoring system supports real-time video storage and can automatically allocate and monitor storage space, avoiding storage space shortages.

[0108] like Figure 3 As shown, the welding molten pool monitoring system also includes an alarm module M400, which triggers an alarm operation when an alarm is required based on the molten pool detection parameters and preset parameter value ranges. This alarm operation occurs when the molten pool detection parameters include the molten pool area, molten pool length, molten pool width, and the distance between the molten pool edge and the bevel. For example... Figure 4 As shown, after obtaining the molten pool detection parameters through the processing module, a standard parameter value range is preset for each parameter. The detected value is compared with the corresponding parameter value range. If the detected value is too high or too low, an alarm operation is triggered. This alarm triggering operation can be done by displaying an alarm prompt on the screen or sending alarm information to the user terminal, etc.

[0109] like Figure 3As shown, the welding pool monitoring system also includes an adaptive control module M500, which is used to determine the type of welding pool detection parameter that exceeds the preset parameter value range when welding parameters need to be adjusted based on the welding pool detection parameters and preset parameter value range. Based on the correspondence between the welding pool detection parameter type and the welding parameter adjustment method, the system adaptively adjusts the welding parameters. Figure 8 As shown, after processing the weld pool identification results using a preset parameter algorithm to obtain the weld pool detection parameters, the following steps are also included: Each weld pool detection parameter is compared with a threshold value (including the upper and / or lower boundary thresholds of the parameter value range) within the preset parameter value range. If at least one weld pool detection parameter exceeds the preset parameter value range, the parameter type exceeding the preset parameter value range is considered an influencing factor. Based on the influencing factor, the situation is classified into weld pool area not meeting requirements, weld pool length not meeting requirements, weld pool width not meeting requirements, and distance between the weld pool and the bevel not meeting requirements, etc. The corresponding welding parameter type requiring adjustment is selected based on the classification, and the welding parameter is adjusted according to the relationship between the weld pool detection parameter exceeding the range and its corresponding preset parameter value range. For example, when the weld pool area is too large, the welding current is reduced; when the weld pool area is too small, the welding current is increased. When the distance between the weld pool and the bevel exceeds the corresponding distance range, the welding torch position is adjusted. Other adjustable welding parameters include welding speed and downward pressure. Through adaptive adjustment of the welding parameters, the weld pool shape, size, and position are adjusted to the set parameter value range. This application achieves real-time quality control by adjusting welding parameters in a closed loop based on real-time feedback from molten pool detection parameters, dynamically optimizing welding parameters according to actual welding results, and continuously adapting to changes in the welding process during real-time control.

[0110] The implementation of this welding pool monitoring system and method will be further introduced below with specific examples.

[0111] Example 1: This system and method are used for real-time monitoring and quality control of MIG welding processes (gas inert gas welding, copper welding wire). This system and method are particularly suitable for industrial welding scenarios requiring high precision and high stability.

[0112] (1) System initialization

[0113] The input module selects camera capture as the image input method. It initializes the camera and sets parameters such as frame rate and resolution. The processing module loads a pre-trained deep learning model specifically for MIG welding. The output module sets the real-time display frame rate and video storage path.

[0114] (2) Image capture and data transmission

[0115] The camera is activated to capture image frames of the molten pool area in real time. The captured image frames are then fed into the inference queue, awaiting further processing.

[0116] (3) Image processing

[0117] The frame selection unit retrieves the current image frame from the inference queue. The model selection unit selects the loaded MIG welding pre-trained model for inference analysis. The inference unit, based on a deep learning model, extracts features from the molten pool region, including parameters such as the area, length, and width of the molten pool. The mask extraction unit extracts masks for the molten pool boundary, welding wire, tungsten electrode, etc., to accurately delineate each region.

[0118] (4) Quality parameter calculation and threshold judgment

[0119] Based on the detection results, the calculation unit calculates key quality parameters of the molten pool (such as molten pool area, width, length, and distance between the molten pool and the bevel). The alarm module determines whether the calculation results exceed a preset quality threshold. If they do, the system will trigger an alarm and enter an adaptive control process.

[0120] (5) Real-time feedback and quality control

[0121] If any parameter exceeds the threshold, the system will automatically adjust the relevant welding parameters, such as the welding current or welding speed, to restore the parameter to the normal range. After adjustment, the welding parameters are monitored in real time to ensure that the adjustment effect meets expectations.

[0122] (6) Video storage and display

[0123] The real-time display unit shows the processed image frames on the monitoring screen to help operators monitor the system in real time. The video storage unit saves the original video and inference video to a designated path for subsequent analysis and quality traceability.

[0124] Through this application, the weld pool monitoring system enables real-time monitoring and quality control of the MIG welding process. When the weld pool parameters are detected to exceed the standard range, the system can automatically adjust the welding parameters, thereby improving the stability and consistency of welding quality, reducing human intervention, and increasing welding efficiency.

[0125] Example 2: The welding pool monitoring system and method are applied to remote monitoring of the welding process based on network transmission. It is suitable for remote welding monitoring needs, especially for situations where multiple locations are collaboratively monitoring the welding process.

[0126] (1) System initialization

[0127] The input module selects network transmission as the image input method. It connects to the receiving end via the RTSP protocol to ensure network transmission stability. The processing module loads the inference model suitable for this welding process. The output module sets the frame rate and storage path for remote monitoring.

[0128] (2) Image acquisition and streaming

[0129] The frame capture subunit continuously captures frames using a camera or screen capture and transmits them to FFmpeg for streaming. The FFmpeg streaming subunit then transmits the image frames as a video stream to a remote monitoring center, enabling real-time monitoring from multiple locations.

[0130] (3) Real-time inference and quality analysis

[0131] The inference unit, based on a deep learning model, analyzes the molten pool detection parameters in the image and feeds the analysis results back to the monitoring interface in real time. It monitors key molten pool detection parameters and welding monitoring parameters (such as temperature and speed) in real time to determine whether thresholds are exceeded.

[0132] (4) Remote alarm and adjustment

[0133] When the monitoring results exceed the threshold, the system will notify the remote operator via an alarm function and record the alarm information. If the system is configured with automated control functions, the system will automatically adjust the welding parameters; otherwise, the operator will adjust them remotely.

[0134] (5) Video storage

[0135] Save the real-time video stream transmitted over the network and the processed video stream to the output folder for easy subsequent analysis and quality traceability.

[0136] In summary, the welding pool monitoring system and method of this embodiment have the following beneficial effects:

[0137] (1) In order to solve the problem that existing welding monitoring systems usually only support a single input method, lack sufficient flexibility, and are difficult to adapt to the needs of various application scenarios, this application can obtain the welding image to be processed from the selected welding image input source, and supports three image input methods: screen capture, camera capture and network transmission. Users can choose the appropriate input source according to the actual scenario needs, which has better applicability and deployment convenience, and increases the flexibility of the welding pool monitoring system in different industrial scenarios.

[0138] (2) To address the problem that most existing welding monitoring systems are limited to a single type of welding process and lack adaptability to multiple welding processes, this application allows for the selection of a matching welding information recognition model for image recognition. The image recognition results are then processed to obtain molten pool detection parameters for monitoring the welding effect. This application can quickly switch between various trained models to adapt to different welding processes, thus meeting diverse process requirements. This function avoids the limitation of a single process and greatly enhances the applicability and flexibility of the welding molten pool monitoring system.

[0139] (3) To address the common problem of insufficient real-time performance in weld quality assessment and process control in existing welding monitoring systems, this application employs a machine learning model to perform real-time inference on the welding images with molten pool areas, and to monitor various molten pool detection parameters in real time. This meets the quality assessment and feedback control requirements of the welding process, ensuring that various molten pool detection parameters (such as molten pool area, length, width, and distance between the molten pool and the bevel) are within the standard range. This real-time performance enables timely detection of welding anomalies, improving the efficiency and accuracy of welding quality control. By dynamically and adaptively adjusting welding parameters based on molten pool detection parameters, manual adjustment is eliminated, reducing human intervention. Combined with deep learning, intelligent feedback and automated adjustment are achieved, providing targeted control in various welding processes and improving the stability, automation, and intelligence of the welding process. This system uses deep learning and online incremental learning technologies to perform real-time analysis of key image signals in the welding process, and combines pre-trained models to achieve rapid adaptation to different welding processes, ensuring the stability and high-quality output of the welding process.

[0140] (4) The welding pool monitoring system supports real-time video storage and can automatically allocate storage space to avoid the problem of insufficient storage space. It also supports the independent saving of original video and inferred video, which is convenient for future analysis and tracing.

[0141] (5) This system uses machine vision and deep learning algorithms to perform real-time monitoring and quality assessment of the molten pool area during welding, and is widely applicable to welding quality control in industries such as aerospace, automobile manufacturing, and shipbuilding. This system overcomes the shortcomings of existing technologies, performs excellently in data fusion and multimodal processing, and is suitable for multiple welding processes and complex industrial environments, providing a new and intelligent solution for welding quality control in industrial production processes.

[0142] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of this application and should not be construed as limiting the specific implementation of this application to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of this application, and all such modifications or substitutions should be considered within the scope of protection of this application.

Claims

1. A welding pool monitoring system, characterized in that, include: The input module is used to acquire the welding image to be processed based on the selected welding image input source; The processing module is used to select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, obtain the welding pool recognition result, and process the welding pool recognition result using a preset parameter algorithm to obtain the pool detection parameters. The welding pool recognition result includes the calibration target position and the detection target position in the welding image to be processed. The calibration target includes the welding wire and / or tungsten electrode, and the detection target includes the pool. The output module is used to push the molten pool detection parameters to the user; The adaptive control module is used to determine the type of weld pool detection parameter that exceeds the preset parameter value range when it is determined that the welding parameters need to be adjusted based on the weld pool detection parameters and the preset parameter value range, and to adaptively adjust the welding parameters according to the correspondence between the weld pool detection parameter type and the welding parameter adjustment method. The processing module processes the weld pool identification result using a preset parameter algorithm through the following steps: The pixel size of the calibration target in the welding image to be processed is determined based on the location of the calibration target in the welding image to be processed. Obtain the actual size of the calibration target, and calculate the conversion relationship from the pixel size in the welding image to the actual size based on the pixel size of the calibration target and the actual size; The pixel size parameters of the target are calculated based on the target location, and the actual size parameters of the target are calculated based on the pixel size parameters and the conversion relationship, which are used as the melt pool detection parameters.

2. The welding pool monitoring system according to claim 1, characterized in that, The input module is also used to receive the user's selection of the welding image input source, and the optional welding image input sources include screen capture input source, camera capture input source and network input source; The input module obtains the welding image to be processed according to the selected welding image input source, including: the input module captures image frames from the selected welding image input source as the welding image to be processed and stores them in the inference queue.

3. The welding pool monitoring system according to claim 1, characterized in that, The processing module is used to receive inference requirement information and select a matching welding information recognition model based on the correspondence between the inference requirement information and the welding information recognition model.

4. The welding pool monitoring system according to claim 1, characterized in that, The detection target also includes the bevel, and the molten pool detection parameters include the actual size parameters of the molten pool and the actual distance parameters between the edge of the molten pool and the bevel.

5. The welding pool monitoring system according to claim 1, characterized in that, The processing module is also used to draw a calibration target contour and a detection target contour in the welding image to be processed according to the welding pool identification result after obtaining the welding pool identification result, so as to obtain the processed welding image and output the processed welding image to the image output queue.

6. The welding pool monitoring system according to claim 5, characterized in that, The output module is also used for: displaying the processed welding images in the image output queue in real time, and / or, The welding image to be processed and the processed welding image are saved as video files and stored in the specified output folder.

7. The welding pool monitoring system according to claim 1, characterized in that, It also includes an alarm module, which is used to trigger an alarm operation when an alarm is required based on the molten pool detection parameters and the preset parameter value range.

8. A method for monitoring a weld pool, characterized in that, The welding pool monitoring system according to any one of claims 1 to 7 includes the following steps: Obtain the welding image to be processed based on the selected welding image input source; Select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, and obtain the welding pool recognition result; The weld pool identification results are processed using a preset parameter algorithm to obtain weld pool detection parameters; The molten pool detection parameters are pushed to the user.

Images