Welding pool monitoring system and method

Abstract

The invention provides a welding pool monitoring system and method, and the system comprises an input module which is used for obtaining a to-be-processed welding image according to a selected welding image input source; the processing module is used for selecting a matched welding information identification model, inputting the to-be-processed welding image into the welding information identification model, obtaining a welding pool identification result, and processing the welding pool identification result by adopting a preset parameter algorithm to obtain welding pool detection parameters; and the output module is used for pushing the molten pool detection parameters to a user. According to the method, the welding image to be processed can be obtained from the selected welding image input source, the matched welding information recognition model is selected for image recognition, processing is conducted according to the image recognition result to obtain the weld pool detection parameters so as to monitor the welding effect, and better applicability and deployment convenience are achieved.

Description

Technical Field

[0001] The present application relates to the field of intelligent welding technology, and in particular to a welding molten pool monitoring system and method. Background Art

[0002] Existing welding quality monitoring and prediction technologies cover different monitoring methods. For example, laser welding monitoring methods based on online incremental learning, weld quality monitoring methods based on machine vision, welding quality prediction methods based on machine learning models, welding quality monitoring based on multi-sensor data fusion, etc. These technologies have achieved the monitoring and quality prediction of the welding process to a certain extent, but there are obvious limitations in monitoring range, data fusion capabilities and adaptability to multiple welding processes. Specifically, the existing technologies generally have the following defects:

[0003] (1) For example, the online incremental learning laser welding monitoring method relies too much on the single sensor signal of the welding process to monitor the welding quality. It lacks the comprehensive collection and fusion processing of multimodal data, and it is difficult to monitor and detect key visual information such as the weld surface morphology, and it is difficult to meet the complex requirements of various welding processes.

[0004] (2) For example, in the weld quality monitoring method based on machine vision, when it uses a machine learning model to detect weld defects, the monitoring scope is limited to the image data of the weld surface, and other key physical parameters of the welding process cannot be effectively obtained. It only relies on a single visual source and lacks support for different welding processes. This limitation makes the existing method poor in adapting to complex industrial environments, especially in the fusion application of multiple data sources and heterogeneous process environments.

[0005] (3) For example, the welding quality prediction method based on machine learning models, which uses machine learning models to predict the quality of welding data, relies too much on the training of historical data models, making it difficult to maintain the accuracy of predictions in the ever-changing real-time welding process. In addition, this method lacks the ability to integrate and process different types of input sources, making it difficult to flexibly apply it in multiple welding process scenarios.

[0006] (4) Although welding quality monitoring with multi-sensor data fusion has also appeared in the prior art, the multi-source data collected generally include parameters such as amplitude, pressure, and power. It lacks real-time collection and fusion of visual information from different sources and cannot support adaptive monitoring of various welding processes, resulting in deficiencies in monitoring the morphology of the welding area. Summary of the invention

[0007] In response to the problems in the prior art, the purpose of the present application is to provide a welding molten pool monitoring system and method, which can obtain the welding image to be processed from a selected welding image input source, and select a matching welding information recognition model to perform image recognition, and process the image recognition results to obtain molten pool detection parameters to monitor the welding effect, which has better applicability and deployment convenience.

[0008] The present application provides a welding pool monitoring system, comprising:

[0009] An input module, used for acquiring a welding image to be processed according to a selected welding image input source;

[0010] A processing module is used to select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, obtain a welding molten pool recognition result, and process the welding molten pool recognition result using a preset parameter algorithm to obtain a molten pool detection parameter;

[0011] The output module is used to push the melt pool detection parameters to the user.

[0012] In some embodiments, the input module is further used to receive a user's selection of a welding image input source, and the optional welding image input sources include a screen capture input source, a camera capture input source, and a network input source;

[0013] The input module obtains the welding image to be processed according to the selected welding image input source, including: the input module captures the image frame from the selected welding image input source as the welding image to be processed and stores it in the inference queue.

[0014] In some embodiments, the processing module is used to receive reasoning requirement information and select a matching welding information recognition model based on the correspondence between the reasoning requirement information and the welding information recognition model.

[0015] In some embodiments, the welding molten pool identification result includes a calibration target position and a detection target position in the welding image to be processed, the calibration target includes the welding wire and / or the tungsten electrode, and the detection target includes the molten pool;

[0016] The processing module uses the following steps to process the welding pool identification results using a preset parameter algorithm:

[0017] Determining a pixel size of the calibration target in the welding image to be processed based on the calibration target position in the welding image to be processed;

[0018] The actual size of the calibration target is obtained, and the conversion relationship from the pixel size in the welding image to be processed to the actual size is calculated according to the pixel size and the actual size of the calibration target;

[0019] The pixel size parameters of the detection target are calculated according to the detection target position, and the actual size parameters of the detection target are calculated according to the pixel size parameters and the conversion relationship as the molten pool detection parameters.

[0020] In some embodiments, the detection target also includes a groove, and the molten pool detection parameters include actual size parameters of the molten pool and actual distance parameters between the edge of the molten pool and the groove.

[0021] In some embodiments, the processing module is also used to draw a calibration target contour and a detection target contour in the welding image to be processed according to the welding pool identification result after obtaining the welding pool identification result, obtain the processed welding image, and output the processed welding image to the image output queue.

[0022] In some embodiments, the output module is further used to: display the processed welding images in the image output queue in real time, and / or,

[0023] The welding image to be processed and the processed welding image are saved as video files respectively and stored in the specified output folder.

[0024] In some embodiments, an alarm module is also included, which is used to trigger an alarm operation when it is determined that an alarm is needed based on the molten pool detection parameters and the preset parameter value range.

[0025] In some embodiments, an adaptive control module is also included for determining the type of molten pool detection parameters that exceeds the preset parameter value range when it is determined that the welding parameters need to be adjusted based on the molten pool detection parameters and the preset parameter value range, and adaptively adjusting the welding parameters based on the correspondence between the molten pool detection parameter type and the welding parameter adjustment method.

[0026] The embodiment of the present application also provides a welding pool monitoring method, which uses a welding pool monitoring system, including the following steps:

[0027] Acquire the welding image to be processed according to the selected welding image input source;

[0028] Select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, and obtain a welding pool recognition result;

[0029] The welding pool identification result is processed by a preset parameter algorithm to obtain the pool detection parameters;

[0030] Push melt pool detection parameters to users.

[0031] The welding pool monitoring system and method provided by the present application have the following advantages:

[0032] By adopting this application, the welding image to be processed can be obtained from the selected welding image input source, and the matching welding information recognition model can be selected by the processing module to perform image recognition, and the molten pool detection parameters are obtained according to the image recognition results to monitor the welding effect, and then the molten pool detection parameters are output to the user, so that the user can know the current welding status in time. This application can provide different welding information recognition models for different welding processes. Users can replace the models according to process requirements, improve the adaptability to different welding processes, and support a variety of different image input methods. Users can choose the most suitable input method according to the application scenario. Therefore, this method has better applicability and deployment convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings.

[0034] Figure 1 is a structural block diagram of a welding pool monitoring system according to an embodiment of the present application;

[0035] Figure 2 is a flow chart of a welding pool monitoring method according to an embodiment of the present application;

[0036] Figure 3 is a detailed structural block diagram of a welding pool monitoring system according to an embodiment of the present application;

[0037] Figure 4 is a detailed flow chart of a welding pool monitoring method according to an embodiment of the present application;

[0038] Figure 5 is a flow chart of capturing image frames through a camera according to an embodiment of the present application;

[0039] Figure 6 This is a flow chart of capturing and sending image frames through network transmission according to an embodiment of the present application;

[0040] Figure 7 is a flow chart of capturing and receiving image frames through network transmission according to an embodiment of the present application;

[0041] Figure 8 This is a flow chart of performing adaptive control after calculating molten pool detection parameters according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar structures, and thus their repeated description will be omitted.

[0043] like Figure 1 As shown, the present application provides a welding pool monitoring system, comprising:

[0044] The input module M100 is used to obtain the welding image to be processed according to the selected welding image input source; in this embodiment, the welding image input source is welding video data, which can be acquired by a camera set on the welding equipment or welding production line, and the image frame is captured from the video data as the welding image to be processed, and added to the reasoning queue for identification and analysis;

[0045] The processing module M200 is used to select a matching welding information recognition model, input the welding image to be processed into the welding information recognition model, obtain a welding molten pool recognition result, and process the welding molten pool recognition result using a preset parameter algorithm to obtain a molten pool detection parameter; specifically, after selecting the specified model, the processing module M200 selects the welding image to be processed from the inference queue and inputs it into the welding information recognition model to obtain the welding molten pool recognition result output by the model;

[0046] The output module M300 is used to push the molten pool detection parameters to the user; wherein, pushing to the user may be in the form of a message to the user terminal, or may be displayed on a display screen for the user to view.

[0047] By adopting the welding molten pool monitoring system of the present application, the welding image to be processed can be obtained from the selected welding image input source through the input module M100, and the matching welding information recognition model can be selected through the processing module M200 to perform image recognition, and the molten pool detection parameters are obtained according to the image recognition results to monitor the welding effect, and then the molten pool detection parameters are output to the user through the output module M300, so that the user can know the current welding status in time. The present application can provide different welding information recognition models for different welding processes. The user can replace the model according to the process requirements, improve the adaptability to different welding processes, and support a variety of different image input methods. The user can choose the most suitable input method according to the application scenario. Therefore, this method has better applicability and deployment convenience.

[0048] like Figure 2 As shown, the present application also provides a welding molten pool monitoring method, which uses a welding molten pool monitoring system, including the following steps:

[0049] S100: acquiring a welding image to be processed according to a selected welding image input source;

[0050] Specifically, an image frame of a welding video is captured from a specified welding image input source and stored in an inference queue as a welding image to be processed;

[0051] S200: selecting a matching welding information recognition model, inputting the welding image to be processed into the welding information recognition model, and obtaining a welding pool recognition result;

[0052] Specifically, after the designated model is selected, a welding image to be processed is selected from the inference queue and input into the welding information recognition model to obtain a welding pool recognition result output by the model;

[0053] S300: The welding molten pool identification result is processed using a preset parameter algorithm to obtain a molten pool detection parameter; the molten pool detection parameter is the result of real-time monitoring and detection during the welding monitoring process;

[0054] S400: Pushing the melt pool detection parameters to the user.

[0055] By adopting the welding molten pool monitoring method of the present application, the welding image to be processed can be obtained from the selected welding image input source through step S100, and the matching welding information recognition model can be selected through step S200 to perform image recognition, and the molten pool detection parameters are obtained through processing according to the image recognition result in step S300 to monitor the welding effect, and then the molten pool detection parameters are output to the user through step S400, so that the user can know the current welding status in time. The present application can provide different welding information recognition models for different welding processes, and the user can replace the model according to the process requirements, which improves the adaptability to different welding processes, and supports a variety of different image input methods. The user can choose the most suitable input method according to the application scenario, so the method has better applicability and deployment convenience.

[0056] Combine the following Figure 3 and Figure 4 The implementation method of the welding pool monitoring system and method in this embodiment is described in detail. It is to be understood that the structural components and steps shown in the drawings and the following description are only examples and are not intended to limit the scope of protection of this application.

[0057] like Figure 3 As shown, the input module M100 is also used to receive the user's selection of a welding image input source, and the optional welding image input sources include a screen capture input source, a camera capture input source, and a network input source. Figure 4As shown, in the step S100, the input module obtains the welding image to be processed according to the selected welding image input source, including: the input module selects a capture area from the selected welding image input source, continuously captures image frames, as the welding image to be processed, and stores them in the inference queue, and at the same time stores the image frames in the original video queue for subsequent original video output. The image frames stored in the original video queue and the inference queue are the same, and are subsequently used for different processing. The input module M100 includes: a screen capture unit, which is used to obtain the welding image to be processed from the screen capture input source; a camera capture unit, which is used to capture the welding image to be processed directly from the camera; and a network transmission unit, which receives the welding image to be processed through a network input source.

[0058] The screen capture unit includes: a capture area selection subunit, in which the user creates a transparent full-screen window through the Tkinter library to prompt the optional capture area, and the user presses the left mouse button to drag and select the screen area to be captured. Tkinter is Python's standard graphical user interface library, which provides the function of creating various graphic components such as windows, buttons, and text boxes, allowing developers to easily build interactive applications; a frame capture subunit, which uses the mss library to continuously capture the screen area to be captured, obtain image frames, and store them in the original video saving queue and reasoning queue. mss is a Python library for cross-platform screenshots, which can easily obtain screenshots and supports multiple operating systems; a configuration management subunit, which supports users to dynamically modify video recording configuration information, including capture frame rate, window transparency, capture frame color and width, etc.; a resource reasoning subunit, which is used to stop recording video when monitoring ends, clean up resources and release memory. When capturing image frames through the screen capture unit, first receive the user's selection of the screen area to be captured, continuously capture the screen area to be captured, obtain image frames, and store them in the original video saving queue and reasoning queue.

[0059] The camera capture unit includes: a camera initialization subunit, which is used to call the camera system library, initialize the camera device, and obtain basic camera information and configuration information; a frame capture subunit, which is used to continuously obtain image frames from the camera and store them in the original video preservation queue and the inference queue; a camera configuration module, which is used to configure camera parameters according to user operations. The configurable camera parameters include, for example, analog gain, exposure time, contrast, gamma value, rotation angle, etc.; a resource cleaning subunit, which is used to release camera resources and clear the queue when the monitoring ends. Figure 5 is a flow chart of capturing image frames through a camera according to an embodiment of the present application. Figure 5 As shown, when the camera capture unit is used to capture image frames, the camera (camera device) parameters are first set, the camera is started, and the image frames are continuously captured through the camera and placed into the inference queue and the original video queue.

[0060] The network transmission unit includes: an RTSP server subunit, which is used to start or stop the RTSP (Real-Time Streaming Protocol) server, and to turn on or off the function of transmitting image frames through the network; a capture area selection subunit, which is used to call the camera through the capture area selection subunit of the direct screen capture module or directly call the camera; a frame capture subunit, which is used to call the camera through the frame capture subunit of the direct screen capture module or directly call the camera, and the captured image frames are directly passed to the FFmpeg (an open source multimedia processing tool) push stream subunit; the FFmpeg push stream subunit is used to use FFmpeg to send the image frames to a specified URL (Uniform Resource Locator) address in the form of a video stream through the RTSP protocol; and a resource cleanup subunit, which is used to stop capturing, close the RTSP server and clean up resources. Figure 6 This is a flowchart of capturing and sending image frames through network transmission according to an embodiment of the present application. Figure 7 FIG. 1 is a flowchart of capturing and receiving image frames through network transmission according to an embodiment of the present application. Figure 6 and Figure 7 As shown, using a network transmission unit to capture image frames includes two processes: sending at the sending end and receiving at the receiving end. The sending end includes: starting the RTSP server, inputting the transmission address and capture frame rate, determining the capture area, continuously capturing image frames, and starting FFmpeg to push the image frames to the specified URL address. The receiving end includes: starting the receiving thread, trying to establish a connection with the specified URL address, connecting to the specified address, obtaining a video stream including image frames from the specified address, decoding the video stream, and storing the decoded video frames in the inference queue and the original video queue.

[0061] like Figure 3As shown, the processing module M200 includes a frame selection unit, a model selection unit, an inference unit, a mask extraction unit, a calculation unit and a contour drawing unit. The frame selection unit is used to obtain the welding image to be processed from the inference queue for subsequent unit processing. The model selection unit is used to determine the selection of the welding information recognition model. The optional welding information recognition model includes, for example, a PT model, an ONNX model, an OpenVINO model, and the like. The PT model is a PT (PyTorch) model file generated by direct training using YOLOv8n-seg, which has the advantages of rapid development and debugging. Its applicable scenarios are: suitable for environments with strong GPU support, facilitating model training and testing. The ONNX model is a PT model converted into an ONNX (Open NeuralNetwork Exchange, an open neural network exchange format) model file, which has good cross-platform compatibility, and its applicable scenarios are: suitable for situations where deployment on multiple hardware devices is required, such as CPUs, GPUs, and mobile devices. The OpenVINO model is a ONNX model converted into an OpenVINO model file, which is optimized for Intel hardware and has a faster inference speed. Its applicable scenarios are: suitable for scenarios requiring high-performance real-time inference, especially in Intel hardware environments. The monitoring system pre-stores a variety of trained welding information recognition models. The welding information recognition model is a machine learning model. It is iteratively trained by collecting sample image data and performing target labeling, so that the welding information recognition model can accurately identify the target to be detected in the image.

[0062] Specifically, the model selection unit is used to receive the reasoning requirement information, and select the matching welding information recognition model according to the correspondence between the reasoning requirement information and the welding information recognition model. The reasoning requirement information can be input by the user, or can be automatically detected by the monitoring system. For example, the reasoning requirement information can be a welding process type (metal inert gas shielded welding MIG, tungsten inert gas shielded arc welding TIG, etc.), and the monitoring system pre-stores the correspondence between the welding information recognition model types corresponding to various welding process types, that is, the information of the model type selected for each welding process type. The model selection unit can receive the welding process type input by the user and match the corresponding model for it. The reasoning requirement information can also be hardware environment information. For example, when the user inputs or the monitoring system detects that the current GPU meets certain performance requirements, the PT model is selected. When the user inputs or the monitoring system detects that it needs to be deployed on multiple hardware devices, the ONNX model is selected. The user can also directly select a certain model according to the needs, that is, the reasoning requirement information can also be directly the welding information recognition model type selected by the user.

[0063] The welding information recognition model corresponding to each welding process type is a deep learning model trained based on the sample data under the welding process type. For example, for MIG welding, the welding images in the historical MIG welding process are collected in advance as sample images, and the molten pool position, welding wire position, groove position, etc. are manually marked in the sample image as the label of the sample image. The prediction is made by the deep learning model dedicated to MIG, and the loss function value is obtained according to the predicted molten pool position, welding wire position, groove position, etc. and the label of the sample image. The deep learning model is trained by reverse iteration based on the loss function value, and the trained deep learning model dedicated to MIG is obtained. The model can be distinguished from other models by numbering or naming it. Similarly, for TIG welding, the welding images in the historical TIG welding process can also be collected in advance as sample images, and a similar method is used to train to obtain a trained deep learning model dedicated to TIG welding. When constructing and training the model, the deep learning models of different welding process types can use the same model type, or different model types can be used according to the needs of different welding monitoring.

[0064] The inference unit is used to input the image to be processed obtained by the frame selection unit into the specified model selected by the model selection unit to obtain the welding pool recognition result output by the model. The welding pool recognition result includes the calibration target position and the detection target position in the welding image to be processed. The calibration target includes the welding wire and / or tungsten electrode, and the detection target includes the molten pool. The calibration target is used to calibrate the conversion relationship from the pixel size to the actual size in the welding image to be processed. Since the actual size of the welding wire and / or tungsten electrode is known, the conversion relationship can be calibrated by its pixel size in the image. The following is an explanation using the calibration target including the welding wire as an example. The mask extraction unit is used to extract the position mask information of the calibration target and the detection target in the image from the molten pool recognition result output by the model for use by the calculation unit and the contour drawing unit.

[0065] The calculation unit is used to process the welding pool identification result using a preset parameter algorithm in the following steps:

[0066] Determining a pixel size of the calibration target in the welding image to be processed based on the calibration target position in the welding image to be processed;

[0067] The actual size of the calibration target is obtained, and the conversion relationship from the pixel size in the welding image to be processed to the actual size is calculated according to the pixel size and the actual size of the calibration target;

[0068] The pixel size parameters of the detection target are calculated according to the detection target position, and the actual size parameters of the detection target are calculated according to the pixel size parameters and the conversion relationship as the molten pool detection parameters.

[0069] In this embodiment, the detection target also includes the groove, and the molten pool detection parameters include the actual size parameters of the molten pool and the actual distance parameters between the edge of the molten pool and the groove. The actual size parameters of the molten pool include, for example, the area, length, and width of the molten pool, and the actual distance parameters between the edge of the molten pool and the groove include, for example, the distances from the upper and lower edges of the molten pool to the upper and lower grooves.

[0070] Taking the calibration target including welding wire and the detection target including molten pool and groove as an example, the calculation process of the calculation module is specifically described as follows:

[0071] (1) Calculation of conversion relationship:

[0072] Through model detection, the bounding box of the welding wire is obtained, and its pixel height in the vertical direction (i.e. corresponding to the diameter of the welding wire) is extracted.

[0073] Pixel size of welding wire L pixels The calculation formula is: pixels =y max -y min

[0074] Among them, y max and min are the pixel coordinates of the lower and upper edges of the welding wire bounding box, respectively.

[0075] Get the actual size L of the welding wire real (equal to the actual wire diameter);

[0076] The conversion ratio from the pixel size in the weld image to be processed to the actual size is calculated as follows:

[0077]

[0078] Where R is the conversion ratio from pixels to millimeters, and the unit is millimeters / pixel.

[0079] (2) Calculation of molten pool area

[0080] Through model recognition, the mask area (mask) of the molten pool is obtained, and the polygonal contour coordinates of the molten pool are extracted;

[0081] The pixel area of ​​the melt pool is calculated using the polygon area calculation formula (Shoelace formula) as follows:

[0082]

[0083] Among them, (x i ,y i ) are the coordinates of the polygon vertices, n is the number of vertices of the polygon, and x n+1 =x1,y n+1 =y1.

[0084] The pixel area of ​​the melt pool is converted into the actual physical area as follows:

[0085] A real =A pixels ×R 2

[0086] Among them, A real is the actual physical area of ​​the molten pool, in millimeters squared.

[0087] (3) Calculation of molten pool length and width

[0088] Through model detection, the bounding box of the molten pool is obtained and its pixel length L in the horizontal direction is extracted pool_pixels and vertical direction W pool_pixel The pixel width is calculated as follows:

[0089] L po_pixe =x max -x min

[0090] W pool_pixels =y max -y min

[0091] Among them, x max and x min is the pixel coordinates of the left and right edges of the melt pool bounding box, y max and min are the pixel coordinates of the upper and lower edges of the melt pool bounding box.

[0092] Use the following formula to convert pixel length and width into actual physical length and width:

[0093] L pool_real =L pool_pixels ×R

[0094] W pool_rea =W pool_pixels ×R

[0095] Among them, L pool_real and W pool_real are the actual length and width of the molten pool, in millimeters.

[0096] (4) Calculation of the distance from the upper and lower edges of the molten pool to the upper and lower slopes

[0097] According to the model recognition results, the key point coordinates of the molten pool and the groove are obtained. The pixel distance D from the upper edge of the molten pool to the groove is calculated. to_pixels The pixel distance D from the lower edge of the molten pool to the groove bottom_pixel as follows:

[0098] D top_pixels =ygroove_top -y pool_top

[0099] D bottom_pixel =y pool_bottom -y groove_bottom

[0100] Among them, y groove_top and groove_bottom is the pixel coordinates of the upper and lower edges of the groove, y pool_top and pool_bottom are the pixel coordinates of the upper and lower edges of the melt pool.

[0101] The following formula is used to convert pixel distance to actual physical distance:

[0102] D to_real =D top_pixels ×R

[0103] D bottom_re =D bottom_pixels ×R

[0104] Among them, D to_rea and D bottom_real They are the actual distances from the upper and lower edges of the molten pool to the upper and lower bevels, respectively, in millimeters.

[0105] The contour drawing unit is used to draw the calibration target contour and the detection target contour in the welding image to be processed according to the welding pool recognition result after obtaining the welding pool recognition result (such as drawing the contours of the welding pool, welding wire, tungsten electrode, groove, etc. in the welding image to be processed according to the extracted mask information), obtain the processed welding image, and output the processed welding image to the image output queue. Figure 4 As shown, in this embodiment, the image output queue includes a real-time display queue and an inference video storage queue, which are used by subsequent output modules.

[0106] like Figure 3 As shown, the output module M300 includes a real-time display unit, a video storage unit and a parameter output unit. The parameter output unit is used to output the calculated key parameters such as the length, width, area of ​​the molten pool, and the distances from the upper and lower edges of the molten pool to the upper and lower bevels. The real-time display unit is used to display the processed welding images in the image output queue in real time. Specifically, the real-time display unit is used to obtain and display image frames from the real-time display queue at a set frame rate, and the user can monitor the current welding status in real time through the display screen.

[0107] The video saving unit is used to save the welding image to be processed and the processed welding image as video files respectively, and store them in the specified output folder. Among them, the video saving unit is used to obtain image frames from the inference video saving queue and the original video saving queue respectively, save them as video files in MP4 format, and store them in the specified output folder. The video saving unit can be further divided into: configuration management subunit, folder management subunit, video writing subunit, storage monitoring subunit, and resource cleaning subunit. The configuration management subunit is used to set the relevant configuration of the output video, including the output folder path, output frame rate, maximum size of a single video (MB), total video size limit (GB), etc. The folder management subunit is used to create a subfolder in the specified output folder according to the current date (year-month-day) to save the output video. The video naming method is the timestamp of the first frame captured (year-month-day-hour-minute-second). The video writing subunit is used to use the video writer to write the image frame into a video file in MP4 format at the set frame rate. The storage monitoring subunit is used to monitor the total size of a single video file and the output folder in real time to ensure the reasonable use of storage space. The resource cleanup subunit is used to stop video writing, close the video writer, clear the frame queue and release resources after monitoring stops. Among them, the video monitoring subunit is used to monitor the total size of a single video file in real time, and determine whether the size of the generated single video file exceeds the preset maximum size of a single video. If so, the video file is split into at least two new video files before saving the output video. The video monitoring subunit is also used to monitor the total size of videos already stored in the output folder in real time. If the total size of the videos already stored is greater than the total video size limit, another storage location is changed. If there is no other storage location, the user is prompted to delete some videos, or the historical videos with the earliest storage time are deleted according to the rules preset by the user. Therefore, the welding pool monitoring system supports real-time video storage, and can automatically allocate and monitor storage space to avoid the problem of insufficient storage space.

[0108] like Figure 3 As shown, the welding pool monitoring system also includes an alarm module M400, which is used to trigger an alarm operation when it is determined that an alarm is needed based on the molten pool detection parameters and the preset parameter value range. When the molten pool detection parameters include the molten pool area, molten pool length, molten pool width, and the distance between the molten pool edge and the groove. Figure 4 As shown, after the molten pool detection parameters are obtained through the processing module, a standard parameter value range is pre-set for each parameter, and the detected value is compared with the corresponding parameter value range. If the detected value is too high or too low, an alarm operation is triggered. This triggering of the alarm operation can be to display an alarm prompt on the display screen, or to send an alarm message to the user terminal, etc.

[0109] like Figure 3As shown, the welding pool monitoring system also includes an adaptive control module M500, which is used to determine the type of molten pool detection parameter that exceeds the preset parameter value range when it is determined that the welding parameters need to be adjusted according to the molten pool detection parameter and the preset parameter value range, and adaptively adjust the welding parameters according to the corresponding relationship between the molten pool detection parameter type and the welding parameter adjustment method. Figure 8 As shown, after the welding molten pool identification result is processed by a preset parameter algorithm to obtain the molten pool detection parameter, the following steps are also included: judge each molten pool detection parameter and compare it with the threshold value in the preset parameter value range (including the upper boundary threshold value and / or the lower boundary threshold value of the parameter value range). If at least one molten pool detection parameter exceeds the preset parameter value range, the parameter type exceeding the preset parameter value range is used as an influencing factor, and the situation is classified according to the influencing factor, which is divided into the molten pool area not meeting the requirements, the molten pool length not meeting the requirements, the molten pool width not meeting the requirements, and the distance between the molten pool and the groove not meeting the requirements. According to the classification situation, the corresponding welding parameter type that needs to be adjusted is selected, and the welding parameter is adjusted according to the size relationship between the molten pool detection parameter that exceeds the range and the corresponding preset parameter value range. For example, when the welding molten pool area is too large, the welding current is reduced, and when the welding molten pool area is too small, the welding current is increased. When the distance between the molten pool and the groove exceeds the corresponding distance range, the welding gun position is adjusted. Other adjustable welding parameters include, for example, welding speed, down force, etc. Through the adaptive adjustment of the welding parameters, the molten pool shape, size and position are adjusted to within the set parameter value range. The present application automatically realizes real-time quality control by adjusting welding parameters in a closed loop according to real-time feedback of molten pool detection parameters, dynamically optimizes welding parameters according to actual welding results, and continuously adapts to changes in the welding process during real-time control.

[0110] The following further introduces the implementation of the welding pool monitoring system and method with reference to specific examples.

[0111] Example 1: The system and method are used for real-time monitoring and quality control of MIG welding process (metal inert gas shielded welding, copper welding wire). The system and method are particularly suitable for industrial welding scenarios with high precision and high stability requirements.

[0112] (1) System initialization

[0113] The input module selects camera capture as the image input method. Initialize the camera and set parameters such as frame rate and resolution. The processing module loads the pre-trained MIG welding-specific deep learning model. The output module sets the real-time display frame rate and video storage path.

[0114] (2) Image capture and data transmission

[0115] The camera is started to capture image frames of the molten pool area in real time. The image frames captured by the camera are passed into the inference queue for further processing.

[0116] (3) Image processing

[0117] The frame selection unit obtains the current image frame from the inference queue. The model selection unit selects the loaded MIG welding pre-trained model for inference analysis. The inference unit extracts features of the molten pool area based on the deep learning model, including parameters such as the area, length, and width of the molten pool. The mask extraction unit extracts masks of the molten pool boundary, welding wire, tungsten electrode, etc. to accurately divide each area.

[0118] (4) Quality parameter calculation and threshold determination

[0119] The calculation unit calculates the key quality parameters of the molten pool (such as the molten pool area, width, length, distance between the molten pool and the groove, etc.) based on the detection results. The alarm module determines whether the calculation result exceeds the preset quality threshold. If exceeded, the system will trigger an alarm and enter the adaptive control process.

[0120] (5) Real-time feedback and quality control

[0121] If a parameter exceeds the threshold, the system will automatically adjust the relevant welding parameters, such as welding current or welding speed, to restore the parameters to the normal range. After adjustment, the welding parameters are monitored in real time to ensure that the adjustment effect meets expectations.

[0122] (6) Video storage and display

[0123] The real-time display unit displays the processed image frames on the monitoring screen to help operators conduct real-time monitoring. The video storage unit saves the original video and inference video to a specified path for subsequent analysis and quality traceability.

[0124] Through the application of this example, the welding pool monitoring system can realize real-time monitoring and quality control of the MIG welding process. When it is detected that the molten pool parameters are out of the standard range, the system can automatically adjust the welding parameters, thereby improving the stability and consistency of welding quality, reducing human intervention, and improving welding efficiency.

[0125] Example 2: The welding pool monitoring system and method are applied to remote monitoring of the welding process based on network transmission, which is suitable for remote welding monitoring needs, especially for collaborative monitoring of the welding process at multiple locations.

[0126] (1) System initialization

[0127] The input module selects network transmission as the image input method. The receiving end is connected through the RTSP protocol to ensure the stability of network transmission. The processing module loads the inference model suitable for the welding process. The output module sets the frame rate and storage path for remote monitoring.

[0128] (2) Image acquisition and streaming

[0129] The frame capture subunit uses a camera or screenshot to continuously capture frames and transmit them to FFmpeg for streaming. The FFmpeg streaming subunit transmits the image frames to the remote monitoring center in the form of video streams to achieve real-time monitoring of multiple locations.

[0130] (3) Real-time reasoning and quality analysis

[0131] The inference unit analyzes the molten pool detection parameters in the image based on the deep learning model and feeds back the analysis results to the monitoring interface in real time. The key molten pool detection parameters and welding monitoring parameters (such as temperature and speed) are monitored in real time to determine whether they exceed the threshold.

[0132] (4) Remote alarm and adjustment

[0133] When the monitoring result exceeds the threshold, the system will notify the remote operator through the alarm function and record the alarm information. If the system is equipped with automatic control function, the system will automatically adjust the welding parameters, otherwise the operator will adjust them remotely.

[0134] (5) Video storage

[0135] Save the real-time video stream transmitted over the network and the processed video stream to the output folder for subsequent analysis and quality tracing.

[0136] In summary, the welding pool monitoring system and method of this embodiment have the following beneficial effects:

[0137] (1) In order to solve the problem that the existing welding monitoring system usually only supports a single input method, lacks sufficient flexibility, and is difficult to adapt to the needs of various application scenarios, the present application can obtain the welding image to be processed from the selected welding image input source, and supports three image input methods: screen capture, camera capture, and network transmission. Users can choose the appropriate input source according to the actual scene requirements. It has better applicability and deployment convenience, and increases the flexibility of the welding pool monitoring system in different industrial scenarios.

[0138] (2) In order to solve the problem that most existing welding monitoring systems are limited to a single type of welding process and lack the ability to adapt to multiple welding processes, this application can select a matching welding information recognition model for image recognition, and process the image recognition results to obtain molten pool detection parameters to monitor the welding effect. This application can quickly switch between various trained models and adapt to a variety of different welding processes, thereby adapting to different process requirements. This function avoids the limitation of a single process and greatly improves the application breadth and flexibility of the welding molten pool monitoring system.

[0139] (3) In order to solve the problem of insufficient real-time performance in weld quality assessment and process control in existing welding monitoring systems, this application uses a machine learning model to perform real-time reasoning on the welding image to be processed with a molten pool area, and monitors various molten pool detection parameters in real time to meet the quality assessment and feedback control requirements in the welding process, ensuring that various molten pool detection parameters in the welding process (such as molten pool area, length, width, position distance between molten pool and groove, etc.) are within the standard range. This real-time performance can detect welding anomalies in a timely manner and improve the efficiency and accuracy of welding quality control. By dynamically and adaptively adjusting the welding parameters according to the molten pool detection parameters, no manual adjustment is required, and human intervention is reduced. In combination with deep learning, intelligent feedback and automatic adjustment are realized, and targeted control is provided in a variety of welding processes, which improves the stability, automation and intelligence level of the welding process. The system uses deep learning and online incremental learning technology to perform real-time analysis of key image signals in the welding process, and combines pre-trained models to achieve rapid adaptation to different welding processes, ensuring the stability and high-quality output of the welding process.

[0140] (4) The welding pool monitoring system supports real-time video storage and can automatically allocate storage space to avoid the problem of insufficient storage space. It also supports independent storage of original videos and inferenced videos to facilitate future analysis and traceability.

[0141] (5) Based on machine vision and deep learning algorithms, the system conducts real-time monitoring and quality assessment of the molten pool area during the welding process, and is widely used in welding quality control in aerospace, automobile manufacturing, shipbuilding and other industrial fields. The system makes up for the shortcomings of existing technologies, performs well in data fusion and multi-modal processing, is suitable for multiple welding processes and complex industrial environments, and provides a new and intelligent solution for welding quality control in industrial production processes.

[0142] The above content is a further detailed description of the present application in combination with specific preferred implementation methods, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. For ordinary technicians in the technical field to which the present application belongs, several simple deductions or substitutions can be made without departing from the concept of the present application, which should be deemed to fall within the scope of protection of the present application.

Claims

1. A welding pool monitoring system, characterized in that: include: An input module, used for acquiring a welding image to be processed according to a selected welding image input source; A processing module, used for selecting a matching welding information recognition model, inputting the welding image to be processed into the welding information recognition model, obtaining a welding molten pool recognition result, and processing the welding molten pool recognition result using a preset parameter algorithm to obtain a molten pool detection parameter; An output module is used to push the molten pool detection parameters to the user.

2. The welding pool monitoring system according to claim 1, characterized in that: The input module is also used to receive a user's selection of a welding image input source, and the optional welding image input sources include a screen capture input source, a camera capture input source, and a network input source; The input module acquires the welding image to be processed according to the selected welding image input source, including: the input module captures image frames from the selected welding image input source as the welding image to be processed, and stores them in the inference queue.

3. The welding pool monitoring system according to claim 1, characterized in that: The processing module is used to receive reasoning requirement information and select a matching welding information recognition model according to the corresponding relationship between the reasoning requirement information and the welding information recognition model.

4. The welding pool monitoring system according to claim 1, characterized in that: The welding molten pool recognition result includes a calibration target position and a detection target position in the welding image to be processed, the calibration target includes the welding wire and / or the tungsten electrode, and the detection target includes the molten pool; The processing module uses the following steps to process the welding pool identification result using a preset parameter algorithm: Determining a pixel size of the calibration target in the welding image to be processed based on the calibration target position in the welding image to be processed; Acquiring the actual size of the calibration target, and calculating a conversion relationship from the pixel size in the welding image to be processed to the actual size according to the pixel size and the actual size of the calibration target; The pixel size parameters of the detection target are calculated according to the detection target position, and the actual size parameters of the detection target are calculated according to the pixel size parameters and the conversion relationship as the molten pool detection parameters.

5. The welding pool monitoring system according to claim 4, characterized in that: The detection target also includes a groove, and the molten pool detection parameters include actual size parameters of the molten pool and actual distance parameters between the edge of the molten pool and the groove.

6. The welding pool monitoring system according to claim 4, characterized in that: The processing module is also used to draw a calibration target contour and a detection target contour in the welding image to be processed according to the welding pool identification result after obtaining the welding pool identification result, obtain a processed welding image, and output the processed welding image to the image output queue.

7. The welding pool monitoring system according to claim 6, characterized in that: The output module is also used to: display the processed welding images in the image output queue in real time, and / or, The welding image to be processed and the processed welding image are saved as video files respectively, and stored in a designated output folder.

8. The welding pool monitoring system according to claim 1, characterized in that: It also includes an alarm module, which is used to trigger an alarm operation when it is determined that an alarm is needed based on the molten pool detection parameters and a preset parameter value range.

9. The welding pool monitoring system according to claim 1, characterized in that: It also includes an adaptive control module, which is used to determine the type of molten pool detection parameter that exceeds the preset parameter value range when it is determined that the welding parameters need to be adjusted based on the molten pool detection parameter and the preset parameter value range, and adaptively adjust the welding parameters based on the correspondence between the molten pool detection parameter type and the welding parameter adjustment method.

10. A welding pool monitoring method, characterized in that: The welding pool monitoring system according to any one of claims 1 to 9 comprises the following steps: Acquire the welding image to be processed according to the selected welding image input source; Selecting a matching welding information recognition model, inputting the welding image to be processed into the welding information recognition model, and obtaining a welding pool recognition result; The welding molten pool identification result is processed using a preset parameter algorithm to obtain a molten pool detection parameter; The melt pool detection parameters are pushed to the user.

Images