Intelligent welding control method and system based on image vision

Abstract

The invention relates to an intelligent welding control method and system based on image vision. The intelligent welding control method comprises the following steps that a current first image and a current second image are obtained; preprocessing the first image and the second image to obtain a preprocessed image; loading the prediction model; performing prediction based on the prediction model and the preprocessed image to obtain a prediction control command and the confidence of the prediction control command; judging whether feedback control needs to be executed based on the confidence coefficient of the predictive control command; if yes, a prediction control command is issued to the lower computer control system to execute feedback control; and if not, the prediction control command is not issued. Intelligent control over welding can be achieved, real-time monitoring and closed-loop control over welding are conducted, meanwhile, intelligent welding control is conducted through image vision, the problem of sensor technology limitation can be effectively solved, and the welding quality and the welding efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline welding, and more particularly to an intelligent welding control method and system based on image vision. Background Art

[0002] Traditional pipeline welding technology mainly relies on manual operation or preset program control, which has the following defects:

[0003] Insufficient real-time monitoring capabilities: Manual welding relies on the operator's experience to judge the state of the molten pool, which can easily lead to welding defects (such as lack of penetration and porosity) due to visual fatigue or environmental interference; and fixed program systems cannot make adaptive adjustments based on dynamic working conditions such as pipeline deformation and weld offset.

[0004] Limitations of sensor technology: Existing automation systems mostly use single laser sensors or infrared detection devices, which are highly sensitive to ambient light (for example, lasers are easily interfered by dust), making it difficult to fully capture the morphology of the molten pool and the dynamic characteristics of the arc.

[0005] Lack of closed-loop control: Existing technologies lack real-time feedback and parameter optimization of the welding process, resulting in unstable welding quality and high rework rate. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an intelligent welding control method and system based on image vision in response to the problems existing in the prior art.

[0007] The technical solution adopted by the present invention to solve the technical problem is to construct an intelligent welding control method based on image vision, which includes the following steps:

[0008] Get the current first image and second image;

[0009] Preprocessing the first image and the second image to obtain preprocessed images;

[0010] Load the prediction model;

[0011] Performing prediction based on the prediction model and the preprocessed image to obtain a predicted control command and a confidence level of the predicted control command;

[0012] determining whether feedback control needs to be performed based on the confidence level of the predicted control command;

[0013] If so, the predictive control command is sent to the lower computer control system to perform feedback control;

[0014] If not, the prediction control command will not be issued.

[0015] In the image vision-based intelligent welding control method of the present invention, preprocessing the first image and the second image to obtain the preprocessed image includes:

[0016] Performing left-right stitching on the first image and the second image to obtain a stitched image;

[0017] Filling is performed on the spliced ​​image to obtain the preprocessed image.

[0018] In the image vision-based intelligent welding control method of the present invention, the determining whether feedback control needs to be performed based on the confidence level of the predicted control command includes:

[0019] comparing the confidence level of the predicted control command with a control threshold;

[0020] If the confidence level of the predicted control command is greater than the control threshold, it is determined that feedback control needs to be performed;

[0021] If the confidence level of the predicted control command is less than the control threshold, it is determined that feedback control does not need to be performed.

[0022] In the image vision-based intelligent welding control method of the present invention, the step of sending the prediction control command to the lower control system to perform feedback control includes:

[0023] The upper computer monitoring system sends the prediction control command to the lower computer control system;

[0024] The lower computer control system sends a control signal to the welding robot body according to the prediction control command;

[0025] The welding robot body performs a welding operation according to the control signal.

[0026] In the image vision-based intelligent welding control method of the present invention, the method further includes:

[0027] The host computer monitoring system starts timing after issuing the prediction control command;

[0028] Determine whether the set control time interval has been reached based on the timing result;

[0029] If the set control time interval is reached, prediction and control are performed again based on the real-time image;

[0030] If the set control time interval is not reached, no action is performed.

[0031] In the image vision-based intelligent welding control method of the present invention, the method further includes:

[0032] During the feedback control process, the lower computer control system generates the control signal according to the minimum displacement based on the predictive control command.

[0033] In the image vision-based intelligent welding control method of the present invention, before acquiring the current first image and the second image, the method includes:

[0034] Acquire historical data of different welding conditions; the historical data includes: first image historical data and second image historical data under different welding conditions;

[0035] Preprocessing the first image history data and the second image history data respectively to obtain training image data under different welding conditions;

[0036] Acquire the same amount of normal image data under normal welding conditions;

[0037] Cleaning the training image and the normal image data to obtain a cleaned training image;

[0038] Classification model training is performed on the cleaned training image to obtain the prediction model.

[0039] The present invention also provides an intelligent welding control system based on image vision, comprising:

[0040] A visual system, wherein the visual system is used to obtain the current first image and the second image;

[0041] The host computer monitoring system is used to:

[0042] Preprocessing the first image and the second image to obtain preprocessed images;

[0043] Load the prediction model;

[0044] Performing prediction based on the prediction model and the preprocessed image to obtain a predicted control command and a confidence level of the predicted control command;

[0045] determining whether feedback control needs to be performed based on the confidence level of the predicted control command;

[0046] If so, the predictive control command is sent to the lower computer control system to perform feedback control;

[0047] If not, the prediction control command will not be issued.

[0048] The image-vision-based intelligent welding control system of the present invention further includes: a welding robot body; the visual system includes: a first image acquisition device and a second image acquisition device;

[0049] The first image acquisition device is provided at the front end of the welding head of the welding robot body, and is used to acquire the first image to obtain the first image;

[0050] The second image acquisition device is arranged at the rear end of the welding head of the welding robot body, and is used to acquire the second image to obtain the second image.

[0051] The image vision-based intelligent welding control system of the present invention further includes: a lower computer control system, a welding power supply, and a control handle;

[0052] The welding power supply is used to provide power to the lower computer control system and the welding robot body;

[0053] The control handle is used to send manual operation commands to the lower computer control system;

[0054] The lower computer control system is used to generate a control signal according to the manual operation command or the predictive control system, and send the control signal to the welding robot body.

[0055] The implementation of the image-based intelligent welding control method and system of the present invention has the following beneficial effects: it includes the following steps: acquiring the current first image and the second image; preprocessing the first image and the second image to obtain a preprocessed image; loading a prediction model; performing a prediction based on the prediction model and the preprocessed image to obtain a prediction control command and the confidence level of the prediction control command; judging whether feedback control needs to be performed based on the confidence level of the prediction control command; if so, issuing a prediction control command to the lower-level control system to perform feedback control; if not, not issuing the prediction control command. The present invention can realize intelligent control of welding, and perform real-time monitoring and closed-loop control of welding. At the same time, it uses image vision for intelligent welding control, which can effectively solve the problem of sensor technology limitations and improve welding quality and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0057] Figure 1 This is a flow chart of an embodiment of an intelligent welding control method based on image vision provided by the present invention;

[0058] Figure 2 1 is a flow chart of another embodiment of the image vision-based intelligent welding control method provided by the present invention;

[0059] Figure 3 This is a logic block diagram of an embodiment of an image-based vision intelligent welding control system provided by the present invention;

[0060] Figure 4 is a schematic diagram of a preprocessed image provided by the present invention;

[0061] Figure 5 This is an operation logic diagram of the image vision-based intelligent welding control system provided by the present invention. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0063] refer to Figure 1 In a preferred embodiment, the image vision-based intelligent welding control method includes the following steps:

[0064] Step S100: Acquire the current first image and second image.

[0065] Optionally, in an embodiment of the present invention, the first image can be obtained by a first image acquisition device 201 installed at the front end of the head of the welding robot body 10, and the second image can be obtained by a second image acquisition device 202 installed at the rear end of the head of the welding robot body 10.

[0066] Step S200: pre-processing the first image and the second image to obtain a pre-processed image.

[0067] Optionally, in an embodiment of the present invention, preprocessing the first image and the second image to obtain the preprocessed image includes: splicing the first image and the second image left and right to obtain a spliced ​​image; and filling the spliced ​​image to obtain the preprocessed image. Specifically, when splicing horizontally, the order of the first image and the second image is randomly adjusted, and then the letter-box operation is used to fill the top and bottom of the spliced ​​image with black, as shown in the following example. Figure 4 As shown. Among them, Figure 4 The front image in the image is the first image, and the back image is the second image.

[0068] Step S300: Load the prediction model.

[0069] Optionally, in an embodiment of the present invention, the prediction model is a model pre-stored in a memory or database, and the prediction model includes a mapping relationship between welding images and control commands. The mapping relationship between welding images and control commands is a one-to-one mapping relationship between welding images and control commands. For example, an OSC left offset corresponds to a welding image, and an OSC right offset corresponds to a welding image. The welding images here are images that have been spliced ​​and cleaned. Furthermore, in an embodiment of the present invention, the prediction model is stored in a memory or database in the form of a file, and when it needs to be called, it can be directly loaded from the memory or database. Optionally, the prediction model can use the ONNX file format.

[0070] Step S400: performing prediction based on the prediction model and the pre-processed image to obtain a predicted control command and a confidence level of the predicted control command.

[0071] Specifically, through this prediction model, after the currently acquired first image and second image are preprocessed to obtain a preprocessed image, the preprocessed image can be directly input into the prediction model. The prediction model is matched and trained based on the mapping relationship between the welding image and the control command, and the control command corresponding to the preprocessed image can be quickly obtained, that is, the predicted control command is output, and the confidence corresponding to the predicted control command is given.

[0072] Step S500: Determine whether feedback control needs to be performed based on the confidence level of the predicted control command.

[0073] Optionally, in some embodiments, determining whether feedback control needs to be executed based on the confidence of the predicted control command includes: comparing the confidence of the predicted control command with a control threshold; if the confidence of the predicted control command is greater than the control threshold, determining that feedback control needs to be executed; if the confidence of the predicted control command is less than the control threshold, determining that feedback control does not need to be executed.

[0074] Step S600: If yes, a prediction control command is issued to the lower computer control system 40 to perform feedback control.

[0075] Optionally, in some embodiments, sending a predictive control command to the lower computer control system 40 to perform feedback control includes: the upper computer monitoring system 30 sends a predictive control command to the lower computer control system 40; the lower computer control system 40 sends a control signal to the welding robot body 10 according to the predictive control command; and the welding robot body 10 performs a welding operation according to the control signal.

[0076] During the feedback control process, the lower-level control system 40 generates a control signal based on the minimum displacement according to the predicted control command. That is, after receiving the predicted control command issued by the upper-level monitoring system 30, the lower-level control system 40 performs feedback control according to the predicted control command. During the feedback control process, the lower-level control system 40 outputs a control signal corresponding to the minimum displacement each time, and sends the control signal to the welding robot body 10, so that the welding robot body 10 can perform the welding operation according to the minimum displacement. The minimum displacement can achieve the minimum accuracy of the mechanical structure or be set according to actual conditions. Each feedback control is only a type of feedback control operation, and the displacement of each operation needs to be set according to the actual control prediction, while taking into account both accuracy and speed requirements.

[0077] Furthermore, in an embodiment of the present invention, the host computer monitoring system 30 begins timing after issuing a predictive control command; determines whether a set control time interval has been reached based on the timing result; if the set control time interval has been reached, prediction and control based on the real-time image are performed again; if the set control time interval has not been reached, no action is taken. That is, after issuing a predictive control command, the host computer monitoring system 30 needs to wait and determine whether the control time interval (i.e., the shortest interval between two intelligent controls) has been reached. If the control time interval has not been reached, the host computer no longer issues predictive control commands, and the intelligent feedback control does not react until the interval has expired (i.e., the control time interval has been reached), at which point prediction is performed again and an action is taken based on the predictive control command, such as executing feedback control or not executing feedback control.

[0078] Step S700: If not, no prediction control command is issued.

[0079] Furthermore, in an embodiment of the present invention, before performing prediction (ie, before obtaining the current first image and the second image), it is necessary to first train the prediction model using historical data.

[0080] Specifically, such as Figure 2 As shown, the training of the prediction model includes the following steps:

[0081] Step S201: Acquire historical data of different welding conditions.

[0082] The historical data includes: first image historical data and second image historical data under different welding conditions.

[0083] Specifically, different welding conditions may include, but are not limited to, installing offset welding machine pipes, setting abnormal welding tracks, etc. A welding condition that requires welder control can be generated by manually interfering with the welding machine, or historical data can be extracted from a large number of welding scenarios. For example, during the historical data acquisition phase, the welder can control the welding through the host computer monitoring system 30 or the buttons on the control handle 50 to output the corresponding control time and control command. At the same time, the control time and control command are recorded by the host computer monitoring system 30, and the host computer monitoring system 30 synchronously acquires the first image historical data and the second image historical data transmitted by the first image acquisition device 201 and the second image acquisition device 202 through the visual system, and finds the visual image corresponding to the time when the control command was issued through the timestamp.

[0084] Step S202: pre-processing the first image history data and the second image history data respectively to obtain training image data under different welding conditions.

[0085] Specifically, the order of the same group of front images and rear images in the first image history data and the second image history data is randomly adjusted when splicing in a horizontal manner, and then the letter-box operation is used to fill the top and bottom of the spliced ​​image with black to obtain a training image. Among them, the same group of front images and rear images refers to the front / rear images corresponding to the same control time and control command. After completing the image preprocessing under all different working conditions, the preprocessed images corresponding to the same welder operation command are saved in a folder to facilitate classification model training. letter-box: "letterbox cropping (processing)" or "black edge cropping (processing)". It refers to adding black edges (or blank areas) above and below the picture while maintaining the aspect ratio of the picture to adapt to different display devices or meet specific aspect ratio requirements, similar to the blank areas above and below the letterbox, so it has such a translation.

[0086] Step S203: Acquire an equal amount of normal image data under normal welding conditions.

[0087] Specifically, after obtaining the pre-processed images corresponding to different operation commands in step S202, an equal number of normal welding images (ie, normal image data) are collected at the same time, and the normal welding images are pre-processed and saved in a normal file.

[0088] Step S204: Clean the training image and normal image data to obtain a cleaned training image.

[0089] Specifically, different welders vary in their understanding of abnormal state boundaries during welding control, leading to varying timings for issuing welding control commands. This can lead to ambiguity in control commands. Therefore, experienced welders, drawing on extensive operational experience, must clean the pre-processed training image data for each control command type, such as OSC control, AVC control, and wire yaw control, eliminating training images corresponding to non-standard or inaccurate control commands. This ultimately results in cleaned training images. OSC control (oscillation control) primarily controls the oscillation of the tungsten electrode in the welding torch. This oscillation control ensures optimal heat distribution during welding, resulting in a more uniform weld pool and reduced weld defects such as porosity and cracks. It also facilitates wide weld bead welding. For example, in applications requiring wide weld seams, OSC control of the torch oscillation ensures sufficient fusion on both sides of the weld, ensuring weld quality. AVC (automatic arc length tracking control) ensures a constant distance between the welding torch and the workpiece. Arc length stability is crucial to weld quality during welding. The AVC system automatically adjusts the distance between the welding torch and the base material, effectively avoiding insufficient welding or overheating caused by improper distance, thereby ensuring precise control of the welding arc length and improving welding quality and efficiency. For example, in pipe welding, even if the pipe surface has certain unevenness, AVC can ensure that the arc length between the welding torch and the pipe remains within the appropriate range, resulting in more stable welding results.

[0090] Step S205: Perform classification model training on the cleaned training image to obtain a prediction model.

[0091] Optionally, in an embodiment of the present invention, yolov8 or other classification models can be used for model training to establish a mapping relationship between the cleaned training image and the control command, that is, a mapping relationship between the welding image and the control command, that is, a prediction model. After obtaining the prediction model, the prediction model can be converted into an ONNX format file. It should be noted that in addition to using an image classification model for training, the present invention can also use a video classification model for training.

[0092] refer to Figure 3 , the present invention also provides an intelligent welding control system based on image vision.

[0093] like Figure 3 As shown, the system includes: a welding robot body 10 , a visual system, a host computer monitoring system 30 (including host computer monitoring software), a slave computer control system 40 , a welding power source 60 and a control handle 50 .

[0094] The visual system is used to acquire the current first image and the second image. Optionally, the visual system includes: a first image acquisition device 201 and a second image acquisition device 202; the first image acquisition device 201 is disposed at the front end of the welding head of the welding robot body 10 and is used to acquire the first image and obtain the first image; the second image acquisition device 202 is disposed at the rear end of the welding head of the welding robot body 10 and is used to acquire the second image and obtain the second image. The first image acquisition device 201 and the second image acquisition device 202 can both be industrial cameras with welding-specific filter modules and can be respectively mounted on the connection structures at the front and rear ends of the welding head. The first image acquisition device 201 and the second image acquisition device 202 respectively capture the welding process from the front and rear ends along the welding travel direction of the welding robot body 10 and transmit the captured images to the host computer monitoring system 30.

[0095] The upper computer monitoring system 30 is used to: pre-process the first image and the second image to obtain a pre-processed image; load the prediction model; perform prediction based on the prediction model and the pre-processed image to obtain a prediction control command and the confidence of the prediction control command; determine whether feedback control needs to be performed based on the confidence of the prediction control command; if so, send a prediction control command to the lower computer control system 40 to perform feedback control; if not, do not send the prediction control command.

[0096] The welding power supply 60 is used to provide power to the lower computer control system 40 and the welding robot body 10; the control handle 50 is used to send manual operation commands to the lower computer control system 40; the lower computer control system 40 is used to generate control signals according to manual operation commands or predictive control systems, and send the control signals to the welding robot body 10.

[0097] Specifically, the upper computer monitoring system 30 can output control commands to the lower computer control system 40, and the lower computer control system 40 outputs corresponding control signals to the welding robot body 10 according to the control commands to control the welding robot body 10 to perform corresponding actions. At the same time, the lower computer control system 40 can also feed back the welding parameters of the welding machine to the upper computer monitoring system 30 in real time to achieve full closed-loop welding control. Its rectifier operation logic is as follows: Figure 5 shown.

[0098] The image-vision-based intelligent welding control system of the present invention forms an integrated interface through the host computer monitoring system 30, monitors the welding parameters of the welding machine in real time, presents the image transmitted by the visual system in real time, and simultaneously records the screen and operation log during the welding process, thereby obtaining welding images and welding operation command (control command) logs.

[0099] Specifically, the specific coordination operation process between the various units in the image-vision-based intelligent welding control system can refer to the above-mentioned image-vision-based intelligent welding control method, which will not be repeated here.

[0100] The present invention can achieve accurate intelligent welding control, which has lower cost, more comprehensive control categories, higher precision and better performance than laser vision intelligent welding.

[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0102] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0103] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0104] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the present invention and implement it accordingly. They are not intended to limit the scope of protection of the present invention. All equivalent variations and modifications within the scope of the claims of the present invention are intended to be covered by the claims of the present invention.

Claims

1. An intelligent welding control method based on image vision, characterized in that: The following steps are involved: Get the current first image and second image; Preprocessing the first image and the second image to obtain preprocessed images; Load the prediction model; Performing prediction based on the prediction model and the preprocessed image to obtain a predicted control command and a confidence level of the predicted control command; determining whether feedback control needs to be performed based on the confidence level of the predicted control command; If so, the predictive control command is sent to the lower computer control system to perform feedback control; If not, the prediction control command will not be issued.

2. The intelligent welding control method based on image vision according to claim 1 is characterized in that: Preprocessing the first image and the second image to obtain preprocessed images includes: Performing left-right stitching on the first image and the second image to obtain a stitched image; Filling is performed on the spliced ​​image to obtain the preprocessed image.

3. The intelligent welding control method based on image vision according to claim 1, characterized in that: The determining whether feedback control needs to be performed based on the confidence level of the predicted control command includes: comparing the confidence level of the predicted control command with a control threshold; If the confidence level of the predicted control command is greater than the control threshold, it is determined that feedback control needs to be performed; If the confidence level of the predicted control command is less than the control threshold, it is determined that feedback control does not need to be performed.

4. The intelligent welding control method based on image vision according to claim 1, characterized in that: The sending of the prediction control command to the subordinate control system to perform feedback control includes: The upper computer monitoring system sends the prediction control command to the lower computer control system; The lower computer control system sends a control signal to the welding robot body according to the prediction control command; The welding robot body performs a welding operation according to the control signal.

5. The intelligent welding control method based on image vision according to claim 1, characterized in that: The method further comprises: The host computer monitoring system starts timing after issuing the prediction control command; Determine whether the set control time interval has been reached based on the timing result; If the set control time interval is reached, prediction and control are performed again based on the real-time image; If the set control time interval is not reached, no action is performed.

6. The image vision-based intelligent welding control method according to claim 4, characterized in that: The method further comprises: During the feedback control process, the lower computer control system generates the control signal according to the minimum displacement based on the predictive control command.

7. The intelligent welding control method based on image vision according to claim 1, characterized in that: Before executing to obtain the current first image and the second image, the method includes: Acquire historical data of different welding conditions; the historical data includes: first image historical data and second image historical data under different welding conditions; Preprocessing the first image history data and the second image history data respectively to obtain training image data under different welding conditions; Acquire the same amount of normal image data under normal welding conditions; Cleaning the training image and the normal image data to obtain a cleaned training image; Classification model training is performed on the cleaned training image to obtain the prediction model.

8. An intelligent welding control system based on image vision, characterized in that: include: A visual system, wherein the visual system is used to obtain the current first image and the second image; The host computer monitoring system is used to: Preprocessing the first image and the second image to obtain preprocessed images; Load the prediction model; Performing prediction based on the prediction model and the preprocessed image to obtain a predicted control command and a confidence level of the predicted control command; determining whether feedback control needs to be performed based on the confidence level of the predicted control command; If so, the predictive control command is sent to the lower computer control system to perform feedback control; If not, the prediction control command will not be issued.

9. The image-vision-based intelligent welding control system according to claim 8, characterized in that: Also includes: Welding robot body; the visual system includes: a first image acquisition device and a second image acquisition device; The first image acquisition device is provided at the front end of the welding head of the welding robot body, and is used to acquire the first image to obtain the first image; The second image acquisition device is arranged at the rear end of the welding head of the welding robot body, and is used to acquire the second image to obtain the second image.

10. The image vision-based intelligent welding control system according to claim 9, characterized in that: Also includes: Lower computer control system, welding power supply and control handle; The welding power supply is used to provide power to the lower computer control system and the welding robot body; The control handle is used to send manual operation commands to the lower computer control system; The lower computer control system is used to generate a control signal according to the manual operation command or the predictive control system, and send the control signal to the welding robot body.

Images