Welding self-adaptive visual control method and device and storage medium

Abstract

The invention relates to a welding self-adaptive visual control method and device and a storage medium. The method comprises the steps that a current electrical parameter combination, a welding image, welding gun relative weld joint position data and surface temperature distribution data in the welding process are collected in real time; inputting the welding image into a pre-trained molten pool recognition model, and outputting internal molten pool form data; the relative welding seam position of the welding gun, the surface temperature distribution data and the internal molten pool form data are input into a pre-trained welding process state space model, a welding state recognition result is output, and the welding state recognition result is selected from normal or abnormal; whether the welding state recognition result is abnormal or not is judged; if yes, according to the real-time evaluation result of the welding quality, a model is generated through a pre-trained strategy, and a new electrical parameter combination is output; the new electrical parameter combination is sent to a welding execution module, and the current electrical parameter combination is replaced with the new electrical parameter combination; therefore, precise self-adaptive adjustment in the welding process is achieved.

Description

Technical Field

[0001] The invention relates to the field of machine vision, and in particular to a welding adaptive vision control method, device and storage medium. Background Art

[0002] In modern industrial manufacturing, welding technology, as an important means of connecting metals or other thermoplastic materials, is widely used in the fields of automobile manufacturing, aerospace, shipbuilding, heavy machinery, etc. With the rapid development of automation and intelligent technology, the automation level of the welding process has been significantly improved. However, the consistency and stability of welding quality is still one of the key factors restricting production efficiency and product quality.

[0003] Traditional welding process control mainly relies on manual experience or preset welding parameters. This method is often difficult to achieve ideal welding results when facing complex and changeable welding environments and material properties. In addition, due to the many influencing factors in the welding process, real-time monitoring and adjustment of welding quality becomes particularly complicated.

[0004] Although some existing automated welding systems can achieve preliminary adjustments to welding parameters, most of them are based on fixed control logic or simple feedback mechanisms, lacking in-depth understanding of the complex dynamic changes in the welding process and the ability to respond in real time. In particular, when it comes to the prevention and correction of welding defects (such as lack of fusion, pores, cracks, etc.), traditional methods often seem inadequate and it is difficult to achieve accurate adaptive adjustment of welding parameters.

[0005] In view of this, a more efficient welding process control method is urgently needed. Summary of the invention

[0006] The main purpose of the present invention is to provide a welding adaptive visual control method, device and storage medium to solve the problem that welding parameters are difficult to achieve accurate adaptive adjustment in the welding process of the prior art.

[0007] To achieve the above object, the present invention provides a welding adaptive visual control method in a first aspect, the method comprising: Real-time collection of current electrical parameter combination, welding image, welding gun relative weld position data and surface temperature distribution data during welding; Inputting the welding image into a pre-trained molten pool recognition model and outputting internal molten pool morphology data; Inputting the welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal; Determining whether the welding state recognition result is abnormal; If yes, a new set of electrical parameter combinations is outputted through a pre-trained strategy generation model based on the real-time evaluation results of the welding quality; The new electrical parameter combination is sent to the welding execution module, and the current electrical parameter combination is replaced by the new electrical parameter combination.

[0008] Furthermore, the step of real-time acquisition of current electrical parameter combination, welding image, welding gun relative weld position and surface temperature distribution data during the welding process includes: Generate and send a collection signal instruction with a timestamp to an electrical parameter collection device, a welding image collection device, a welding gun position collection device and a surface temperature distribution collection device; after receiving the collection signal instruction, the electrical parameter collection device collects an electrical parameter combination and embeds the timestamp into the electrical parameter combination to form an electrical parameter combination with a timestamp; after receiving the collection signal instruction, the welding image collection device collects a welding image and embeds the timestamp into the welding image to form a welding image with a timestamp; after receiving the collection signal instruction, the welding gun position collection device collects position data of the welding gun relative to the weld and embeds the timestamp into the position data to form position data with a timestamp; and after receiving the collection signal instruction, the surface temperature distribution collection device collects surface temperature distribution data and embeds the timestamp into the surface temperature distribution data to form surface temperature distribution data with a timestamp; receiving the time-stamped electrical parameter combination, the time-stamped welding image, the time-stamped position data, and the time-stamped surface temperature distribution data; Correlation matching is performed on the electrical parameter combination, welding image, position data and surface temperature distribution data with the same time stamp respectively; All electrical parameter combinations, welding images, position data and surface temperature distribution data after correlation matching in the welding process are sorted and stored in time series.

[0009] Furthermore, the pre-training step of the molten pool identification model includes: Acquire a historical welding image data set; the historical welding image data set stores a plurality of historical welding images and welding electrical parameters corresponding to each of the historical welding images in an associated manner; Extracting the characteristic parameters of the molten pool in each of the historical welding images by a preset image processing algorithm, wherein the characteristic parameters include at least one of length, width, area, aspect ratio, and brightness; Assigning a corresponding internal molten pool morphology annotation to each of the historical welding images to obtain an annotated welding image data set, wherein the annotated welding image data set stores a mapping relationship between molten pool characteristic parameters, annotations, and corresponding welding electrical parameters of each historical welding image, wherein the annotations include at least one of molten pool morphology classification and molten pool development trend; Dividing the labeled welding image dataset into a training set and a validation set; The training set is input into a convolutional neural network to train a molten pool recognition model, and the mapping relationship between the welding image and the internal molten pool morphology is learned through forward propagation, back propagation and optimization processes.

[0010] Further, the step of inputting the welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, outputting a welding state recognition result, wherein the welding state recognition result is selected from a normal or abnormal state, comprises: The welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data are respectively subjected to data normalization preprocessing to obtain preprocessed position data, preprocessed temperature distribution data and preprocessed molten pool morphology data; The preprocessing position data, the preprocessing temperature distribution data and the preprocessing molten pool morphology data are integrated to form a comprehensive welding feature vector; Inputting the welding comprehensive feature vector into the welding process state space model, wherein the welding process state space model predefines the boundary between a normal welding state and an abnormal welding state; According to the mapping result between the welding comprehensive feature vector and the boundary, it is judged whether the welding state is normal, and the welding state recognition result is output.

[0011] Furthermore, before the step of outputting a set of new electrical parameter combinations according to the real-time evaluation results of welding quality through a pre-built reinforcement learning model, the method further comprises: Get real-time assessment of welding quality; The step of obtaining a real-time evaluation result of welding quality comprises: Continuously and synchronously collect continuous frame welding images and 3D point cloud data during the welding process, where each frame welding image and the corresponding 3D point cloud data are assigned a unique timestamp; Inputting the welding images frame by frame into a pre-trained convolutional neural network model for feature extraction to extract weld image features; Inputting the weld image features frame by frame into a pre-trained surface defect detection model for defect recognition, and outputting a single-frame image defect detection result; and Input the weld image features of adjacent frames into a pre-trained welding anomaly recognition model for anomaly recognition, and output the inter-frame welding state recognition results; Comparing the geometric shape of the weld with a preset ideal geometric shape through the three-dimensional point cloud data to obtain a geometric shape evaluation result; The single-frame image defect detection results, the inter-frame welding state recognition results and the geometric shape evaluation results within a preset time length based on the same start timestamp and the same end timestamp are input into the pre-trained deep learning model, and the comprehensive quality score of the single-segment welding process is output. The comprehensive quality score is used to evaluate the welding quality.

[0012] Furthermore, the pre-training step of the strategy generation model includes: The reward function is designed as: R(s,a) = w1 × f 1(Quality(s,a))+ w2× f 2(Stability(s,a))-w3 × C (a) Among them, Quality(s,a) is the single-frame image defect detection result, Stability(s,a) is the inter-frame welding state recognition result, C (a) is the comprehensive quality score of a single-stage welding process; w1 is the weight of the defect detection result of a single-frame image, w2 is the weight of the welding state recognition result between frames, and w3 is the weight of the comprehensive quality score of a single-stage welding process; A pre-collected strategy training set is used, a reinforcement learning algorithm is selected to train the strategy generation model, and the reward value of the reward function is adjusted to update the parameters of the strategy generation model to output the action with the highest reward value in the current state; the strategy training set includes states, actions and corresponding reward value mapping relationships; the states include single-frame image defect detection results, inter-frame welding state recognition results and a single-segment welding process comprehensive quality score, and the actions include electrical parameter combinations.

[0013] Further, f 1(Quality(s,a)) = q1 × D(s,a)+q2 × W(s,a)+q3 × I(s,a) Where D(s,a) is the depth index of the weld, W(s,a) is the width index of the weld, I(s,a) is the integrity index of the weld, q1 is the depth index weight of the weld, q2 is the width index weight of the weld, and q3 is the integrity index weight of the weld.

[0014] Furthermore, after the step of sending the new electrical parameter combination to the welding execution module and replacing the current electrical parameter combination with the new electrical parameter combination, the method further includes: Obtaining a real-time evaluation result of a new welding quality corresponding to the new electrical parameter combination; The new real-time evaluation result of the welding quality is updated to the strategy generation model accordingly to achieve optimization of the strategy generation model.

[0015] A second aspect of the present invention provides a welding adaptive visual control device, the device comprising: Data acquisition module, used to collect the current electrical parameter combination, welding image, welding gun relative weld position data and surface temperature distribution data in real time during the welding process; A molten pool recognition module, used to input the welding image into a pre-trained molten pool recognition model and output internal molten pool morphology data; A state recognition module, used for inputting the relative weld position of the welding gun, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal; A judgment module, used for judging whether the welding state recognition result is abnormal; A strategy generation module, for outputting a set of new electrical parameter combinations through a pre-trained strategy generation model according to a real-time evaluation result of welding quality if the welding state recognition result is abnormal; The parameter adjustment module is used to send the new electrical parameter combination to the welding execution module, and replace the current electrical parameter combination with the new electrical parameter combination.

[0016] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned steps of welding adaptive visual control when executed by a processor.

[0017] The welding adaptive visual control method provided by the present invention ensures comprehensive and accurate monitoring of the welding state by real-time collection of multi-dimensional data in the welding process, including electrical parameter combinations, welding images, welding gun positions and surface temperature distributions; through pre-trained molten pool recognition models and welding process state space models, instant analysis and judgment of the welding state are achieved, greatly improving the real-time and accuracy of control; when abnormal welding state is detected, based on the real-time evaluation results of welding quality, the model is generated through pre-trained strategies, and the optimal electrical parameter combination is automatically output and adjusted. This intelligent adaptive adjustment capability effectively solves the problem of fixed welding parameters and difficulty in adapting to complex working conditions in traditional welding methods, and significantly improves the stability and consistency of welding quality. Automated and intelligent welding process control reduces the need for manual intervention and reduces the impact of human factors on welding quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic flow chart of a welding adaptive visual control method in one embodiment of the present invention; Figure 2 is a schematic structural diagram of a welding adaptive visual control device in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0019] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0021] Reference Figure 1 , an embodiment of the present invention provides a welding adaptive visual control method, the method comprising: S1, real-time collection of current electrical parameter combination, welding image, welding gun relative weld position data and surface temperature distribution data during welding; S2, inputting the welding image into a pre-trained molten pool recognition model, and outputting internal molten pool morphology data; S3, inputting the relative weld position of the welding gun, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal; S4, determining whether the welding state recognition result is abnormal; S5. If yes, then based on the real-time evaluation results of welding quality, a model is generated through a pre-trained strategy to output a new set of electrical parameter combinations; S6. Send the new electrical parameter combination to the welding execution module, and replace the current electrical parameter combination with the new electrical parameter combination.

[0022] In this embodiment, in the above step S1, the current electrical parameter combination includes welding current, welding voltage, welding speed, etc., and these data are obtained in real time through the electrical parameter monitoring interface of the welding power supply. The welding image can use a high-resolution industrial camera to capture the real-time video stream of the welding area to ensure that the changes in the molten pool and the surrounding area can be clearly observed. The relative weld position data of the welding gun can be measured in real time by a position sensor (such as a laser rangefinder or encoder) installed on the welding gun. The relative position between the welding gun and the weld includes horizontal offset, vertical offset and angle, etc. The surface temperature distribution data can be obtained by using an infrared thermal imager to perform non-contact temperature measurement on the welding area to obtain a temperature distribution map of the welding surface and the surrounding environment.

[0023] In the above step S2, the pre-trained molten pool recognition model can be constructed using a deep learning model, such as a convolutional neural network (CNN). The molten pool recognition model is trained on a large number of welding image data sets, and can accurately identify the internal molten pool area in the welding image and extract the morphological features of the internal molten pool, such as the width, length, area and shape of the molten pool. After the real-time collected welding image is input into the molten pool recognition model, the molten pool recognition model will output the internal molten pool morphological data under the current welding state.

[0024] In the above step S3, the welding process state space model is a comprehensive judgment model based on deep learning. It integrates multi-source data such as the position of the welding gun relative to the weld, the surface temperature distribution, and the internal molten pool morphology to comprehensively analyze the welding process. The welding process state space model learns the difference between normal welding state and abnormal welding state during the training phase, and can accurately identify whether the current welding state is normal or abnormal. After the data obtained in the above steps S1 and S2 are input into the welding process state space model, the model will output the recognition result of the welding state.

[0025] In the above step S5, if the welding state recognition result is "abnormal", it indicates that there are problems in the current welding process that may affect the welding quality. According to the real-time evaluation results of the welding quality, the welding parameters are automatically adjusted through a pre-trained strategy generation model (such as a reinforcement learning model, etc.) to generate a set of optimal new electrical parameter combinations to improve the welding quality and restore the normal welding state.

[0026] In the above step S6, the newly generated electrical parameter combination is sent to the welding execution module (such as a welding power supply controller) through the communication interface, instructing it to use the new electrical parameter combination for welding, thereby realizing real-time adaptive adjustment of the welding parameters and ensuring the stability of the welding process and the reliability of the welding quality.

[0027] In a specific embodiment, the step S1 of real-time acquisition of current electrical parameter combination, welding image, welding gun relative weld position and surface temperature distribution data during welding process includes: S101, generating and sending a collection signal instruction with a timestamp to an electrical parameter collection device, a welding image collection device, a welding gun position collection device and a surface temperature distribution collection device; after receiving the collection signal instruction, the electrical parameter collection device collects an electrical parameter combination and embeds the timestamp into the electrical parameter combination to form an electrical parameter combination with a timestamp; after receiving the collection signal instruction, the welding image collection device collects a welding image and embeds the timestamp into the welding image to form a welding image with a timestamp; after receiving the collection signal instruction, the welding gun position collection device collects position data of the welding gun relative to the weld and embeds the timestamp into the position data to form position data with a timestamp; and after receiving the collection signal instruction, the surface temperature distribution collection device collects surface temperature distribution data and embeds the timestamp into the surface temperature distribution data to form surface temperature distribution data with a timestamp; S102, receiving the electrical parameter combination with a timestamp, the welding image with a timestamp, the position data with the timestamp, and the surface temperature distribution data with the timestamp; S103, respectively performing correlation matching on the electrical parameter combination, welding image, position data and surface temperature distribution data with the same timestamp; S104, sorting and storing all the electrical parameter combinations, welding images, position data and surface temperature distribution data after correlation matching in the welding process according to time series.

[0028] In this embodiment, in order to ensure that the data collected during the welding process can accurately correspond to the welding state at the same time point, a collection signal instruction with a high-precision timestamp is first generated. The instruction is sent to the electrical parameter collection device, the welding image collection device, the welding gun position collection device and the surface temperature distribution collection device simultaneously through the data bus or wireless communication.

[0029] Since each acquisition device performs data acquisition operations almost simultaneously after receiving the acquisition signal instruction with the same timestamp, it can be considered that the data records with the same timestamp correspond to the welding state at the same time point. By comparing the timestamps, the electrical parameter combination, welding image, welding gun position data and surface temperature distribution data with the same timestamp are associated and matched to form a complete data packet, which reflects the welding state at that time point.

[0030] Finally, the associated matched data packets are sorted in the order of timestamps and stored in the database for subsequent data analysis and processing. By querying the database, detailed data at any time point in the welding process can be quickly obtained, providing a data basis for the evaluation and improvement of welding quality.

[0031] In a specific embodiment, the pre-training step of the molten pool identification model includes: S201, obtaining a historical welding image data set; the historical welding image data set stores a plurality of historical welding images and welding electrical parameters corresponding to each of the historical welding images in an associated manner; S202, extracting the characteristic parameters of the molten pool in each of the historical welding images by a preset image processing algorithm, wherein the characteristic parameters include at least one of length, width, area, aspect ratio, and brightness; S203, assigning a corresponding internal molten pool morphology annotation to each of the historical welding images, and obtaining an annotated welding image data set, wherein the annotated welding image data set stores a mapping relationship between molten pool characteristic parameters, annotations, and corresponding welding electrical parameters of each historical welding image, wherein the annotations include at least one of molten pool morphology classification and molten pool development trend; S204, dividing the annotated welding image data set into a training set and a verification set; S205, inputting the training set into a convolutional neural network to train a molten pool recognition model, and learning the mapping relationship between the welding image and the internal molten pool morphology through forward propagation, back propagation and optimization processes.

[0032] In this embodiment, in the above step S201, in order to train the molten pool recognition model, it is first necessary to collect a large number of historical welding image data sets. These data sets can be derived from image records of historical welding processes, and each image records the state of the molten pool at a certain moment in the welding process. At the same time, these data sets also store the welding electrical parameters (such as current, voltage, etc.) corresponding to each historical welding image, which are crucial for understanding the relationship between welding conditions and molten pool morphology.

[0033] In the above step S202, the above preset image processing algorithm can adopt the image recognition algorithm in the prior art, including edge detection, threshold segmentation, morphological operation and other steps, to process each historical welding image to extract the key characteristic parameters of the molten pool. These characteristic parameters include but are not limited to the length, width, area, aspect ratio and brightness of the molten pool, so as to fully describe the geometric morphology and physical properties of the molten pool.

[0034] In the above step S203, the annotation work can be completed by experts or technicians with professional knowledge of welding, who assign corresponding internal molten pool morphology annotations to each image according to the morphological characteristics and development trends of the molten pool. The annotation content includes, for example, the classification of the molten pool morphology (such as normal, abnormal, etc.) and the development trend of the molten pool (such as stable, expanding, shrinking, etc.). After the annotation is completed, the molten pool characteristic parameters, annotations and corresponding welding electrical parameters are associated and stored to form an annotated welding image data set. The data set contains the mapping relationship between the welding image and the internal molten pool morphology, which is used to train the molten pool recognition model.

[0035] In the above step S204, the training set is used in the training process of the model, that is, the parameters of the model are updated by learning the data in the training set; the validation set is used to evaluate the performance of the model during the training process, so as to timely adjust the training strategy and optimize the model structure.

[0036] In the above step S205, the training set is input into a convolutional neural network (CNN) to train the molten pool recognition model. CNN extracts feature information from the image through structures such as convolutional layers, pooling layers, and fully connected layers, and learns the mapping relationship between the input image and the output label. During the training process, the error between the model output and the true annotation is calculated through forward propagation, and then the error is passed back to each layer in the network through the back propagation algorithm, and the weight parameters of each layer are updated. This process is repeated for multiple iterations until the performance of the model on the validation set reaches the preset standard. The molten pool recognition model obtained through training can accurately identify the internal molten pool morphology in the welding image.

[0037] In a specific embodiment, the step S3 of inputting the welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal, comprises: S301, performing data normalization preprocessing on the relative weld position of the welding gun, the surface temperature distribution data and the internal molten pool morphology data, respectively, to obtain preprocessed position data, preprocessed temperature distribution data and preprocessed molten pool morphology data; S302, fusing the preprocessed position data, preprocessed temperature distribution data and preprocessed molten pool morphology data to form a comprehensive welding feature vector; S303, inputting the welding comprehensive feature vector into the welding process state space model, wherein the welding process state space model predefines the boundary between a normal welding state and an abnormal welding state; S304. Determine whether the welding state is normal according to the mapping result between the welding comprehensive feature vector and the boundary, and output a welding state recognition result.

[0038] In this embodiment, in the above step S301, for the welding gun relative weld position data, normalization can be achieved by converting it into an offset relative to the weld centerline and scaling it; for the surface temperature distribution data, the maximum and minimum normalization method can be used to map the temperature value to the [0,1] interval; for the internal molten pool morphology data, since it contains multiple characteristic parameters (such as length, width, area, etc.), each characteristic parameter can be normalized separately, or the characteristic parameters can be combined into a characteristic vector and then normalized as a whole.

[0039] In the above step S302, after normalization preprocessing, the preprocessed position data, preprocessed temperature distribution data and preprocessed molten pool morphology data are fused to form a welding comprehensive feature vector containing multi-dimensional information. Common fusion methods include feature splicing, weighted summation, principal component analysis, etc.

[0040] In the above step S303, the welding comprehensive feature vector is input into the pre-trained welding process state space model. The welding process state space model is a model built based on deep learning, which predefines the boundaries between normal welding state and abnormal welding state. The above boundaries are determined by analyzing a large amount of historical welding data and extracting the characteristic laws therein, which can effectively distinguish between normal and abnormal welding states. In the process of training the welding process state space model, the mapping relationship between the welding comprehensive feature vector and the welding state has been learned, so the current welding state can be predicted based on the input welding comprehensive feature vector.

[0041] In the above step S304, according to the mapping result of the welding comprehensive feature vector and the predefined boundary, it is judged whether the current welding state is normal, and the welding state recognition result is output. If the welding comprehensive feature vector falls within the boundary of the normal welding state, it is judged as a normal welding state; otherwise, it is judged as an abnormal welding state. The recognition result can be fed back to the welding operator or control system in real time so that corresponding measures can be taken in time to adjust the welding parameters or stop the welding process, thereby ensuring the welding quality and safety. It can also enter the subsequent steps to achieve adaptive adjustment of welding parameters.

[0042] In a specific embodiment, before the step S5 of outputting a set of new electrical parameter combinations according to the real-time evaluation results of welding quality through a pre-built reinforcement learning model, the method includes: S7. Obtaining real-time evaluation results of welding quality; The step S7 of obtaining the real-time evaluation result of welding quality comprises: S701, continuously and synchronously collecting continuous frames of welding images and three-dimensional point cloud data during the welding process, wherein each frame of welding image and corresponding three-dimensional point cloud data are assigned a unique timestamp; Specifically, the above step S701 includes generating and sending an acquisition signal instruction with a timestamp to a welding image acquisition device and a three-dimensional point cloud data acquisition device, wherein the welding image acquisition device acquires a welding image after receiving the acquisition signal instruction, and embeds the timestamp into the welding image to form a welding image with a timestamp, and the three-dimensional point cloud data acquisition device acquires three-dimensional point cloud data after receiving the acquisition signal instruction, and embeds the timestamp into the three-dimensional point cloud data to form three-dimensional point cloud data with a timestamp; receiving the welding image with a timestamp and the three-dimensional point cloud data with a timestamp; respectively associating and matching the welding image and the three-dimensional point cloud data with the same timestamp; and sorting and storing all the welding images and three-dimensional point cloud data after associating and matching in the welding process according to the time series.

[0043] S702, inputting the welding image frame by frame into a pre-trained convolutional neural network model for feature extraction to extract weld image features; Specifically, step S702 includes cropping each of the welding images in timestamp order to extract the weld image from the welding image and remove the background image; graying, removing noise and normalizing each of the weld images to obtain a preprocessed weld image; inputting the preprocessed weld image frame by frame into the encoder layer of the pretrained convolutional neural network model, and extracting features of the weld image layer by layer through the convolution layer, activation function, pooling layer and residual connection of the ResNet architecture encoder to generate a series of feature maps, which contain feature information at different levels from low to high levels; passing the feature map generated by the encoder to the decoder of the convolutional neural network model through a short-circuited path, and using a multi-scale feature fusion mechanism to fuse feature maps of different scales, and outputting the welding image features frame by frame.

[0044] S703, inputting the weld image features frame by frame into a pre-trained surface defect detection model for defect recognition, and outputting a single-frame image defect detection result; Specifically, step S703 includes inputting the weld image features frame by frame into an encoder of a pre-trained surface defect detection model, wherein the encoder of the surface defect detection model adopts a ResNet50V2 architecture, and gradually extracts a pyramid image feature map through a preset number of stages of ResNet50V2, wherein each stage enhances image feature propagation through residual connection, wherein the resolution of the pyramid image feature map is gradually reduced, while the number of channels is gradually increased for the output feature map of each stage; on the pyramid image feature map, anchor points and detection heads are respectively established on each output feature map, and candidate detection frames are generated through a series of 3x3 convolutions; the attribute values ​​of the candidate detection frames are detected by each detection head, wherein the attribute values ​​include spatial coordinates, the length and width of the circumscribed rectangle, and the probability of whether there is a defect in the frame; all candidate detection frames for which defects are detected are processed by a non-maximum suppression algorithm to remove redundant frames with an overlap higher than a preset threshold, and obtain a positioning frame for each defect; the defect type is identified for the defect in the positioning frame of each defect to obtain the defect type; and the single-frame image defect detection result is output, wherein the defect detection result includes the spatial coordinates of the positioning frame of each defect, the length and width of the outer rectangle, and the defect type.

[0045] S704, inputting the weld image features of adjacent frames into a pre-trained welding anomaly recognition model for anomaly recognition, and outputting the inter-frame welding state recognition result; Specifically, step S704 includes inputting the weld image features of adjacent frames into the encoder of the welding anomaly recognition model, the encoder of the welding anomaly recognition model adopts MobileNet-V2 as the main architecture, and re-encodes the input weld image features of adjacent frames through multi-layer convolution and pooling operations of MobileNet-V2 to extract deep image features; calculates the difference in deep image features of adjacent frames through a feature matching algorithm, and uses the feature difference of adjacent frames as a new feature dimension, which is fused with the deep image features of the previous frame in the adjacent frames to generate a fused feature containing dynamic information between frames; based on the fused feature, a classifier is used to identify whether there is an abnormality in the welding state between frames, and an abnormal welding state recognition result is output.

[0046] S705, comparing the geometric shape of the weld with a preset ideal geometric shape through the three-dimensional point cloud data to obtain a geometric shape evaluation result; Specifically, the step S705 includes acquiring three-dimensional point cloud data of the workpiece to be welded; selecting a corresponding ideal weld shape from a predefined standard library based on the three-dimensional point cloud data of the welded workpiece; acquiring electrical parameters with timestamps for the welding process in real time; during the welding process, according to the real-time electrical parameters with timestamps, calculating the ideal geometric shape of the welded workpiece at each timestamp through the mapping relationship of the ideal weld model generation model; for the three-dimensional point cloud data at each timestamp, loading or generating the corresponding preset ideal geometric shape in real time; comparing the geometric shape of the weld corresponding to the three-dimensional point cloud data with the preset ideal geometric shape in real time through a preset comparison algorithm, calculating and outputting a real-time geometric shape evaluation result, wherein the evaluation result includes at least one of weld height, weld width, weld height deviation, and weld width deviation.

[0047] S706. Input each of the single-frame image defect detection results, each of the inter-frame welding state recognition results, and each of the geometric shape assessment results within a preset time length based on the same start timestamp and the same end timestamp into a pre-trained deep learning model, and output a comprehensive quality score for a single-segment welding process, wherein the comprehensive quality score is used to evaluate the welding quality.

[0048] Specifically, the step S706 includes extracting features from each of the single-frame image defect detection results, each of the inter-frame welding state recognition results, and each of the geometric shape assessment results within a preset time length based on the same start timestamp and the same end timestamp to obtain data features respectively; fusing the data features from different sources to obtain a multi-dimensional feature vector; inputting the multi-dimensional feature vector into a pre-trained deep learning model for quality assessment, mapping the output value to a preset score range to obtain a comprehensive welding quality score; mapping the output value to a preset welding quality grade to obtain a welding quality grade.

[0049] In a specific embodiment, the pre-training step of the strategy generation model includes: S501. Design the reward function as follows: R(s,a) = w1 × f 1(Quality(s,a))+ w2× f 2(Stability(s,a))-w3 × C (a) Among them, Quality(s,a) is the single-frame image defect detection result, Stability(s,a) is the inter-frame welding state recognition result, C(a) is the comprehensive quality score of a single-stage welding process; w1 is the weight of the defect detection result of a single-frame image, w2 is the weight of the welding state recognition result between frames, and w3 is the weight of the comprehensive quality score of a single-stage welding process; S502. Use the pre-collected strategy training set, select the reinforcement learning algorithm to train the strategy generation model, adjust the reward value of the reward function and update the parameters of the strategy generation model to output the action with the highest reward value in the current state; the strategy training set includes the mapping relationship between states, actions and corresponding reward values; the states include single-frame image defect detection results, inter-frame welding state recognition results and a comprehensive quality score of a single-segment welding process, and the actions include electrical parameter combinations.

[0050] In this embodiment, the reward function R(s,a) is designed to be a weighted sum of the single-frame image defect detection results, the inter-frame welding state recognition results, and the comprehensive quality score of a single-segment welding process. s is the real-time evaluation result of the current welding quality, and a is the action in this state; Quality(s,a) represents the result obtained by single-frame image defect detection after taking action a under the current real-time evaluation result s of welding quality; Stability(s,a) represents the result obtained by inter-frame welding state recognition after taking action a under the current real-time evaluation result s of welding quality, which reflects the stability of the welding process; C(a) represents the comprehensive quality score of a single-segment welding process, which is a comprehensive evaluation of the welding quality in a welding process. w1, w2, and w3 are the weights of the single-frame image defect detection results, the inter-frame welding state recognition results, and the comprehensive quality score of a single-segment welding process, respectively, which are used to adjust the relative importance of different factors in the reward function.

[0051] Among them, each single-frame image defect identified will generate a negative reward, and the abnormal welding state between frames will also bring negative rewards, while the comprehensive quality score of a single-segment welding process that meets expectations will correspond to a positive reward. The above three reward components combined with weight superposition can obtain a single-step cumulative reward signal. In multi-step decision-making, a cumulative reward weight reduction mechanism is further set, that is, as the welding process progresses, the contribution of the reward value of the current step to the total reward gradually decreases. Thereby strengthening the model's attention to the initial state of the welding process and preventing quality shocks. After reaching the maximum number of execution steps, a set of parameter combinations with the largest accumulation of reward signals on the path is selected to form the final adaptive control strategy, output the corresponding electrical parameter combination, and adjust the input of the subsequent welding execution module online to stabilize the welding process at the point with the minimum quality loss.

[0052] In the above step S502, the strategy training set includes multiple state-action-reward value mapping relationships, which can be obtained through actual welding data collection or simulation. During the training process, the reinforcement learning algorithm selects actions according to the current state and updates the parameters of the strategy generation model according to the reward value calculated by the reward function.

[0053] The above-mentioned states include single-frame image defect detection results, inter-frame welding state recognition results and single-segment welding process comprehensive quality scores, which are provided by the real-time data acquisition system during the welding process and input into the strategy generation model after preprocessing and feature extraction in the above steps.

[0054] The above actions include a combination of electrical parameters (such as current, voltage, welding speed, etc.), which are control instructions output by the strategy generation model and are used to adjust the operating parameters of the welding equipment to optimize the welding quality.

[0055] The above reward value is calculated according to the reward function and is used to evaluate the impact of the action taken in the current state on the welding quality. The higher the reward value, the more conducive the current action is to improving the welding quality.

[0056] During the training process, the reinforcement learning algorithm continuously tries different action combinations and updates the parameters of the strategy generation model based on the reward value, so that actions with higher reward values ​​can be selected under the same state. Through multiple iterative training, the strategy generation model can learn how to select the appropriate electrical parameter combination to optimize welding quality under different states.

[0057] Specifically, f 1(Quality(s,a)) = q1 × D(s,a)+q2 × W(s,a)+q3 × I(s,a) Where D(s,a) is the depth index of the weld, W(s,a) is the width index of the weld, I(s,a) is the integrity index of the weld, q1 is the depth index weight of the weld, q2 is the width index weight of the weld, and q3 is the integrity index weight of the weld.

[0058] In this embodiment, D(s,a) can be extracted from multiple frames of welding images by image processing technology (through edge detection and contour extraction of the image) to accurately measure the depth of the weld. Welds that are too deep or too shallow may affect the strength and sealing of the welded joint. W(s,a) can be extracted from the welding image by image processing technology, and the width index is directly related to the load-bearing capacity and appearance quality of the welded joint. I(s,a) reflects whether there are defects such as cracks, lack of fusion, slag inclusions, etc. in the weld, which can be evaluated by image recognition technology, such as defect detection algorithms, pattern recognition methods, etc. The integrity index is directly related to the reliability and service life of the welded joint.

[0059] The weights q1, q2, and q3 can be determined based on factors such as welding process requirements, material properties, and actual welding experience, so that f1(Quality(s,a)) can comprehensively and accurately evaluate the defect detection results in a single-frame welding image.

[0060] In a specific embodiment, after the step S6 of sending the new electrical parameter combination to the welding execution module and replacing the current electrical parameter combination with the new electrical parameter combination, the method further includes: S8, obtaining a real-time evaluation result of a new welding quality corresponding to the new electrical parameter combination; S9, updating the new real-time evaluation result of welding quality to the strategy generation model accordingly, so as to optimize the strategy generation model.

[0061] In this embodiment, after the welding execution module receives the new electrical parameter combination and starts to execute the welding task, the real-time data in the welding process will be continuously collected. The collected real-time data is processed and analyzed by the aforementioned pre-set welding quality evaluation method to obtain a new real-time evaluation result of the welding quality.

[0062] The new real-time evaluation results are added as feedback to the training set of the policy generation model. According to the principle of reinforcement learning, the model will recalculate the value function and policy function of each state based on the new data and reward function, thereby updating the model parameters.

[0063] Through the above steps, the strategy generation model can continuously learn from the actual welding process and optimize its decision-making ability, thereby improving welding quality and efficiency.

[0064] The embodiment of the present invention also provides a welding adaptive visual control device, the device comprising: The data acquisition module 10 is used to collect the current electrical parameter combination, welding image, welding gun relative weld position data and surface temperature distribution data in real time during the welding process; A molten pool identification module 20, used to input the welding image into a pre-trained molten pool identification model and output internal molten pool morphology data; A state recognition module 30, for inputting the welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal; A judging module 40 is used to judge whether the welding state recognition result is abnormal; A strategy generation module 50 is used to output a new set of electrical parameter combinations through a pre-trained strategy generation model according to the real-time evaluation result of the welding quality if the welding state recognition result is abnormal; The parameter adjustment module 60 is used to send the new electrical parameter combination to the welding execution module, and replace the current electrical parameter combination with the new electrical parameter combination.

[0065] For the specific implementation of each module in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0066] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned welding adaptive visual control method is implemented.

[0067] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0068] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0069] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0070] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0071] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A welding adaptive visual control method, characterized in that: The method comprises: Real-time collection of current electrical parameter combination, welding image, welding gun relative weld position data and surface temperature distribution data during welding; Inputting the welding image into a pre-trained molten pool recognition model and outputting internal molten pool morphology data; Inputting the welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal; Determining whether the welding state recognition result is abnormal; If yes, then a new set of electrical parameter combinations is outputted through a pre-trained strategy generation model based on the real-time evaluation results of the welding quality; The new electrical parameter combination is sent to the welding execution module, and the current electrical parameter combination is replaced by the new electrical parameter combination.

2. The welding adaptive visual control method according to claim 1, characterized in that: The step of real-time acquisition of current electrical parameter combination, welding image, welding gun relative weld position and surface temperature distribution data during welding process includes: Generate and send a collection signal instruction with a timestamp to an electrical parameter collection device, a welding image collection device, a welding gun position collection device and a surface temperature distribution collection device; after receiving the collection signal instruction, the electrical parameter collection device collects an electrical parameter combination and embeds the timestamp into the electrical parameter combination to form an electrical parameter combination with a timestamp; after receiving the collection signal instruction, the welding image collection device collects a welding image and embeds the timestamp into the welding image to form a welding image with a timestamp; after receiving the collection signal instruction, the welding gun position collection device collects position data of the welding gun relative to the weld and embeds the timestamp into the position data to form position data with a timestamp; and after receiving the collection signal instruction, the surface temperature distribution collection device collects surface temperature distribution data and embeds the timestamp into the surface temperature distribution data to form surface temperature distribution data with a timestamp; receiving the time-stamped electrical parameter combination, the time-stamped welding image, the time-stamped position data, and the time-stamped surface temperature distribution data; Correlation matching is performed on the electrical parameter combination, welding image, position data and surface temperature distribution data with the same time stamp respectively; All electrical parameter combinations, welding images, position data and surface temperature distribution data after correlation matching in the welding process are sorted and stored in time series.

3. The welding adaptive visual control method according to claim 1, characterized in that: The pre-training step of the molten pool identification model includes: Acquire a historical welding image data set; the historical welding image data set stores a plurality of historical welding images and welding electrical parameters corresponding to each of the historical welding images in an associated manner; Extracting the characteristic parameters of the molten pool in each of the historical welding images by a preset image processing algorithm, wherein the characteristic parameters include at least one of length, width, area, aspect ratio, and brightness; Assigning a corresponding internal molten pool morphology annotation to each of the historical welding images to obtain an annotated welding image data set, wherein the annotated welding image data set stores a mapping relationship between molten pool characteristic parameters, annotations, and corresponding welding electrical parameters of each historical welding image, wherein the annotations include at least one of molten pool morphology classification and molten pool development trend; Dividing the labeled welding image dataset into a training set and a validation set; The training set is input into a convolutional neural network to train a molten pool recognition model, and the mapping relationship between the welding image and the internal molten pool morphology is learned through forward propagation, back propagation and optimization processes.

4. The welding adaptive visual control method according to claim 1, characterized in that: The step of inputting the welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected as normal or abnormal, comprises: The welding gun relative weld position, the surface temperature distribution data and the internal molten pool morphology data are respectively subjected to data normalization preprocessing to obtain preprocessed position data, preprocessed temperature distribution data and preprocessed molten pool morphology data; The preprocessing position data, the preprocessing temperature distribution data and the preprocessing molten pool morphology data are integrated to form a comprehensive welding feature vector; The welding comprehensive feature vector is input into the welding process state space model, wherein the welding process state space model predefines the boundary between the normal welding state and the abnormal welding state; based on the mapping result between the welding comprehensive feature vector and the boundary, whether the welding state is normal is judged, and the welding state recognition result is output.

5. The welding adaptive visual control method according to claim 1, characterized in that: The step of outputting a set of new electrical parameter combinations according to the real-time evaluation results of welding quality through a pre-built reinforcement learning model includes: Get real-time assessment of welding quality; The step of obtaining a real-time evaluation result of welding quality comprises: Continuously and synchronously collect continuous frame welding images and 3D point cloud data during the welding process, where each frame welding image and the corresponding 3D point cloud data are assigned a unique timestamp; Inputting the welding images frame by frame into a pre-trained convolutional neural network model for feature extraction to extract weld image features; Inputting the weld image features frame by frame into a pre-trained surface defect detection model for defect recognition, and outputting a single-frame image defect detection result; and Input the weld image features of adjacent frames into a pre-trained welding anomaly recognition model for anomaly recognition, and output the inter-frame welding state recognition results; Comparing the geometric shape of the weld with a preset ideal geometric shape through the three-dimensional point cloud data to obtain a geometric shape evaluation result; The single-frame image defect detection results, the inter-frame welding state recognition results and the geometric shape evaluation results within a preset time length based on the same start timestamp and the same end timestamp are input into the pre-trained deep learning model, and the comprehensive quality score of the single-segment welding process is output. The comprehensive quality score is used to evaluate the welding quality.

6. The welding adaptive visual control method according to claim 5, characterized in that: The pre-training step of the strategy generation model includes: The reward function is designed as: R(s,a) = w1 × f 1(Quality(s,a))+ w2× f 2(Stability(s,a))-w3 × C (a) Among them, Quality(s,a) is the single-frame image defect detection result, Stability(s,a) is the inter-frame welding state recognition result, C (a) is the comprehensive quality score of a single-stage welding process; w1 is the weight of the defect detection result of a single-frame image, w2 is the weight of the welding state recognition result between frames, and w3 is the weight of the comprehensive quality score of a single-stage welding process; A pre-collected strategy training set is used, a reinforcement learning algorithm is selected to train the strategy generation model, and the reward value of the reward function is adjusted to update the parameters of the strategy generation model to output the action with the highest reward value in the current state; the strategy training set includes states, actions and corresponding reward value mapping relationships; the states include single-frame image defect detection results, inter-frame welding state recognition results and a single-segment welding process comprehensive quality score, and the actions include electrical parameter combinations.

7. The welding adaptive visual control method according to claim 6, characterized in that: f 1(Quality(s,a)) = q1 × D(s,a)+q2 × W(s,a)+q3 × I(s,a) Where D(s,a) is the depth index of the weld, W(s,a) is the width index of the weld, I(s,a) is the integrity index of the weld, q1 is the depth index weight of the weld, q2 is the width index weight of the weld, and q3 is the integrity index weight of the weld.

8. The welding adaptive visual control method according to claim 1, characterized in that: After the step of sending the new electrical parameter combination to the welding execution module and replacing the current electrical parameter combination with the new electrical parameter combination, the method further includes: Obtaining a real-time evaluation result of a new welding quality corresponding to the new electrical parameter combination; The new real-time evaluation result of the welding quality is updated to the strategy generation model accordingly to achieve optimization of the strategy generation model.

9. A welding adaptive visual control device, characterized in that: The device comprises, Data acquisition module, used to collect the current electrical parameter combination, welding image, welding gun relative weld position data and surface temperature distribution data in real time during the welding process; A molten pool recognition module, used to input the welding image into a pre-trained molten pool recognition model and output internal molten pool morphology data; A state recognition module, used for inputting the relative weld position of the welding gun, the surface temperature distribution data and the internal molten pool morphology data into a pre-trained welding process state space model, and outputting a welding state recognition result, wherein the welding state recognition result is selected from normal or abnormal; A judgment module, used for judging whether the welding state recognition result is abnormal; A strategy generation module, for outputting a set of new electrical parameter combinations through a pre-trained strategy generation model according to a real-time evaluation result of welding quality if the welding state recognition result is abnormal; The parameter adjustment module is used to send the new electrical parameter combination to the welding execution module, and replace the current electrical parameter combination with the new electrical parameter combination.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of welding adaptive visual control according to any one of claims 1 to 8 are implemented.

Images