Method and system for determining welding defects

Abstract

The application provides a welding defect determination method. According to an embodiment of the application, a plurality of process data sets corresponding to each welding process in a plurality of welding processes are obtained, thereby obtaining the plurality of process data sets. A corresponding process curve is constructed based on each process data set in the plurality of process data sets, thereby obtaining a plurality of process curves. The process curve is a two-dimensional image of the value of the process data in the process data set and time. Recognition information of at least one defect category existing in each welding process in the plurality of welding processes is received. Each curve in the plurality of process curves is associated with the at least one defect category. Mixed model training is performed according to the plurality of process curves associated with the defect category, thereby obtaining a welding defect determination model. The welding defect determination model can determine whether a defect exists in the welding process.

Description

Technical Field

[0001] This application relates to the field of machine vision, and in particular to a method and system for determining welding defects. Background Technology

[0002] Welding is the process in industrial production that has the greatest impact on the overall quality of a product; its quality plays a crucial role in the overall product quality. However, existing quality inspection methods generally employ manual sampling inspection after welding, non-destructive testing of weld joints using ultrasonic waves, and semi-destructive and fully destructive manual inspection. These traditional manual methods cannot detect weld joint quality problems in real time. Although related technologies have adopted artificial intelligence to build welding defect judgment models, these models typically require a large number of defective and normal samples. Furthermore, these judgment models do not consider the transferability to changes in operating conditions or new application scenarios. Therefore, when operating conditions or application scenarios change, it is often necessary to re-accumulate and adapt these models, affecting the feasibility of welding solutions. Summary of the Invention

[0003] This application provides a method and system for determining welding defects that can solve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of this application provide a method for determining welding defects. The method includes: acquiring process datasets corresponding to each welding process in a plurality of welding processes, thereby obtaining a plurality of process datasets; constructing corresponding process curves based on each process dataset in the plurality of process datasets to obtain a plurality of process curves, wherein the process curves are two-dimensional images of the relationship between the values ​​of process data in the process datasets and time; receiving identification information for at least one defect category existing in each welding process in the plurality of welding processes; associating each of the plurality of process curves with the at least one defect category; and training a hybrid model based on the plurality of process curves associated with the defect categories to obtain a welding defect determination model, wherein the welding defect determination model is capable of determining whether a defect exists in the welding process.

[0005] In one embodiment, each of the plurality of process datasets contains at least dynamic resistance values.

[0006] In one embodiment, the determination method further includes: deploying the welding defect determination model on an edge device or a server device.

[0007] In one embodiment, the hybrid model training includes training a semi-supervised anomaly detection model that is adversarial to generative processes using the multiple process curves associated with the defect category.

[0008] In one embodiment, after training an adversarial generation semi-supervised anomaly detection model using the multiple process curves associated with the defect category, the determination method further includes: training a classification model on the trained semi-supervised anomaly detection model to obtain a welding defect determination model.

[0009] In one embodiment, training a semi-supervised anomaly detection model for adversarial generation using multiple process curves associated with defect categories includes: acquiring feature data for each process curve among the multiple process curves associated with defect categories; performing a reconstruction operation on each process curve using the acquired feature data to obtain a corresponding reconstructed curve for each process curve; classifying the multiple reconstructed curves to obtain a first set of reconstructed curves and a second set of reconstructed curves, wherein the first set of reconstructed curves corresponds to welding processes with defects, and the second set of reconstructed curves corresponds to welding processes without defects; and comparing the multiple process curves associated with defect categories with the first reconstructed curve, and optimizing the reconstruction operation based on the comparison result.

[0010] In one embodiment, the classification model training of the trained semi-supervised anomaly detection model includes: training the classification model of the trained semi-supervised anomaly detection model based on the difference between the curve corresponding to the defect category "no defect" and other curves in the multiple process curves.

[0011] In one embodiment, each of the plurality of process datasets includes at least one of the following items: workstation, welding torch, weld point number, and operation time. After obtaining the defect determination model, the determination method further includes: converting the first working condition data into second working condition data through working condition transfer training.

[0012] Secondly, embodiments of this application provide a welding defect determination system, the system comprising:

[0013] An information acquisition unit is configured to acquire process datasets corresponding to each welding process in multiple welding processes, thereby obtaining multiple process datasets; a defect category receiving unit is configured to acquire identification information of at least one defect category existing in each welding process in the multiple welding processes; a welding process data analysis unit is configured to construct corresponding process curves based on each process dataset in the multiple process datasets to obtain multiple process curves, wherein the process curves are two-dimensional images of the relationship between the values ​​of process data in the process datasets and time; and to associate each of the multiple process curves with the at least one defect category; a hybrid model training unit is configured to perform hybrid model training based on the multiple process curves associated with the defect categories to obtain a welding defect determination model, wherein the welding defect determination model can determine whether there is a defect in the welding process.

[0014] In one embodiment, the system further includes a welding defect determination model deployment unit configured to deploy the welding defect determination model on an edge device or a server device.

[0015] In one embodiment, the hybrid model training unit further includes a semi-supervised anomaly detection model training unit, which is configured to: acquire feature data of each process curve among multiple process curves associated with the defect category; reconstruct each process curve using the acquired feature data to obtain a reconstructed curve corresponding to each process curve; classify the multiple reconstructed curves to obtain a first reconstructed curve set and a second reconstructed curve set, wherein the first reconstructed curve set corresponds to welding processes with defects, and the second reconstructed curve set corresponds to welding processes without defects; and compare the multiple process curves associated with the defect category with the first reconstructed curve, and optimize the reconstruction operation based on the comparison result.

[0016] In one embodiment, the hybrid model training unit further includes a classification model training unit, configured to perform classification model training on the trained semi-supervised anomaly detection model based on the difference between the curve corresponding to the defect category "no defect" among the multiple process curves and other curves among the multiple process curves.

[0017] In one embodiment, the welding defect determination system further includes a working condition transfer unit, configured to convert first working condition data into second working condition data through working condition transfer training after obtaining the welding defect determination model.

[0018] Compared with the prior art, this application has the following advantages:

[0019] According to the embodiments of this application, the process curves of the welding process can be associated with defect categories, and a hybrid model can be trained based on multiple process curves to obtain a welding defect judgment model. This allows for model training with only a small number of welding processes, achieving good training results and reducing the number of defect and normal samples required for model training. Furthermore, the embodiments of this application can perform condition migration of the hybrid model based on actual welding conditions, fully utilizing the original welding condition model to achieve data judgment for new welding conditions and / or scenarios, realizing cross-condition adaptability and thus improving the generalization ability of the welding process defect judgment model.

[0020] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0021] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0022] Figure 1 This is a schematic diagram of the basic structure of a generative adversarial network according to an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of the basic structure of a cyclic adversarial generative network according to an embodiment of this application.

[0024] Figure 3 This is a schematic flowchart of a welding defect determination method according to an embodiment of this application.

[0025] Figure 4 This is a flowchart illustrating a semi-supervised anomaly detection model method for training adversarial generation according to an embodiment of this application.

[0026] Figure 5a This is a flowchart illustrating a work condition transfer training method according to an embodiment of this application.

[0027] Figure 5b This is a schematic diagram of the basic structure of a work condition transfer training method according to an embodiment of this application.

[0028] Figure 6 This is a structural diagram of a welding defect determination system according to an embodiment of this application.

[0029] Figure 7 This is a block diagram of an electronic device used to implement embodiments of this application. Detailed Implementation

[0030] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0031] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0032] This application relates to applications in the fields of neural networks and computer vision. To better understand the solutions in this application, the relevant terms and concepts in the fields of neural networks and computer vision that may be involved in this application will be introduced below.

[0033] Deep neural networks, also known as multilayer neural networks, can be understood as neural networks with multiple intermediate layers. Based on the position of different layers, deep neural networks can be divided into three categories: input layers, intermediate layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are intermediate layers, or hidden layers. The layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer.

[0034] Convolutional Neural Network (CNN): A deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers; this feature extractor can be viewed as a filter. A convolutional layer is a layer of neurons in a CNN that performs convolutional processing on the input signal. In a convolutional layer of a CNN, a neuron may only be connected to some of its neighboring neurons. A convolutional layer typically contains several feature planes, each composed of a matrix of neural units. Neural units on the same feature plane share weights, referred to as the convolutional kernel. These shared weights can be understood as the way image information is extracted regardless of location. The convolutional kernel can be initialized as a matrix of random size, and during the training process of the CNN, it can learn appropriate weights. Furthermore, the shared weights can reduce the connections between layers in the CNN. It can be understood that the training process of a neural network is learning how to control spatial transformations, more specifically, learning the weight matrix. The goal of training a neural network is to make its output as close as possible to the expected value. This can be achieved by comparing the current network's prediction with the expected value, and then updating the weight vector of each layer based on the difference (of course, the weight vector is usually initialized before the first update, i.e., pre-configured parameters for each layer in the deep neural network). For example, if the network's prediction is too high, the values ​​of the weights in the weight matrix are adjusted to lower the prediction. This adjustment continues until the neural network's output value is close to or equal to the expected value. Specifically, the difference between the neural network's prediction and the expected value can be measured using a loss function or an objective function. Taking the loss function as an example, a higher output value (loss) indicates a greater difference; training a neural network can be understood as a process of minimizing the loss as much as possible.

[0035] Generative Adversarial Networks (GANs): GANs can be based on convolutional neural networks. In a GAN, two networks are trained adversarially. Figure 1 This is a schematic diagram 100 illustrating the basic structure of a generative adversarial network according to an embodiment of this application. Figure 1As shown, the generator (generator network) G receives a random noise z and generates an image using this noise. The generated image is denoted as G(z). The discriminator (discriminator network) D takes the real image and the generated image as input. The output of the discriminator D (e.g., X = G(z)) represents the probability of the real image. If the output is 1, it means that the probability of the generated image being judged as a real image is 100%. If the output is 0, it means that the probability of the generated image being judged as a real image is 0% (or in other words, the generated image is judged as not being a real image 100%). The adversarial generative network (GGN) first initializes the network parameters of the generator and the discriminator. The semi-supervised anomaly detection model training based on GGN includes: fixing the generator's network parameters, inputting the generated image G(z) and the real image into the discriminator, and training the discriminator to classify the generated image and the real image; based on the classification already completed, fixing the discriminator's network parameters and updating and optimizing the generator's network parameters to reduce the difference between the generated image and the real image, thus making it impossible for the discriminator to distinguish between generated and real images; the core of the discriminator is a binary classifier used to distinguish between generated and real images. Next, fixing the generator's network parameters again and updating the discriminator's network parameters again, enabling the discriminator to distinguish between generated and real images. The network parameters of the discriminator and the generator are trained alternately. Through this mutual adversarial competition, the functionality and performance of the generator and discriminator will continuously improve over time. The loss of the discriminator is a function of the generator's quality: if the discriminator is fooled by the generator's reconstructed output, the loss rate is high. The loss of the generator is also a function of the discriminator's quality: if the generator cannot fool the discriminator, the loss rate is high. During the training phase, the training process of the generator and the discriminator is a competitive and adversarial process, with their performance improving iteratively until they reach a Nash equilibrium in game theory (i.e., the generated image has the same probability of being distinguished from the real image). At this point, both the discriminator D and the generator G have been optimized.

[0036] Cycle GAN: Essentially, a Cycle GAN is a ring network consisting of two mirror-symmetric adversarial generative networks (GANs). The two GANs share two generators and each has its own discriminator. Sample images can be transformed without pairing. The key feature of a Cycle GAN is that it uses a loop to first transform the image from one domain to another, and then back again. If both transformations are accurate, the transformed image should be nearly identical to the input image. Through this loop, the Cycle GAN pairs the images before and after the transformation to improve the transformation accuracy.

[0037] Figure 2 This is a schematic diagram 200 of the basic structure of a cyclic adversarial generative network according to an embodiment of this application. Figure 2 As shown, G and F are generators, and D... X D Y This is the discriminator. During the training of the generator, D... X D Y The parameters are fixed, only G and F are adjustable, and the loss function of the cycle GAN network is as follows:

[0038] Minimize Loss GAN

[0039] =Minimize E x～pdata(x) [log(1-D y (G(x)))]+MinimizeE y～pdata(y) [log(1-D X (F(y)))]

[0040] =MaximizeE x～pdata(x) [logD Y (G(x))]+MaximizeE y～pdata(y) [logD X (F(y))] (1)

[0041] In formula (1), E represents the expected probability, x represents the sampled sample in domain x, pdata(x) represents the sampled sample in the image library of domain x, y represents the sampled sample in domain y, and pdata(y) represents the sampled sample in the image library of domain y. The meaning of this function is that in order to minimize the loss function of the cycle GAN network, it is necessary to minimize [logD]. Y (G(x))] and [logD X [F(y))] is maximized, that is, the parameters of G are adjusted to make D... Y The higher the score given to the data generated by G, the better. Adjust the parameters of F to make D... X The higher the score for the image generated by F, the better.

[0042] As mentioned above, Minimize Loss GAN ensures that the images generated by the generator become increasingly realistic (the conditions become increasingly similar to another type of image). The ultimate goal of the Minimize Loss cycle is to make F(G(x)) = x and G(F(y)) = y. The Minimize Loss cycle ensures that the content of the images generated by the generator remains largely unchanged.

[0043] During the training of the discriminator, the parameters G and F are fixed, and D is adjusted.X D Y The parameters for D X Regarding the update process (at this time, G, F, D) Y With all parameters fixed, the loss function of Cycle GAN is as follows:

[0044] Maximize Loss = Maximize Loss GAN

[0045] =Maximize GAN(F, D) x (X, Y)

[0046] =Maximize E x～pdata(x) [logD X (x)+MaximizeE y～pdata(y) |log(1-D X (F(y)))|

[0047] =MaximizeE x～pdata(x) [logD X (x)]+MinimizeE y～pdata(y) [logD X (F(y))] (2)

[0048] In formula (2), E represents the expected probability, x represents the sampled sample in domain x, pdata(x) represents the sampled sample in the image library of domain x, y represents the sampled sample in domain y, and pdata(y) represents the sampled sample in the image library of domain y. The meaning of this function is that to maximize the loss function of the cycle GAN network, logD needs to be maximized. X Maximize (x) to make [logD X [F(y))] is minimized, i.e., the discriminator D is trained. X At that time, D should be maximized. X The value of (x) should be minimized (so that the discriminator gives a high score to the true signal x). At the same time, D should be minimized. X The value of (F(y)) (causes the discriminator to assign a low score to the generated signal F(y)) is increased, thereby improving the discriminator's discrimination ability. Update D Y The process is similar and will not be described in detail here.

[0049] MQTT (Message Queuing Telemetry Transport) is a lightweight communication protocol based on the publish / subscribe model. Built on top of the TCP / IP protocol, it can provide real-time and reliable messaging services for connecting remote devices with minimal code and limited bandwidth.

[0050] Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data on a website.

[0051] Figure 3 This is a schematic flowchart of a welding defect determination method 300 according to an embodiment of this application. The welding defect determination method may include the following steps S301 to S305. The steps in the welding defect determination method will be described below.

[0052] In step S301, process datasets corresponding to each welding process in multiple welding processes are obtained, thereby obtaining multiple process datasets.

[0053] In one embodiment, the process dataset includes at least one of the following: dynamic resistance value, voltage value, current value, power value, and pressure value. The dynamic resistance value during the welding process reflects, to some extent, the growth process of the weld nugget, and is closely related to the welding quality. In the initial stage of welding, due to the low temperature and high material hardness, the contact resistance is relatively high. As the temperature of the welding zone increases, the material hardness decreases, the workpiece contact area increases, and the effective conductive area increases, leading to a decrease in contact resistance and thus a rapid decrease in the dynamic resistance value. Simultaneously, changes in the voltage, current, power, and pressure values ​​during the welding process are also reflected in the changes in the dynamic resistance value.

[0054] In one embodiment, the process data can be acquired from a third-party data acquisition system. This system can collect real-time welding data of the weld joint during the welding process. The real-time welding data represents the change in welding data of the weld joint over a time series. The real-time welding data may be, for example, real-time welding current and / or real-time welding voltage. The dynamic resistance value can be calculated by comparing the real-time detection data of welding current and welding voltage.

[0055] Those skilled in the art should understand that the dynamic resistance values, voltage values, current values, power values, and pressure values ​​described above are merely illustrative examples of the data items included in the process dataset and are not intended to limit the scope of the process dataset. Those skilled in the art can selectively set the data items included in the process dataset according to actual circumstances, as long as the technical principles of this application are achieved.

[0056] Next, proceed to step S302. In step S302, a corresponding process curve is constructed based on each of the multiple process datasets to obtain multiple process curves, wherein the process curve is a two-dimensional image of the relationship between the values ​​of process data in the process dataset and time.

[0057] In one embodiment, constructing the corresponding process curve based on each of the plurality of process datasets includes: parsing the collected welding process data, and using, for example, third-party automatic plotting software, to create a two-dimensional image of the relationship between the values ​​of the collected welding process data and time, wherein the two-dimensional image is the welding process curve corresponding to the welding process data. The parsing can be performed using methods including, but not limited to, MQTT or Kafka message queues. The parsing process includes extracting information related to workstation, welding torch, weld point number, time, etc., from the collected JSON (JavaScript Object Notation) string as associative information.

[0058] Next, proceed to step S303. In step S303, identification information for at least one defect category existing in each of the plurality of welding processes is received. The at least one defect category includes at least one of the following categories:

[0059] Incomplete soldering, false soldering, spatter, burrs, no defects.

[0060] In one embodiment, the user identifies at least one defect category for each welding process and labels the corresponding defect category for each welding process. For example, there may be three welding processes: welding process A, welding process B, and welding process C. After each welding process is completed, the user identifies the defect category of that welding process based on experience. For example, if the user identifies the defect in welding process A as a cold weld, then the user labels welding process A as a cold weld; if the user identifies the defect in welding process B as spatter, then the user labels welding process B as spatter; if the user identifies that welding process C has no defects, then the user labels welding process C as defect-free.

[0061] "Cold weld" refers to a weld nugget diameter that is smaller than the specified minimum diameter. A cold weld can appear as a whitish weld. Causes of cold welds include one or more of the following: insufficient current, insufficient welding time, weld points being too close together, current shunting during welding, current shunting caused by the copper backing plate, a large electrode tip leading to increased contact area between the workpiece and the electrode tip, weld point distortion, and impurities on the electrode tip surface or the plate material.

[0062] The causes of "false soldering" are similar to those of "cold soldering". Compared to cold soldering, the surface of a false solder joint appears to be without defects, but in fact, it has no weld pillar at all, causing the solder joint to fall off with the slightest touch, making it more prone to detachment than cold soldering.

[0063] "Splatter" refers to the phenomenon where solder flies off the solder joint during the welding process due to gravity, heat, equipment aging, operational errors, and / or other factors. Spattered solder can adhere to the surface of the workpiece and / or the workpiece, potentially causing a short circuit.

[0064] "Bursts" refer to the burrs formed on the outer surface of the weld when the current is concentrated at the weld seam and heated to a molten state, and then subjected to compressive force, causing excess metal and oxides to accumulate on the upper part of the weld seam. Other main causes of burrs include: excessive welding current, insufficient spot welding time, the formation of edge welds, impurities or oil on the workpiece surface, and an insufficiently small electrode tip end face.

[0065] "Defect-free" means that the weld joint is free from the aforementioned defects and other welding defect categories that affect the normal operation of the workpiece and / or equipment.

[0066] In one embodiment, the user's identification of the soldering category can be achieved by observing the wetness of the solder joint surface, the degree of solder buildup, and the corresponding position of the soldered end of the device to the pad based on experience after soldering. For example, a poor solder joint will cause the solder on the surface of the solder joint to appear whitish, and a solder joint with burrs will have an uneven and rough solder surface. The user's identification of the soldering category also includes associating the defect category of the solder joint with the soldering process data before acquiring the soldering process data through a third-party data acquisition system.

[0067] The defect categories described above are merely illustrative and are not intended to limit the welding defects involved in the technical principles of this application. The technical principles of this application can be applied to various existing welding defects as well as future welding defects in this welding field.

[0068] Next, proceed to step S304. In step S304, each of the plurality of process curves is associated with the at least one defect category.

[0069] In one embodiment, based on the defect category of each welding process received in step S303, the corresponding defect category can be marked in the image of the process curve corresponding to the welding process, thereby establishing a correlation between the process curve and the corresponding defect category.

[0070] Next, proceed to step S305. In step S305, a hybrid model is trained based on multiple process curves associated with the defect category to obtain a welding defect determination model, which can determine whether a defect exists in the welding process.

[0071] In some embodiments, for example, a user can train a welding defect assessment model using welding processes numbered 1-10 between 9:00 and 10:00 on day A. Then, the user can use the model between 9:00 and 18:00 on day A to determine whether welding processes numbered 11-100 have defects. The defect assessment process can involve inputting the process curves corresponding to welding processes numbered 11-100 into the model, which can output two results: whether a defect exists or not.

[0072] In one embodiment, after step S305, the welding defect determination method 300 may further include: deploying the welding defect determination model on an edge or server-side device. Deploying the welding defect determination model on the edge or server-side device forms a callable API interface, which accepts real-time welding process data to achieve online weld quality determination. The device may be, for example, a mobile phone, tablet computer, laptop computer, augmented reality (AR) / virtual reality (VR) device, in-vehicle terminal, etc., or it may be a server or cloud device.

[0073] In one embodiment, the hybrid model training includes training a semi-supervised anomaly detection model that is adversarial to generative processes using the multiple process curves associated with the defect category. Figure 4 This is a flowchart 400 illustrating a semi-supervised anomaly detection model training adversarial generation method according to an embodiment of this application. The semi-supervised anomaly detection model training adversarial generation method may include: step S401, acquiring feature data for each process curve among multiple process curves associated with a defect category; step S402, performing a reconstruction operation on each process curve using the acquired feature data to obtain a corresponding reconstructed curve for each process curve; step S403, classifying the multiple reconstructed curves to obtain a first reconstructed curve set and a second reconstructed curve set, wherein the first reconstructed curve set corresponds to welding processes with defects, and the second reconstructed curve set corresponds to welding processes without defects; and step S404, comparing the multiple process curves associated with the defect category with the first reconstructed curve, and optimizing the reconstruction operation based on the comparison result.

[0074] Steps S401 to S404 will be described separately below.

[0075] In one embodiment, obtaining the first reconstructed curve set can also involve: pre-training a semi-supervised anomaly detection model using a massive amount of normal curve samples, making the semi-supervised anomaly detection model a pre-trained model. The pre-trained model constructs the first reconstructed curve set by learning the features of normal samples.

[0076] In step S401, feature data for each of the multiple process curves associated with the defect category is obtained. For example, a process curve can be represented by a three-dimensional matrix, where the length and width of the three-dimensional matrix can represent the image size, and the depth of the three-dimensional matrix can represent the color channels of the image. The process curve can be a black and white image with a depth of 1. The inner product (element-wise multiplication and summation) of the three-dimensional matrix and the filter matrix (a set of fixed weights: because the multiple weights of each neuron are fixed, it can also be understood as a constant filter) is performed to obtain the feature matrix corresponding to each process curve.

[0077] In step S402, the acquired feature data is used to reconstruct each process curve to obtain a reconstructed curve corresponding to each process curve. In one embodiment, the reconstruction operation can employ, for example, average pooling, where the average value within the feature matrix represents the value of the entire matrix region. Alternatively, max pooling can be used, where the maximum value of the feature matrix represents the value of the entire matrix region. Pooling can effectively reduce the size of the feature matrix, thereby reducing the number of parameters and ultimately accelerating computation and preventing overfitting. Pooling can also be viewed as a type of convolution operation.

[0078] In one embodiment, the reconstruction operation may include normalizing the distribution of input values ​​for each layer of the neural network to force a deviated distribution back to a more standard distribution, thereby improving the speed of feature learning and training.

[0079] In one embodiment, the reconstruction operation may include using, for example, a ReLU activation function (a non-linear activation function whose output is also a linear function if a linear function is used) to retain useful features and filter out irrelevant features.

[0080] In one embodiment, the reconstruction operation may include constructing a memory enhancement module, which stores feature matrices (vectors) generated during the training process that can represent different normal samples.

[0081] In step S403, multiple reconstructed curves are classified to obtain a first set of reconstructed curves and a second set of reconstructed curves. The first set of reconstructed curves corresponds to welding processes with defects, and the second set of reconstructed curves corresponds to welding processes without defects. The classification operation may include: weighted summation of the feature matrix (vector) in the memory enhancement module and calculation of a new intermediate feature vector. After multiple rounds of convolutional and pooling layers, the final classification result in the semi-supervised anomaly detection model is generally given by one or two fully connected layers. After several rounds of convolutional and pooling layers, the information in the image can be considered to have been abstracted into features with higher information content. We can regard the convolutional and pooling layers as an automatic image feature extraction process. After the extraction is completed, fully connected layers are still needed to complete the classification task.

[0082] In step S404, the multiple process curves associated with the defect category are compared with the first reconstruction curve, and the reconstruction operation is optimized based on the comparison results.

[0083] In some embodiments, the multiple process curves associated with defect categories, such as those associated with defects like cold welds, false welds, spatter, and burrs, are compared with the first reconstructed curve. If the curve trends of the two differ significantly, the reconstruction operation can be optimized by adjusting the parameters of the reconstruction operation to reduce the difference between the reconstructed first curve and the multiple process curves associated with the defect categories. In one embodiment, after training an adversarial generative semi-supervised anomaly detection model using the multiple process curves associated with the defect categories, the determination method further includes: training a classification model on the trained semi-supervised anomaly detection model to obtain a welding defect determination model.

[0084] The classification model is based on the trained semi-supervised anomaly detection model, and further utilizes the differences between normal samples and defective samples for classification training. The classification model training is performed on the trained semi-supervised anomaly detection model based on the differences between the curve corresponding to the defect category "no defect" and other curves in the multiple process curves. In the classification model, after the network parameters are consistent with the aforementioned semi-supervised anomaly detection model, similarity-based classification training can be performed using, for example, round-robin training and cross-entropy loss function, to quantify the differences between the first reconstructed curve and the second reconstructed curve. These differences can be fed back into the pre-trained model obtained from the semi-supervised anomaly detection training to optimize the discriminator's recognition ability.

[0085] According to the embodiments of this application, the process curves of the welding process are associated with defect categories, and a hybrid model is trained based on multiple process curves to obtain a defect judgment model. Because this training process uses a generative adversarial network (GAN), the generator in the GAN can reconstruct a large number of process curves based on feature data from a small number of process curves for model training. Therefore, only a small number of welding processes are needed to train the model and achieve good training results, thereby reducing the number of defect and normal samples required for model training. Furthermore, a memory enhancement module and classification model training are introduced to improve the accuracy of the hybrid model training by strengthening the feature matrix (vector) of normal samples.

[0086] In one embodiment, due to factors such as the material of the weld joint and / or workpiece, and the different models of the controller welding clamps, the previously trained welding defect judgment model is often only applicable to specific working conditions and / or scenarios. When the working conditions and / or scenarios change, the generalization performance of the welding defect judgment model is often affected, and the accuracy of the model in judging defects also decreases. Furthermore, the time and cost required for training the welding process data acquisition and detection model for new working conditions are significant, which cannot meet the requirements for rapid deployment in industrial production. To address the issue of decreased detection performance after changes in working conditions, this solution further introduces working condition transfer training after the hybrid model to address the problem of working condition changes. The working condition transfer training includes transferring the data y under the new working condition Y to the working condition transfer model G. YX This is a model algorithm that is transferred to working condition X, so that the original judgment model still has the judgment ability of the new working condition model.

[0087] In one embodiment, each of the plurality of process datasets includes at least one of the following items: workstation, welding torch, weld spot number, and operation time. After obtaining the welding defect determination model, the determination method further includes: converting the first working condition data into second working condition data through working condition transfer training. The first working condition data can be data under a new working condition, and the second working condition data can be data under the original working condition.

[0088] The model capable of achieving the aforementioned work condition transfer can be a work condition transfer model based on CycleGAN. CycleGAN is a neural network that can learn two data transformation functions between two domains. One of these two transformation functions is G... XY (x), the G XY (x) transforms a given sample x∈X into elements of the domain Y. The other of the two transformation functions is G. YX(y) transforms sample elements y∈Y into elements of domain X. By iteratively generating the identification method, the working conditions between the two domains can ultimately be transferred. In the welding scenario, the domains refer to different working conditions. Since the mechanism causing defects does not change due to changes in working conditions, this method can still achieve good defect identification for the transferred data.

[0089] Figure 5a This is a flowchart 500a illustrating a working condition transfer training method according to an embodiment of this application. The working condition transfer training method may include the following steps S501 to S505. Figure 5b This is a schematic diagram 500b of the basic structure of a work condition transfer training method according to an embodiment of this application. The following will be combined with... Figure 5b The steps in the method of working condition transfer training are explained.

[0090] In step S501, a first process dataset corresponding to each welding process in multiple welding processes in the first welding scenario is obtained, and a second process dataset corresponding to each welding process in multiple welding processes in the second welding scenario is obtained, wherein the first process dataset and the second process dataset correspond to different and / or the same working conditions.

[0091] "First welding scenario" and "second welding scenario" can refer to different environments and / or different working conditions during the welding process. For example, different environments can refer to welding time, ambient temperature, and ambient humidity, while different working conditions can refer to work station, weld material, welding requirements, operation time, and welding clamp model.

[0092] The “first process dataset” and the “second process dataset” correspond to the “first welding scenario” and the “second welding scenario” mentioned above, respectively. The meanings of the “first process dataset” and the “second process dataset” are the same as those described above. Figure 3 The described "process datasets" are the same or similar, and will not be repeated here.

[0093] Next, proceed to step S502. In step S502, a corresponding process curve is constructed based on each process dataset in the plurality of process datasets to obtain a first process curve set A corresponding to the first process dataset and a second process curve set B corresponding to the second process dataset.

[0094] In one embodiment, it is assumed that in the first welding scenario, there are a total of 3 welding processes, and a dynamic resistance curve is obtained for each welding process. Therefore, a total of 3 dynamic resistance curves a1, a2, and a3 are obtained, which constitute the first process curve set A. Similarly, it is assumed that in the second welding scenario, there are a total of 3 welding processes, and a dynamic resistance curve is obtained for each welding process. Therefore, a total of 3 dynamic resistance curves b1, b2, and b3 are obtained, which constitute the second process curve set B. It should be noted that the number of welding processes listed here is merely illustrative and not restrictive.

[0095] The construction operations for the "first process curve set" and the "second process curve set" are the same as described above. Figure 3 The processes described are the same or similar in terms of curve construction, and will not be repeated here.

[0096] Next, proceed to step S503. In step S503, the first process curve set A is input into the working condition transition model, and a first welding scene reconstruction curve set X (including reconstruction curves x1, x2, and x3) is generated by the first generator G1. Based on the first welding scene reconstruction curve set X, a second welding scene reconstruction curve set A' (including reconstruction curves a1', a2', and a3') with the same working condition as the first process curve set A is generated by the second generator G2. The first welding scene reconstruction curve set X is determined by the first discriminator D1 to have the same working condition as the second process curve set B.

[0097] Next, proceed to step S504. In step S504, the second process curve set B is input into the working condition transition model. The second generator G2 generates a second welding scenario reconstruction curve set Y (including reconstruction curves y1, y2, and y3). Based on the second welding scenario reconstruction curve set Y, the first generator G1 generates a first welding scenario reconstruction curve set B' (including reconstruction curves b1', b2', and b3') with the same working condition as the second process curve set B. The second welding scenario reconstruction curve set Y is determined by the second discriminator D2 to have the same working condition as the first process curve set A. The first discriminator D1 is used to determine whether the input data corresponds to the working condition of the second welding scenario, and the second discriminator D2 is used to determine whether the input data corresponds to the working condition of the first welding scenario.

[0098] Next, proceed to step S505. In step S505, a loss function is determined based on the first process curve set, the second process curve set, the first generator, the second generator, the first discriminator, and the second discriminator, and the loss function is minimized to obtain the working condition migration model.

[0099] Corresponding to the application scenarios and methods provided in the embodiments of this application, the embodiments of this application also provide a welding defect judgment system 600. Figure 6 This is a structural diagram 600 of a welding defect determination system according to an embodiment of this application. The welding defect determination system may include: an information acquisition unit 601, a defect category receiving unit 602, a welding process data analysis unit 603, a hybrid model training unit 604, a working condition transfer unit 605, and a welding defect determination model deployment unit 606. The various components of the welding defect determination system 600 are described below:

[0100] The information acquisition unit 601 can be used to acquire the process dataset corresponding to each welding process in multiple welding processes. The acquisition method of the welding process data acquisition unit 601 can be referred to the above. Figure 3 The steps described in S301 will be performed and will not be repeated here. The information acquisition unit 601 can be implemented by, for example, sensors and / or third-party data acquisition systems. Those skilled in the art can adapt the information acquisition unit 601 according to the specific application scenario.

[0101] The defect category receiving unit 602 can be used to acquire identification information of at least one defect category existing in each of the plurality of welding processes. The defect category receiving unit 602 can be implemented by, for example, a third-party data acquisition system, and those skilled in the art can adaptively select the defect category receiving unit 602 according to the specific application scenario.

[0102] The welding process data analysis unit 603 is configured to construct a corresponding process curve based on each of the plurality of process datasets to obtain multiple process curves, wherein each process curve is a two-dimensional image showing the relationship between the values ​​of process data in the process dataset and time; and to associate each of the plurality of process curves with the at least one defect category. The analysis method of the welding process data analyzer 602 can be referred to the above. Figure 3 The steps described in S302 will be performed and will not be repeated here. The welding process data analysis unit 603 can be, for example, third-party automatic drawing software. Those skilled in the art can adapt the welding process data analysis unit 603 according to the specific application scenario.

[0103] The hybrid model training unit 604 can be used to train a hybrid model based on multiple process curves associated with defect categories to obtain a welding defect determination model, which can determine whether a defect exists in the welding process. In some embodiments, the hybrid model training unit 604 may further include the following model training units: a semi-supervised anomaly detection model training unit 6041 and a classification model training unit 6042.

[0104] The semi-supervised anomaly detection model training unit 6041 can be used to: acquire feature data of each process curve among multiple process curves associated with the defect category; reconstruct each process curve using the acquired features to obtain a corresponding reconstructed curve for each process curve; classify the multiple reconstructed curves to obtain a first reconstructed curve set and a second reconstructed curve set, wherein the first reconstructed curve set corresponds to welding processes with defects, and the second reconstructed curve set corresponds to welding processes without defects; and compare the multiple process curves associated with the defect category with the first reconstructed curve, and optimize the reconstruction operation based on the comparison result. The training method of the semi-supervised anomaly detection model training unit 6041 can be referred to above. Figure 4 The steps described will be followed, and will not be repeated here.

[0105] The classification model training unit 6042 can be used to train a classification model on the pre-trained semi-supervised anomaly detection model based on the differences between the curve corresponding to the defect category "no defect" among the multiple process curves and other curves among the multiple process curves. In the classification model training unit 6042, after the network parameters are kept consistent with those of the aforementioned semi-supervised anomaly detection model training unit 6041, similarity-based classification training can also be performed using, for example, round-robin training and cross-entropy loss function, to quantify the differences between the first process curve and the second process curve. These differences can be fed back into the semi-supervised anomaly detection model to optimize the discriminator's recognition ability.

[0106] The working condition transfer unit 605 can be used to convert the first working condition data into the second working condition data through working condition transfer training after obtaining the welding defect judgment model. The training method of the working condition transfer unit 605 can be referred to above. Figure 5a The steps described will be followed, and will not be repeated here. The working conditions can be at least one of the following items included in each process dataset: workstation, weld material, welding requirements, operation time, and welding clamp model, etc.

[0107] The welding defect judgment model deployment unit 606 can be used to deploy the welding defect judgment model on edge or server devices. After the welding defect judgment model is deployed on the edge or server device, a callable API interface is formed. The API interface accepts real-time welding process data to achieve online weld quality judgment. The device can be, for example, a mobile phone, tablet, laptop, augmented reality (AR) / virtual reality (VR) device, in-vehicle terminal, etc., or it can be a server or cloud device.

[0108] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0109] Figure 7 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 7 As shown, the electronic device includes a memory 701 and a processor 702. The memory 701 stores a computer program that can run on the processor 702. When the processor 702 executes the computer program, it implements the method described in the above embodiments. The number of memories 701 and processors 702 can be one or more.

[0110] The electronic device also includes:

[0111] The communication interface 703 is used to communicate with external devices and perform data exchange and transmission.

[0112] If the memory 701, processor 702, and communication interface 703 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0113] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0114] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0115] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0116] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0117] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0118] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0119] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0120] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0121] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0122] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0123] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0124] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0125] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0126] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining welding defects, the method comprising: obtaining a plurality of process data sets, each process data set corresponding to a welding process, respectively; constructing a process curve based on each process data set in the plurality of process data sets to obtain a plurality of process curves, the process curve being a two-dimensional image of values of process data in the process data set versus time; receiving identification information of at least one defect category existing in each welding process in the plurality of welding processes; associating each curve in the plurality of process curves with the at least one defect category, respectively; performing hybrid model training based on the plurality of process curves associated with the defect category to obtain a welding defect determination model capable of determining whether a welding process has defects; the hybrid model training comprises training a generative adversarial semi-supervised anomaly detection model using the plurality of process curves associated with the defect category; after training the generative adversarial semi-supervised anomaly detection model using the plurality of process curves associated with the defect category, the method further comprises performing classification model training on the trained semi-supervised anomaly detection model to obtain the welding defect determination model; the training of the generative adversarial semi-supervised anomaly detection model using the plurality of process curves associated with the defect category comprises: obtaining feature data of each process curve in the plurality of process curves associated with the defect category; reconstructing each process curve based on the obtained feature data to obtain a reconstructed curve corresponding to each process curve, respectively; classifying the plurality of reconstructed curves to obtain a first reconstructed curve set and a second reconstructed curve set, wherein the first reconstructed curve set corresponds to welding processes having defects, and the second reconstructed curve set corresponds to welding processes not having defects; and comparing the plurality of process curves associated with the defect category with the first reconstructed curve, and optimizing the reconstruction operation based on the comparison result; the classification model training on the trained semi-supervised anomaly detection model comprises performing classification model training on the trained semi-supervised anomaly detection model based on differences between curves in the plurality of process curves corresponding to the defect category "no defects" and other curves in the plurality of process curves.

2. The method of claim 1, wherein each process data set in the plurality of process data sets comprises at least a dynamic resistance value.

3. The determination method according to claim 1, further comprising: The welding defect determination model is deployed on a device at an edge or a server.

4. The determination method according to claim 1, wherein Each process data set in the plurality of process data sets comprises at least one of the following: a station, a welding gun, a welding point number, and an operation time, wherein after obtaining the welding defect determination model, the method further comprises converting first working condition data to second working condition data through working condition migration training.

5. A welding defect determination system, the system comprising: an information obtaining unit configured to obtain a plurality of process data sets, each process data set corresponding to a welding process, respectively; a defect category receiving unit configured to receive identification information of at least one defect category existing in each of the plurality of welding processes; a welding process data analysis unit configured to construct a corresponding process curve based on each of the plurality of process data sets to obtain a plurality of process curves, the process curve being a two-dimensional image of a value of process data in the process data set versus time; and associate each of the plurality of process curves with the at least one defect category respectively; a hybrid model training unit configured to perform hybrid model training according to the plurality of process curves associated with the defect category to obtain a welding defect determination model capable of determining whether a welding process has a defect; the hybrid model training unit performing hybrid model training includes training a generative adversarial semi-supervised anomaly detection model using the plurality of process curves associated with the defect category; and after training the generative adversarial semi-supervised anomaly detection model using the plurality of process curves associated with the defect category, performing classification model training on the trained semi-supervised anomaly detection model to obtain the welding defect determination model; wherein the hybrid model training unit further includes a semi-supervised anomaly detection model training unit configured to: obtain feature data of each of the plurality of process curves associated with the defect category; reconstruct each of the process curves by the obtained feature data to obtain a respective reconstructed curve for each of the process curves; classify the plurality of reconstructed curves to obtain a first reconstructed curve set and a second reconstructed curve set, wherein the first reconstructed curve set corresponds to welding processes having defects, and the second reconstructed curve set corresponds to welding processes not having defects; and compare the plurality of process curves associated with the defect category with the first reconstructed curve set, and optimize the reconstruction operation according to a comparison result; the hybrid model training unit further includes: a classification model training unit configured to perform classification model training on the trained semi-supervised anomaly detection model according to a difference between curves in the plurality of process curves corresponding to the defect category "no defect" and other curves in the plurality of process curves.

6. The welding defect determination system according to claim 5, further comprising: a welding defect determination model deployment unit configured to deploy the welding defect determination model on a device at an edge or a server.

7. The welding defect determination system according to claim 5, further comprising: a working condition migration unit configured to convert first working condition data to second working condition data through working condition migration training after obtaining the welding defect determination model.

Images