Corrective welding method and seam processing device for corrective welding

Abstract

The invention relates to a (first) method for the corrective welding of workpieces which have at least one faulty welding seam, having the following steps: - providing the workpiece which has at least one faulty welding seam; - capturing image data of the workpiece using a camera device; - carrying out a first artificial intelligence-based image analysis of the image data in order to identify the position of the at least one welding seam; - carrying out a second artificial intelligence-based image analysis of the image data in order to identify at least one fault class of the at least one welding seam; and - checking whether a corrective welding of the at least one welding seam is possible on the basis of the identified seam position and the identified fault class of the at least one welding seam.

Description

[0001] Method for corrective welding, seam processing device for corrective welding

[0002] The invention relates to a method for corrective welding and a seam processing device for corrective welding.

[0003] When welding workpieces, defects in the weld seam can occur, which can be categorized into different defect classes. If a defect occurs in a weld seam, the workpieces can be subjected to corrective welding in a post-processing station. Currently, corrective welding is performed manually by a worker, as the positioning of the defective workpiece cannot be resolved algorithmically for all defect classes. Furthermore, the worker decides whether corrective welding is possible and, if so, with which welding parameters and / or scanner geometry the corrective welding should be performed.

[0004] Due to their cognitive abilities, humans are particularly predestined for the "manual" or human evaluation of the workpiece with regard to the position and defect class of the weld seam. However, human evaluation of the workpieces requires training and experience, is subjective, error-prone, and time- and resource-consuming. Therefore, "manual" evaluation is unsuitable in many cases when reliable, high-throughput analysis of large quantities of workpieces in real time is required. Automated, fast, and reliable computer-assisted image recognition and analysis is desirable. The invention is therefore based on the object of providing a method for corrective welding of workpieces with at least one defective weld seam, which method is fast, reliable, and / or at least partially automatable.

[0005] The object underlying the invention is achieved by a method having the features of claim 1. The method, in particular the computer-implemented method, for corrective welding of workpieces having at least one defective weld seam, comprising the following steps, in particular in the stated order. A defective workpiece having at least one defective weld seam is provided in a seam processing device. Image data of the workpiece, in particular of the weld seam, is acquired by means of a camera device. Subsequently, a first image analysis based on artificial intelligence is carried out for the image data in order to identify a seam position of the at least one weld seam.Accordingly, the absolute or relative seam position of the weld seam is known, wherein the relative seam position relates to the seam processing device, in particular the camera device and / or the laser device. In addition, a second image analysis based on artificial intelligence is carried out on the image data to identify at least one defect class of the at least one weld seam. Consequently, the defect class of the weld seam, i.e. the type of weld seam defect, is known. Depending on the identified seam position and / or the identified defect class of the at least one weld seam, a check is then carried out to determine whether a corrective weld of the at least one weld seam is possible (correction suitability test). It is conceivable that some welding defects or defect classes are not suitable for corrective welding. The suitability can be determined, for example,depend on the capability and parameters of a seam processing device, in particular a laser device.

[0006] As a result, it is possible to quickly, automatically, and reliably determine whether a faulty weld seam can be subjected to corrective welding, especially if it is beneficial. The classification of welding defects is deterministic and is not subject to fluctuations due to the subjective assessment of different operators. The subsequent determination of the welding parameters can still be performed manually.

[0007] If the test shows that a corrective weld is possible, particularly if it is sensible, then it is advantageous to determine the welding parameters based on the identified seam position and / or the identified defect class of at least one weld. This allows for faster and more reliable post-processing and allows for the automation of further steps of the process. The determination of the welding parameters is based on the information obtained for the test to determine whether a corrective weld is possible, in particular the seam position and / or defect class.

[0008] The description further includes a method for corrective welding of workpieces with at least one defective weld seam, comprising the following steps, in particular in the order mentioned:

[0009] - Providing the workpiece with at least one defective weld;

[0010] - Capturing image data of the workpiece using a camera device;

[0011] - performing an artificial intelligence-based first image analysis for the image data to identify a seam position of the at least one weld seam;

[0012] - performing a second image analysis based on artificial intelligence for the image data to identify at least one defect class of the at least one weld seam; and

[0013] - Determining welding parameters depending on the identified seam position and / or the identified defect class of the at least one weld seam.

[0014] Computer-aided image analysis solutions include artificial intelligence (AI) methods based on machine learning (ML), particularly deep learning techniques that utilize artificial neural networks (ANNs). When artificial intelligence (AI) is mentioned here, this refers, for example, to machine learning (ML) methods, preferably deep learning. The currently most widely used method for training AI is so-called "supervised learning." In this method, the AI ​​is trained using training data for one or more specific goals or tasks. For training, the AI ​​is presented with combinations of training images and the associated result for which the AI ​​is to be trained—i.e., a sample solution (“labels,” “annotations,” “ground truth”) for the task to be solved using the images.This combination of training images and sample solutions enables the student to learn the task given to her, check and correct her results and thus complete a successful training session.

[0015] For the purposes of the invention, "defect classes" are understood to mean various types of weld defects. Weld defects can include, for example: pores and bubbles, inclusions, cracks, interruptions and voids, incomplete penetration or fusion, overfilling or underfilling, distortion or deformation, tarnishing, and oxidation. Depending on the seam position and / or defect class of the weld defect, different welding strategies can be selected. The welding strategy is determined, among other things, by the welding parameters of a seam processing device, in particular a laser device.

[0016] An advantageous further development provides that the first image analysis based on artificial intelligence and / or the second image analysis based on artificial intelligence comprises a deep convolutional network. Convolutional neural networks (CNNs) are a class of deep, feedforward neural networks used in image processing (e.g., image recognition), sound processing (e.g., speech recognition), and similar areas. A convolutional layer in a deep convolutional network applies a convolution operation to its input and passes the result to the next layer. In general, deep convolutional networks use relatively little preprocessing, meaning that the network learns the filters that were developed by hand in traditional algorithms, making them less dependent on existing knowledge and human effort in feature design.

[0017] The artificial intelligence for the first image analysis and the artificial intelligence for the second image analysis can be designed as a joint artificial intelligence.

[0018] To determine the seam position of the weld seam, image data, in particular one or more camera images, are preferably captured by the camera device. The AI ​​for component position and / or seam position detection detects the seam position of the weld seam using a deep convolutional network. In this case, a decision is made in the image data with pixel precision as to whether a pixel belongs to the weld seam or not. Depending on the workpiece, other classes (e.g., unwelded hairpin surfaces in the case of hairpins) can also be taken into account in the component position detection and transmitted as information to the downstream correction suitability test and / or parameter adjustment. After an exact determination of the seam position, the workpiece, in particular the at least one weld seam, can preferably be aligned towards a laser device in order to ensure optimal corrective welding.

[0019] The AI ​​for defect class detection, especially the classifier, can be a deep convolutional network, as with the AI ​​for component location and / or seam position detection, pre-trained with representative data from each defect class. The trained network calculates a probability vector for each input, indicating how likely the current input belongs to the respective defect class. With this approach, new defect classes are learned through a new training process using all the data.

[0020] Alternatively or additionally, the AI ​​for defect class detection can be trained using a few-shot / one-shot training approach. This involves training the AI ​​in advance, preferably once, and learning a specific metric to distinguish data. The metric can be viewed as a more abstract capability relative to the deep convolutional network. This trained model is preferably located on a single processing unit and requires only a few sample data samples of a defect class to perform the classification. Furthermore, this approach is capable of learning new defect classes during operation. Computationally intensive training of the network is preferably not necessary. By implementing the few-shot / one-shot training approach, it is also easy to introduce new defect classes and welding strategies directly on the system.

[0021] To improve classification, sensor data is preferably acquired using a sensor device, with the defect class being identified further depending on the sensor data. The sensor device preferably performs optical coherence tomography and / or a light section and / or a triangulation method and / or a 3D reconstruction.

[0022] It is further advantageous if the second image analysis based on artificial intelligence identifies the seam shape and / or the center of mass of the at least one weld seam, and the welding parameters are determined depending on the identified seam shape and / or the identified center of mass. Accordingly, the reliability of the method is further increased.

[0023] A further advantageous development of the invention provides that information for checking whether corrective welding is possible and / or for the welding parameters is transmitted to a laser device. It is advantageous if the corrective welding is carried out using a laser device with the determined welding parameters. The result of the classification is transmitted together with the parameters from the first AI to an algorithm which selects the welding parameters based on the classification. In addition, based on the sensor data, such as shape and / or center of mass, numerical parameters can be adapted to the respective welding situation. The result of this algorithm is transmitted to the laser, and the corrective welding is carried out using the parameters.

[0024] The camera device preferably captures 2D image data. Preferably, a third image analysis based on artificial intelligence is performed, in which 3D image data is generated from the 2D image data. The 2D and / or 3D image data are preferably used to identify the seam position and / or the defect class and / or the welding parameters.

[0025] The object underlying the invention is also achieved by a seam processing device having the features of claim 10. The seam processing device for corrective welding of workpieces with at least one defective weld seam comprises a camera device for capturing image data of the workpiece, a computing unit for performing image analyses, and a laser device for performing corrective welding of the at least one weld seam.The computing unit is configured to perform a first image analysis based on artificial intelligence of the image data to identify a seam position of the at least one weld seam, to perform a second image analysis based on artificial intelligence for the image data to identify a defect class of the at least one weld seam, and to check, depending on the identified seam position and the defect class of the at least one weld seam, whether a corrective weld of the at least one weld seam is possible.

[0026] It is advantageous if the computing unit is further configured to determine welding parameters depending on the identified seam position and defect class of the at least one weld seam, wherein the laser device is configured to perform the corrective welding of the at least one weld seam using the determined welding parameters. The computing unit can further be configured to control the laser device.

[0027] It is further advantageous if the seam processing device further comprises a sensor device for generating sensor data, in particular based on optical coherence tomography and / or a light section and / or a triangulation method and / or a 3D reconstruction. The computing unit preferably identifies the defect class further as a function of the sensor data. The laser device preferably comprises a laser beam source for generating a laser beam, processing optics for deflecting the laser beam, and a camera device, wherein the camera device is arranged coaxially to the laser beam on the processing optics. Preferably, either a scanner (PFO) or a fixed optics can be used as the processing optics. With the fixed optics, an axis system must be responsible for tracking the optics.

[0028] The setup preferably corresponds to a standard Trumpf VisionLine setup on a processing optics with a laser. VisionLine 3D can optionally be used with an OCT sensor if the 3D information provides important information for defect classification. Alternatively, the 3D information can be calculated from the 2D image using another AI. The VisionLine camera is directed coaxially from above onto the component. The VisionLine panel PC can be used as the processing unit and calculates both the neural networks and the algorithm for determining the welding strategy. In an initial expansion stage, this strategy selection can be made using a look-up table approach (Active Process Logic - APL) in VisionLine. The result is transmitted from VisionLine to the laser device via the network connection.

[0029] Further details and advantageous embodiments of the invention can be found in the following description, on the basis of which embodiments of the invention are further described and explained.

[0030] They show:

[0031] Fig. 1 shows a schematic sequence of training a deep convolutional network for a first and / or a second image analysis;

[0032] Fig. 2 shows a schematic sequence of training an AI for one and / or a second image analysis using a few-shot / one-shot training approach; and

[0033] Fig. 3 shows a schematic sequence of a method for corrective welding.

[0034] To identify a seam position and / or a defect class of at least one defective weld, a first and second image analysis as shown in Fig. 3 are used. The image analysis is based on an AI, which can be trained as shown in Fig. 1 and / or Fig. 2.

[0035] The AI ​​shown in Fig. 1 is based on a deep convolutional network. Training such an AI requires data preparation 10. This includes collecting data from labeled or designated examples, such as weld seams on a workpiece and / or weld seams with welding defects of different defect classes. To make the model more robust, additional examples can be generated by subjecting the existing examples to, for example, rotation, mirroring, or cropping. Furthermore, the dataset can be divided into training, validation, and test sets. A model architecture 12 is then determined. The types and number of layers the network should have are selected. This includes convolutional layers, pooling layers, activation functions, and fully connected layers.Training then takes place 14 by first passing an input dataset containing workpieces and / or welds through the network, with all calculations being performed in each layer. The result of the calculations is compared with the labeled value, i.e., the actual weld and / or the actual defect class, to calculate the error. The loss quantifies how well or poorly the model performs the task. Finally, the error is passed backward through the network to adjust the weights in each layer. After training, the network is validated and adjusted 16. The network is tested on the validation set to check its performance. Depending on the error rate in the validation set, parameters such as the learning rate, number of layers, or type of activation function can be adjusted. Finally, the network is evaluated on the test set. Various performance metrics (accuracy, Fl score, etc.)) are calculated to assess the quality of the network.

[0036] The AI ​​shown in Fig. 2 is trained using a few-shot / one-shot training approach. The data is also prepared in step 20, whereby few-shot and one-shot learning require a dataset with far fewer examples per class. The dataset is also divided into training, validation, and test sets. In step 22, a base model is selected, which is fine-tuned for the specific task—in this case, identifying the weld and / or defect class. In step 24, an architecture is selected that is particularly suited to identifying welds and / or defect classes. Siamese networks, triplet networks, and meta-learning, such as MAML (Model-Agnostic Meta-Learning), are suitable for this purpose. Training in step 26 is carried out using episodic training: in each training episode, a small selection of classes and examples is randomly selected.The model is then briefly trained on this small dataset. Loss functions specialized for the small dataset, such as triplet loss, can be used to train the network by encouraging it to place similar objects closer together and dissimilar objects farther apart in the feature space. The network is then validated (28). Finally, testing occurs according to step 30, where the network is evaluated using a test set or new classes it has not yet seen. Performance metrics are particularly important for assessing the model's ability to generalize to new, unknown data.

[0037] The AI ​​trained according to Fig. 1 and / or 2 can be used for the method for corrective welding according to Fig. 3. For this purpose, according to step 100, a workpiece with at least one defective weld seam is provided. Image data of the workpiece are acquired according to 102 by means of a camera device. Subsequently, a first image analysis based on an artificial intelligence, in particular trained in a method according to Fig. 1 and / or 2, is carried out according to step 104 in order to identify a seam position of the at least one weld seam from the image data. In addition, according to step 106, a second image analysis based on an artificial intelligence, in particular trained in a method according to Fig. 1 and / or 2, is carried out in order to identify at least one defect class of the at least one weld seam from the image data, in particular the same image data.

[0038] Depending on the seam position and / or the defect class, a check is carried out in step 108 to determine whether corrective welding of the at least one weld seam is possible. If the assessment is positive, welding parameters are further determined in step 110 depending on the identified seam position and / or the identified defect class of the at least one weld seam. Subsequently, the seam position and the welding parameters are fed to a laser device, which, in step 112, uses the welding parameters to perform corrective welding of the defective weld seam. Finally, the workpiece or the weld seam can be run through the process again to check whether the welding defect has been rectified and / or whether an existing welding defect can be corrected again.

Claims

Patent claims 1. A method for corrective welding of workpieces having at least one defective weld, comprising the following steps: Providing the workpiece with at least one defective weld seam; capturing image data of the workpiece using a camera device; - performing an artificial intelligence-based first image analysis for the image data to identify a seam position of the at least one weld seam; - performing a second image analysis based on artificial intelligence for the image data to identify at least one defect class of the at least one weld seam; and Depending on the identified seam position and the identified defect class of the at least one weld seam, check whether a corrective weld of the at least one weld seam is possible.

2. The method according to claim 1, wherein the method, if a corrective welding of the at least one weld seam is possible, further comprises the following step: Determining welding parameters depending on the identified seam position and / or the identified defect class of the at least one weld seam.

3. The method according to claim 1 or 2, wherein the first image analysis based on artificial intelligence and / or the second image analysis based on artificial intelligence comprises a deep convolutional network.

4. Method according to one of the preceding claims, wherein training data of the individual error classes are provided to the artificial intelligence of the second image analysis for training and / or wherein, in a few-shot / one-shot training approach, training data is provided to the artificial intelligence of the second image analysis for learning a specific metric to distinguish image data.

5. Method according to one of the preceding claims, wherein sensor data are acquired by at least one sensor device, and wherein the identification of the error class is further carried out as a function of the sensor data.

6. The method according to claim 5, wherein an optical coherence tomography and / or a light section and / or a triangulation method and / or a 3D reconstruction is carried out by means of the sensor device.

7. Method according to one of the preceding claims, wherein the seam shape and / or the center of mass of the at least one weld seam is identified in the second image analysis, and wherein the welding parameters are determined as a function of the identified seam shape and / or the identified center of mass.

8. Method according to one of the preceding claims, wherein information for checking whether a corrective weld is possible and / or for the welding parameters is transmitted to a laser device, and wherein the corrective weld is carried out by means of the laser device with the welding parameters.

9. Method according to one of the preceding claims, wherein the camera device captures 2D image data, and wherein a third image analysis based on artificial intelligence is carried out in order to generate 3D image data from the 2D image data.

10. Seam processing device for corrective welding of workpieces with at least one defective weld seam, comprising: - a camera device for capturing image data of the workpiece, - a computing unit for performing image analyses, - a laser device for performing a corrective welding of the at least one weld seam, wherein the computing unit is configured to carry out: - a first image analysis of the image data based on artificial intelligence for identifying a seam position of the at least one weld seam, - a second image analysis based on artificial intelligence for the image data to identify a defect class of the at least one weld seam, - a check, depending on the identified seam position and the defect class of the at least one weld seam, as to whether a corrective weld of the at least one weld seam is possible.

11. Seam processing device according to claim 10, wherein the computing unit is further configured to carry out: - a determination of welding parameters depending on the identified seam position and defect class of at least one weld seam, The laser device is configured to perform the corrective welding of the at least one weld seam using the determined welding parameters. Seam processing device according to claim 10 or 11, further comprising a sensor device for generating sensor data, in particular based on optical coherence tomography and / or a light section and / or a triangulation method and / or a 3D reconstruction, wherein the computing unit further identifies the defect class as a function of the sensor data. Seam processing device according to one of claims 10 to 12, wherein the laser device comprises: - a laser beam source for generating a laser beam, - a processing optics for deflecting the laser beam, and - a camera device, wherein the camera device is arranged coaxially to the laser beam on the processing optics.

Images