Monitoring of Laser Processing Processes Using Deep Convolutional Neural Networks

Abstract

To surely and quickly detect a processing error without a complicated parameterization process.SOLUTION: The present invention relates to a system for monitoring a laser machining process for machining a workpiece, the system comprising a computing unit configured to determine an input tensor based on current data of the laser machining process and to determine an output tensor based on the input tensor by using a transfer function, where the output tensor includes information on a current machining result, and where the transfer function between the input tensor and the output tensor is formed by a trained neural network.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a system for monitoring a laser processing process for machining a workpiece, and a machining system for machining a workpiece by a laser beam, the machining system including such a system for monitoring a laser processing process for machining a workpiece. The present disclosure further relates to a method for monitoring a laser processing process for machining a workpiece.

Background Art

[0002] In a machining system for machining a workpiece by a laser beam, a laser beam emitted from an end of a laser light source or a laser optical fiber is focused or collimated onto the workpiece to be machined by beam guiding and focusing optics. Machining may include, for example, laser cutting, soldering, or welding. The laser processing system may include, for example, a laser processing head.

[0003] In particular, in laser welding or soldering of a workpiece, it is important to continuously monitor the welding or soldering process and to ensure the quality of the process. Current solutions for monitoring such laser processing processes typically include what is called in-process monitoring.

[0004] In-process monitoring, i.e., monitoring of the laser processing process, is typically carried out in such a way that several signals or parameters of the laser processing process, such as temperature values, plasma radiation, the laser output of the laser processing head, the amount and type of backscattered laser output, are recorded and evaluated independently of each other. For example, measured values of signals or parameters are continuously measured or detected over a certain period to obtain signals corresponding to the parameters. The geometry of the vapor capillary (also called keyhole) and the molten pool surrounding the vapor capillary is also monitored by image processing and evaluation during the laser processing process.

[0005] After this, the individual signals are successively processed and classified, and for each of the individual signals, various setting values for the filtering process, the calculation of the median or average value, the envelope curve, the threshold value, etc. must be set by an expert. When the signal is classified, the signal is inspected as to whether it meets a certain error criterion. For example, it is inspected whether the signal is below or above a predefined threshold value. For this purpose, the individual signal is compared with a predefined reference curve around which a so-called envelope curve is arranged. Another criterion is, for example, the integral of the signal over the envelope curve.

[0006] If the signal meets the predefined error criterion during the laser processing process, an error is output by in-process monitoring. This means that the in-process monitoring generates a notification that a machining error has occurred.

[0007] Therefore, the classification of the signals and the monitoring of the geometries of the keyhole and the melt pool describe the quality of the laser processing process. Based on the classification of the signals or parameter curves and the monitoring of the geometries of the keyhole and the melt pool, machining errors are detected and classified, and based on this, the workpiece to be machined is labeled or classified, for example, as "good" (i.e., suitable for further machining or sale) or "bad" (i.e., as scrap). Furthermore, during the execution of the laser processing process, the control parameters of the process can be affected by monitoring the signals or parameters, or the geometries of the keyhole and the melt pool. Therefore, the laser processing process can be controlled.

[0008] In conventional systems, since the characteristics describing quality greatly depend on the materials used, the laser output applied, the welding speed, etc., signal processing and classification are complex. This means that the classification of signals has to be adapted using a large number of parameters. Adopting laser processing for new materials or changes in machining processes requires changes in classification parameters and image processing parameters. For example, each time the laser processing is adjusted due to product changes, it is necessary to reset or readjust the parameters again.

[0009] Due to this complexity, both signal processing and signal classification are carried out separately for each signal or each parameter, that is, independently of other signals or parameters. Therefore, the setting of parameters for both signal processing and signal classification, such as the creation of an envelope curve, etc., and image processing have to be carried out by experts.

[0010] The complexity of these systems is very high due to the number of parameters to be set, so it can only be judged and carried out by experts in the art whether which features of the signal or parameter curves or the geometry of the keyhole and the melt pool can be used to monitor the laser processing process and detect machining errors.

[0011] Therefore, the training of experts is complex and very long. Furthermore, the setting and readjustment of parameters require long production interruptions in manufacturing for customers of laser processing systems. Moreover, the risk of inaccurate parameterization is high.

[0012] Therefore, in conventional systems, the laser processing system is monitored only based on individual signal or parameter curves and the monitoring of the keyhole and melt pool geometry. Therefore, monitoring that takes into account all signals or parameters of the laser processing process and at the same time takes into account the keyhole and melt pool geometry is not carried out.

[0013] FIG. 3A shows an exemplary signal representing the plasma emission of a laser processing process, such as a welding process, and a signal of the temperature of the laser processing process within each envelope curve. Both signals are within the envelope, and thus the welding is classified as accurate. This is because the integral of the signal exceeding the envelope is less than a defined threshold.

[0014] In FIG. 3B, the signals are clearly above their respective envelope curves. With the corresponding parameterization of the threshold for the integral exceeding the envelope curve, the signals are classified as defective and machining errors are detected.

[0015] FIG. 3C shows the curves of three different signals. Each signal curve is individually classified as accurate. However, in reality, there are defective welds. To recognize this, the curves of multiple signals must be used, and in some cases, an expert must conduct an inspection.

Prior Art Documents

Patent Documents

[0016]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0017] An object of the present invention is to reliably, quickly, and without a complex parameterization process detect processing errors. Another object of the present invention is to detect machining errors during the execution of a laser processing process, preferably in real time.

[0018] Furthermore, an object of the present invention is to provide a system that automates the detection of machining errors and thus preferably enables real-time process monitoring.

[0019] Another object of the present invention is also to provide a system that can be quickly and easily adapted to detect machining errors in changing environments or situations such as changed machining processes or different workpiece materials.

[0020] Another object of the present invention is to provide a system in which the detection of machining errors is carried out while taking into account a large number of parameters.

[0021] Furthermore, an object of the present invention is to provide a system in which the detection of machining errors is carried out (so-called "end-to-end" processing or analysis) based on raw data recorded from a machining area on a workpiece surface.

Means for Solving the Problems

[0022] These objects are achieved by the subject matter of the independent claims. Advantageous embodiments and further developments are the subject matter of the corresponding dependent claims.

[0023] The present invention is based on the concept that the machining result of a laser machining process, specifically the determination of the detection of machining errors and the determination or characterization of the machining area, is carried out using a deep neural network that receives the current sensor data, control data, and / or image data of the laser machining process as input data, preferably raw data.

[0024] According to one aspect of the present disclosure, a system for monitoring a laser machining process for machining a workpiece is provided. The system includes a computing unit configured to obtain an input tensor based on the current data of the laser machining process and obtain an output tensor including information about the current machining result based on the tensor by a transfer function, wherein the transfer function between the input tensor and the output tensor is formed by a supervised neural network. The machining result may include information about machining errors and / or machining areas of the workpiece.

[0025] Therefore, the system can directly and independently obtain the machining results of the laser processing process. For example, it can be determined whether there are machining errors in the workpiece machined by the laser processing system. Furthermore, it can be determined whether the machined area of the workpiece has a predetermined feature or a predetermined geometry, for example, whether a keyhole is formed, or what range the molten pool has. Based on this, the parameters of the laser processing process can be set to avoid further errors. Therefore, the laser processing process or the laser processing system can be controlled using a system for monitoring.

[0026] Generally, the use of a neural network forming a transfer function has the advantage that the system can autonomously detect whether there are machining errors and what machining errors exist. Therefore, there is no longer a need to preprocess the detected sensor data to make it accessible for error detection. Furthermore, it is not necessary to characterize the machining quality or define an error criterion indicating some machining error. It is also not necessary to specify or adapt the parameterization of the error criterion. This simplifies the monitoring of the laser processing process. An expert in laser processing does not need to perform or accompany the aforementioned steps. The system for monitoring the laser processing process according to the aspects disclosed herein independently, i.e., automatically, performs the detection of machining errors and the determination of the geometry of the keyhole and the molten pool, and the system can be easily adapted.

[0027] Therefore, using the output tensor, the system can include information about the current machining results of the current monitoring area, such as the state of the machined area itself, for example, the range of the machined area, the presence of a so-called keyhole or vapor capillary, the presence of a molten pool, the position and / or depth of the keyhole in the molten pool, the range or shape of the molten pool.

[0028] Furthermore, the system may detect machining errors and be able to indicate their type. The output tensor may include, for example, at least one of the following information: the presence of at least one machining error, the type of machining error, the probability of a certain type of machining error, the position of the machining error on the surface of the machined workpiece. The type of machining error may be at least one of the following: pores, holes, lack of weld penetration through the workpiece, false friends, spatter, or gaps.

[0029] Accordingly, the computing unit may be configured to determine an output tensor for the current machining area of the laser machining process while the laser machining process is still in progress. By directly determining the machining result, it may be possible to monitor the laser machining process in real time. The computing unit may be configured to form the output tensor in real time and output control data to the laser machining system that performs the laser machining process. In the simplest case, the output tensor may include information on whether the machining of the workpiece is good or bad. Based on this information, the laser machining process may be controlled, for example, by adapting the process parameters accordingly. For example, the laser output may be increased or decreased, the focusing position of the laser may be changed, and the distance between the machining head of the laser machining system and the workpiece may be changed.

[0030] The transfer function between the input tensor and the output tensor is formed by a supervised neural network or a trained neural network. In other words, the computing unit may include a neural network. The neural network may be trained by error feedback or backpropagation.

[0031] The neural network can be a supervised deep neural network, such as a supervised deep convolutional neural network or a convolutional network. The convolutional network can have from 10 to 40 convolutional layers, preferably 34 convolutional layers. Further, the convolutional network can have at least one so-called "fully connected" layer.

[0032] The neural network can be configured for transfer learning. In other words, the neural network can be adapted to the changed requirements of a changed laser processing process. Specifically, the computing unit can be configured to adapt the neural network to the changed laser processing process, for example, by transfer learning based on training data.

[0033] The training data can include test data of the changed laser processing process for determining a corresponding input tensor and a predetermined output tensor, and the predetermined output tensor is associated with the test data and includes information about the corresponding previously determined machining result of the changed laser processing process. The machining result can include information about machining errors, for example, identified by an expert. To adapt or train the neural network, the training data can include multiple sets of such test data and associated output tensors. The test data can be based on the values of sensor parameters detected by at least one sensor unit during a previous laser processing process and / or the values of control parameters used during a previous laser processing process.

[0034] Therefore, the neural network forming the transfer function can be adapted to the changed situation or the changed laser processing process. For this purpose, the transfer function is modified. The changed situation may include, for example, that the workpiece to be machined has different materials, different degrees of contamination and / or thickness, or that the parameters of the laser processing change. In transfer learning, the training dataset used to train or teach the neural network, or a reduced training dataset, can be supplemented with new examples.

[0035] Therefore, the use of a trained neural network configured for transfer learning within a system for detecting machining errors according to the aspects described herein has the advantage that the system can be quickly adapted to the changed situation.

[0036] The input tensor can include or be composed of the current data of the laser processing process as raw data. Therefore, there is no need to preprocess the current data before the input tensor is created. Therefore, the data processing steps preceding the formation of the input tensor can be omitted. The neural network directly obtains the output tensor based on the raw data.

[0037] The computing unit can be configured to incorporate a plurality of current data of the machining process corresponding to the same current time point in the input tensor and associate them with the output tensor by means of the transfer function. For the simultaneous processing of all relevant current data of the laser processing process, the determination of the machining result can be made more reliably and quickly. This makes it possible to monitor the laser processing process more reliably and precisely.

[0038] The input tensor may include current data of the laser processing process, and the current data of the laser processing process includes acquired sensor data and / or control data, for example, includes 512 samples, and each sample is associated with a time point. The sensor data and control data are also referred to as process data hereinafter. The input tensor is formed from each current data by placing a window every 256 samples on top of the 512 samples. This ensures sample overlap between two input tensors created one after another. An image is captured for each sample, and the image may be associated with each sample of the sensor data and / or control data via the time point of image capture. Thus, each input tensor may include data such as, for example, sensor data, image data, and / or control data of the machining process corresponding to each time point. That is, sensor data or image data is recorded at each time point, and control data is applied to the machining process at each time point by the control unit of the laser processing system. The input tensor generated in this way may include the last n acquired sensor data and / or the last n acquired image data and / or the last n used control data for a given time point during the execution of the laser processing process. In the simplest case, n = 1.

[0039] According to one embodiment, n = 512, the first input tensor includes current sensor and / or control data, in other words, current process data, and the second input tensor includes current image data. That is, the first input tensor includes the last 512 samples or values of respective sensor data, control data, and / or image data. At a typical sampling rate of 100 kHz, a data set of dimension m×512 is generated every 5.12 ms. Here, m represents the number of m different types of data including (acquired) sensor data and (received or used) control data. The first input tensor of dimension m×512 is formed from these data sets every 2.56 ms. At a corresponding image acquisition speed of approximately 391 images / s (i.e., an image is recorded every 2.56 ms), a second input tensor of image data can be generated for each first input tensor of process data. For example, a 512×512 pixel image is acquired corresponding to the input tensor of current process data. Accordingly, the input tensor of image data generated correspondingly thereto has a dimension of 512×512 in this case.

[0040] The current sensor data may include one or more of temperature, plasma radiation, luminance of reflected laser light or backscattered laser light at various wavelengths, keyhole depth, and / or the distance between the laser processing head that performs the laser processing process and the workpiece. The control data may include the output power of the laser on the laser processing head, the focusing position, the focusing diameter, the position of the laser processing head, the machining speed, and / or the path signal. The image data may include an image of the surface of the workpiece, for example, an image of the machining area of the workpiece. The machining area may include a molten pool and / or a keyhole.

[0041] The path signal can be a control signal of a laser processing system that performs a laser processing process, and the path signal controls the movement of the laser processing head relative to the workpiece. By including the path signal in the determination of the machining signal, for example, the position on the workpiece where the resulting machining error occurred can be quickly and easily pinpointed. This is because it is known at which point in time and which area of the workpiece has been machined or is being machined by the laser processing system. Thus, the system can indicate the point in time when an error occurred during the laser processing process. Alternatively, the system can calculate the point in time based only on a known machining speed, a known point in time as a defined start time, and the time mapping of the input tensor. The time mapping of the input tensor is obtained from the generation speed of the input tensor and the number of input tensors generated since the start time.

[0042] Furthermore, the system may include at least one sensor unit for detecting current sensor data of the laser processing process during the laser processing process. Thus, the sensor data detected by the sensor unit represents the values of parameters detected or measured by the sensor unit, such as physical parameters such as temperature. The at least one sensor unit may include a temperature sensor, an optical sensor, or a plasma sensor. The sensor unit may further include a distance sensor, such as a triangulation system and / or an OCT ("optical coherence tomography") system. The distance sensor can be used to determine the distance to the surface of the workpiece, for example, the distance between the laser processing head of the laser processing system and the workpiece surface.

[0043] Furthermore, the system comprises at least one image detection unit for detecting current image data of the machining area of the workpiece during the laser processing process. The image detection unit may comprise a camera or a camera system, specifically a 2D and / or 3D camera system, preferably one having incident light LED illumination. The image detection unit may comprise a stereo camera system. Preferably, the image data corresponds to a two-dimensional image or two-dimensional imaging of a section of the workpiece surface including the machining area of the laser processing process. The machining area may include a so-called melt pool and keyhole. In other words, the image data may include images of the melt pool and keyhole.

[0044] The acquisition speeds of the image detection unit and the sensor unit may be the same. In other words, the data of the image detection unit and the sensor unit may be correlated for their respective predefined periods. The image detection unit and the sensor unit may always acquire their respective data at the same time point. For example, when the temperature sensor performs a temperature measurement, the image detection unit may record an image of the workpiece.

[0045] The current data of the laser processing process may include current sensor data and / or current image data and / or current control data of the laser processing process. Sensor data and control data are hereinafter also referred to as process data. Sensor data represents the value of at least one parameter detected or measured by the sensor unit. Control data represents the value of at least one control parameter of the laser processing process or the laser processing system.

[0046] Preferably, the computing unit includes at least one interface configured to receive the current data. The at least one interface can be configured to receive, for example, training data for training or adapting a neural network, or control data of a laser processing system, and / or sensor data of a sensor unit, and / or image data from an image detection unit. Thus, the system can be configured to receive, for example via the interface, the values of at least one control parameter from a control mechanism of a laser processing system that performs a laser processing process.

[0047] The network architecture for classifying sensor data is usually different from the network architecture for classifying image data.

[0048] According to one embodiment, in order to use image data and process data simultaneously, the respective neural networks for classifying the process data and the image data are interconnected after the last or the second last hidden layer. The feature representations of the input tensors of the image data and the process data are arranged in the last hidden layer of the respective networks. The classification of these connected features is performed in the following fully connected layer.

[0049] This procedure has the advantage that only a few layers need to be trained when training the entire network, and the network trained for the process data can be reused during use restricted to only the process data. For this procedure, the networks for the image data and the process data are trained separately. Thus, the network is learning the mapping of the input tensor consisting of image data and the input tensor consisting of process data to a feature vector.

[0050] Assuming that x is a natural number, the input tensor may include current data for the past x time points during the execution of the laser processing process. For each of the x time points, the input tensor may include the image data corresponding to this time point and sensor or control data, i.e., process data. The time points x may be equidistant from each other, for example, each may be 256 ms or 512 or 1024 ms. The input tensor may be mapped to an output tensor from a transfer function, i.e., the image data and the control data or sensor data are processed by a common transfer function.

[0051] According to one embodiment, two branches of the network are passed, and the features of each input tensor are connected in one layer. One branch of the network has image data, and the other branch of the network has process data and the input tensor. This approach is called so-called "feature-level fusion". Since the combination of image data and process data in a tensor may not be effective in certain situations, both networks can be easily separated again and used individually.

[0052] According to another aspect of the present disclosure, a laser processing system for machining a workpiece with a laser beam is provided, and the machining system includes a laser processing head for emitting a laser beam onto the workpiece to be machined and a system for detecting machining errors according to one of the aspects described herein. Preferably, the detection unit is arranged on the laser processing head.

[0053] According to another aspect, a method for monitoring a laser processing process for machining a workpiece is provided, the method including the steps of obtaining one or more input tensors based on current data from the laser processing process and obtaining an output tensor based on the one or more input tensors using a transfer function, the output tensor including information about the current machining result, and the transfer function between the input tensor and the output tensor being formed by a supervised neural network.

[0054] The present invention will be described in detail below with reference to the drawings.

Brief Description of the Drawings

[0055]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0056] Unless otherwise specified, the same reference numerals are used hereinafter for elements that are the same and have the same effects.

[0057] FIG. 1 shows a schematic view of a laser processing system 100 for machining a workpiece by a laser beam according to an embodiment of the present disclosure. The laser processing system 100 is configured to perform a laser processing process according to an embodiment of the present disclosure.

[0058] The laser processing system 100 includes a laser processing head 101, specifically a laser cutting head, a laser soldering head, or a laser welding head, and a system 300 for detecting machining errors. The laser processing system 100 includes a laser device 110 for supplying a laser beam 10 (also referred to as a "machining beam" or a "machining laser beam").

[0059] The laser processing system 100, or a part of the laser processing system 100 such as the machining head 101, for example, may be movable along the machining direction 20 according to an embodiment. The machining direction 20 may be the cutting, soldering, or welding direction and / or the moving direction of the laser processing system 100 such as the machining head 101 with respect to the workpiece 1. Specifically, the machining direction 20 may be a horizontal direction. The machining direction 20 may also be referred to as a "feed direction".

[0060] The laser processing system 100 is controlled by a control unit 140 configured to control the machining head 101 and / or the laser device 110.

[0061] The system 300 for monitoring the laser processing process includes a computing unit 320. The computing unit 320 is configured to obtain an input tensor based on the current data of the laser processing process and use a transfer function to obtain an output tensor including information about the current machining result of the laser processing process based on the input tensor.

[0062] In other words, the output tensor can be the result of one or more arithmetic operations and can include information on whether an error occurred when the workpiece 1 was machined by the laser processing system 100 and what errors occurred. Further, the output tensor can include information on the type, position, and size of the errors on the workpiece surface 2. The output tensor can also include information on the machining area of the workpiece 1, such as the size, shape, or extent of the keyhole and / or the melt pool.

[0063] According to one embodiment, the computing unit 320 is combined with a control unit 140 (not shown). In other words, the functions of the computing unit 320 can be combined with the functions of the control unit 140 in a common processing device.

[0064] According to one embodiment, the system 300 further comprises at least one sensor unit 330 and an image detection unit 310.

[0065] The at least one sensor unit 330 is configured to detect the values of the parameters of the laser processing process performed by the laser processing system 100, generate sensor data from the detected values, and send the sensor data to the computing unit 320. The detection can be performed continuously or in real time. According to one embodiment, the sensor unit 330 can be configured to detect the values of a plurality of parameters and transfer the values to the computing unit 320. The values can be detected simultaneously.

[0066] The image detection unit 310 is configured to detect the image data of the machined surface 2 of the workpiece 1 and / or the machining area of the laser processing process. The machining area can be defined as the area of the workpiece surface where the laser beam 10 hits the workpiece surface at the current time, the material of the workpiece surface is melted, and / or there are ablation holes or perforation holes in the material. Specifically, the machining area can be defined as the area of the workpiece surface where a molten pool and / or a keyhole are formed. According to one embodiment, the image detection unit 310 is disposed on the machining head 101. For example, the image detection unit 310 can be disposed on the downstream side of the machining head 101 with respect to the machining direction 20. The image detection unit 310 can also be disposed coaxially with the laser beam 10 and / or the measurement beam 13 to be described later. The computing unit 320 is configured to receive the image data detected by the image detection unit 310 and the sensor data detected by the sensor unit 330, and form an input tensor based on the current image data and the current sensor data.

[0067] Optionally, the laser processing system 100 or the system 300 includes a measurement device 120 for measuring the distance between the end of the machining head 101 and the workpiece 1 to be machined. The measurement device can include an optical coherence tomograph, specifically, an optical low coherence tomograph.

[0068] The laser device 110 can include a collimator lens 112 for collimating the laser beam 10. The coherence tomograph can include collimator optics 122 configured to collimate the optical measurement beam 13 and focusing optics 124 configured to focus the optical measurement beam 13 onto the workpiece 1.

[0069] FIG. 2 shows a block diagram of a system 300 for monitoring a laser processing process according to one embodiment.

[0070] System 300 includes a computing unit 320, at least one sensor unit 330, and an image detection unit 310. The computing unit 320 is connected to the sensor unit 330 and the image detection unit 310. As a result, the computing unit 320 can receive the image data detected by the image detection unit 310 and the sensor data detected by the sensor unit 320.

[0071] According to one embodiment, the computing unit 320 includes a processor for obtaining an output tensor. The transfer function is typically stored in the memory (not shown) of the computing unit 320 or implemented as a circuit, such as an FPGA. The memory can be configured to store other data, such as the obtained output tensor.

[0072] The computing unit 320 may include an input / output unit 322, and the input / output unit 322 specifically includes a graphical user interface for interfacing with the user. The computing unit 320 may include a data interface 321, and through the data interface 321, the computing unit can send the output tensor to an external location, such as another computing unit, a computer, a PC, an external storage unit such as a database, a memory card, or a hard drive. The computing unit 320 may further include a communication interface (not shown), and the computing unit can communicate with a network using the communication interface. Further, the computing unit 320 can graphically display the output tensor on the output unit 322. The computing unit 320 can be connected to the control unit 140 of the laser processing system 100 to send the output tensor to the control unit 140.

[0073] The computing unit 320 may further be configured to receive control data from the control unit 140 of the laser processing system 100 via the interface 321 and further incorporate the control data into the input tensor. The control data may include, for example, the output power of the laser device 110, the distance between the machining head 101 and the surface of the workpiece 1, the feed direction and speed at respective predetermined times.

[0074] The computing unit 320 forms one or more input tensors for the transfer function from the current data. According to the present invention, one or more input tensors are formed from the current data. This means that the current data is not pre - processed by the computing unit 320, the sensor unit 330, or the image detection unit 310.

[0075] The transfer function is formed by a supervised neural network, i.e., a pre - trained neural network. In other words, the computing unit includes a deep convolutional neural network. The output tensor is created by applying the transfer function to one or more input tensors. Thus, the output tensor is obtained from one or more input tensors using the transfer function.

[0076] The output tensor includes information or data about the current machining result of the laser processing process. The machining result may include, for example, the resulting machining error and / or information about the machined area of the workpiece. The information about the current machining error may include whether there is at least one machining error, the type of at least one machining error, the position of the machining error on the surface of the machined workpiece 1, and / or the size or extent of the machining error. The information about the machined area may be the position and / or size of the keyhole, the position and / or size and / or geometry of the melt pool. According to one embodiment, the output tensor may also include the probability that a certain type of machining error has occurred, or the confidence level at which the system has detected a certain type of machining error.

[0077] The image detection unit 310 may include a camera system or a stereo camera system, for example, one having incident light LED illumination. According to the present invention, the image data corresponds to a two-dimensional image of a section of the workpiece surface. In other words, as shown by way of example in FIG. 4 and described in detail below, the detected or recorded image data represents a two-dimensional image of the workpiece surface.

[0078] According to one embodiment, the computing unit 320 may be configured to graphically display an input tensor and / or an output tensor on the output unit 322. For example, the computing unit 320 may graphically display the sensor data and / or image data included in the input tensor as a curve, as shown in FIGS. 3A to 3C, or as a two-dimensional image of the workpiece 1, as shown in FIG. 4, and may superimpose these with the information included in the output tensor.

[0079] FIG. 4 shows an illustration of image data according to one embodiment. More precisely, FIG. 4 shows exemplary images of the melt pool and keyhole at 850 nm, together with overlaid geometric shape data. As they are included as information in the output tensor determined by the computing unit 320, cross 2a indicates the center of the keyhole, cross 2b indicates the center of the melt pool, line 2c indicates the contour of the melt pool, and line 2d indicates the contour of the keyhole. The surrounding rectangle 2e ("bounding box") indicates the calculated size of the melt pool.

[0080] In the case of deviation from a predetermined geometry or size of the melt pool, information indicating that the machining result of the laser processing process, for example welding, is classified as "defective" may be included in the output tensor. In this case, the system 300 for monitoring the laser processing process may output an error.

[0081] In a conventional system, target specifications or reference values for the size of the surrounding rectangle 2e must be specified or memorized. A two-stage morphological operation ("blob analysis") is performed for calculation. Parameters required for this, such as binary thresholds, must be specified by an expert in a conventional system. In this method, changes to the welding process require changes to the parameters by a skilled expert. According to the monitoring system described herein, these drawbacks are avoided.

[0082] FIG. 5 shows a block diagram of a deep convolutional neural network 400 according to the first embodiment.

[0083] According to the embodiment shown in FIG. 5, the input tensors 405, 415 include various types or kinds of sensor data, control data, and image data of a laser processing system. For example, the input tensor 405 for image data has dimensions "image height in pixel units" × "image width in pixel units", and the input tensor 415 for sensor data and / or control data has dimensions "number of types of sensor data and / or control data" × "number of samples". Therefore, the image data forms the input tensor 405 for the "branch" of the neural network 400 that reduces the image data to significant features. The sensor data forms the input tensor 415 for the branch of the neural network 400 that calculates significant features from the sensor data.

[0084] The sensor data can be, for example, the temperature measured by one or more temperature sensors, the plasma emission measured by the corresponding sensor, the luminance of the laser light reflected or backscattered on the workpiece surface measured by an optical sensor, the wavelength of the reflected laser light or backscattered laser light, or the distance between the laser processing head and the workpiece measured by a distance sensor.

[0085] The control data can be a control signal generated by a control mechanism to cause a laser processing system to perform a laser processing process. The control data can include the focusing position and the focusing diameter of the laser beam or the path signal, and the path signal represents a position signal that specifies the relative position of the laser processing head of the laser processing system with respect to the workpiece.

[0086] The sensor data and / or the control data directly form the input tensor 415 of the deep convolutional neural network. Similarly, the image data directly forms the input tensor 405 of the deep convolutional neural network. This means that a so-called "end-to-end" mapping or analysis is performed between the input tensors 405, 415 and the output tensor. Since the image data and the process data are classified by the network within this deep convolutional neural network, it is called so-called "feature-level fusion".

[0087] The computing unit can be configured to combine, for each at time point n, a set of sensor data, control data, and / or image data corresponding to each time point in the respective input tensors 405, 415, and map it as a whole to the output tensor using the transfer function 420.

[0088] According to one embodiment, the detection speed of the image detection unit for detecting the image data and the detection speed of the sensor unit for detecting the sensor data can be the same, and the image detection unit and the sensor unit perform detection at the same time point respectively.

[0089] The output tensor 430, and thus the output layer, has dimensions corresponding to the information contained therein. The output tensor 430 includes, for example, at least one of the following information: the existence of at least one machining error, the type of machining error, the location of the machining error on the surface of the machined workpiece, the probability of a certain type of machining error, the spatial and / or planar extent of the machining error on the surface of the machined workpiece, the location and / or size of the keyhole, the location and / or size and / or geometry of the melt pool.

[0090] The output tensor 430 can be transferred to a control unit (not shown) for each laser processing process. Using the information contained within the output tensor 430, the control unit can adapt the laser processing process, for example, by adapting various parameters of the laser processing process.

[0091] The computing unit can be configured to form the output tensor 430 in real time. Thus, the laser processing process can be directly controlled using the system for monitoring the laser processing process described herein.

[0092] FIG. 6 shows a block diagram of a deep convolutional neural network 600 according to another embodiment suitable for mapping an input tensor including current sensor and / or control data to an output tensor.

[0093] According to the embodiment shown in FIG. 6, the input tensor 630 includes 512 measurements or samples of four different types of sensor data and / or control data of the laser processing system, i.e., process data. The sensor data and control data directly form the input tensor 630 of the deep convolutional neural network. This means that a so-called "end-to-end" mapping or analysis is performed between the input tensor 630 and the output tensor 640.

[0094] Thus, the input layer or input tensor 630 has dimensions of 4×512.

[0095] The transfer function formed by the deep neural network should contain information about the current monitoring error, i.e., the machining error that occurred at the time the sample was obtained. The output tensor 640 should include, for example, the information "error present / absent", the presence or probability of the error "hole", the presence or probability of the error "splash", the presence or probability of the "gap" error, and the presence or probability of the "false friend / lack of weld penetration" error. Therefore, the output tensor 640 or the output layer has a dimension of 1×5.

[0096] Therefore, the deep convolutional neural network 600 according to the embodiment shown in FIG. 6 maps the input tensor 630 of dimension 4×512 to the output tensor 640 of dimension 1×5: R 2048 →R 5 。

[0097] From the inspection of the output tensor 640 or the values contained therein, the machined workpiece can be classified as "good" or "bad" using a pre-defined classification algorithm. In other words, depending on the situation, the workpiece can be classified as suitable for sale or further machining ("good"), or classified as scrap, or marked for post-machining purposes ("bad").

[0098] The deep convolutional neural network 600 (abbreviated as "CNN" hereinafter) (Deep Convolutional Neural Net) may include a plurality of convolutional layers 610 that perform convolution with multiple cores. Further, the CNN 600 may include a "fully connected" layer or block 620 and / or a "Leaky ReLu" block or layer 650. As shown in FIG. 6, the CNN includes, for example, 21 convolutional layers, and at least some of the convolutional layers include normalization (batch normalization) and residual blocks.

[0099] FIG. 7 shows a block diagram of a deep convolutional neural network 700 according to one embodiment suitable for mapping an input tensor including current image data to an output tensor.

[0100] In the case of the deep convolutional neural network 700 according to the embodiment shown in FIG. 7, the input tensor 730 includes an image of a workpiece having a size of 512×512 pixels, for example, an image of a machining area of the workpiece. That is, the input layer 730 has a dimension of 512×512.

[0101] The input tensor 730 includes the detected raw data of the image data. These raw image data directly form the input tensor of the deep convolutional neural network. This means that a so-called "end-to-end" mapping or analysis is performed between the input tensor 730 and the output tensor 740. The features of the keyhole or the melt pool are not calculated or parameterized in the intermediate steps.

[0102] The transfer function is for providing information about whether a keyhole exists, and / or information about the centroid or center position of the keyhole, information about the rectangle around the keyhole, and / or information about the rectangle around the melt pool.

[0103] Therefore, the output tensor 740 includes the values "Pkeyhole" (keyhole presence / absence), "XKeyhole" (position of the centroid or center of the keyhole in the x direction), "YKeyhole" (position of the centroid or center of the keyhole in the y direction), "dXKeyhole" (size of the keyhole in the x direction), "dYKeyhole" (size of the keyhole in the y direction), "Xmelt_pool" (position of the centroid or center of the melt pool in the x direction), "Ymelt_pool" (position of the centroid or center of the melt pool in the y direction), "dXmelt_pool" (size of the melt pool in the x direction), and "dYmelt_pool" (size of the melt pool in the y direction). Therefore, the output tensor 740 or output layer includes nine values and thus has a dimension of 1×9.

[0104] Therefore, according to the embodiment shown in FIG. 7, the neural network 700 maps an input tensor 730 of dimension 512×512 to an output tensor 740 of dimension 1×9:R 262144 →R 9 。

[0105] As shown in FIG. 7, the CNN includes, for example, 34 convolutional layers 710, and at least some of the convolutional layers include normalization ("batch normalization") and so-called residual blocks. The convolutional network also has two so-called "fully connected" layers 720. The neural network 700 is serialized in the last layer 750 and mapped to the output tensor 740 by a sigmoid activation function.

[0106] By normalizing the output of the layer, the problem of gradients that "explode" or "vanish" can be avoided. The behavior in the inference process is less sensitive to data from other distributions.

[0107] Normalization usually includes the mean value and standard deviation over a "mini-batch". Its effect is regularization.

[0108] According to one embodiment, these parameters are used as hyperparameters in a trained deep convolutional neural network: "Batch Normalization", "Accelerating Deep Network Training by Reducing Internal Covariate Shift" (by Sergey Ioffe, Christian Szegedy).

[0109] In FIG. 7, the "Convolution 32 3×3" block represents a convolution block or convolution layer having 32 different 3×3 convolution filter masks. That is, the block "Convolution 32 3×3" generates a tensor of dimension m×n×32 from an input tensor 730 of dimension m×n×c, where m represents the height, n represents the width, and c represents the number of channels. In the case of a single-channel (c = 1) input tensor having a height m = 512 and a width n = 512, an output tensor 740 of dimension 512×512×32 is formed, and as a result, the output tensor includes 32 images of dimension 512×512. The same applies to other convolution blocks.

[0110] The notation " / 2" within the convolution block of FIG. 7 describes a "stride" of 2. That is, the filter core is shifted forward by 2 pixels, and as a result, the dimension is reduced by half. The information above the block, for example, "512×512", describes the dimensions m×n of the tensor without the number of channels.

[0111] The notation "residual block" specifies that the output (1) of the previous layer is added (1 + 2) to the result of the output layer before the value is passed through the activation function.

[0112] FIG. 8 shows a deep convolutional neural network for classifying image data and process data. According to one embodiment, neural network 800 includes neural networks 845, 855 according to the embodiments of FIGS. 6 and 7. In this case, neural network 800 is created by chaining or coupling or linking at least one fully connected layer 830 and optionally another fully connected layer 820 with what is called a "Leaky ReLu" activation function. The mapping of the last fully connected layer 830 to output tensor 840 can be performed by a sigmoid activation function. The two input tensors 805, 815 of each neural network 845, 855 are mapped to output tensor 840, and output tensor 840 has the following components: P(error), P(hole), P(spatter), P(gap), P(false friend), "x_keyhole", "y_keyhole", "dx_keyhole", "dy_keyhole", "x_melt_pool", "y_melt_pool", "dx_melt_pool", and "dy_melt_pool". P represents the probability of a certain type of machining error. Thus, output tensor 840 of neural network 800 has a dimension of 1×13.

[0113] The neural networks used in the embodiments of FIGS. 5 to 8 are pre-trained deep convolutional neural networks or supervised deep convolutional neural networks. In other words, before delivery of the system for detecting machining errors, the CNN has learned from examples of "good" and "bad" machined workpiece surfaces, or examples of "good" and "bad" welds or solders or weld seams. In other words, the CNN has learned to classify machined workpiece surfaces as "good" or "bad", or to detect machining errors, locate the positions of machining errors, classify machining errors according to the types of machining errors, and determine the sizes of machining errors.

[0114] In the case of in-process monitoring, the system should reliably determine whether the machined workpiece surface has machining errors or what geometric characteristics the machining area has. Preferably, the system detects what errors exist (e.g., pores, holes, protrusions, spatter, adhesion, or lack of weld penetration, or "false friends"), and in some cases, also locates the position of the machining error and may indicate the size of the machining error on the workpiece surface. To train the CNN and set the hyperparameters, an input dataset and a corresponding output tensor are supplied to the CNN. The specified input dataset includes, for example, sensor, image, and / or control data of the laser processing process as described above. A corresponding predetermined output tensor or result tensor is associated with each predefined input dataset. This output tensor contains the desired results of the CNN for each laser processing process for each input dataset.

[0115] According to the embodiment illustrated in FIG. 8, individual networks for image data and process data are used to train the network for the image data and process data. After training, the last fully connected layer contains the representation of significant features. Using the parameters of each individual network thus obtained, the networks are chained as described, and only the layers after chaining are trained.

[0116] In other words, the corresponding predetermined output tensor includes information about the classification of machining errors present on a section of the machined workpiece surface and / or information about the geometric features of the machining area. This mapping of the output tensor to each respective predetermined input data set is performed manually (so-called "labeling" of the detected sensors, images, and control data). That is, a predetermined mapping of the sensors, images, and control data to the results of the transfer function is performed. For example, whether machining errors occurred in the laser machining process used as the basis for the input data set, what types of errors exist, for example, using a two-dimensional coordinate system with x and y coordinates, where on the machined workpiece surface the machining errors exist, the size of the machining errors in the x and y directions, whether keyholes and / or melt pools exist, where the keyholes and / or melt pools are relative to each other or relative to the current machining point, what area and / or what semi-axes the keyholes and / or melt pools have, etc. are specified in the output tensor.

[0117] Next, the transfer function formed by the CNN is determined by an optimization method and stored within the system 300, preferably in the memory of the computing unit 320. The optimization process is carried out in a "backpropagation" process using, for example, Adam optimization. For inference, the CNN provides a mapping of the input data set to the machining results.

[0118] According to one embodiment, the following parameters are used as hyperparameters within the trained network: "Batch Normalization", "Accelerating Deep Network Training by Reducing Internal Covariate Shift" (by Sergey Ioffe, Christian Szegedy).

[0119] The teacher-aided deep folding neural network is configured to be adaptable to a changed situation or a changed laser processing process by so-called transfer learning. The basic training of the network is carried out before the commissioning of the system. In the case of changes to the machining process after commissioning, only what is called transfer learning is carried out. A changed situation can be, for example, that the workpiece to be machined changes when, for example, the material changes. The thickness or material composition of the workpiece surface can also change slightly. Furthermore, other process parameters can be used to machine the workpiece. This can cause other machining errors. For example, the probability of various types of machining errors can change, or the machining errors can be formed differently. This means that the neural network has to be adapted to the changed situation and the resulting change in machining errors.

[0120] Transfer learning proceeds in the same way as the initial teaching of the neural network. However, usually only some specific convolutional layers of the deep convolutional neural network, specifically the last two to three convolutional layers, are adapted in transfer learning. The number of parameters of the neural network that are changed is significantly less than when training or teaching the neural network. This enables the customer to complete transfer learning quickly, usually in less than one hour. This means that in transfer learning, the entire neural network is not retrained or reteached.

[0121] The system 300 can receive the training data required for transfer learning via the interface 321.

[0122] The training data may include a test data set of the modified laser processing process. During transfer learning, the computing unit forms a corresponding input tensor from the test data set. Further, the training data includes a predetermined output tensor associated with each test data set, and includes information about the corresponding machining results of the modified laser processing process previously determined by an expert.

[0123] For example, the test data set includes sensor data detected when a machining error occurred during the previous laser processing process, and the associated output tensor includes information about the error, such as the type of error, the position and extent of the machining error on the workpiece.

[0124] Figure 9 shows a method for monitoring a laser processing process for machining a workpiece. The first step 910 includes obtaining an input tensor based on the current data from the laser processing process. In the second step 920, an output tensor is obtained based on the input tensor using a transfer function, and the output tensor includes information about the current machining result. The transfer function is pre-obtained and formed by a supervised neural network.

[0125] A method for monitoring a laser processing process while a workpiece is being machined may be implemented. According to one embodiment, the method is executed once across the entire machined workpiece surface.

[0126] The use of a neural network that forms a transfer function has the advantage that the system can independently detect whether machining errors exist and which machining errors exist. Therefore, it is no longer necessary to preprocess the received current data such as image data and sensor data in order to make it accessible for error detection. Furthermore, it is not necessary to extract features that characterize the processing quality or any machining errors from the detected data. Furthermore, it is not necessary to determine which of the extracted features are necessary or relevant for the evaluation of machining quality or the classification of machining errors. It is also not necessary to specify or adapt the parameterization of the extracted features for classifying machining errors. Thereby, the determination or evaluation of machining quality or machining errors by a laser processing system is simplified. It is not necessary for an expert in laser processing to perform the above steps or to accompany the above steps.

Claims

1. A system (300) for monitoring a laser machining process for machining a workpiece (1), comprising: a computing unit (320) configured to determine an input tensor based on current data of the laser machining process and to determine an output tensor based on the input tensor using a transfer function, the output tensor comprising information about a current machining result; a sensor unit (330) for detecting process data representative of parameters of the laser processing process; an image detection unit (310) for detecting image data of the workpiece (1) processed by the laser processing process; Equipped with the transfer function between the input tensor and the output tensor is formed by a supervised neural network; the input tensor includes, as raw data, the process data, including current control data for the laser processing process, and the image data; The supervised neural network includes a first neural network to which the process data including the current control data is input, and a second neural network different from the first neural network to which the image data is input, the first neural network and the second neural network being coupled to each other via at least a common output layer; the supervised neural network is configured to map input tensors of the first neural network and input tensors of the second neural network to a common output tensor; the at least one output layer includes at least one fully connected layer; A system (300).

2. 2. The system (300) of claim 1, comprising: the process data of the laser processing process includes at least one of temperature, plasma emission, laser power, brightness of reflected or backscattered laser light, wavelength of reflected or backscattered laser light, distance of a laser processing head performing the laser processing process relative to the workpiece, keyhole depth, focus position, focus diameter, path signal, and / or an image of a surface of the workpiece; A system (300).

3. 3. A system (300) according to claim 1 or 2, comprising: the process data is based on a plurality of samples, and the image data includes an image; the images correspond in time to the samples of the process data; A system (300).

4. A system (300) according to any one of claims 1 to 3, comprising: the machining result comprises information about machining errors and / or machined areas of the workpiece (1), A system (300).

5. A system (300) according to any one of claims 1 to 4, comprising: the output tensor comprises one of the presence of at least one machining error, a type of machining error, a location of the machining error on a surface of the machined workpiece, a probability of a certain type of machining error, and a spatial and / or planar extent of the machining error on the surface of the machined workpiece; A system (300).

6. A system (300) according to any one of claims 1 to 5, comprising: the computing unit (320) is configured to form the output tensor in real time and output control data to a laser processing system (100) that performs the laser processing process. A system (300).

7. A system (300) according to any one of claims 1 to 6, comprising: The supervised neural network can be adapted to the altered laser processing process by transfer learning based on training data. A system (300).

8. 8. The system (300) of claim 7, further comprising: The training data is test data of the altered laser processing process to determine a corresponding input tensor; a predetermined output tensor associated with each of the test data and containing information about a corresponding previously determined machining result of the altered laser machining process; A system (300) comprising:

9. A laser processing system (100) for machining a workpiece with a laser beam, comprising: a laser processing head (101) for emitting a laser beam (10) onto a workpiece (1) to be machined; A system (300) according to any one of claims 1 to 8, A laser processing system (100) comprising:

10. A method (800) for monitoring a laser machining process for machining a workpiece, comprising: detecting (805) process data representative of parameters of the laser processing process; detecting (815) image data of the workpiece machined by the laser machining process; determining (820) an output tensor based on an input tensor using a transfer function, said output tensor containing information about a current machining result; Including, the transfer function between the input tensor and the output tensor is formed by a supervised neural network; the input tensor includes, as raw data, the process data, including current control data for the laser processing process, and the image data; The supervised neural network includes a first neural network to which the process data including the current control data is input, and a second neural network different from the first neural network to which the image data is input, the first neural network and the second neural network being coupled to each other via at least a common output layer; the supervised neural network is configured to map input tensors of the first neural network and input tensors of the second neural network to a common output tensor; the at least one output layer includes at least one fully connected layer; 13. A method (800).

Images