Method and system for controlling a laser welding process

Abstract

The present application relates to a method and a control system for a laser welding machine for welding at least two metal pieces separated by a welding joint, wherein the machine comprises an electronic unit for the processing and automatic recognition of a welding joint, configured according to a supervised learning model for object recognition in images, arranged to receive as input at least one image of a welding region and to identify therein an area of adjacent pixels indicative of a welding joint. In a learning phase, the supervised learning object recognition model is trained by means of a series of training data which each comprise a training image of a welding region including at least one welding joint and respective classification data adapted to mark at least one area of adjacent training pixels, indicative of the welding joint in the training image. In an operating step, at least one current welding region image of the machine is acquired and provided as input to the electronic processing and automatic recognition unit to identify an area of adjacent pixels indicative of a welding joint in the welding region image, and to control at least one processing parameter of the machine based on spatial geometric and morphological properties of the identified welding joint.

Description

[0001] Method and system for controlling a laser welding process

[0002] The present invention relates to the laser processing of materials, in particular metal materials, and more specifically improvements in a method and in a system for laser welding metal materials.

[0003] More specifically, the invention relates to a method and a system for controlling a laser welding machine for welding at least two metal pieces separated by a welding joint, according to the preamble of claims 1 and 15.

[0004] In the description and in the claims which follow, the terms "metal material" and "metal piece" (or plural forms thereof) are used to identify any manufactured item, such as a sheet or an elongated profile having a cross-section which is either closed - for example of circular, rectangular or square hollow shape - or open - for example a flat section or a section in the shape of L, C, U, H, I, etc. The metal pieces to be welded may be of the same metal material or of separate metal materials.

[0005] The term "welding joint" refers to a generic joining line or track between the metal pieces, which may be in close contact with each other or be separated by a channel or groove. The configuration of the welding joint may be that of a butt joint, in which the metal pieces face each other along the welding joint, or a more complex configuration, for example that of a lap joint, in which the metal pieces are at least partially overlapping along the joint line.

[0006] In the industrial processing of materials, and of metal sheets and profiles in particular, the laser is used as a thermal tool for a wide variety of applications which depend on the interaction parameters of the laser beam with the material being processed, specifically on the energy density per incidence volume of the laser beam on the material and on the interaction time interval. By directing a laser beam with power in the order of kW or tens of kW along a welding joint between two pieces of metal material made of the same material or of different materials, a process of localized fusion of the material(s), and / or of a possible filler material, is achieved, with consequent welding of the pieces.Laser welding machines are well known and comprise machines for industrial processing with an ytterbium-doped fiber laser source, or further with a direct diode laser source, adapted to emit a single-mode or multimode laser beam, with an optical path of the laser beam in optical fiber. From the emission source, the laser beam is routed along an optical beam transport path toward a processing head comprising an optical system for focusing the laser beam along an optical axis of propagation incident on the material being processed. The processing head may be moved in space, along the two dimensions (X, Y axes) of an underlying work plane (or a generic work surface) and also in height (Z axis), i.e., the distance thereof from the work plane or surface may be modified, and it may be oriented in rotation about said axes, in particular to maintain a condition of tangentiality of the direction of movement of the head with respect to the welding trajectory.

[0007] A machine for industrial processing with a laser beam launched into an optical fiber, according to the prior art, is shown diagrammatically in Figure 1.

[0008] It comprises an emission source 10, such as a laser generator device capable of launching a laser beam B into a transport fiber, for example a ytterbium-doped fiber laser, or a direct diode laser, adapted to emit a single-mode or multimode laser beam, and an optical fiber guide 12 adapted to conduct the laser beam emitted from the emission source toward a processing head 14 arranged near work pieces WP, for example metal pieces to be welded along a welding joint S. At the work head, the laser beam emerging from the fiber with a controlled divergence thereof is collimated by a dioptric collimation system 20 and reflected by a catoptric system 22 before being focused through an optical focusing system 16, generally consisting of a focusing lens, along an optical propagation axis, incident on the material, through the emission nozzle 18.

[0009] In welding processes, a filler material in powder or wire may be supplied near the welding area, to be melted by the laser radiation, obtaining a bond between the parts to be joined following the re-solidification of the material itself.

[0010] In the context of a welding process, the quality of the processing depends on the interaction of the laser beam with the materials being processed and, possibly, with a filler material andan assist gas, which interaction in turn depends on the type of material being processed, filler material, and assist gas (and, more generally, on the processing atmosphere). The interaction of the laser beam B with the materials being processed depends on the welding trajectory -i.e., on the evolution in space of the welding joint S between the metal pieces WP, and on the width of said welding joint. Different metal alloys respond differently to the processing laser beam, and further differences in the execution of the process are observed as a result of the surface conditions of the material, for example, the oxidation state thereof or other surface modifications such as the presence of metal surface coating layers or conditions of protective oils, and of the process atmosphere.

[0011] The control of the propagation direction of the laser beam on the processing area of the material, for example as a function of the welding trajectory to be followed, brings advantages to the processing process. For example, the control of the power distribution of a laser beam over time along a welding trajectory may allow the power distribution to be localized or expanded where needed in relation to the gap of the welding joint of the pieces being processed or to the welding trajectory to be tracked.

[0012] It is evident that the control of the propagation direction of the welding laser beam must be as precise as possible, so that the indicated advantages may be obtained. Accordingly, it is necessary that the movement of the processing head and the position of the proximal end thereof with respect to the material, that is, the position of the laser beam output with respect to the pieces, and, in particular, with respect to the point of incidence of the laser beam on the material, be controlled with extreme precision and in real time as a function of the current position along the welding trajectory. Otherwise, there is a risk of focusing the laser power in a non-optimal position with respect to the welding joint between the pieces to be welded.

[0013] For these reasons, in the field of laser processing it is desirable to be able to accurately determine in advance the welding trajectory which the processing head must follow and to know the local conditions of the pieces to be welded along the welding trajectory, for example the mutual distance thereof, i.e., the width of the welding joint.

[0014] The calculation of the welding trajectory is necessary for determining in advance thetrajectory itself and consequently controlling the processing head in real time, without the need to periodically stop or slow down the process, which would entail stoppage of movement of the processing head and / or shutdown of the laser, with a consequent slowdown in the execution of the process and poor quality of the resulting weld.

[0015] The estimation of the welding width is instead necessary to determine, for example, whether the welding process must occur with or without addition of material, or by modifying the focusing of the laser beam in the neighbourhood of the welding joint, or also whether to increase / decrease the power of the laser beam at determined points of the welding trajectory, the movement speed of the welding head, the dwell time of the laser beam on a predetermined welding area.

[0016] Current techniques for controlling welding processes are based on the acquisition of a temporal sequence of images of the material (or materials) being processed, on the recognition of a welding joint between two or more pieces being processed starting from the acquired images, and on controlling the movement of the processing head with respect to the pieces being processed, or, more generally, on the control of the relative movement between the processing head and the pieces being processed (which also comprises controlling rotary -tilting tables, lathes, or other devices on which the pieces being processed are positioned and fastened, possibly keeping the processing head fixed) for tracking the welding joint, as well as on controlling the possible addition of material as a function of the width of the welding joint. As a function of the trend and the width of the welding joint, it is possible to control the localized emission of the welding laser beam at the junction between the pieces being processed, along the extension line thereof (which may be straight, curved, broken or mixed, open or closed), as well as the focusing of the laser beam on different processing planes through controlling the distance of the processing head from the pieces being processed or the focusing of the beam.

[0017] An example of the current techniques is shown in Figure 2, where a sequence of process steps (a)-(e) is highlighted.

[0018] In Figure 2(a) an image of a welding region between two facing pieces Pl and P2 is shown,in which a welding joint between the pieces Pl and P2 is indicated by S, a welding pool at the current time by W, i.e., an area in which the material forming the pieces Pl and P2 is locally molten, thereby forming the union between the two pieces, or a laser emission area at the current welding point where the welding laser beam is incident. An area of interest or area of welding advancement is indicated with A, i.e. the area in which it is intended to determine the direction and width of the welding joint S, which will be reached by the processing head in a predetermined time interval in the future (time horizon of the welding process), for example in the order of hundreds of milliseconds.

[0019] In current techniques, the determination of the direction and the width of a possible slit present along the welding joint within a predetermined time horizon of the welding process is based on the acquisition of a real grayscale image of the area of interest A, the transformation into a gradient image along a predetermined direction of exploration of the image (Figure 3b), the extraction of features from the image through a technique adapted to recognize linear structures in the image, such as, by way of example, the Hough transform (Figure 3c), so as to determine the orientation and position of the welding joint in the image, and identify the edges of the welding joint based on the gradient of the pixel values determined along said image direction of exploration, generally perpendicular to the estimated direction of the welding joint (Figure 3d) for estimating the width of the welding joint. For example, with reference to Figure 3d, in the lower diagram the profile along the y axis of the pixel value in a predetermined section x is shown, and in the upper diagram the profile of the first derivative of the pixel value, the minimum and maximum of which identify, in the gradient image, the edges SI and S2 of the welding joint S.

[0020] The midpoint of the local width of the welding joint thus identified is calculated along the image x axis (Figure 3e) as a new welding point in the current time horizon of the welding process, and a welding trajectory T is calculated by interpolation with the welding points determined in the preceding steps.

[0021] Disadvantageously, this technique fails in the case of non-optimal image acquisition conditions, and exhibits a high sensitivity to the material types being processed, which determine images in which the welding joint is not easily resolvable by the described imageprocessing algorithms, and long welding trajectory inference times.

[0022] The present invention has the object of providing a method and a system for controlling a laser welding process, which can be implemented efficiently and on a production line, i.e., ensuring real-time control of the welding process.

[0023] A further object of the invention is to provide a method and a system for controlling a laser welding machine which ensures good performance under different operating conditions and with different materials.

[0024] A still further object of the invention is to provide a method and a system for controlling a welding process which is scalable, i.e. which can be extended to different use cases (different materials, welding trajectories and geometries of the welding joint, also including three-dimensional geometries).

[0025] According to the present invention, such objects are achieved by virtue of a control method of a laser welding machine having the features recited in claim 1.

[0026] Particular embodiments form the subject matter of the dependent claims, the content of which is to be understood as an integral part of the present description.

[0027] A further object of the invention is a control system for a laser welding machine as claimed.

[0028] In summary, the present invention relates to a method and a system for controlling a laser welding process of two or more pieces separated by a welding joint, for example, but not limited to, abutting or facing each other, preferably of metal materials, i.e., for controlling a laser welding machine along a welding path, the fundamental feature of which is the optimization of the recognition of the welding joint starting from the acquired images of a processing area, through artificial vision techniques with deep learning. Among these, techniques for object classification using supervised learning are preferably employed. In particular, the present invention provides a method for estimating both the development trajectory of the welding joint between the pieces being processed, and the local width of thewelding joint, simultaneously and in real time during the execution of the welding process.

[0029] Unlike the prior art, which uses a deterministic image analysis approach for the recognition of the welding joint, the invention is based on the use of deep-learning artificial intelligence tools, configured to identify a welding joint in an image, i.e., at least one segment of a development trajectory of the welding joint through an object recognition technique, namely geometric shapes, representative of the joint, and to estimate the width of such a joint, within a predetermined time horizon of the welding process.

[0030] More specifically, starting from an image, such as a two-dimensional image, of a welding region or of an area of interest of the welding region, in a position ahead of the current welding point, i.e. an image which is a representation - for example two-dimensional - of a welding region in which a trajectory of a welding joint develops downstream of the current welding point, acquired by imaging means integrated on the welding head, for example a video camera preferably coaxial with respect to the welding laser beam, optionally assisted by a diffuse light illuminator, a welding joint is identified by an object recognition algorithm or model based on an instance segmentation activity, i.e. an artificial vision technique adapted to identify objects in the image, delineating the precise boundaries of each object at the pixel level. The algorithm is trained based on a set of training images which include overview images of processing zones of a material, i.e. zones of development or execution of a welding process, i.e. further welding regions comprising a geometric development trajectory (which can also be identified as a spatial evolution trajectory) of a welding joint, and further optionally include a laser welding area in progress, and again optionally a filler material, representative of a plurality of predetermined types or classes of welding joints in a plurality of image acquisition conditions of the welding region, of different materials and material states. The image acquisition conditions comprise lighting conditions of the pieces being processed, image focusing conditions, presence of artifacts in the images, such as an illuminated area corresponding to the laser emission in progress, the presence of filler material and of a nozzle for supplying an assist gas. In the training images, the welding joint is, for example, identified by the separation line between two facing pieces or - in the case of a lap joint - by the shadow line projected by the edge of the upper piece onto the lower piece.The set of images comprises a plurality of images pre-classified in a supervised manner, possibly including a plurality of synthetic images obtained by alteration of a plurality of original images, according to a data augmentation technique, to take into account possible alterations in the materials being processed, for example scratches or striations of the materials, in the arrangement of the welding joint with respect to the imaging means, in the processing environment or in the image acquisition means and in the acquisition conditions of the images during the operating cycle of the machine, as well as to increase the quantity of images fortraining. Advantageously, the spatial orientation capability of the welding head and of maintaining a condition of tangency of the movement direction thereof with respect to the welding trajectory ensures image acquisition with a field of view unaltered by the joint geometry.

[0031] Starting from the geometric shape of the welding joint thus identified, the corresponding medial axis is computed through post-processing, and the position of a subsequent welding point along the trajectory, at a predetermined distance, determined as a function of the image acquisition and processing time, from the current point of incidence of the laser processing beam, is extracted.

[0032] In the neighbourhood of the point corresponding to the position of incidence of the laser, which corresponds to the center of the image in the event of data acquired by means of a camera which is coaxial with respect to the laser beam, a region of interest of the image is extracted, which is analyzed by a further classification algorithm or model, preferably a convolutional neural network, for estimating the width of the welding joint, in the case of butt-welding j oints.

[0033] The identification of the point of the welding trajectory and the width of the welding joint are provided as input to a control logic of the welding machine for controlling at least one processing parameter of the machine (trajectory, local motion, power distribution of the welding laser beam) based on spatial geometric properties of the identified welding joint (such as the position of the welding j oint and the orientation thereof with respect to the welding head) and of the morphological properties thereof (such as the joint type, the gap thereof, and thecoupling conditions of the pieces being processed), for example for the tracking of a welding trajectory and to control the welding laser beam in a point incidence configuration along the welding joint or in a configuration of static beam shaping or wobbling beam based on the estimated joint width. The term wobbling beam means, in general, a laser beam the time evolution of which at a point of the welding traj ectory follows a circular, elliptical, or otherwise cyclic path according to a predetermined law of motion in the neighbourhood of the welding point, so as to deliver greater thermal power to the material or to cause the melting of a greater area of material of the pieces being processed, and possibly of a filler material, sufficient to fill the gap of the welding joint.

[0034] An improvement of the invention includes the estimation of the welding joint gap with a variable position of the welding head along the Z axis, i.e., with a variable distance of the welding head with respect to the welding plane.

[0035] The processing parameters of the machine may also comprise the pressure of the assist gas, a pulse repetition frequency of the welding laser beam, the duration of such pulses, the power thereof, the advancement speed of the wire of filler material.

[0036] Advantageously, the method which is the subject matter of the invention allows achieving control of a welding process with high performance in terms of robustness and efficiency, i.e. with inference times for the welding trajectory which are much shorter than in the prior art, starting from the acquisition of a real image of the area of interest of the welding process, and is capable of achieving, in real time and simultaneously, the tracking of the welding trajectory and the estimation of the width of the welding joint where the welding is to occur.

[0037] A further advantage of the invention is the possibility of correcting, in real time, during the welding process, a welding trajectory programmed prior to processing.

[0038] The method of the invention can be implemented by a system which can be adapted to a laser processing machine, which may be easily integrated therein, also sharing part of the processing resources already present on the machine and dedicated to the control of the emission and focusing of the laser beam, and of the laser processing process as a whole.Further features and advantages of the invention will be set out in greater detail in the following detailed description of one embodiment thereof, given by way of non-limiting example, with reference to the accompanying drawings, in which:

[0039] Figure 1, described above, is an example of a laser processing machine according to the prior art;

[0040] Figures 2a-2e, described above, show steps of a welding joint recognition method according to the prior art;

[0041] Figure 3 shows an example structure of a processing head of the laser welding machine in accordance with the invention;

[0042] Figure 4 is a flow diagram of the method which is the subject matter of the invention; Figure 5 is a depiction of some steps for estimating a horizon point of the welding trajectory in the method which is the subject matter of the invention;

[0043] Figure 6 is a diagram over time of the execution steps of the method which is the subject matter of the invention; and

[0044] Figure 7 is a flow diagram of a training method of a control system for a laser welding process configured to perform the method which is the subject matter of the invention.

[0045] The method which is the subject matter of the invention is implemented, for example, by a machine having the configuration shown in Figure 1, the welding head of which is shown diagrammatically in Figure 3.

[0046] In Figure 3, a welding head 14 is exemplified according to a currently preferred embodiment. In the figure, elements or components identical or functionally equivalent to those illustrated in Figure 1 have been designated with the same reference numerals. The laser beam B, generated by the emission source 10 and transported to the processing head by means of an optical path in optical fiber 12, i incident, collimated, on the reflecting deflector element 22, which deflects the optical propagation axis thereof into a direction of incidence on the material being processed. The optical focusing system 16 is intermediate between the reflecting deflector element 22 and a protective window (not shown) arranged downstream, near the nozzle 18, adapted to protect the focusing system from possible melt spatter, and comprises a lens holder group to which mechanical adjustment mechanisms are coupled forcalibrating the positioning of the lens in the beam propagation direction (Z axis).

[0047] Arranged coaxially with the optical propagation axis of the welding laser beam B exiting the nozzle 18 are imaging means of the welding region, for example a photo camera or monochrome video camera 30 with a resolution of 2064 x 1544 pixels, adapted to collect a vision beam V of the welding area through the nozzle 18. The welding area is preferably illuminated by an LED illuminator with a wavelength which can be detected by the photo camera / video camera but different from that of the process laser, for example of 660 nm. The optical chain associated with the imaging means preferably provides an optical resolution of 20 pm / px, to distinguish small-size details, such as, for example, zero-gap welding joints, and depth of field of 15 mm, to obtain in-focus images from the coaxial camera even when the processing laser beam B is defocused (up to ±5 mm) for technological requirements.

[0048] The pieces being processed may be positioned and fastened on a fixed table or a rotary -tilting table, more generally on supports with controlled motion.

[0049] The method which is the subject matter of the invention is described below with reference to Figure 4, in which a flow diagram of one of a plurality of iterations of the method is shown, implemented by an electronic processing unit, accessory or integrated into a laser welding machine, according to predetermined supervised estimation models. Simultaneously, reference will be made to Figure 5, which shows representative images of some steps of the method.

[0050] Indicated at 100 is a step of acquiring an image of the welding region in which the trajectory of the welding joint between the pieces being processed develops, which occurs through the camera 30 integrated with the welding head. A portion of said image is shown in the upper image of Figure 5, in which W indicates the current welding point, and S indicates the welding joint.

[0051] At 120 is indicated a recognition method of an item in the image of the welding area, belonging to a predefined class of items representative of a welding joint included in thewelding area. For example, starting from an image of a welding area having a size of 2064 x 1544 pixels, the electronic processing unit executes an estimation algorithm based on a predetermined supervised image analysis model with instance segmentation, in a currently preferred embodiment the Yolo v.8 algorithm, comprising a step 140 of executing a model for estimating the position of the welding joint, and a step 160 of applying a binary segmentation mask to the image, by virtue of which each pixel of the image is associated with the welding joint or with the background (the material to be welded), i.e., the pixels belonging to the identified welding joint are classified differently from the pixels of the remaining image area. In Figure 5, in the second image from the top, a segmentation mask indicative of the identified welding joint is indicated with M.

[0052] At step 180, a median axis of the object which has been recognized as a welding joint is computed.

[0053] At step 200, removal of anomalous values that emerged from the calculation of the median axis is performed, obtaining an estimated welding trajectory, indicated with T in the third image of Figure 5, and, at the subsequent step 220, the horizon point of the welding trajectory, subsequent to the horizon point calculated in a previous iteration, at a predetermined distance f from the current welding position, is extracted, indicated with P in the fourth image of Figure 5. The current welding position is identified starting from the centroid of the contour of the current welding point W, conventionally identified by the lighting area of the laser emission incident on the welding joint.

[0054] The removal of the outliers which emerged from the calculation of the median axis is performed through a deterministic correction process of the prediction of the supervised model. With reference to a reference system of the image of the welding region based on the welding joint, in which the X axis is directed tangentially to the estimated welding trajectory, which can be approximated as linear along the local extension direction of the welding joint in the neighbourhood of the current welding position or of the spatial horizon for estimating the welding trajectory, by virtue of the reduced field of vision of the coaxial camera and of the constant orientation of the direction of movement of the welding head tangent to the trajectory, it is possible to control the variability of the Y coordinate along the aforesaidtrajectory.

[0055] Specifically, in the single image at the i-th iteration, the X and Y coordinates at the subsequent (i+l)-th iteration are calculated as follows:

[0056] output > ^prediction

[0057] ,t+l {output . p ; prediction 1 prediction output > i11£ -1 ITT

[0058]

[0059] Vj

[0060]

[0061] prediction

[0062] Vi-i-'l otherwise D e N:

[0063] namely, the X coordinate is determined as corresponding to the coordinate predicted at the (i+ 1 )-th iteration, while the Y coordinate is determined as corresponding to the coordinate predicted at the (i+l)-th iteration only if the difference with respect to the mean of the Y coordinates at the preceding iterations is lower than a predetermined threshold (D), otherwise the Y coordinate predicted at the i-th iteration is considered. By way of example, the predetermined threshold D is a number of pixels between 2 and 5.

[0064] The estimated horizon point of the welding trajectory is provided at step 240 to a welding head movement control module for welding trajectory tracking, possibly resorting to a coordinate transformation between the image reference system and the welding head reference system. It should be noted that, if the movement direction of the welding head is oriented tangentially to the welding trajectory underway, the welding joint search direction is determined directly by the orientation of the head with respect to the pieces, and since the field of vision of the camera remains unchanged, the exploration direction corresponds to the vertical direction in the acquired image, while the welding joint is oriented horizontally.

[0065] At step 260, the electronic processing unit extracts an area of interest in the neighbourhood of the horizon point of the estimated welding trajectory, for example centered around said point, of a size smaller than the welding region, for example 330 x 250 pixels. On the area of interest, at step 280, the electronic processing unit executes an estimation algorithm based on a predetermined supervised image classification model, for example a convolutional neural network such as MobileNet, for estimating the width of the welding joint. The image classification model comprises, at step 300, an execution step of a classification model of the welding joint for widths, and, at step 320, the extraction of a width value in real units ofmeasurement.

[0066] The welding joint width value is provided as input at step 340 to a welding laser beam power distribution control module, for example a module adapted to control the movement of the welding laser beam B in the neighbourhood of the horizon point of the welding trajectory if the estimated width of the welding joint exceeds a predetermined threshold value. The movement of the laser beam in the neighbourhood of the welding point is synchronized, at step 360, with the control of the movement of the welding head for tracking the welding trajectory. As a person skilled in the art can understand, in place of the physical movement of the propagation direction of the laser beam and incidence on the materials being processed, it is possible to implement control of the power distribution of the laser beam, by widening the incidence section thereof on the materials being processed or by modifying the transverse power distribution of the beam according to forms not exclusively Gaussian, for example according to an annular or semi-annular shape with variable angular orientation.

[0067] Figure 6 shows a diagram over time of the execution steps of the above-described method.

[0068] In Figure 6a along the time axis t, the time intervals of performance of three successive iterations of the method described with reference to Figure 4 are indicated. A first iteration of the method commences at the time instant tO. The time interval employed by the imaging means for acquiring an image of the welding region is indicated with team. taigindicates the time interval employed by the electronic processing unit for the analysis of the image and the execution of the identification and classification algorithms recalled in the preceding part of the present discussion, in this specific case to identify the welding joint and estimate the width thereof. The waiting time interval necessary for any corrections of the estimation results is indicated with tidi- The sum of these intervals is indicated with tset and represents the identification time interval of a subsequent point of the welding trajectory.

[0069] Finally, the time interval necessary for the actuator devices of the welding head to react to the position and width information of the welding joint and move the welding head or the support plane of the pieces being processed, or modify the transverse power distribution of the laser to track the welding trajectory is indicated with tact.The sum of these intervals is indicated with tresp and represents the overall actuation time interval of the welding process control method at each iteration.

[0070] As is evident in the figure, a second iteration of the control method of the welding process may be initiated at the time interval tl, subsequent to the instant tO by a time equal to l / fcam, where fcam is the image acquisition frequency. It is also evident from the figure that the time between successive actuation instants of the actuating devices of the welding head, corresponding to 1 / fctri, is equivalent to l / fcam. Otherwise, the second iteration of the method is initiated at a later time which is the greater of l / fcamand 1 / fctri, to ensure that image acquisition occurs without overlaps and that the welding control process can be initiated after the acquisition of the subsequent image.

[0071] In Figure 6b, an enlargement of the image acquisition time interval tcamis shown, comprising a first time interval texpof exposure of the image acquisition means for acquiring the image, i.e., the time during which the sensor of the image acquisition means is active and collects light information, and a second time interval treadout of transferring data acquired by the image acquisition means to the processing system.

[0072] Based on the above-described execution times of the welding process control method, it is possible to define a prediction horizon of the welding trajectory f, defined as a distance from the current welding position, which depends on the overall time interval of execution of the welding process control method at each calculation iteration of a horizon point of the welding trajectory, tresp, and on the advancement speed of the welding head along the welding joint, or more simply welding speed, vWeid, according to the following expression

[0073]

[0074] The time interval of actuation of the method for controlling the welding process at each iteration, tresp, must be at least greater than the sum of the image acquisition time interval tcam, the time interval employed by the electronic processing unit for image analysis and execution of the identification and classification algorithms, taig, and the operating timeinterval of the actuating devices of the welding head, tact (tidi is comparatively negligible).

[0075] Assuming, by way of example, an inference time of the supervised estimation models (the position of the welding joint and the width thereof) not greater than 10ms, assuming that the execution of the estimation models can occur in parallel with the acquisition of a subsequent image by the capturing means, whereby team < taig, assuming an action time of the movement head of 30ms and a welding speed of 100 mm / s, a capability of the method which is the subject matter of the invention is obtained, namely to estimate the parameters of the welding joint within a horizon of 5mm, i.e. at a point of the joining trajectory 5mm ahead in time with respect to the current welding position. The spatial estimation horizon of the welding trajectory is preferably configurable based on the processing requirements.

[0076] The models described are trained in a supervised manner for the determination of parameters such as the width of the welding joint, as well as the position and corresponding orientation thereof in the acquired image, starting from a collection of data comprising a plurality of images of welding regions which are representative of the desired classifications, acquired during welding tests by one or more training image acquisition modules which are identical or similar to the image acquisition module employed in the execution of the welding process.

[0077] Preferably, the preliminarily classified collection of data is divided into a first training dataset, comprising 70% of the collection of data, and a second validation dataset, comprising the remaining 30% of the collection of data.

[0078] The described image analysis and convolutional neural network models are trained in a supervised manner according to the flow diagram of Figure 7.

[0079] 1000 designates an acquisition step of a learning image representative of a welding region, for which the position and extension of the welding joint S are known, from a welding start point, I, near the current welding point, to a welding end point, E, along a welding direction c which may be locally variable if the welding joint does not follow a straight path.

[0080] The learning image is used in a first learning branch 1100 of the identification model of thewelding joint, and in a second learning branch 1200 of the estimation model of the width of the welding joint.

[0081] Specifically, at step 1110 a classification label is assigned to the image, marking the welding joint therein, i.e., marking the pixels belonging to the welding joint, as shown in the associated image. The set of values assigned to the pixels of the image forms a training dataset of the welding joint identification model at step 1120, and this is provided as input to the selected model, for example to the Yolo v.8 architecture, at step 1130, to provide, at step 1140, the trained model. Different validation images of welding regions may then be submitted to the trained model at step 1150 to evaluate the reliability of the model.

[0082] The learning branch 1200 of the model for estimating the width of the welding joint is analogous.

[0083] At step 1210, an extraction of the area of interest of the welding region is performed, corresponding in size to the area of interest analyzed by the model. At step 1220 a classification label is assigned to the image, corresponding to the welding joint width value present in the image, measured in a separate step, as shown in the associated image. The assigned value supplements a training dataset of the model for estimating the width of the welding joint at step 1230, and this is provided as input to the selected model, for example to the MobileNet v.l architecture, at step 1240, to provide the trained model at step 1250. Different validation images of areas of interest may therefore be submitted to the trained model at step 1260, obtained from the validation images of welding regions prepared at step 1150, to assess the reliability of the model.

[0084] Advantageously, the set of training images comprises images including laser emission at different power levels, images in which the laser emission is absent, and images in which no welding joint is present, for different materials, for example aluminum 6500 T6 and AISI 301LN stainless steel. Likewise advantageously, the set of training images is acquired during test welding processes with and without feeding filler material, and also with wobbling of the laser beam in the neighbourhood of the welding position, at predetermined different wobbling frequencies and amplitudes.The set of validation images are in turn images comprising laser emission at different power levels, images in which laser emission is absent, images in which the welding joint exhibits irregular widths along the extension direction and straight and curved trajectories, images with different materials in different surface conditions (for example, with striations) and for different image acquisition conditions, for example in different lighting conditions.

[0085] Advantageously, the plurality of images of welding regions and welding joints representative of the different classifications required of the system, forming the data set, comprises at least one plurality of primary images of welding regions acquired by one or more training image acquisition modules from real tests, and a plurality of synthetic images obtained by alteration of said plurality of primary images according to a data augmentation technique. The plurality of synthetic images includes respective altered welding regions. The alteration of the images of the welding regions may be obtained by altering the dimensional, positional, or optical properties of the welding joint, and, more specifically, the altered welding regions include, by way of example, at least one of welding regions which are translated, rotated, reduced or enlarged, blurred, with modified light intensity. The use of synthetic images with altered welding regions advantageously enables conducting broader and more robust training of the models, and providing greater generalization to the models.

[0086] It should be noted that the proposed embodiment for the present invention in the foregoing discussion is purely by way of explanation and not by way of limitation of the present invention. A person skilled in the art will be able to readily implement the present invention in different embodiments which, however, do not depart from the principles set forth herein, and are therefore included in the present patent.

[0087] This applies in particular to the possibility of adapting the welding joint width estimate to the current distance between the welding head and the material being processed, which may be estimated based on the current focal distance of the imaging means by comparison with a predetermined focal distance corresponding to a predetermined reference distance between the welding head and the material, for example determined according to the teachings of Italian patent application no. 102019000023181 or Italian patent application no.102020000006880 in the name of Adige S.p.A. and the Milan Polytechnic Institute.

[0088] The foregoing discussion makes the advantages of the method and system according to the invention apparent, i.e., the ability to identify a welding joint and estimate the width thereof in little time, for example in the order of 10ms, compatible with determining a welding trajectory in real time, during a welding process. Furthermore, the use of identification techniques of the welding joint and estimation of the respective width, based on machine learning models with training data augmentation based on synthetic images, allows creating a robust method for controlling a welding process, i.e., tolerant to the different conditions and materials employed in the process, and easily scalable to new materials, new geometries, and new joint and process configurations, through a simple and rapid execution of a new training procedure, as shown in Figure 7.

[0089] Further advantages lie in the possibility of rapidly identifying errors in the operating conditions of the welding process, for example by detecting the absence of a piece of material to be welded or the arrangement thereof at an excessive distance from the piece facing it, adjacent thereto, or overlapped therewith, and errors in the configuration of the welding system, for example the absence of filler material or of an assist gas nozzle in the case of processing operations which instead require it. Not only, but the present invention also enables monitoring and controlling the consistency of predictions based on artificial vision exploiting a comparative analysis of the predictions obtained from images acquired at separate time instants, taking data into analysis such as the advancement speed of the process and the image acquisition frequency. This can be achieved by extracting, from images captured at successive moments, points corresponding to the same spatial information along the welding joint, belonging to the overlapping welding trajectory segments between the sequences of trajectory points predicted in two or more consecutive iterations of the process, comparing the predictions of welding trajectory points generated in each iteration with the preceding ones, and detecting any discrepancies or anomalies in the estimated position of the welding joint. In order for this technique to function correctly, a spatial and temporal synchronization mechanism is required. Temporal synchronization ensures that the images are analyzed at a frequency compatible with the advancement speed of the welding head, while spatial synchronization ensures that the points extracted at successive instantscorrespond to the same actual position of the joint. In the event of inconsistent variations with respect to past predictions, the electronic processing unit may activate correction mechanisms to ensure the accuracy and stability of the monitoring, for example in the event of significant deviations. Thereby, the objective of improving the reliability and repeatability of the predictions as well as optimizing weld execution is achieved, reducing the risk of errors by the vision system.

[0090] Naturally, without prejudice to the principle of the invention, the embodiments and the implementation details may be widely varied with respect to what has been described and shown purely by way of non-limiting example, without thereby departing from the scope of protection of the invention defined by the appended claims.

Claims

CLAIMS1. A method for controlling a laser welding machine for welding at least two metal pieces separated by a welding joint in a welding region, said machine being configured for irradiating a welding laser beam along said welding joint according to controlled processing parameters,wherein said machine comprises:electronic means for processing and automatically recognizing a welding joint, configured according to a supervised learning model for object recognition in images, arranged to receive as input at least one image of a welding region including a development trajectory of a welding joint and to identify therein an area of adjacent pixels indicative of a welding joint; andmeans for controlling the laser welding machine arranged to control at least one processing parameter of said machine,the method being characterized in that it comprises the operations of:in a learning phase, training said supervised learning obj ect recognition model by means of a series of training data each comprising:a training image of a welding region including at least one development trajectory of a welding joint; andrespective classification data adapted to mark at least one training area of adjacent pixels, indicative of the welding joint in the training image; andin an operating phase, the iteration of the following steps:acquiring at least one image of the current welding region of the machine including a development trajectory of a welding joint;providing said at least one acquired image of said welding region as input to said electronic processing and automatic recognition means;identifying an area of adjacent pixels indicative of a welding joint in the welding region image by means of said electronic processing and automatic recognition means; and controlling at least one processing parameter of the machine based on spatial and morphological geometric properties of the identified welding joint.

2. The method according to claim 1, wherein said supervised learning objectrecognition model is an instance segmentation object recognition model.

3. The method according to claim 1 or 2, wherein controlling at least one processing parameter of the machine based on spatial and morphological geometric properties of the identified welding joint includes determining the position of a point of a welding trajectory that the laser beam must track at a predetermined distance from the current welding point.

4. The method according to claim 3, wherein determining the position of a point of a welding trajectory at a predetermined distance from the current welding point includes calculating the median line of the identified welding joint and determining the position of the welding trajectory point subsequent to a welding trajectory point identified in the previous iteration, at a distance from the current welding point determined as a function of the acquisition time of a welding region image and the identification time of the pixel area indicative of the welding joint in the welding region image by means of the supervised learning object recognition model.

5. The method according to claim 4, comprising a correction of the position of the welding trajectory point if at least one of its coordinates in an image reference system is further away from the respective coordinate of the welding trajectory point identified in the previous iteration by a predetermined threshold.

6. The method according to claim 3, further comprising:acquiring at least one image of an area of interest in the neighbourhood of a welding trajectory point identified along a welding joint;by means of a supervised learning computational regression model, estimating the width of the identified welding joint at the determined welding trajectory point; and controlling said processing parameters based on the estimated width of the welding joint,wherein said supervised learning computational regression model is trained using a series of training data each comprising:at least one image of an area of interest in the neighbourhood of a welding trajectory point identified along the welding joint; anda respective classification data element indicative of the width of the welding joint at said identified welding trajectory point.

7. The method according to claim 6, wherein controlling the processing parameters based on the estimated width of the welding joint includes controlling the oscillation of the laser beam in the neighbourhood of the current welding point, or controlling the transverse power distribution of the laser beam in the neighbourhood of the current welding point.

8. The method according to claim 6, wherein said supervised learning computational regression model includes a trained convolutional neural network.

9. The method according to claim 6, comprising the determination of the acquisition distance of said welding region image from the welding joint and the correction of the estimate of the width of the identified welding joint based on the determined acquisition distance of said welding region image from the welding joint.

10. The method according to any one of the preceding claims, wherein at least one training image of a welding region includes at least one of a laser beam irradiation area incident on the welding joint, an assist gas emission nozzle, a feature of a filler material.

11. The method according to any one of the preceding claims, wherein at least one training image of a welding region is free of a welding joint.

12. The method according to any one of the preceding claims, comprising synthetic training images of a welding region that include altered training images of a welding region, said altered training images of a welding region comprising at least one of training images of a welding region that are translated, rotated, reduced or enlarged in scale, distorted, incomplete, blurred, with modified light intensity.

13. The method according to any one of the preceding claims, wherein the acquisition of said at least one welding region image occurs coaxially with the irradiation of the welding laser beam.

14. The method according to any one of the preceding claims, wherein the acquisition of said at least one welding region image occurs simultaneously with the irradiation of the welding laser beam.

15. A control system for a laser welding machine for welding at least two metal pieces separated by a welding joint in a welding region, said machine being configured for irradiating a welding laser beam along said welding joint according to controlled processing parameters,the system comprising:image acquisition means adapted to acquire at least one image of the current welding region of the machine, including a development trajectory of a welding joint;electronic means for processing and automatically recognizing a welding joint, configured according to a supervised learning model for object recognition in images, arranged to receive as input at least one image of a welding region including a development trajectory of welding joint and to identify therein an area of adjacent pixels indicative of a welding joint; andmachine control means coupled to said electronic processing and automatic recognition means, arranged to control at least one processing parameter of the machine based on spatial and morphological geometric properties of the identified welding joint, wherein said supervised learning object recognition model is trained by means of a series of training data each comprising:a training image of a welding region including at least one development trajectory of a welding joint; andrespective classification data adapted to mark at least one area of adjacent training pixels, indicative of the welding joint in the training image.

16. The system according to claim 15, wherein said supervised learning object recognition model is an instance segmentation object recognition model.

17. The system according to claim 15 or 16, wherein the control means are arranged to determine the position of a point of a welding trajectory that the laser beam must track at apredetermined distance from the current welding point.

18. The system according to claim 17, wherein the control means are arranged to calculate the median line of the identified welding joint and determine the position of the welding trajectory point subsequent to a welding trajectory point identified in the previous iteration, at a distance from the current welding point determined as a function of the acquisition time of a welding region image and the identification time of the pixel area indicative of the welding joint in the welding region image.

19. The system according to claim 18, wherein the control means are arranged to correct the position of the welding trajectory point if at least one of its coordinates in an image reference system is further away from the respective coordinate of the welding trajectory point identified in the previous iteration by a predetermined threshold.

20. The system according to claim 17, further comprising electronic processing means configured according to a supervised learning computational regression model, arranged to receive as input at least one image of an area of interest in the neighbourhood of a welding trajectory point identified along a welding joint and to estimate the width of the identified welding joint at the determined welding trajectory point,wherein said machine control means are coupled to said electronic processing means configured according to a supervised learning computational regression model and arranged to control at least one processing parameter of the machine based on the estimated width of the welding joint,wherein said supervised learning computational regression model is trained using a series of training data each comprising:at least one image of an area of interest in the neighbourhood of a welding trajectory point identified along a welding joint; anda respective classification data element indicative of the width of the welding joint at said identified welding trajectory point.

21. The system according to claim 20, wherein said machine control means are arranged to control the oscillation of the laser beam in the neighbourhood of the current welding point,or control the transverse power distribution of the laser beam in the neighbourhood of the current welding point.

22. The system according to claim 20, wherein said supervised learning computational regression model includes a trained convolutional neural network.

23. The system according to claim 20, wherein said electronic processing means are arranged to determine the distance of the image acquisition means from the welding joint and to correct the estimate of the width of the identified welding joint based on the determined distance of the image acquisition means from the welding joint.

24. The system according to any one of claims 15 to 23, wherein at least one training image of a welding region includes at least one of a laser beam irradiation area incident on the welding joint, an assist gas emission nozzle, a feature of a filler material.

25. The system according to any one of claims 15 to 24, wherein at least one training image of a welding region is free of a welding joint.

26. The system according to any one of claims 15 to 25, comprising synthetic training images of a welding region that include altered training images of a welding region, said altered training images of a welding region comprising at least one of training images of a welding region that are translated, rotated, reduced or enlarged in scale, distorted, incomplete, blurred, with modified light intensity.

27. The system according to any one of claims 15 to 26, wherein said image acquisition means are coaxial to the welding laser beam.

Images