Position detection of joining partners

Abstract

The invention relates to a method for determining the position of joining partners for a joining process, the method comprising the steps of: • providing (S10) the joining partners in a joining position; • capturing (S20) a real image of the positioned joining partners; • generating (S30) a false colour image based on the real image; and • determining (S40), based on the false colour image, at least one item of information about the position of at least one of the joining partners. The invention also relates to a device and to a computer programme product for implementing the method.

Description

[0001] Position detection of joining partners

[0002] Field of the invention

[0003] The present invention relates to the field of joining. In particular, the invention relates to a method for determining the position of joining partners for a joining process.

[0004] State of the art

[0005] When welding component joints, it is important to know the exact position of the joining partners to be welded. Based on this position information, the weld path (especially the position of the weld path) can be precisely adjusted to create the best possible weld. Clamping devices are generally used for rough positioning of the joining partners. However, reliable, precisely reproducible positioning of the joining partners is often not possible even with the available clamping devices. In laser welding, computer programs with position detection algorithms are therefore used for precise position detection. These programs are designed to recognize the exact position or positioning of the joining partners relative to one another based on a camera image. However, detection only works accurately if the image areas of the camera image relevant for position detection stand out from the surrounding image areas with high contrast.

[0006] In so-called hairpin technology, which is used in the manufacture of stators for electric motors or generators, the ends of two adjacent hairpins are welded together to create plug-in coils, for example. The ends are roughly positioned in a joining position using a positioning device. To better determine the exact position of the hairpin ends relative to each other, it is known to use hairpins with flat, cut end surfaces. Due to sufficient contrast, the position of the flat end surfaces of the hairpins can be reliably detected using known position detection algorithms, as shown in Figure 2a. However, creating flat joining surfaces often involves additional effort and thus increased costs.

[0007] The present invention is based on the object of improving the possibilities for automated position detection of joining partners using known position detection algorithms.

[0008] The invention

[0009] The problem underlying the invention is solved by the subject matter of the independent claims. Further possible embodiments of the invention are specified in the subclaims, the description, and the drawings. Features, advantages, and possible embodiments presented in the description for one of the subject matter of the independent claims are to be regarded at least analogously as features, advantages, and possible embodiments of the respective subject matter of the other independent claims, as well as any possible combination of the subject matter of the independent claims, optionally in conjunction with one or more of the subclaims.

[0010] According to a first aspect of the invention, a method for determining the position of joining partners for a joining process is provided. The method can preferably be preceded by a joining process for joining the joining partners and serve to prepare the joining process. In a first step of the method, the joining partners are provided in a joining position. For rough alignment of the joining partners, the joining partners can be fixed in the joining position by means of a positioning device. The positioning device generally allows some play of movement of the joining partners, which is why the joining partners can, for example, have a small offset (e.g., a few millimeters) or a small distance (e.g., a gap of a few millimeters or a few tenths of a millimeter) from one another. In a second method step, a real image of the joining partners positioned in the joining position is recorded. A camera can be used for this purpose, in particular.The real image is a raster graphic that depicts the camera's field of view. In a subsequent, third method step, a false-color image is generated from the real image. In the context of the present disclosure, a false-color image is understood to be a digital raster graphic (also referred to as a pixel graphic) whose pixels can only assume one of at least two predetermined colors, in particular black or white and / or a shade of gray in between. When using shades of gray as colors in a false-color image, care must generally be taken to ensure that each shade of gray has a clearly recognizable contrast to every other color used. When only two colors are used, the false-color image can be a binary image. The false-color image can visually highlight information relevant to the joining of the joining partners, in particular regarding the position and / or geometry of the joining partners.Based on the false-color image, at least one piece of information regarding the position of at least one of the joining partners is then determined (in a fourth method step). In particular, the position of at least one of the joining partners can be determined with respect to the position and / or orientation of a joining optics, in particular a laser welding head. The reference point for the position of the joining optics can, in particular, be a tool center point (TCP) of the joining optics. To determine the position information or in addition to determining the position information, for example, a respective shape and / or size of the joining partners can be determined.

[0011] According to a preferred variant, the false-color image can be generated from the real image using a deep convolutional neural network (also called a convolutional neural network or "CNN" for short), in particular a modified U-Net.

[0012] For more reliable processing of image data, even with irregular contrast ratios, an intelligent filter is used according to the invention before image analysis (in particular position detection), which generates the false color image. In this false color image, the features relevant to the joining process are shown in one color (e.g. white) and the features irrelevant to the joining process (e.g. the clamping device or sections of the joining partners irrelevant to the joining) are shown in the other color (e.g. black). It is understood that more than just two feature classes can also be represented using the false color image, with a separate color preferably being used to represent each feature class, each color having a sufficiently clear contrast to every other color used.Due to the optimal contrast between relevant and irrelevant features, the proposed method enables particularly robust detection of the position or positioning of joining partners for optimized preparation of a joining process. In particular, computer programs with known component position detection algorithms can be used to easily and automatically identify information on the position of the joining partners based on the false-color image and derive specific joining settings for a subsequent joining process.

[0013] Preferably, based on the determined position information, the relative position of the joining partners can be checked in a fifth process step. In this case, process steps four and five can proceed essentially in parallel.

[0014] Furthermore, it can be provided that a warning is issued if, when checking the position information and / or information relating to the geometry of at least one of the joining partners, an inadmissible position and / or geometry of at least one of the joining partners is determined. For example, a warning can be issued if at least one of the joining partners has a shape or size that lies outside the specified limits for the shape or size of the joining partner in question. Furthermore, a warning can be issued, for example, if the joining partners have an inadmissible offset from one another or are positioned too far apart from one another. In addition to or alternatively to issuing a warning in the event of a critical geometry or position of the joining partners, it can be provided that a downstream joining process cannot be carried out without additional, manual approval.The checking step may further comprise specifying processing settings for joining the joining partners. In particular, based on the position information, a movement path or the position of a predetermined movement path can be specified, along which movement path a laser beam is moved over the surface of the joining partners for local melting.

[0015] The method may comprise, in a further method step, joining the joining partners using the specified processing settings in a laser welding process. In this case, the claimed method may be implemented as part of a laser welding process.

[0016] In particular, if the claimed method is implemented within the framework of a laser welding process, the real image can be recorded from a direction perpendicular to a processing plane. In other words, the real image can represent a top view of the processing point of the joining partners. The processing plane is typically located in the area of ​​the surface of the joining partners and, in a laser welding process, extends approximately transversely to the propagation direction of the processing laser beam in the area of ​​a beam focus of the processing laser beam. In a laser welding process, the real image can be recorded using a camera through the beam path of the processing laser beam. By aligning the camera used concentrically to the beam path of the processing laser beam, the real image always shows the field of view of the processing optics.In this way, the described method can also be used to easily determine the relative position of the joining partners to the position of the processing optics or to the position of a tool center point of the joining device used.

[0017] The false-color image generated from the real image can preferably have a resolution that corresponds to the resolution of the real image. To generate the false-color image, each pixel of the real image can be assigned one of at least two predefined colors. The color assignment can preferably be based on the assignment of the respective pixel to a predefined feature class. Thus, at least two feature classes can be predefined to generate the false-color image. For example, one feature class can be defined as the surface of at least one of the joining partners, while a second feature class is defined as the background. Using artificial intelligence (AI) trained to assign pixels to one of the defined classes, a probability of each pixel belonging to one of the predefined feature classes can be determined.If the probability is greater than a specified threshold, the corresponding pixel can be assigned to the respective feature class. As already described above, the AI ​​can preferably be implemented as a deep convolutional neural network, preferably a U-Net / U-Net derivative, which has been optimized / trained for the specified problem and runtime. It is understood that more than two feature classes can also be specified for generating the false-color image. For example, different joining partners or, in addition, the components of a positioning device can each be assigned to different feature classes and thus different colors of the false-color image.

[0018] The joining partners can, for example, be workpieces to be welded together. For example, the joining partners can be metallic, rod-shaped conductor elements, so-called hairpins (essentially U-shaped wires), which are to be welded together to form an electrical connection. The hairpins can be made of copper or a copper-containing material, or, for example, of aluminum or a material containing aluminum.

[0019] According to a further aspect of the invention, a laser welding device for joining two joining partners is provided. The laser welding device comprises a positioning device for positioning the joining partners in a joining position. The positioning device can, for example, be a holder that fixes the joining partners in a specific position (with some play) relative to one another. The laser welding device further comprises a laser beam source for providing a processing laser beam and a laser welding head with laser welding optics for aligning and focusing the processing laser beam in the direction of the joining partners. The laser beam source can, for example, be a solid-state laser, in particular a fiber laser or a disk laser. The laser beam can be guided from the laser beam source to the laser welding head, at least in part, by means of an optical fiber.For example, a scanner optic or a fixed optic can be used as the laser welding optic. The laser welding device further comprises an image capture device for capturing a real image of the joining partners in the joining position. The image capture device can in particular be a camera, for example a grayscale camera with a resolution in the megapixel range. The laser welding device further comprises an image processing device for generating a false-color image based on the real image, as well as a measuring device for determining at least one piece of information regarding the position of at least one of the joining partners based on the false-color image.

[0020] The laser welding device can further comprise a verification device for verifying, based on the at least one piece of position information, the position and / or geometry of at least one of the joining partners. The image processing device, the measuring device, and the verification device can each be implemented as a computer program (product) on a common computer, which can also be used, for example, to control the camera.

[0021] The checking device can further be configured to output a warning. A warning can be output, for example, if, during checking of the position and / or geometry, an impermissible position for joining the joining partners and / or an impermissible geometry of at least one of the joining partners is detected. In addition to or alternatively to the warning, the checking device can be configured to send a signal to a control device of the laser welding device that prohibits a welding process due to impermissible position or geometry information of the joining partners until the welding process has been manually released. The checking device can further be configured to determine processing settings for laser welding the joining partners using the laser welding device based on the at least one piece of position information.In particular, the checking device can send instructions for positioning and moving the laser processing beam to the control device of the laser welding device based on a detected relative position of the joining partners to the laser welding head.

[0022] According to a further aspect of the invention, a computer program product is provided which contains computer-readable instructions for carrying out a method according to one of the variants described above on a joining device, in particular on a laser welding device according to one of the variants described above.

[0023] Examples of implementation

[0024] The following description of preferred embodiments, in conjunction with the drawings, serves to explain the invention in more detail.

[0025] They show:

[0026] Fig. 1 Schematically shows the sequence of a method according to the invention;

[0027] Fig. 2 A real image of two hairpins in a welding position, wherein the ends of the hairpins to be welded have flat end surfaces;

[0028] Fig. 3a A real image of two hairpins in a welding position, where the ends of the hairpins to be welded have uneven end surfaces;

[0029] Fig. 3b A false color image based on the real image according to Figure 3a (here a binary image); and

[0030] Fig. 4 Schematic of a laser welding device according to the invention. Identical or functionally identical elements are provided with the same reference numerals in the figures.

[0031] Figure 1 schematically shows the sequence of a method according to the invention for determining the position of joining partners 20a, 20b (cf., for example, Fig. 4) for a joining process. The method comprises, in a first step S10, providing the joining partners 20a, 20b in a joining position. In a second step S20, a real image of the positioned joining partners is recorded (cf. also real image 22, see Fig. 3a). In a third step S30, a false-color image is generated on the basis of the real image 22 (cf. also binary image 24, see Fig. 3b). In a fourth step S40, at least one piece of information relating to the position of at least one of the joining partners 20a, 20b is determined on the basis of the false-color image 24.

[0032] In an optional fifth step S50, the position and / or geometry of at least one of the joining partners 20a, 20b can be checked based on the information obtained in step S40.

[0033] In an optional, sixth method step S60, a warning can be issued if, during the checking S50 of the position and / or the geometry, an inadmissible position for joining the joining partners 20a, 20b and / or an inadmissible geometry of at least one of the joining partners 20a, 20b is determined.

[0034] Preferably, based on the verified position and / or geometry information of the joining partners 20a, 20b, settings for a joining process can be determined automatically, and the joining partners 20a, 20b can be welded together in a seventh step S70, in particular in a laser welding process, using the determined settings. Determining settings for the joining process can in particular comprise determining the positioning (or position) of a predetermined welding path and / or determining coordinates for the starting point of the welding path. Figure 2 shows a plan view of the joining surfaces of two hairpins 20a, 20b to be welded together. Hairpin technology refers to a winding technology for stators in electric motors and generators, in which a plurality of essentially U-shaped conductor elements - the so-called"Hairpins" - are welded together in pairs at their ends. The term "hairpin" comes from the U-shape of the curved conductor elements, which is reminiscent of a hairpin. To weld two hairpins 20a, 20b, a focused processing laser beam L (see Fig. 4) is moved along a processing path (e.g., circular pendulum motion) over the joining surfaces at the ends of the hairpins 20a, 20b. The energy introduced by the processing laser beam L transforms the material of the hairpins 20a, 20b into a molten state, so that a common molten bead is formed, which connects the two hairpins 20a, 20b to each other. Since, depending on the design, more than 200 hairpin pairs may have to be welded for an electric motor, it is necessary that the process be fast and robust.For a good welding result, it is important that the welding process, and in particular the position of the weld path, is coordinated with the position of the hairpin ends to be joined. For precise position detection of the hairpins 20a, 20b, image processing programs are used which determine the position of the hairpins 20a, 20b based on a recorded image, in particular relative to the position or orientation of the welding optics used. Based on the detected position of the joining partners 20a, 20b, the position and / or starting point of the weld path for the welding process is then adjusted. For automated image processing, an image of the hairpin ends with the highest possible contrast is required, in which the contour of the hairpin ends relevant for welding stands out from the background (e.g. remaining hairpin areas, clamping device, etc.).Therefore, it is known from the prior art to cut the ends of the hairpins cleanly, so that they have a surface that is as shiny and flat as possible. This surface reflects the light from a camera illumination and stands out clearly from the surroundings in the camera's real image 22, so that the generated real image 22 can be used for automated position detection using a conventional position detection algorithm.

[0035] Since the production of hairpins 20a, 20b with bare, flat end surfaces requires complex additional processing steps (e.g., cutting off the hairpin ends with an expensive cutting system), the present invention proposes a solution by means of which even uneven or dirty joining partners (e.g., the hairpin ends, see Fig. 3a) can be welded together reliably and precisely using known position detection algorithms. The solution according to the invention is explained in more detail below with reference to Figures 3a, 3b, and 4 using the hairpin example. Figure 3a shows a real image 22 of the ends of two hairpins 20a, 20b that are positioned in a positioning device 12 for laser welding. In this case, the ends of the hairpins 20a, 20b have a roof-shaped structure, which results from a previous processing step of the hairpins 20a, 20b.Due to the surface properties of the hairpin ends, the real image 22 according to Figure 3a does not have a sufficiently strong contrast between those image areas 222 showing hairpin ends and the adjacent image areas 224, which is why the position of the hairpin ends relative to one another cannot be reliably determined using conventional position detection algorithms.

[0036] The real image 22 is captured, as is also known from the prior art, by a camera 18 (for example, a monochrome industrial camera with a resolution of 1.5 MPx) of a laser welding system 10 (see Fig. 4). The optical axis of the camera 18 is coupled, in particular via a semi-transparent mirror, into the welding optics 162 in the laser welding head 16 of the laser welding device 10. In this way, the camera image shows the processing area of ​​the processing laser beam L at all times. The processing laser beam L is, in turn, fed from a laser beam source 14 via a fiber optic cable to the laser welding head 16 and focused onto the ends of the hairpins 20a, 20b via the welding optics 162, which can be designed, for example, as scanner optics with movable scanner mirrors or lenses. The laser welding device 10 can be controlled by a central control device 19.The real image 22 captured by the camera 18 (see Fig. 3a) is transferred to a deep convolutional neural network (convolutional neural network) of an image processing device 182, which processes the real image 22 and generates as output a false-color image 24, here in the form of a binary image 24, with the same resolution as the real image 22. A U-Net / U-Net derivative is preferably used as the neural network for the image transformation, which has been optimized for the task at hand and the runtime. The features relevant to the welding process are highlighted in the generated binary image 22. This means that those pixels of the real image 22 that can be assigned (with sufficient probability) as a feature class to a surface of one of the hairpins 20a, 20b are colored white, and those pixels of the real image 22 that are assigned to the "background" feature class are colored black.As a result, the image areas 242 recognized as the surface of the hairpins 20a, 20b are displayed in white and stand out clearly from the black background 244.

[0037] The binary image 24 is now transferred, instead of the real image 22, to a measuring device 182 according to the conventional procedure, which determines information on the position and / or geometry of the hairpins 20a, 20b. Using a checking device 182, the acquired information can then be checked for its suitability for the welding process, and welding settings (in particular the shape and / or position of the welding path to be traced by the laser beam L) for the laser welding process can be determined based on the position information. For this purpose, the checked information on the position of the hairpin ends can be transferred from the checking device to the central control device 19, which controls the laser welding device 10 according to the specified welding settings.When checking the position and / or geometry of the hairpin ends, it is possible to check, in particular, whether predetermined position limits of the hairpin ends are being observed, whether a gap between the hairpins 20a, 20b lies within a specified tolerance range, or, for example, whether the cross-section of a hairpin 20a, 20b exceeds or falls below a specified size range. If it is determined that a specified tolerance limit is not being observed, the checking device 182 can issue a warning signal and / or stop an ongoing joining process in conjunction with the control device 19.

[0038] According to the structure of the laser welding device 10 shown in Figure 4, the image processing device, the measuring device, and the monitoring device can be implemented in a common computing unit 182. The computing unit 182 can be a panel PC for image monitoring. The applicant markets an image monitoring system suitable for implementing the present invention under the name VisionLine.

[0039] The neural network utilized by the image processing device 182 can be viewed in the system described here as an intelligent filter that highlights the features of the captured real image 22 relevant to the welding process for automated image processing, thus simplifying position detection. Detection is not limited to one feature or feature class, but can detect any number of feature classes in an image and highlight them in a single false-color image with two or more colors to reduce processing time. By determining the feature area with pixel precision (here, the surface of the hairpin ends), downstream algorithms can perform plausibility checks and compare the calculated / detected area with an expected area.This is difficult to achieve with classical intensity-based algorithms and can also have a positive effect on the robustness of the process.

[0040] The position detection of joining partners 20a, 20b according to the invention enables process-reliable joining of the joining partners 20a, 20b. Furthermore, in some processes, the effort (cost) for pre-processing the joining partners 20a, 20b (e.g., the accurate cutting of the hairpin ends) can be reduced.

Claims

Patent claims 1. Method for determining the position of joining partners (20a, 20b) for a joining process, the method comprising the steps: Providing (S10) the joining partners (20a, 20b) in a joining position; Taking (S20) a real image (22) of the positioned joining partners (20a, 20b); Generating (S30) a false-color image (24) based on the real image (22); and Determine (S40), based on the false color image (24), at least one piece of information about the location of at least one of the joining partners (20a, 20b).

2. Method according to claim 1, wherein the false color image (24) is generated from the real image (22) using a deep convolutional neural network, in particular a modified U-Net.

3. The method of claim 1 or 2, further comprising: Check (S50), based on at least one positional information, a relative position of the joining partners (20a, 20b) to each other.

4. The method of claim 3, further comprising: Issue (S60) a warning message if, during the check (S50) of the position information or information concerning the geometry of at least one of the joining partners (20a, 20b), an impermissible position and / or an impermissible geometry of at least one of the joining partners (20a, 20b) is detected.

5. Method according to claim 3 or 4, wherein the checking step (S50) comprises setting machining settings for joining the joining partners (20a, 20b).

6. The method of claim 5, further comprising: Joining (S70) the joining partners (20a, 20b) using the specified processing settings in a laser welding process.

7. Method according to one of the preceding claims, wherein the real image (22) is recorded from a direction perpendicular to a processing plane.

8. Method according to one of the preceding claims, wherein the false color image (24) has a resolution corresponding to a resolution of the real image (22).

9. Method according to one of the preceding claims, wherein to generate the false color image (24) each pixel of the real image (22) is assigned one of at least two predetermined colors, in particular black or white and / or at least one intermediate shade of gray.

10. Method according to claim 9, wherein the color assignment is carried out by assigning the respective pixel to a predetermined feature class.

11. Method according to one of the preceding claims, wherein the joining partners (20a, 20b) are metallic, rod-shaped conductor elements, so-called hairpins, which are to be welded together to form an electrical connection.

12. Laser welding device (10) for joining two joining partners (20a, 20b), the laser welding device (10) comprising: A positioning device for positioning the joining partners in a joining position; A laser beam source (14) for providing a processing laser beam (L); A laser welding head (16) with a laser welding optic (162) for Aligning and focusing the processing laser beam (L) towards the joining partners (20a, 20b); An image capture device (18) for capturing a real image (22) of the joining partners (20a, 20b) in the joining position; An image processing device (182) for generating a false-color image (24) based on the real image (22); and a measuring device (182) for determining at least one piece of information about the position of at least one of the joining partners (20a, 20b), based on the false-color image (24).

13. Laser welding device (10) according to claim 12, further comprising: A verification device (182) for checking, based on at least one position information, the position and / or geometry of at least one of the joining partners (20a, 20b), wherein the verification device (182) is further configured to issue a warning message if, during the verification of the position and / or geometry, an impermissible position and / or geometry of at least one of the joining partners (20a, 20b) is detected.

14. Laser welding device (10) according to claim 13, wherein the verification device (182) is further configured to determine processing settings for laser welding of the joining partners (20a, 20b) by means of the laser welding device (10) based on the at least one position information.

15. Computer program product containing computer-readable instructions for carrying out a method according to any one of claims 1 to 11 on a joining device.

Images