Computer-implemented method for generating a generation image for optical inspection of a weld seam and device for carrying out the method

Abstract

Computer-implemented method for generating a generation image (13) for optical inspection of a weld seam, comprising the following steps: a. Taking pictures (1; 2; 3; 4; 5; 6) along the weld seam (S1); b. Determination of several subgroups of the images (1; 2; 3; 4; 5; 6), wherein each of the subgroups relates to a region of the weld, wherein the region concerned is shown in all images (1; 2; 3; 4; 5; 6) of the respective subgroup (S2); c. Determination of subgroup pixels in the subgroups that correspond to the respective affected area (S2); d. Generation of generation pixels from the subgroup pixels for each of the areas (S2); and e. Generation of the generation image (13) from the generation pixels (S2).

Description

[0001] The invention relates to a computer-implemented method for generating a generation image for optical inspection of a weld seam according to claim 1 and a device for carrying out the method according to claim 10.

[0002] It is known from the prior art to visually inspect a weld seam for quality. EP 2 061 621 B1 discloses a method for the optical assessment of weld quality during welding. Different image areas are read out with different exposure times.

[0003] DE 21 2018 000 067 U1 discloses a laser marking system comprising an imaging device, a marking device, and a processing device. The system enables the acquisition of images from the marking area, with the imaging device creating multiple partial images. These are then combined by the processing device into a single overall image. Subsequently, the workpiece is identified within the overall image, and the marking is applied at a specific position relative to the workpiece. The processing device corrects perspective distortions and performs the marking.

[0004] DE 10 2020 211 408 A1 describes a welding information display device worn on the user's body, comprising a display unit, at least one camera unit for capturing welding images, a sensor unit for detecting light intensity in the welding area, and an environmental management unit for acquiring environmental data. The device can capture welding parameters such as speed, direction, temperature, and distance between the welding torch and the workpiece and display this information to the user in real time. The sensor unit can be partially mounted on the welding torch, and the environmental management unit provides additional image information about the welding environment.The system is designed to provide welding and environmental information during the welding process, but does not offer computer-aided image processing for optical inspection of the weld after completion of the welding process, nor does it offer algorithmic generation of a composite image from multiple individual images of the weld.

[0005] AT 508 696 A1 describes a monitoring module for an arc welding process, in which a camera, a light source, and a control unit are arranged in a single housing. The control unit ensures that the light source is activated during image acquisition by the camera. Additional features include a compressed air supply for cleaning and cooling, a removable cover, a microphone, and various control and regulation options for the light source and image acquisition. The module can be attached to the torch and is specifically designed for process monitoring during welding. However, no computer-implemented methods for image processing or for generating a composite image from multiple individual images along a weld seam are disclosed, nor is there any software-based evaluation or verification of the weld seam based on such image data.

[0006] WO 2013 / 147919 A1 discloses a computer-aided method for automatically determining the centerline of an object in an image. The method divides the image into several test areas and defines a limited number of scanning and detection directions, one of which is assigned to each test area. Each test area is then examined by scanning the image in the assigned scanning direction and performing a detection step in the assigned detection direction to determine a localized centerline pixel. Subsequently, a collection of these centerline pixels is gathered to obtain the centerline within the test area, the coordinates of which can be stored. Finally, the length of the centerline can be calculated by summing all localized centerline pixels from all test areas.The document also describes an application of the procedure for inspecting the quality of a weld, in which a series of images are taken and the length of the weld's centerline is calculated to compare it with a reference length.

[0007] In contrast, the present invention aims to create an image that is better suited for the optical inspection of the weld seam.

[0008] This problem is solved by a method according to claim 1 and by a device according to claim 10. Embodiments of the invention are specified in the dependent claims.

[0009] The method according to claim 1 comprises capturing images along the weld seam. The images can be, for example, individual frames of a video. The weld seam can be, for example, a laser weld. The images can be captured, in particular, along the entire length of the weld seam.

[0010] Several subgroups of images are defined. Each subgroup relates to a specific area of ​​the weld. The affected area is shown in all images of the respective subgroup. Images can belong to multiple subgroups if, for example, several of the areas are depicted in those images. Thus, there can be one subgroup for each area of ​​the weld.

[0011] Within the subgroups, subgroup pixels are identified that correspond to the respective affected area. This is advantageous because the images may depict more than just the affected area. In this description, a subgroup pixel corresponding to the affected area is understood to be, in particular, a pixel from one of the images in the respective subgroup that represents the area in that image. For example, this could be only a single pixel per image. In this case, the affected area consists of only a single pixel.

[0012] From the subgroup pixels, generation pixels are created for each of the areas. If the area consists of only a single pixel, it is possible that only a single generation pixel will be created for that area. Thus, multiple subgroup pixels can be used to generate each generation pixel. This results in more information being available for generating the generation pixels. For example, if there is little or no information about a specific area of ​​the weld on one or more images, information from other images depicting the same area can be used to at least partially compensate for this lack of information. This is advantageous, for instance, for reflections on the weld that are present on only one or some, but not all, images of one of the subgroups.

[0013] A generated image is created from the generated pixels. Due to the at least partially compensated information deficit compared to the individual images, this generated image is better suited for the visual inspection of the weld. This inspection can be carried out by a person or by a computer program. In both cases, the weld is easier to inspect due to the higher information content. For example, if reflections are the cause of the lower information content in some of the images, both a person and an artificial intelligence can better inspect the respective area for weld quality once the reflections have been at least partially eliminated.

[0014] According to one embodiment of the invention, images can be captured with a recording device while the weld is being produced with a welding device. The recording device can move together with the welding device during image capture. This is particularly advantageous for reliably capturing images of a large portion or even the entire weld. Furthermore, the number of mechanically moving components and drives can be reduced. Preferably, the recording device moves at the same speed as the welding device. It is particularly preferred that the welding device and the recording device are moved by the same drive.

[0015] According to one embodiment of the invention, a displacement, particularly a translational displacement, of the individual recorded images relative to one another can be determined. The determined displacement can be used to identify the subgroups. If the magnitude of the displacement of the images relative to one another is known, it is also possible to determine which images depict a specific area of ​​the weld seam, thus enabling the identification of the subgroups. The displacement can be determined, in particular, for images recorded directly one after the other. The displacement can be determined, in particular, as a number of pixels. For example, the displacement can be determined as the number of pixels by which a subsequent image is shifted relative to a specific image. In this way, the subgroup pixels can be determined particularly effectively.It is possible to determine the speed of the recording device when determining the displacement. It is also possible to determine the displacement of the images relative to each other using the speed of the recording device. For example, the speed and distance of the recording device from the weld seam may be known, allowing the displacement of the images relative to each other to be determined using these two parameters.

[0016] According to one embodiment of the invention, the displacement of the individual images relative to each other can be determined by calculating the displacement of a region captured in both images by a certain number of pixels. This region could, for example, be a section of the weld seam. It is also possible that the region is adjacent to the weld seam. The number of pixels can be determined, in particular, using the following formula: D=1mn∑i=0m∑j=0n(xi,j(1)−xi,j(2))2

[0017] where x (1) a pixel of the first of the two images and x (2) A pixel of a second of the two images. Both pixels together form a pixel pair. The expression x (1) - x (2)This describes the distance between the two pixels. Using the indices i and j, this distance is calculated for all pixel pairs in the region under consideration. Each individual distance is squared. Then, all the squared distances are summed.

[0018] According to one embodiment of the invention, the subgroup pixels can each have a subgroup brightness value. The generating pixels can each have a generating brightness value. When generating the generating pixels, the median of the subgroup brightness values ​​of the subgroup pixels used in the generation process can be used. Preferably, the generating brightness value is the median of the subgroup brightness values ​​of the subgroup pixels used in the generation of the respective generating pixel.

[0019] According to one embodiment of the invention, the weld seam can be checked using the production image.

[0020] According to one embodiment of the invention, the weld seam can be inspected by a computer program. The computer program can, for example, include artificial intelligence.

[0021] According to one embodiment of the invention, the number of generating pixels for each of the areas can correspond to the number of subgroup pixels for the respective area divided by the number of images of the respective subgroup.

[0022] According to one embodiment of the invention, each area can consist of a single pixel. This can mean that the number of subgroup pixels corresponding to an area corresponds to the number of images in the respective subgroup. Only a single generating pixel is then produced from each of the subgroups.

[0023] The device according to claim 10 comprises a digital electronic data storage device and a digital electronic processing unit, which may also be referred to as a processor. Instructions are stored in the data storage device. The processing unit is configured to read and execute the instructions. The instructions are configured to cause the processing unit, upon execution of the instructions, to carry out a method according to an embodiment of the invention.

[0024] Further features and advantages of the present invention will become clear with reference to the following description of preferred embodiments and the accompanying figures. The same reference numerals are used for identical or similar components and for components with identical or similar functions. Fig. 1A and B are two schematic top views of a section of the weld in images taken at different times; Fig. 2 a schematic representation of several images taken successively of different sections of the weld; Fig. 3 a schematic matrix representation of several images and a generating image according to an embodiment of the invention; Fig. 4 a schematic top view of a production image according to an embodiment of the invention; and Fig. 5 a schematic view of a method according to an embodiment of the invention.

[0025] In images 1 and 2 in the Fig. 1A and Fig. 1B highlights a section of the weld seam. Fig. In 1A, this area is designated as 14. Fig. In Figure 1B, this area is designated as 15. This is the same area of ​​the weld. However, the position of area 14 in Figure 1 is shifted relative to the position of area 15 in Figure 2 because the camera and welding equipment were moved together between the taking of Figures 1 and 2. The shift between Figures 1 and 2 can be calculated using the following formula: D=1mn∑i=0m∑j=0n(xi,j(1)−xi,j(2))2

[0026] where x (1) a pixel of the area 14 and x (2) A pixel in area 15. Both points together form a pixel pair. The expression x (1) - x (2) This describes the distance between the two pixels. Using the indices i and j, this distance is calculated for all pixel pairs in the considered areas 14 and 15. Each individual distance is squared. Then, all the squared distances are summed.

[0027] In Fig. Figure 2 shows how several images 1 to 6 are shifted relative to each other due to the movement of the recording device during image capture. This represents only a portion of all captured images. Since these are primarily individual frames from a video, several thousand images can be captured in total.

[0028] In Fig. 3 are images 1 to 6 from Fig. 2 are shown vertically compressed. Images 1 to 6 are part of a subgroup. The individual images from Fig. 2 are thereby cut off at the image boundaries. As a result, the original image 1 is in Fig. 2 is divided into several areas. It is in Fig. 3 is represented by pixels 7-12. As a result, all images of the same weld area taken at different times during production, and thus captured in different images, are displayed one below the other, row by row. It is possible and preferred that many more images are part of the subgroup.

[0029] In the next step, all images 1-6 are merged into a single display layer. This can also be described as superimposing the lines. A generation image 13 is created. The generation image has the same height as the original images 1-6, a newly created width resulting from the total movement of the lens, and a thickness in the drawing plane of six pixels in this example. A median of the brightness values ​​of the six pixels for each point of a defined height and width from generation image 13 is calculated. A generation pixel is then created with this calculated median as its brightness value. The steps described above are performed for different areas and thus also for different subgroups of the images. The generation image 13 is then created from all the generation pixels created in this way.Image 13, the generated image, can then be used for visual inspection of the weld. Areas that may be difficult to see in individual images 1 to 6 are more clearly visible in image 13, as information from other images relating to the same weld area was used in calculating the generated image points.

[0030] Such a production image of a weld seam is in Fig. 4 are shown as examples. In comparison with the Fig. 1A and Fig. In 1B, it can be seen that more details of the weld are visible, as reflections present in individual images have less of an impact on the visual impression.

[0031] The individual process steps are described in Fig.Figure 5 clearly illustrates this process. In the first step, S1, images are acquired along the weld seam. In the second step, S2, the image subgroups are determined, with each subgroup representing a specific area of ​​the weld seam. Furthermore, in step S2, the subgroup image points corresponding to each affected area are identified. The generation image points are then created from these subgroup image points. The brightness value for each generation image point can be calculated as the median of the subgroup image points used for generation. The generation image is then created from these generation image points.

[0032] The generated image is used in step S3 to visually inspect the weld. This step can be performed by a human or a computer program, particularly artificial intelligence. Both humans and computer programs can better assess the weld quality using the generated image than if the unprocessed images were used for inspection. In the optional step S4, individual areas of the weld are then flagged as visually deficient, indicating that the weld in these areas should be examined more closely and potentially reworked.

Claims

[1] Computer-implemented method for generating a generation image (13) for optical inspection of a weld seam, comprising the following steps: a. Taking pictures (1; 2; 3; 4; 5; 6) along the weld seam (S1); b. Determination of several subgroups of the images (1; 2; 3; 4; 5; 6), wherein each of the subgroups relates to a region of the weld, wherein the region concerned is shown in all images (1; 2; 3; 4; 5; 6) of the respective subgroup (S2); c. Determination of subgroup pixels in the subgroups that correspond to the respective affected area (S2); d. Generation of generation pixels from the subgroup pixels for each of the areas (S2); and e. Generation of the generation image (13) from the generation pixels (S2). [2] Method according to claim 1, characterized by, that the images (1; 2; 3; 4; 5; 6) are taken with a recording device while the weld is produced with a welding device, wherein the recording device is moved together with the welding device when taking the images (1; 2; 3; 4; 5; 6). [3] Method according to any one of the preceding claims, characterized by , that a shift of the individual recorded images (1; 2; 3; 4; 5; 6) relative to each other is determined, whereby the determined shift is used in the determination of the subgroups. [4] Method according to the previous claim, characterized by , that the displacement of the individual images (1; 2; 3; 4; 5; 6) relative to each other is determined by determining, in the case of two images taken directly one after the other (1; 2; 3; 4; 5; 6), a displacement of a region recorded in both images (1; 2; 3; 4; 5; 6) by a number of pixels. [5] Method according to any one of the preceding claims, characterized by , that the subgroup pixels each have a subgroup brightness value and the generation pixels each have a generation brightness value, wherein the median of the subgroup brightness values ​​of the subgroup pixels used in the respective generation is used in the generation of the generation pixels. [6] Method according to any one of the preceding claims, characterized by , that the weld is checked using the production image (13) (S3). [7] Method according to the previous claim, characterized by that the weld inspection is carried out by a computer program. [8] Method according to any one of the preceding claims, characterized by , that the number of generating pixels for each of the areas corresponds to the number of subgroup pixels for the respective area divided by the number of images (1; 2; 3; 4; 5; 6) of the respective subgroup. [9] Method according to any one of the preceding claims, characterized by that each area consists of one pixel. [10] Device comprising a digital electronic data storage device and a digital electronic processing unit, wherein instructions are stored in the data storage device, wherein the processing unit is configured to read and execute the instructions, wherein the instructions are configured to cause the processing unit, when executing the instructions, to carry out a method according to one of the preceding claims.

Images