Welding training system for performing virtual manual welding process

Abstract

In order to provide a welding training system (1) which is improved in terms of object recognition and object tracking compared to the prior art, a mixed reality helmet (800) with an RGB camera (803) and an infrared camera (804) is provided in the welding training system (1) according to the invention, the infrared field of view (810) being larger than the RGB field of view (809), and arranging infrared reflection reference marks (71, 73) on the training workpiece (4) and on the training hand-held torch (6) in a reference pattern (72, 74) that personalizes the training workpiece (4) and the training hand-held torch (6), and capturing the infrared reflection reference marks by an infrared camera (804) for object identification and object tracking.

Description

Technical Field

[0001] The invention relates to a welding training system and a method for performing a virtual manual welding process. Background Art

[0002] Welding, whether it is gas fusion welding, gas welding, manual arc welding, inert gas welding, plasma welding or laser welding, as the most important joining process in modern production technology, plays an important role in many modern production processes and manufacturing processes. For example, the construction of modern means of transportation such as motor vehicles (Kfz), railways or aircraft is unthinkable without modern high-precision welding technology. As in modern production technology in general, at least semi-automated welding processes ("robotic welding") are increasingly used in welding technology, which can save time and resources in many cases.

[0003] However, despite the increasing use of automated welding processes, the traditional form of welding, so-called "manual welding", remains an important part of a variety of different technical processes. In manual welding, a manual welder performs the required welding work by manually moving a handheld welding torch. As is known, manual welding is divided into a number of different manual welding processes, such as manual arc welding (electrode welding), WIG welding (tungsten inert gas welding), MIG welding (metal inert gas welding) or MAG welding (metal active gas welding), depending on the type of material used.

[0004] The motivations and reasons for using manual welding instead of automated robotic welding are varied. Welding tasks are often very complex, individual and unique, so that it is not possible to automate the welding process at a reasonable cost. Automation is also not possible in many cases when welding work is carried out in outdoor areas. In other cases, automation of the welding process would not be justified for economic reasons due to the high initial investment, mainly through the design of the welding control program.

[0005] Since the work of manual welders is increasingly focused on complex and individual welding tasks according to the above description, and the ever-increasing demands for faster, more precise and more economical welding processes also affect the manual welders who carry out the work, it is not difficult to see that the training of manual welders is a demanding and costly process that requires a lot of practice time, extensive technical support from experienced professionals and, in particular, consumable materials or training materials.

[0006] In order to support and simplify the training process for manual welders, and in addition to making the training of beginners as safe as possible and saving consumables or training materials, so-called welding training systems for virtual welding have been developed. The welding training system and the so-called "virtual welding" that can be carried out with the welding training system make it possible to simulate and repeatedly practice complex welding tasks and difficult situations safely, economically and realistically. The safety risks faced by beginners during welding, especially due to the hot and dazzling arc, can be very high, which is completely eliminated on welding training systems or "welding simulators". With the welding training system for virtual welding, manual welders to be trained can learn and practice basic welding skills / hands-on skills on common training workpieces. In addition, expensive consumables such as practice parts made of different metals / steels / alloys (and their partly complicated preparation work), welding wire and / or shielding gas and energy can be saved by virtual welding.

[0007] The basic components of a welding training system are a training handheld welding torch, a training workpiece, a welding simulator and an electronic display screen (electronic screen). During virtual welding, the training handheld welding torch moves along the training workpiece. The welding simulator determines the so-called virtual weld from the motion data of the training handheld welding torch and from the set welding parameters such as welding current, wire feed speed, etc., which is displayed on the display screen. The specific design scheme, in particular the collection of the motion data of the training handheld welding torch or the display of the virtual weld on the display screen, can be realized in different ways. The prior art provides a variety of schemes for this purpose.

[0008] Therefore, US 2020 / 0265750 A1 describes a welding training system in which markings are applied to both the training workpiece and the training handheld welding torch. For object recognition and object tracking, ie, detecting motion data, the markings are captured with the aid of an RGB camera, ie, an optical camera.

[0009] EP 2863376 A1 describes a concept for simulating different welding methods (MIG, MAG), wherein the results of these simulations are displayed on a display screen of video glasses within the framework of an augmented reality method ("AR"). To this end, one or more cameras record the surrounding environment and display a so-called mixed reality (real and virtual images superimposed on each other) to the manual welder.

[0010] WO 2020 / 167812 A1 describes a welding helmet that can be used in a virtual manual welding process. The welding helmet comprises an electronic screen, i.e. an electronic display, and a plurality of cameras (RGB (optical), thermal imaging, infrared). WO 2020 / 167812 A1 also describes markings that are added to the training workpiece and the training handheld welding torch. These markings are captured by an optical camera for object recognition and object tracking.

[0011] US 2021 / 0158724 A1 describes a camera sensor having a plurality of adjustable lenses, filters and other optical components for performing simulated welding training. The analysis of sensor data detected by the camera sensor is disclosed, and these sensor data can mainly describe markings for identifying objects, so as to determine the position, orientation and movement of the training workpiece and the training handheld welding torch from these sensor data.

[0012] Welding training systems known from the prior art, in particular welding training systems such as those disclosed in the cited documents, preferably use RGB cameras (optical cameras) for object recognition and object tracking, and in this case usually use the same RGB camera that is also used to capture the field of view (sometimes also called visual field) of the manual welder. This situation leads to a number of disadvantages. On the one hand, only components that are located in the field of view of the manual welder (that is, the field of view of the RGB camera) can be tracked, i.e., identified and tracked. Therefore, the tracking range is limited by the manual welder's line of sight guidance. Therefore, problems arise when the manual welder wants to observe the training workpiece from different observation angles.

[0013] In the case of welding training systems of the prior art, the manual welder must also maintain a minimum distance from the given training workpiece during the virtual welding process. In the case of tracking performed with an RGB camera, maintaining a minimum distance is necessary in order to always be able to capture a sufficiently large area of ​​the object to be tracked. This provision sometimes severely limits the applicability of welding training systems.

[0014] The field of view of the RGB camera that simultaneously tracks the object and captures the manual welder's field of view cannot be extended arbitrarily. Previous attempts, for example using wide-angle lenses such as fisheye lenses, have shown that the use of such lenses can cause dizziness and discomfort to manual welders. RGB camera-based tracking also depends in many cases on a given illumination level, which can sometimes produce severe functional impairments when illumination levels change. Summary of the invention

[0015] It is therefore an object of the present invention to provide a welding training system with improved object recognition and object tracking.

[0016] This object is achieved by the features of the independent claims. The independent claims describe in this case a welding training system and a method for carrying out a virtual welding process on the welding training system according to the invention.

[0017] The welding training system of the present invention has a training workpiece and a movable training handheld welding torch. According to the present invention, a first number of multiple infrared reflective reference marks are provided on the training workpiece, and these reference marks are arranged on the training workpiece with a first reference pattern personalized for the training workpiece. In addition, a second number of multiple infrared reflective reference marks are provided, and these reference marks are arranged on the training handheld welding torch with a second reference pattern personalized for the training handheld welding torch.

[0018] Furthermore, the welding training system according to the invention also has a mixed reality helmet, on which a mixed reality display, an RGB camera and an infrared camera are provided. In this case, it is very important that the infrared camera has a spatial infrared field of view for recording / capturing infrared images of infrared reflective reference marks located in the infrared field of view, which infrared field of view is larger than the RGB field of view of the RGB camera. Therefore, in combination with the use of the infrared reflective reference marks according to the invention, object recognition and object tracking can be improved compared to welding training systems known in the prior art. The manual welder can get much closer to the training workpiece, can perform significantly faster movements with the training handheld welding torch, and thus even in this scenario, accurate and correct object recognition and object tracking, in particular accurate and correct object recognition and object tracking of the training handheld welding torch, can be ensured.

[0019] By means of the welding training system according to the invention, tracking, ie object recognition and object tracking in particular of the training workpiece and the training hand welding torch, can be ensured precisely and reliably, even in dynamic phases, such as rapid movements of the training hand welding torch.

[0020] With the infrared camera of the invention, a significantly larger tracking area can be captured than in the prior art. The position of the component to be tracked is also almost no longer restricted within the scope of the invention and a minimum distance to the training workpiece no longer has to be maintained.

[0021] Furthermore, the welding training system of the present invention also has a simulation unit, which is configured to determine the geometry and / or shape and / or type of the training hand-held welding torch - such as a MIG hand-held welding torch or a MAG hand-held welding torch or a TIG hand-held welding torch or a manual arc welding torch - and the training workpiece and the temporal change of the spatial position of the training hand-held welding torch and the training workpiece relative to the helmet reference point based on the infrared images of the first and second reference patterns captured by the infrared camera. On this basis, the simulation unit makes it possible to determine the temporal change of the position of the virtual welding electrode and to determine therefrom a virtual weld seam on the training workpiece, using which a mixed reality image of the virtual manual welding process is generated, which is ultimately displayed on a mixed reality display that is also arranged on the mixed reality helmet.

[0022] The type and specification of the welding electrodes used in the actual welding process are known to depend on the type of welding process. For example, in arc welding, the arc burns between the workpiece and the welding electrode, for which a wire electrode or a strip electrode or a melting rod electrode or a non-melting welding electrode (e.g. a tungsten-based welding electrode) that melts under a shielding gas can be provided. Within the scope of the present invention, the welding electrodes required for the simulation welding process, i.e., for example, a wire electrode or a strip electrode or a melting rod electrode or a non-melting welding electrode that melts under a shielding gas, are taken into account in the simulation in the form of a virtual welding electrode. Preferably, the type of the virtual welding electrode can be identified simultaneously with the type of the training handheld welding torch, wherein the temporal position change and the spatial position of the virtual welding electrode can be advantageously determined based on the temporal change of the determined spatial position of the training handheld welding torch.

[0023] Within the scope of the invention, much more accurate information about the spatial position of the components that are important for the virtual manual welding process - in particular the training handheld welding torch and the training workpiece - is provided, which information can also be used in the simulation unit to simulate the determination of the virtual weld seam, so that the virtual manual welding process can be designed to be much more realistic. The greater realism improves the training effect of the manual welder, which is another parameter that the invention improves compared with the known prior art. Within the scope of the invention, the RGB camera is used as a so-called "live observation" of the environment, and the tracking is achieved by the infrared camera. In this case, it is very important that the infrared camera can be equipped with a larger field of view, which has the advantage of a much larger tracking range, but avoids the above-mentioned disadvantages for the manual welder who is working.

[0024] An important advantage of the welding training system of the invention is that, due to the use of an infrared camera for tracking, in particular movable components can be equipped with smaller, passive and therefore more economical markings than in the prior art, but can still be reliably and accurately identified.

[0025] The movable component to be tracked also requires significantly less hardware than components known in the prior art, particularly since the tag is passive and therefore does not require power. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Refer to the following Figure 1 to 1 2 describes the present invention in detail, and the accompanying drawings show advantageous structural designs of the present invention by way of example, schematically and non-limitingly. Among them:

[0027] Figure 1 A possible structural design of the welding training system of the present invention is shown;

[0028] Figure 2 A schematic block diagram showing a simulation model in a simulation unit of the present invention;

[0029] FIG3 shows a welding training system of the prior art;

[0030] Figure 4 The mixed reality helmet of the present invention is shown;

[0031] Figure 5 A front view of a mixed reality helmet and an adapter for fixing an infrared camera and an RGB camera on the mixed reality helmet at a predetermined spatial distance;

[0032] Figure 6 The field of view of the RGB camera and the infrared camera arranged on the mixed reality helmet according to the present invention is shown;

[0033] Figure 7 A training handheld welding torch designed according to the present invention is shown with infrared reflective reference marks;

[0034] Figure 8 showing design possibilities for a first reference pattern for personalizing a training workpiece;

[0035] Fig. 9 A side view showing a construction design of a first reference pattern for personalizing a training workpiece, having recesses for arranging infrared reflective reference marks;

[0036] Fig.10 A detailed view showing a recess for arranging an infrared reflective reference mark;

[0037] Fig.11a , Fig.11b , Fig.11c Showing that a menu or interactive menu on a mixed reality display of the mixed reality helmet of the present invention is activated when a mixed reality target object is captured;

[0038] Fig.12a , 12b A weld seam and a weld root are shown. DETAILED DESCRIPTION

[0039] Figure 1 The welding training system 1 according to the present invention for performing a virtual manual welding process is shown as a possible design. The basic components of the welding training system 1 are a training workpiece 4, a manually movable training handheld welding torch 6, a mixed reality helmet 800 and a simulation unit 9 with a schematically shown welding torch holder 17.

[0040] Mixed reality in the present context refers to the mixture of the natural perception of the manual welder 2 and the artificially generated (“computer-generated”) perception. The natural perception of the manual welder 2 is captured within the scope of the present invention by the RGB camera 803 and represented by the RGB image generated by the RGB camera 803 .

[0041] As described in detail below, the simulation unit 9 is used as the CPU or computing unit of the welding training system 1 shown. How the CPU and the simulation unit 9 of the welding training system 1 can be constructed can be obtained from a plurality of documents of the prior art. Specifically, for example, EP 2 863 376 A1, US 202 / 0265750 A1 or WO 2020 / 167812 A1 provide relevant descriptions.

[0042] Within the scope of the invention, the simulation unit 9 can also be integrated in a real welding device (also referred to as a "welding power source" or "power source" by those skilled in the art), to which the training handheld welding torch 6 can be connected like a conventional real welding torch. In this case, in particular, interfaces provided on the real welding device or its housing, for example interfaces for connecting a real handheld welding torch, can also be used without modification when performing a virtual welding process. In the case of a virtual welding process being performed by means of a simulation unit 9 integrated in a real welding device, the power supply to the real welding device, for example the power supply to the training handheld welding torch 6, is disconnected.

[0043] The training handheld welding torch 6 can be, in particular, a real handheld welding torch, which can be adapted to perform a virtual manual welding process, but can still be adapted to perform a real welding process, such as a real tungsten inert gas (WIG) welding process.

[0044] To arrange the training artifact 4, Figure 1 In the embodiment shown, a workpiece holder 5 is provided, on which the training workpiece 4 can be mounted. With the aid of a workpiece holder 5, various so-called welding positions can be simulated in particular. As is known, a welding position describes the position and / or orientation and / or arrangement of the workpiece, welding torch and welding electrode, and thus the position and / or orientation and / or arrangement of the weld seam during the welding process. In this regard, the standard DIN EN ISO 6947 or the ASME code section IX (QW-120) defines a series of possible welding positions, such as PA (horizontal welding of butt welds and fillet welds), PB (horizontal welding of fillet welds, horizontal vertical position) or PC (transverse position or transverse weld, horizontal welding on a vertical wall). In this case, in particular when welding vertical welds, difficult results may occur, such as melt downflow, which can also be taken into account within the scope of the invention and displayed as part of the virtual manual welding process.

[0045] In order to arrange the training workpiece 4 on the workpiece holder 5, a variety of different mounting methods can be considered. Therefore, the training workpiece 4 can be mounted on the workpiece holder 5 in a magnetic manner, or screwed, firmly clamped, firmly bonded, cast, wedged on the workpiece holder 5, or simply placed thereon. In addition, the workpiece 4 can generally also be mounted in a form-locked, force-locked and / or material-locked manner. In the case of magnetic mounting, at least one metal part can be configured for the training workpiece 4, and at least one magnet can be configured for the workpiece holder 5 to provide a magnetic holding force. It is also conceivable to configure a magnet in the training workpiece 4 and / or a metal part in the workpiece holder 5. Here, the space in the welding training system 1 in which the training workpiece 4, optionally the tool holder 5, optionally the training handheld welding torch 6 and the welding torch holder 17 are arranged is referred to as a 3D welding environment 3.

[0046] However, the workpiece holder 5 is not an indispensable component of the welding training system 1 of the present invention. Moreover, the welding torch holder 17 is not an indispensable component of the welding training system 1 of the present invention. Therefore, the training workpiece 4 can also be placed freely and loosely. The working mode of the welding training system 1 of the present invention is not affected by the arrangement of the training workpiece 4.

[0047] The use of only one single training workpiece 4 is also purely exemplary. It is entirely possible to provide a second additional training workpiece 4 or a plurality of additional training workpieces 4. These training workpieces 4 can have the same or different shapes. In the case of simulating a welding process with the goal of connecting a plurality of workpieces on the welding training system 1, it can be provided that a plurality of training workpieces 4 are used. Here, the basic operating mode of the welding training system 1 remains unchanged, so the invention is described below based on only one training workpiece 4 without limiting its generality.

[0048] The principle of training a manual welder 2 implemented with the basic components of a training workpiece 4, a training handheld welding torch 6, a mixed reality helmet 800 and a simulation unit 9 provides that a virtual weld seam 13 is generated on the training workpiece 4 by manually moving the training handheld welding torch 6, and the virtual weld seam 13 is displayed to the manual welder 2 together with the training workpiece 4. In this case, the virtual weld seam 13 does not actually exist, but is determined by simulating an actual welding process when the movement data of the handheld welding torch 6 and the training workpiece 4 are given in advance. For visualization purposes, the virtual weld seam 13 is displayed as part of a mixed reality image 12 sequence of virtual manual welding processes on a mixed reality display 801 arranged on the mixed reality helmet 800. Possible variant embodiments of the mixed reality display 801 are well known in the prior art, and preferably a liquid crystal display or a liquid crystal display screen can be used for this purpose.

[0049] By using Figure 1 The welding training system 1 shown in the figure can learn a variety of different welding processes without safety risks, such as MIG (Metal Inert Gas Welding) manual welding process, MAG (Metal Active Gas Welding) manual welding process, WIG (Tungsten Inert Gas Welding) manual welding process, plasma manual welding process or manual arc welding process.

[0050] In order to make the impression obtained on the welding training system 1 as similar as possible to the actual welding situation, the manual welder 2 can also wear welding gloves and / or protective gear during the virtual welding process, so that a high degree of consistency between actual welding and virtual welding is also achieved in terms of the clothing to be worn by the manual welder 2. For this purpose, it is also conceivable within the scope of the present invention to integrate the mixed reality helmet 800 into a real welding helmet.

[0051] In order to realize the above basic principle of virtual welding, an RGB camera 803 ( Figure 4 ), which optically captures the components used in virtual manual welding, in particular the training handheld welding torch 6 and the training workpiece 4, provided that these components are located within the field of view of the RGB camera 803. In this case, the RGB camera 803 can be designed as a traditional digital camera, that is, an optical instrument or photographic device for recording moving images. The RGB camera 803 can be configured with suitable or required objective lenses, apertures, optical filters, etc.

[0052] A large number of digital camera designs are available from the prior art, for example DE 10 2005 053 276 A1 or US 10,877,266 B2. For the RGB camera 803 used within the scope of the present invention, it is important that it can capture "visible light", i.e. light in the visible part of the electromagnetic spectrum. As is known, visible light includes electromagnetic waves with wavelengths in the range of 400 nm to 780 nm.

[0053] The RGB camera 803 is arranged on the mixed reality helmet 800 and can capture RGB images of objects in the field of view (hereinafter referred to as "RGB field of view" 809) of the RGB camera 803. Here, the term "field of view" refers to an area in the field of view of an optical device in which an object, event or change can be perceived by the optical device and thus can be recorded by the RGB camera 803. The terms "field of view" and "viewing angle" are well known to those skilled in the art of video / camera technology.

[0054] In order to perform a specific virtual manual welding process and to visualize it, it is necessary that at least a portion of the training workpiece 4 and at least a portion of the trained handheld welding torch 6 are located within the RGB field of view 809 of the RGB camera 803 at at least one time point during the manual welding process. In this case, it is advantageous that at least three, preferably at least four or particularly preferably at least five infrared reference markers 71, 73 are in the RGB field of view 809 of the RGB camera 803. However, this condition does not exclude that the manual welder 2 briefly moves away from the training workpiece 4 during the virtual manual welding process and turns his sight to the ground for a short time, for example. However, the manual welder 2 usually works facing the training workpiece 4, so the above requirements are usually met from beginning to end.

[0055] in this case, Figure 1 The simulation unit 9 of the welding training system 1 shown in FIG. 1 is configured to determine the geometry and / or shape and / or type of the training handheld welding torch 6 and the training workpiece 4 and in particular the temporal changes in the spatial position of the training handheld welding torch 6 and the training workpiece 4 during the execution of the virtual welding process. In order to achieve this identification and tracking (also referred to as "tracking" in the prior art) of the spatial position of the training handheld welding torch 6 and the training workpiece 4, a first number of multiple infrared reflective reference marks 71 (hereinafter referred to as "infrared reflective" reference marks 71) and a second number of multiple infrared reflective reference marks 73 are provided within the scope of the welding training system 1 of the present invention. As described in detail below, the infrared reflective reference marks 71, 73 of the present invention can be configured to be much smaller than the solutions known in the prior art, and preferably the diameter or the maximum size or the maximum distance between two points of the edge of a reference mark 71, 73 is in the range of only a few millimeters.

[0056] "Infrared" in the context of this application refers to infrared radiation, also called IR radiation, which corresponds to the well-known electromagnetic radiation in the spectral range between visible light and the longer-wavelength terahertz radiation. Specifically, this refers to light with a wavelength between 780 nm and 1 mm, which corresponds to a frequency range of 300 GHz to 400 THz, or 10 cm -1 Up to 12800cm -1 Infrared-reflecting reference marks are characterized in that they re-reflect preferably more than 70%, or very preferably more than 80%, or particularly preferably more than 90%, or most preferably more than 99% of the infrared radiation impinging on them.

[0057] In this case, a first number of multiple infrared reflective reference marks 71 are arranged on the training workpiece 4 with a first reference pattern 72 that personalizes the training workpiece 4. A second number of multiple infrared reflective reference marks 73 are arranged on the training handheld welding torch 6 with a second reference pattern 74 that personalizes the training handheld welding torch 6.

[0058] A "reference pattern" is a certain geometric arrangement of a plurality of infrared-reflecting reference marks. In this case, "individualized" means that a reference pattern is unique, so that a reference pattern is different from other reference patterns. The reference pattern is preferably individualized when viewed from different viewing directions, wherein only a part of the reference pattern is visible from each viewing direction.

[0059] In order to capture the infrared reflective reference marks 71, 73, the present invention provides an infrared camera 804 specifically designed for this purpose.

[0060] Infrared camera 804 is specifically an optical device similar to a conventional camera, which can receive and process infrared radiation and further reproduce the infrared radiation as an image. Light in a spectral range outside the infrared range is not recorded by infrared camera 804. For this purpose, infrared camera 804 can also be configured with a corresponding optical filter.

[0061] According to the present invention, the infrared camera 804 and the RGB camera 803 are arranged on the mixed reality helmet 800 at a predetermined, fixed spatial distance. Figure 4 ). Here, the infrared camera 804 has its own field of view, hereinafter referred to as the “infrared field of view” 810, just like the RGB camera 803. The infrared camera 804 is capable of capturing infrared images of the infrared reflective reference marks 71, 73 located in the infrared field of view 810. While capturing the infrared reflective reference marks 71, 73, at least part of the reference patterns 72, 74 that personalize the training workpiece 4 and the training handheld welding torch 6 is also captured.

[0062] A main feature of the infrared camera 804 used according to the present invention is that its infrared field of view 810 is larger than the RGB field of view 809 of the RGB camera 803, preferably larger in at least one solid angle. According to the present invention, the infrared field of view 810 overlaps the RGB field of view 809.

[0063] During the virtual manual welding process, at least a portion of the first reference pattern 72 and at least a portion of the second reference pattern 74 are located in the infrared field of view 810. This situation will be discussed separately below.

[0064] In order to assign the reference patterns 72, 74 to a specific training workpiece 4 or training handheld welding torch 6, a database, database entries or lists can be stored in the simulation unit 9, which associate different reference patterns 72, 74 with descriptions of the training workpiece 4 or training handheld welding torch 6. If the infrared camera 804 now captures a reference pattern 72, 74 or a part of the reference pattern 72, 74 from which the entire reference pattern 72, 74 can be inferred, the underlying training workpiece 4 and its features can be identified by comparison with the provided database entries. In this case, the database entries can include descriptions of the geometry, dimensions, materials, etc. of the training workpiece 4.

[0065] The same applies to the training handheld welding torch 6, which can also be identified by comparing the captured reference pattern 74 with the corresponding database entry.

[0066] Preferably, infrared images and RGB images are continuously captured at discrete time points k. Figure 1 The same is true within the scope of the structural design of the present invention shown. "Continuously" in this case means repeatedly capturing infrared images and RGB images at each discrete time point. It should be noted that the RGB camera 803 and the infrared camera 804 can also capture at different rhythms within the scope of the present invention and therefore have different scanning times.

[0067] According to the above description, the continuously captured infrared images form a continuously captured part of the first and second reference patterns 72, 74, from which a continuous description of the spatial position of the training workpiece 4 and the training handheld welding torch 6 can be obtained in the simulation unit 9. For this purpose, the captured RGB image and the captured infrared image are transmitted to the simulation unit 9. For this purpose, the RGB camera 803 and the infrared camera 804 can be connected to the simulation unit 9, for example, via a cable to transfer the recorded image data. However, within the scope of the present invention, a wireless connection ("wireless") of the RGB camera 803 and the infrared camera 804 to the simulation unit 9 can also be considered. In order to wirelessly connect the RGB camera 803 and the infrared camera 804 to the simulation unit 9, known solutions such as Bluetooth, Wireless LAN or ZigBee can be used.

[0068] Finally, the relative movement between the training hand-held welding torch 6 and the training workpiece 4 required for the virtual welding process can be determined in a known manner from the continuous description of the spatial position of the training workpiece 4 and the training hand-held welding torch 6. The acceleration and velocity of the training workpiece 4 and the training hand-held welding torch 6 can also be advantageously determined from the change in their position, which can also be taken into account when determining the virtual weld seam 13.

[0069] It should be noted that the spatial position of the training handheld welding torch 6 and the training workpiece 4 and the resulting position change are determined relative to a helmet reference point 802 fixed as a fixed point on the mixed reality helmet 300. However, the helmet reference point 802 moves with the mixed reality helmet 800, so that the position description has great flexibility. The specific composition of the position description based on the helmet reference point 802 will be discussed in detail below.

[0070] In addition, the spatial position of the training handheld welding torch 6 or the training workpiece 4 and the resulting position change can not only be determined, but also evaluated. On this basis, guidance can be displayed on the mixed reality display 801, or prompts can be displayed on the mixed reality display 801, for example, if the manual welder 2 incorrectly positions the training workpiece 4, then prompts or guidance for correcting the positioning can be issued.

[0071] In order to determine a continuous description of the spatial position of the training workpiece 4 and the training handheld welding torch 6, geometric reference points on the training workpiece 4 and the training handheld welding torch 6 can first be determined. For example, the geometric reference points on the training workpiece 4 and the training handheld welding torch 6 can be determined based on reference patterns 72, 74 applied to the training workpiece 4 and the training handheld welding torch 6. On this basis, position vectors can be determined - for example three-dimensionally - which describe the relative position of the geometric reference points on the training workpiece 4 and the training handheld welding torch 6 relative to the pre-defined geometric helmet reference points 802. These position vectors can be determined from the image data captured by the infrared camera 804.

[0072] The temporal change of the image data of the infrared camera 804 can determine the temporal change of these position vectors, from which the position change of the training workpiece 4 and the training handheld welding torch 6 can be directly derived. Therefore, the position change or position change curve can be understood as a time series of position vectors, which describe the relative position of the training workpiece 4 and the training handheld welding torch 6 relative to the helmet reference point 802.

[0073] The orientation of the training workpiece 4 and the training handheld welding torch 6 relative to the helmet reference point 802 can also be determined in an advantageous manner from the reference patterns 72, 74. In order to also determine the orientation of the training workpiece 4 and / or the training handheld welding torch 6 in the 3D welding environment 3 or in general in three-dimensional space, other sensors can also be provided, which provide additional sensor data that can be used for this purpose. These sensors will be discussed separately below, with the help of which, for example, the welding positions (PA, PB, ...) mentioned at the beginning can also be determined.

[0074] If the geometry of the training workpiece 4 and the training hand welding torch 6 is known, the temporal change of the corresponding spatial position can be determined based on the orientation of the training workpiece 4 and the training hand welding torch 6 in conjunction with the determined position change.

[0075] In addition, Figure 1 In the context of the welding training system 1 shown, it is provided that, based on a change in the position of the training hand welding torch 6 and the training workpiece 4, taking into account at least one predefined training welding parameter p s , a virtual weld seam 13 produced by a manual welder on a training workpiece 4 during a virtual manual welding process is determined. The virtual weld seam 13 is preferably determined by simulating an actual manual welding process, wherein under the following boundary conditions, an actual handheld welding torch, preferably equipped with a corresponding welding electrode, is moved relative to the training workpiece 4 according to a determined temporal position change of the training handheld welding torch 6, wherein the corresponding welding electrode is represented in the simulation by a virtual welding electrode. In this case, in particular in the case of MIG welding / MAG welding, it can also be taken into account that the length of the virtual welding electrode is variable. For this purpose, a suitable welding simulation model 91 can be run on the simulation unit 9, which, according to the position change (curve) of the training handheld welding torch 6 and the training workpiece 4 and therefore also based on the temporal position change (curve) of the virtual welding electrode, taking into account at least one pre-given training welding parameter p s In order to simulate the actual manual welding process, the prior art provides a series of solutions, such as in RU 2694147 C1, WO 2020 / 056388 A1 or US 2015 / 0352794A1.

[0076] The simulation unit 9 is designed as a computer or computer-based hardware. The simulation model 91 can be run as software executed on the simulation unit 9. Figure 2 A possible configuration of the simulation model 91 is schematically shown in FIG. 9 with the aid of a block diagram. The simulation model 91 is arranged in the simulation unit 9 and acquires the image data B of the RGB camera 803. RGB , image data B of infrared camera 804 IR And training welding parameters p s As input signals, the simulation model 91 determines therefrom a virtual weld seam 13 . According to the invention, this virtual weld seam 13 is then displayed on a mixed reality display 801 of the mixed reality helmet 800 .

[0077] The simulation model 91 can advantageously be designed to also take into account the addition of filler materials that are common in welding technology. For this purpose, a further sub-simulation model can be used depending on the selected welding type, which simulates the addition of filler materials by means of a model.

[0078] In this case, the at least one predetermined training welding parameter p s It may correspond to the welding current intensity or welding voltage, or welding speed, or workpiece geometry, or waiting time, or preheating temperature, or wire feed speed, or arc length, or welding type, or shielding gas setting, in particular shielding gas flow rate. However, it is also conceivable within the scope of the invention to select other training welding parameters p s Since welding is a complex physical process in which usually multiple variables need to be specified, usually more than one single training welding parameter p is specified. s , but multiple training welding parameters p s .

[0079] The training welding parameters p s The virtual manual welding process can also be changed in an advantageous manner, for which purpose a suitable - preferably tactile - operating element, such as a rotary potentiometer, can be provided on the simulation unit 9. For example, the training welding parameter p can be (preferably only) adjusted using the tactile operating element. s The simulated shielding gas and other training welding parameters p s For example, the adjustment can be carried out at another location, such as on a tablet computer as portable display element 14 , which will be discussed separately below.

[0080] In a final step, the determined virtual weld seam 13 and the welding progress associated therewith must be graphically processed and displayed. For this purpose, a mixed reality display 801 is provided on the mixed reality helmet 800. In the graphical display, the corresponding current time point T in the virtual manual welding process should be noted. akt According to the invention, it is provided that at least until the current time point T of the virtual manual welding process akt The virtual weld 13 determined so far is located at the current time point T akt The part existing in the RGB field of view 809 is superimposed on the part at the current time point T akt The RGB image exists in order to generate a virtual manual welding process at the current time point T akt Of course, the virtual weld 13 is superimposed with the correct spatial orientation and position relative to the training workpiece 4 so that the virtual weld 13 is correctly displayed on the training workpiece 4.

[0081] The mixed reality image 12 determined according to the above description is then transmitted to the mixed reality display 801 set on the mixed reality helmet 800 for displaying it on the mixed reality display 801 as part of the mixed reality image 12 sequence of the virtual manual welding process.

[0082] like Figure 1 As shown in FIG. 1 , the mixed reality image 12 can also be displayed on an additional display element 14. This enables, for example, an instructor to observe a virtual welding process. In this case, the display element 14 can also display a mixed reality image 12 of the virtual welding process observed from the instructor's perspective for the instructor. In addition, the instructor gives optical prompts or pointers to the manual welder 2, which can be transmitted from the display element 14 to the mixed reality display 801 in real time.

[0083] The steps of determining the position change of the training handheld welding torch 6 and the training workpiece 4, determining the virtual weld 13 and generating the mixed reality image 12 are preferably performed in the simulation unit 9, that is, in the CPU or computing unit of the welding training system 1 in question. The simulation unit 9 can be mainly implemented by microprocessor-based hardware, such as by a microcontroller and an integrated circuit (ASIC, FPGA). However, other schemes for forming the simulation unit can also be considered. Therefore, multiple simulation units 9 can also be set, or the simulation unit 9 can also be connected to multiple other simulation units 9 via a computing cloud.

[0084] The simulation unit 9 can be designed to perform further complex calculations in addition to calculating the virtual weld 13. For example, in the simulation unit 9, based on the determined virtual weld 13, a virtual metallographic structure and / or a virtual stress-induced deformation and / or a virtual grain structure of the metallographic structure can be determined and displayed as required in a subsequent mechanical simulation, which can appear as a result of the virtual weld 13 in the training workpiece 4.

[0085] Furthermore, the simulation unit 9 can also be designed to determine, based on the subsequent mechanical simulation, so-called welding quality parameters which describe the quality and / or quality of the virtual weld seam 13. Welding quality parameters can describe weld seam thickness, fatigue strength, weld seam transition or edge offset, as described, for example, in the standard DIN EN ISO 581. It can advantageously be provided that the determined welding quality parameters are displayed on the mixed reality display 801 in addition to the virtual manual welding process.

[0086] The simulation unit 9 can also be configured to superimpose a predetermined virtual ideal weld seam 131 (in the example of FIG. 1 ) which is to be generated by a virtual manual welding process on the existing RGB image. Figure 1 and Fig.11ac). Based on this, the mixed reality display 801 can of course be configured to display an RGB image with a virtual ideal weld 131 superimposed thereon. With respect to the virtual ideal weld 131, the simulation unit 9 can also be configured to determine one or more weld quality parameters, which can, for example, describe the deviation between the determined virtual weld 13 and the pre-given ideal weld 131. In this way, an objective and reproducible evaluation of the virtual weld 13 produced by a manual welder can be performed.

[0087] In order to evaluate the produced virtual weld 13, a deviation threshold value can advantageously be predefined. If the deviation between the determined virtual weld 13 and the predefined ideal weld 131 exceeds the predefined deviation threshold value, the simulation unit 9 or another element of the welding training system 1 can be designed to draw the attention of the manual welder 2 to this deviation and thus to the welding defect associated therewith by means of acoustic and / or optical signals. These acoustic and / or optical signals are preferably displayed by means of a mixed reality helmet 800, which can be equipped with suitable speakers for this purpose.

[0088] Furthermore, the simulation unit 9 may also be configured to be capable of data backup and / or welding recording, so that the learning manual welder 2 can check the virtual manual welding process again, analyze defects and record the learning progress.

[0089] The simulation unit 9 can also be arranged in a particularly preferred manner on the mixed reality helmet 800. The simulation unit 9 can therefore be a component of the mixed reality helmet 800 or an integral, ie inseparable, component of the mixed reality helmet 800. Thus, for example, a separate simulation unit 9 can be omitted.

[0090] Unlike the actual manual welding process, the virtual manual welding process usually does not require any preparation or post-processing of the training workpiece. Especially in the case of steel welding, the welding result often depends decisively on the weld preparation. One of the most important prerequisites for good welding is the cleanliness of the weld edges. These weld edges must not only have a metallic luster, i.e., be free of oxides and / or scales, but also must not be contaminated by grease, oil or other organic matter, which can lead to carburization and inclusions in the weld. No preparation and post-processing are required on the training workpiece 4. However, in an advantageous embodiment, the present invention allows a deliberately pre-set in the simulation unit 9 of the present invention of an inadequately prepared manual welding process and to take it into account in the simulation for calculating the virtual weld 13, in particular in order to show the manual welder 2 to be trained the consequences associated with an inadequately prepared manual welding process.

[0091] Figure 1The structural design of the welding training system 1 shown also has an optional additional display element 14. However, the additional display element 14 is not an essential component of the welding training system 1 of the present invention, but is only used to display the virtual manual welding process for other personnel other than the manual welder 2, such as an instructor or another trainer. For this purpose, the additional display element 14 can preferably be designed as a display element that can be separated from the simulation unit 9, such as in the form of a tablet computer, which can be removed from the simulation unit 9 and can be reinstalled on the simulation unit, for example, via a plug connection. However, an additional display element 14 is naturally not a necessary prerequisite for the welding training system 1 of the present invention.

[0092] In a display element 14, preferably in a display element 14 implemented as a tablet computer, the possibility of setting different front ends, i.e. different graphical user interfaces, can be provided. In this case, the front end of the display element 14 can be adapted to an actual welding device simulated in a simulation or to corresponding operating units of an actual welding device simulated in a simulation, so that operating units of different actual welding devices can be realistically simulated. The mixed reality display 801 of the mixed reality helmet 800 can also be provided with the possibility of changing the front end in the 3D view, in particular in order to adapt the front end to the actual welding device simulated in the simulation. Preferably, the front end can be adapted to the training welding parameters p taken into account in the simulated manual welding process. s Synchronization and coordination, since not every front end necessarily has the same training welding parameters p s .

[0093] In addition to displaying the virtual manual welding process for other persons besides the manual welder 2, it can also be provided that the virtual manual welding process is controlled via a display element 14, preferably via a portable tablet computer. In particular, it can also be provided that the virtual manual welding process is controlled exclusively via a portable tablet computer. If a removable display element 14 is provided, preferably no further display elements are provided on the rest of the simulation unit 9, except for a few status LEDs for signaling and / or displaying the status of the simulation unit 9.

[0094] Compared to the welding training system 1 of the present invention discussed so far, a welding training system of the prior art is shown in FIG3 . It is obvious here that, in order to identify the welding torch 21 and the workpiece 22 , large-area markers (e.g. so-called “image markers”, such as ArUco markers well known in the field of image processing, which are usually larger than 1 cm in diameter) are always used, which are captured by the RGB camera 23 .

[0095] The design shown in Figure 3 has the obvious disadvantage that only components located in the RGB field of view of the RGB camera 23 can be tracked, i.e., identified and traced. Therefore, the so-called tracking area marked by the circle 24 in Figure 3 is limited by the sight guidance of the manual welder 2 and the field of view of the RGB camera.

[0096] The field of view of the RGB camera that simultaneously tracks the object and captures the field of view of the manual welder 2 is also not arbitrarily extendable. Attempts to do so - for example using wide-angle lenses such as fisheye lenses - have shown that the use of such lenses can cause dizziness and discomfort to the manual welder 2. Tracking based on RGB cameras is also very dependent on the given illumination, which can sometimes lead to severe functional impairments when the illumination changes.

[0097] The welding training system 1 of the present invention solves the above-mentioned problems of the prior art, and the mixed reality helmet 800 in particular plays a decisive role in solving these problems. Figure 4 A possible structural design of a mixed reality helmet 800 of the present invention is shown in FIG. The mixed reality helmet 800 shown has a mixed reality display 801 for displaying a mixed reality image 12 sequence of a virtual manual welding process.

[0098] It goes without saying that the mixed reality display 801 is of course arranged on the mixed reality helmet 800 in such a way that a person wearing the mixed reality helmet 800 has the mixed reality display 801 in the field of vision.

[0099] The mixed reality helmet 800 shown is equipped with a predefined geometric helmet reference point 802. The resulting coordinate system for describing the spatial position of the training workpiece 4 and the training handheld welding torch 6 is not static and cannot be changed during the virtual manual welding process. Instead, the helmet reference point 802 can be used as the origin of a movable coordinate system that moves with the movement of the manual welder 2 and therefore with the movement of the mixed reality helmet 800. In contrast, in the known solutions of the prior art, for example, the RGB camera is arranged in a fixed position on a tripod in the space of the welding training system, only static coordinate systems can be used. The reference point 802 on the mixed reality helmet 800 can significantly enhance the tracking effect, that is, the effect of identifying and tracking objects (such as a training handheld welding torch) and better adapt to the given welding situation in the specific case.

[0100] In addition, the mixed reality helmet 800 may also have a weight compensation element 807, as shown in FIG. Figure 4 The weight compensation element 807 utilizes the torque T KThe torque T formed by the weight or weight distribution of the RGB camera 803 and / or the weight of the infrared camera 804 and / or the weight of the mixed reality display 801 and acting perpendicular to the line of sight 10 of the mixed reality helmet 800 is offset. g .

[0101] As far as the present situation is concerned, it has been found that the mixed reality helmet 800, especially as in Figure 4 As shown in , a considerable total weight of the mixed reality helmet 800 may sometimes be present when it is connected to the simulation unit 9 by means of a cable.

[0102] The asymmetrically distributed heavy weight on the manual welder 2's head will cause stress on the manual welder 2's back and neck for obvious reasons. Although the total weight of the mixed reality helmet 800 cannot be reduced by the weight compensation element 807, it is distributed more symmetrically on the manual welder 2's head. In this way, the duration of the training unit can be greatly extended, thereby increasing the intensity of welding training and achieving better training results and progress.

[0103] In this case, the line of sight 10 refers to a straight line along which the manual welder 2 can see the training workpiece 4 when not wearing the mixed reality helmet 800. In a preferred design of the mixed reality helmet 800, the line of sight 10 can correspond to the optical axis of the RGB camera 803.

[0104] As in Figure 4 As shown in FIG. 8 , the RGB camera 803 has at least one RGB lens 831, whose optical axis 806 intersects the display surface 805 facing the manual welder 2 at a display cut angle γ at the display intersection point 808. In particular, when the display cut angle γ is 90 degrees, the optical axis 806 and the line of sight 10 can coincide. In principle, it is advantageous to arrange the RGB camera 803 and the mixed reality display 801 in such a way that the display cut angle γ is between 70 degrees and 110 degrees, or between 80 degrees and 100 degrees, or between 85 degrees and 95 degrees.

[0105] If the line of sight 10 and the optical axis 806 do not overlap, the mixed reality helmet 800 is preferably configured so that the line of sight 10 and the optical axis 806 have a common intersection point and diverge at an angle less than 10 degrees, or diverge at an angle less than 5 degrees, or diverge at an angle less than 1 degree.

[0106] The weight compensation element 807 can advantageously additionally ensure a predefined cable routing of the cables for connecting the simulation unit 9 to the mixed reality helmet 800. In particular, it can be ensured in this way that the cables do not touch the manual welder 2 and thus do not interfere with the virtual manual welding.

[0107] In a preferred embodiment, the shape of the weight compensation element 807 can be changed, and / or the position of the weight compensation element on the mixed reality helmet 800 can be adjusted. In particular, the weight compensation element 807 can be designed to be foldable and / or removable.

[0108] Furthermore, loudspeakers (not shown) can advantageously be provided in the mixed reality helmet 800, which simulate the acoustic noises expected during the actual manual welding process. As is known, acoustic effects are an essential element in the welding process. Thus, when performing arc welding, the acoustic noise can be related to arc characteristics, such as the power transmitted by the arc or the so-called torch angle of attack, from which the angle between the arc and the workpiece can be derived. In order to generate the acoustic noise, in the simulation unit 9, in addition to the simulation model 91 simulating the welding process, an acoustic model can also be provided. In this case, the loudspeakers can be arranged to provide spatial stereo sound, so that the virtual manual welding process can be easily demonstrated to the manual welder 2 in a particularly realistic ("immersive") manner.

[0109] Figure 5 A front view of a mixed reality helmet 800 and an adapter 30 for fixing an infrared camera 804 in a defined position relative to an RGB camera 803 are shown. With regard to the positioning of the RGB camera 803 and the infrared camera 804 on the mixed reality helmet 800, it has been found that they should preferably be mounted in a mutually arranged manner that is as rigid as possible.

[0110] In particular, with the help of Figure 4 and Figure 5 As can be seen, the RGB camera 803 and the infrared camera 804 are vertically offset relative to each other. This vertical offset must be taken into account when the virtual weld seam determined from the infrared image is superimposed on the RGB image obtained with the RGB camera. It has been shown here that although the temporal variation of this vertical offset can be taken into account, this requires determining the vertical offset and using significantly more computing power.

[0111] exist Figure 4 and Figure 5 In the embodiment, the RGB camera 803 and the infrared camera 804 each include two optical lenses. In this case, the use of two optical lenses in the RGB camera 803 enables the capture of so-called stereoscopic photographs or images of the RGB field of view 809. As is known, a stereoscopic image is an image reproduction with a spatial depth impression. In this case, the stereoscopic image can be used to be able to stereoscopically demonstrate a virtual manual welding process to the manual welder 2. In this case, in an advantageous embodiment, two stereoscopically offset mixed reality images 12 can be simultaneously displayed in real time on the mixed reality display 801 of the mixed reality helmet 800.

[0112] In addition, further sensors may be provided on the RGB camera 803 and / or the mixed reality helmet 800 in order to be able to detect the aforementioned positions of the training workpiece 4, the training handheld welding torch 6, the mixed reality target object T in the 3D welding environment 3 or generally in three-dimensional space. These sensors may be, for example, gyroscopes, acceleration sensors and / or proximity sensors. The sensor data measured by such sensors may be advantageously transmitted to the simulation unit 9, in which the positions in three-dimensional space may be determined from the transmitted sensor data.

[0113] The adapter 30 is therefore used to advantageously ensure an always (as much as possible) constant vertical offset between the RGB camera 803 and the infrared camera 804 .

[0114] Figure 5 The embodiment of the adapter 30 shown has a rigid mounting frame 33 on which two mounting elements 31 , 32 are arranged in a defined spatial position. The RGB camera 803 is fixed by the first mounting element 31 and the infrared camera 804 is fixed by the second mounting element 32 .

[0115] also, Figure 6 The fields of view 809, 810 of the RGB camera 803 and the infrared camera 804 are shown as they would appear in the welding training system 1 according to the invention. It can be seen that the infrared field of view 810 is designed to be much larger than the RGB field of view 809. In order to ensure that objects captured by the RGB camera 803, such as the training workpiece 4 or the training handheld welding torch 6, can be taken into account when determining the virtual weld 13 according to the invention based on the infrared image captured by the infrared camera 804, it is necessary to make the camera fields of view 809, 810, i.e. the RGB field of view 809 and the infrared field of view 810 at least partially overlap. In this case, "overlap" means that the RGB field of view and the infrared field of view at least partially overlap each other.

[0116] In a particularly advantageous embodiment of the present invention, it can be provided that the infrared field of view 810 completely surrounds the RGB field of view 809, that is, the RGB field of view 809 is completely included in the infrared field of view 810. However, this is not an essential prerequisite within the scope of the present invention.

[0117] With the help of Figure 6, it can be seen that the main advantage of the present invention is the use of two video / camera systems. In the illustrated case, the reference pattern 72 provided for personalizing the training workpiece 4 is outside the RGB field of view 809 of the RGB camera 803. In this case, the training workpiece 4 cannot be identified and tracked based solely on the RGB image, i.e., the training workpiece 4 cannot be tracked. However, due to the larger infrared field of view, the reference pattern 72 can still be captured, and in this case, the virtual manual welding process can also be performed without problems.

[0118] Reference Figure 7 Further discussion of the infrared reflective reference marks important to the present invention. However, Figure 7 For this purpose, different embodiments of a training handheld welding torch 6 and training supplementary material 7 are shown, which can be used to simulate a manual electrode welding method and a WIG welding method. However, the following description also applies to the reference pattern 72 of the training workpiece 4 .

[0119] According to the invention, individual reference patterns 72 , 74 are designed in such a way that capturing only a portion of the respective reference pattern 72 , 74 is sufficient to identify the object to which the respective reference pattern 72 , 74 is applied.

[0120] Depending on the type and design of the training handheld welding torch 6, it may be necessary to take into account different types of welding electrodes in the form of virtual welding electrodes in the simulation in the simulation unit 9. In this case, the type of virtual welding electrode can preferably be identified with the type of training handheld welding torch 6. However, it is also conceivable to include a welding electrode as a component of the training handheld welding torch 6 and to also provide it with an infrared-reflecting reference mark 73, so that the welding electrode can also be directly identified and then taken into account as a virtual welding electrode in the simulation.

[0121] Furthermore, various checks can be provided with regard to the specifically discussed training handheld welding torch 6, which are preferably carried out in the simulation unit 9. Thus, checks can be provided for the interfaces of the training handheld welding torch 6, such as the grounding interface and / or the welding torch interface, or a check can be made whether the correct training handheld welding torch 6 is used, which is particularly important in the case of MIG / MAG welding. Furthermore, it can be checked whether the training handheld welding torch 6 is connected with the correct electrical polarity, or whether the grounding interface is correctly connected to the training workpiece 4, or whether the grounding interface is substantially correctly connected and / or whether the welding circuit is closed. The results of the checks or these checks can be displayed graphically on the mixed reality display 801, wherein in particular attention can be drawn to misconnections or missing connections, and furthermore, prompts for appropriate corrections can be provided in the case of misconnections or missing connections.

[0122] To this end, a first distance can be set between infrared reflection reference marks 71 in a first reference pattern 72 for personalizing the training workpiece 4, and the infrared reflection reference marks 71 are spaced apart from respectively adjacent infrared reflection reference marks 71 by the first distance, and a second distance different from the first distance can be set in a second reference pattern 72 for personalizing the training workpiece 4.

[0123] Other features of the pattern can also be used to distinguish the reference patterns 72, 74 used, such as different grid structures. The infrared reflective reference marks 71, 73 can also constitute fractals that are different from each other. For example, different fractals can be generated based on different Mandelbrot sets. However, the reference pattern can also be generated using a random generator or determined empirically when designing the welding training system 1.

[0124] An important advantage of the present invention using infrared reflective reference marks is that much smaller reference marks can be used compared to the prior art. Specifically, the diameter of the infrared reflective reference marks 71, 73 can be selected in the range between 0.1mm and 5mm, or in the range between 0.1mm and 3mm, or in the range between 0.1mm and 1mm.

[0125] The diameter of the infrared reflective reference marks 71, 73 can also be used to distinguish different reference patterns 72, 74. Thus, an infrared reflective reference mark 71 having a first diameter (e.g., 0.5 mm, 1.5 mm, 2.5 mm, or 3.5 mm) can be used in the first reference pattern 72, and an infrared reflective reference mark 74 having a second diameter (e.g., 0.1 mm, 1.1 mm, 2.1 mm, or 3.1 mm) can be used in the second reference pattern 74.

[0126] In order to reliably ensure the capture of objects personalized by infrared reflective reference marks, it is necessary to ensure that the infrared reflectivity of the surface portion of the object not covered by the infrared reflective reference marks 71, 73 is less than the infrared reflectivity of the mark itself, that is, the infrared reflectivity of the surface portion of the training workpiece 4 located between the infrared reflective reference marks 71, 73, the surface portion of the training handheld welding torch 6 located between the infrared reflective reference marks 71, 73, and the surface portion of the mixed reality target object T located between the infrared reflective reference marks 71, 73 are less than the infrared reflectivity of the infrared reflective reference marks 71, 73, and 75.

[0127] In another embodiment, the reflective properties of the reference mark, the surface of the training workpiece 4 and the surface of the training handheld welding torch 6 can also be configured opposite to the embodiments described so far. Therefore, the reflectivity of the surface of the training workpiece 4 and the surface of the training handheld welding torch 6 can be configured to be greater than the reflectivity of the reference mark. In this way, a reference pattern that can be detected by an infrared camera can also be realized on the training workpiece 4 and the training handheld welding torch 6.

[0128] It should be noted that the training handheld welding torch 6 can be designed as an actual handheld welding torch, for example, an actual MIG handheld welding torch, an actual MAG handheld welding torch, an actual TIG handheld welding torch, or an actual manual arc welding torch. Therefore, within the scope of the present invention, an actual handheld welding torch can be removed from an actual welding system for actual welding, and the handheld welding torch can be configured with an infrared reflective reference mark 73 having a reference pattern 74 personalized for the handheld welding torch, so that the actual handheld welding torch is used as the training handheld welding torch 6 during the virtual manual welding process.

[0129] In a preferred design of the training handheld welding torch 6, operating elements S1, S2, such as switches, buttons, slide controls, etc., can be provided on the training handheld welding torch 6 as on a real handheld welding torch. Such operating elements S1, S2 are used in real handheld welding torches to control the welding process, for example to start, i.e., to ignite the arc, or to change the welding current intensity, or to change the wire feed speed. Preferably, the training handheld welding torch 6 can have the same operating elements S1, S2, and the control signals generated by these operating elements S1, S2 are transmitted to the simulation unit 9, for example, by means of a cable connection. In the simulation unit 9, the effects of these control signals can then be taken into account in the simulation of the manual welding process, i.e., to start or interrupt the simulated welding, to accelerate or decelerate the simulated wire feed, or to increase or decrease the simulated welding current.

[0130] In this regard, it has been recognized that even if the RGB camera 803 is so close to the training workpiece 4 that it will come into contact with the training workpiece 4, the virtual manual welding process can still be perfectly performed due to the structure of the reference marks 71, 73 according to the present invention.

[0131] Figure 8 A number of possible configurations of first reference patterns 72 are shown for individualizing different training workpieces 4. It can be seen that IR reference markings 71 can also be applied to the end faces and side faces of training workpieces 4 due to their small size. Figure 7 The description of the design of the infrared reflective reference mark 71 in Figure 8 Infrared reflective reference mark 71 on the training workpiece 4 in.

[0132] The infrared reflective reference marks 71, 73 can be attached to the surface of the training workpiece 4 in the form of an adhesive film, or can be drawn, printed, embossed, or applied to the surface of the training workpiece 4 by a coating method. In addition, such infrared reflective reference marks 71, 73 can also be provided on the surface of the training workpiece 4 by introducing energy to the surface (such as by laser engraving).

[0133] Fig. 9 Another possible design of the first reference pattern 72 for individualizing the training workpiece 4 is shown. Fig. 9 The reference pattern 72 shown emphasizes two dimples 16 arranged on the upper part of the training workpiece 4. The dimples 16 are depressions, i.e., recesses, on the surface of the training workpiece 4. Infrared reflective reference marks 71 are arranged in these depressions, but due to the depressions, the infrared reflective reference marks can only be captured by the infrared camera 804 from a specific viewing angle.

[0134] Fig.10 A detailed view of a recess 16 for arranging an infrared-reflecting reference mark 71 is shown.

[0135] Since the infrared reflective reference mark 71 arranged in the recess 16 can only be captured by the infrared camera 804 from a specific viewing angle, when the reference mark is detected, the spatial orientation of the training workpiece 4 can be inferred, for example. Fig. 9 In the case shown, two infrared-reflecting reference marks 71 arranged in the recess 16 can be seen, which makes it possible to infer an orientation of the training workpiece 4 in which the longitudinal side of the training workpiece 4 faces the infrared camera 804. However, if only one infrared-reflecting reference mark 71 arranged in the recess 16 is visible, an orientation of the training workpiece 4 can be assumed in which the transverse side faces the infrared camera 804.

[0136] also, Figures 11a to 11c It is shown that the menu M on the mixed reality display of the mixed reality helmet 800 is activated when the infrared camera 804 captures the mixed reality target object T. In order to activate the menu M, a third number of multiple infrared reflective reference marks 75 are arranged on the mixed reality target object T in a third reference pattern 76 that personalizes the mixed reality target object T. If at least a part of the third reference pattern 76 is captured by the infrared camera 804, it triggers the display of the menu M on the mixed reality display 801. To this end, the mixed reality target object T can be arranged at a desired position in the 3D welding environment 3.

[0137] In a preferred structural design of the present invention, the orientation of the menu M may depend on the spatial orientation of the mixed reality target object T, so that when the spatial orientation of the mixed reality target object T changes, the spatial orientation of the menu M also changes accordingly.

[0138] In another preferred embodiment of the invention, a further reference pattern 77 can also be arranged on the training workpiece 4 , which can also lead to the display of the menu M. As in the case of the mixed reality target object T, the orientation of the displayed menu M can also be dependent on the spatial orientation of the training workpiece 4 in this case.

[0139] A series of functions can be implemented with the aid of the displayed menu M. Thus, the menu M can be designed for modifying the at least one training welding parameter ps, so that ultimately the entire virtual manual welding process can be modified and changed.

[0140] However, the above-described display of the menu M can also be performed on another display element 14, which is detachably connected to the simulation unit 9, for example in the form of a tablet computer. In this way, in addition to the manual welder 2, other persons can also follow the menu and optionally also operate it.

[0141] In an advantageous embodiment of the invention, it can be provided that a smartphone is synchronized with the welding training system 1 and that the smartphone display is used to display the menu.

[0142] In another advantageous embodiment of the invention, the training handheld welding torch 6 can be equipped with a further operating element which allows the displayed menu M to be operated.

[0143] For this purpose, the orientation of the training handheld welding torch 6 can first be determined in the manner already described. On this basis, a menu item can be determined in the menu M that is identified by the orientation of the training handheld welding torch 6, for example, by determining a menu item that intersects a straight line determined during the identification process and starting from the training handheld welding torch 6. If a menu item is identified, it can be activated and / or deactivated by operating the further operating element.

[0144] As in Fig.11a and 11bAs shown in FIG. 1 , on the one hand, the menu items of the menu M can be operated / confirmed by pointing to a virtual straight line of the menu starting from the menu item (the virtual straight line can also be displayed here) and then pressing the operating elements S1, S2. In addition, the "dwell-based pointing" method ("point and load") established in the field of human-computer interaction can also be used to select menu items, and a menu item can also be generated by virtual contact training handheld welding torch 6.

[0145] also, Fig.12a and Figure 12b Also shown is a so-called weld root 15 as a further advantageous design of the invention. Fig.12a The lower side I of the training workpiece 4 is shown in FIG. Figure 12b The opposite upper side II is shown in FIG. Figure 12b On the upper side II shown, a virtual weld seam 13 can be seen, which forms a so-called weld root 15 on the lower side I.

[0146] Specifically, the simulation unit 9 can be advantageously configured to first determine a virtual weld root 15 generated by a virtual manual welding process on the training workpiece 4, or to determine a virtual penetration weld, or to determine a virtual longitudinal crack or a virtual transverse crack that penetrates the weld seam 13. Based on such a determination, the simulation unit 9 can advantageously be configured to superimpose the determined virtual weld root 15 or the determined virtual penetration weld or the determined virtual longitudinal crack or the determined virtual transverse crack on the current time point T akt Existing RGB images.

[0147] The demonstration of the weld root 15 or transverse cracks or longitudinal cracks is important in particular when analyzing the training workpiece 4 during a virtual manual welding process, for example by interrupting the welding and observing the training workpiece 4 from all sides, for example from the bottom, or by turning the training workpiece 4 over. In this case, it is possible to observe or check whether the weld root 15 welds the workpiece through, making the virtual welding process more realistic.

Claims

1. A welding training system (1) for performing a virtual manual welding process, the welding training system comprising - Training Artifacts (4); - Removable training handheld welding torch (6); - a first number of infrared reflective reference marks (71) arranged on the training workpiece (4) in a first reference pattern (72) personalizing the training workpiece (4); - a second number of infrared reflective reference marks (73) arranged on the training hand-held welding torch (6) in a second reference pattern (74) personalizing the training hand-held welding torch (6); - Mixed reality helmet (800), the mixed reality helmet includes a) predefined geometric helmet reference points (802); b) a mixed reality display (801) for displaying a sequence of mixed reality images (12) of a virtual manual welding process; c) an RGB camera (803) having a spatial RGB field of view (809) for capturing an RGB image of an object located in the RGB field of view (809), wherein at least a portion of a training workpiece (4) and at least a portion of a training handheld welding torch (6) during the virtual manual welding process are located in the RGB field of view (809), and the mixed reality display (801) and the RGB camera (803) are arranged along a line of sight (10) of the mixed reality helmet (800); d) an infrared camera (804), the infrared camera being arranged in a predetermined relative position relative to the RGB camera (803), the infrared camera comprising a spatial infrared field of view (810), for capturing an infrared image of an infrared reflective reference mark (71, 73) located in the infrared field of view (810), wherein the infrared field of view (810) is larger than the RGB field of view (809), and the infrared field of view (810) at least partially overlaps with the RGB field of view (809), wherein at least a portion of the first reference pattern (72) and at least a portion of the second reference pattern (74) are located in the infrared field of view (810) during the execution of the virtual manual welding process; - A simulation unit (9), the simulation unit being composed of a) determining the geometry and / or shape and / or type of the training handheld welding torch (6) and the training workpiece (4) and the temporal changes in the spatial positions of the training handheld welding torch (6) and the training workpiece (4) relative to the helmet reference point (802) based on the portions of the first and second reference patterns (72, 74) captured by the infrared image; b) taking into account at least one previously specified training welding parameter (p s ) and taking into account the temporal position change of the virtual welding electrode of the training handheld welding torch (6) determined according to the temporal change of the spatial position of the training handheld welding torch (6), determining a virtual weld seam (13) on the training workpiece (4); c) at least until the current time point T of the virtual manual welding process akt The virtual weld (13) determined so far is located at the current time point T akt The part of the RGB field of view (810) that exists at the current time point T akt The existing RGB images are superimposed to generate a virtual manual welding process at the current time point T akt Mixed reality images (12); d) transmitting the determined mixed reality image (12) to the mixed reality display (801), so as to display the mixed reality image on the mixed reality display (801) as part of a sequence of mixed reality images (12) of the virtual manual welding process.

2. The welding training system (1) according to claim 1, characterized in that: A mixed reality target object (T) is provided, a third number of infrared reflective reference marks (75) are arranged on the mixed reality target object to personalize a third reference pattern (76) for the mixed reality target object (T), and a mixed reality display (801) is configured to display, on the mixed reality display (801), instructions for changing the at least one training welding parameter (p) when at least a portion of the third reference pattern (76) is captured by an infrared camera (804). s ) menu, and at least one training welding parameter (p s ) can be modified by operating a menu displayed on the mixed reality display (801).

3. The welding training system (1) according to claim 1 or 2, characterized in that: The diameter of the infrared reflective reference mark (71, 73, 75) is in the range between 0.1 mm and 5 mm, or in the range between 0.1 mm and 3 mm, or in the range between 0.1 mm and 1 mm.

4. A welding training system (1) according to any one of the preceding claims, characterized in that The infrared reflectivity of the portion of the surface of the training workpiece (4) located between the infrared reflective reference marks (71, 73, 75), the portion of the surface of the training handheld welding torch (6) located between the infrared reflective reference marks (71, 73, 75), and the portion of the surface of the mixed reality target object (T) located between the infrared reflective reference marks (71, 73, 75) is less than the infrared reflectivity of the infrared reflective reference marks (71, 73, 75), and vice versa.

5. The welding training system (1) according to claim 1, characterized in that: The mixed reality helmet (800) has a weight compensation element (807) that counteracts a torque (T) generated by the weight of the RGB camera (803) and / or the weight of the infrared camera (804) and / or the weight of the mixed reality display (801) and acting perpendicularly to the line of sight (10) of the mixed reality helmet (800). g ).

6. The welding training system (1) according to claim 5, characterized in that: The geometry of the weight compensating element (807) can be varied and / or the position of the weight compensating element on the mixed reality helmet (800) can be adjusted.

7. A welding training system (1) according to any one of the preceding claims, characterized in that The at least one previously defined training welding parameter (p s ) corresponds to the workpiece geometry or welding current intensity or welding voltage or welding speed or waiting time or preheating temperature or wire feed speed or arc length or welding type.

8. A welding training system (1) according to any one of the preceding claims, characterized in that The simulation unit (9) has a display element (14), preferably a display element that can be separated from the simulation unit (9), for displaying a sequence of mixed reality images (12) of a virtual manual welding process.

9. A welding training system (1) according to any one of the preceding claims, characterized in that A workpiece holder (5) is provided, on which the training workpiece (4) can be mounted.

10. The welding training system (1) according to claim 9, characterized in that: The training workpiece (4) can be mounted on the workpiece holder (5) in a magnetic manner.

11. A welding training system (1) according to any one of the preceding claims, characterized in that The simulation unit (9) is arranged on the mixed reality helmet (800).

12. A welding training system (1) according to any one of the preceding claims, characterized in that The simulation unit (9) is designed to determine, in a subsequent mechanical simulation, based on the determined virtual weld seam (13), a virtual metallographic structure and / or a stress-induced virtual deformation and / or a virtual grain structure of the metallographic structure, which can be generated as a result of the virtual weld seam (13) in the training workpiece (4).

13. The welding training system (1) according to claim 9, characterized in that: The simulation unit (9) is designed to determine a welding quality parameter describing the quality and / or quality of the virtual weld seam (13) by means of one of the variables determined in a subsequent mechanical simulation.

14. A welding training system (1) according to any one of the preceding claims, characterized in that The simulation unit (9) is configured to superimpose a virtual ideal weld (131) that is predetermined and needs to be generated through a virtual manual welding process on a current RGB image, and the mixed reality display (801) is configured to display the RGB image superimposed with the virtual ideal weld (131).

15. The welding training system (1) according to claim 11, characterized in that The simulation unit (9) is designed to determine a weld seam quality parameter which describes a deviation between a determined virtual weld seam (13) and a previously defined ideal weld seam (131).

16. A welding training system (1) according to any one of the preceding claims, characterised in that The simulation unit (9) is configured to determine a virtual weld root (15) generated on a training workpiece (4) by a virtual manual welding process, or to determine a virtual penetration weld, or to determine a virtual longitudinal crack or a virtual transverse crack, and the simulation unit (9) is configured to superimpose the determined virtual weld root (15) or the determined virtual penetration weld or the determined virtual longitudinal crack or the determined virtual transverse crack on the training workpiece (4) at a current time point T. akt Existing RGB images.

17. A welding training system (1) according to any one of the preceding claims, characterised in that The training workpiece (4) has at least one recess or at least one depression, and a reference mark (71) of a first reference pattern (72) is arranged in the at least one recess or in the at least one depression.

18. A welding training system (1) according to any one of the preceding claims, characterised in that The training handheld welding torch (6) corresponds to an actual WIG handheld welding torch or an actual MIG handheld welding torch or an actual MAG handheld welding torch or an actual manual arc welding torch.

19. A welding training system (1) according to any one of the preceding claims, characterised in that The mixed reality helmet (800) is provided with another sensor, preferably a gyroscope and / or an acceleration sensor and / or a proximity sensor, which detects sensor data for determining the spatial orientation of a training workpiece (4) and / or a training handheld welding torch (6) and / or a mixed reality target object (T) in three-dimensional space. The sensor data detected by the other sensor is transmitted to a simulation unit (9), and the simulation unit (9) determines the spatial orientation of the training workpiece (4) and / or the training handheld welding torch (6) and / or the mixed reality target object (T) in three-dimensional space based on the transmitted sensor data.

20. Method for performing a virtual manual welding process using a welding training system (1) according to any one of the preceding claims, wherein - a manual welder (2) wearing a mixed reality helmet (800) of the welding training system (1); - a manual welder (2) manually moves a training handheld welding torch (6) of the welding training system (1) to generate a virtual weld (13) on a training workpiece (4) of the welding training system (1); - using an RGB camera (803) of the welding training system (1) to capture an RGB image of a portion of a training handheld welding torch (6) and a training workpiece (4) that is located in an RGB field of view (809) of the RGB camera (803); - using an infrared camera (804) of the welding training system (1) to capture an infrared image of the infrared reflective reference markers (71, 73) located in the infrared field of view (810) of the infrared camera (804); - In the simulation unit (9) of the welding training system (1): a) determining the geometry and / or shape and / or type of the training handheld welding torch (6) and the training workpiece (4) based on the captured infrared image; b) determining the temporal change of the spatial position of the training handheld welding torch (6) and the training workpiece (4) relative to the helmet reference point (802) of the mixed reality helmet (800) of the welding training system (1) based on the portions of the first and second reference patterns (72, 74) captured by the infrared image; c) taking into account at least one previously specified training welding parameter (p s ) and the position change of the virtual welding electrode over time, determining a virtual weld seam (13) on the training workpiece (4) according to the change of the spatial position of the training handheld welding torch (6) over time; d) at least until the current time point T of the virtual manual welding process akt The virtual weld (13) determined so far is located at the current time point T akt The part existing in the RGB field of view (809) is superimposed on the current time point T akt The RGB image exists in order to generate a virtual manual welding process at the current time point T akt Mixed reality images (12); - displaying the determined mixed reality image (12) as part of a sequence of mixed reality images (12) of a virtual manual welding process on a mixed reality display (801) of a mixed reality helmet (800) of the welding training system (1).

Images