Method for projecting a laser image on a workpiece, laser projection system and lighting device
Patent Information
- Application Number
- PCT/EP2024/073091
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-08-16
- Filing Date
- 2024-08-16
- Publication Date
- 2025-02-20
Smart Images

Figure EP2024073091_20022025_PF_FP_ABST
Abstract
Description
[0001] Method for projecting a laser image onto a workpiece, laser projection system and lighting device
[0002] DESCRIPTION
[0003] The invention relates to a method for projecting a laser image onto a workpiece as well as to a laser projection system and a lighting device which are designed and configured for carrying out the method according to the invention or for use in the method according to the invention.
[0004] Laser projectors are used in a wide variety of technical fields, particularly in industrial applications, to project outlines and markings onto objects or workpieces. These lines can typically be clearly projected in monochrome, for example, red, green, or blue, or in multicolor, particularly mixed-color, projection onto a flat or curved surface, or onto a surface of any other contour. CAD data often serves as the basis for the projection, which is then processed into projection data and transmitted to one or more projectors. Industrial laser projection systems create virtual but visible templates on the objects to be processed, which do not touch or contaminate the surfaces.Technically, projection is typically achieved using one or more galvanometer drives, each of which allows a mirror to be adjusted in its orientation relative to a laser beam. The laser beam is deflected by the mirror, with the angle of rotation of the mirror determining the direction of deflection. If two or more galvanometers are combined, any optically accessible point in the room within the beam range of the laser projector can be illuminated.
[0005] To precisely align the laser projection onto an object, the position of the object in relation to the laser projector must be known. If the object and laser projector are placed at fixed and unchangeable locations, the spatial relationship can be specified to the laser projection system. Due to the fixed locations of the object and projector, the projection can always be precisely projected onto the object. In industrial applications, however, it is often necessary to move workpieces during processing, for example in order to be able to process different areas of the workpiece. Furthermore, it can also be specified that the workpiece is not only placed at fixed locations or in fixed orientations, but is moved to different locations and in different orientations in relation to the laser projection system depending on the application.
[0006] To enable precise laser projection even on workpieces that can be moved flexibly in space, both temporally and spatially, it is necessary to determine the spatial relationship between the workpiece and the laser projection unit. To determine the spatial relationship between the laser projection unit and the workpiece, it is known to place so-called target markers in a fixed spatial relationship to the workpiece, which are optically detected and recognized by a detection device of the laser projection system. These target markers typically carry a retroreflector and / or coded markings that can be detected and / or identified by the detection device or a control device of the laser projection system.The target markers are illuminated by a light source, and the light reflected by the target markers is detected by the detection device. The position of various target markers relative to the detection device can thus be determined. With knowledge of the spatial relationship between the detection device and the laser projector or laser projection unit, precise control of the laser relative to the workpiece can ultimately be achieved.
[0007] The object of the invention is to provide a method for projecting a laser image of a laser projection system and a lighting device according to the type described above, with the aim of improving the adaptability of the system to different applications.
[0008] The object is achieved by a method according to claim 1, as well as by a laser projection system according to claim 9 and a lighting device according to claim 10. Advantageous embodiments are specified in the subclaims. According to the invention, a method for projecting a laser image onto a workpiece, comprising the steps of: providing a laser projection system, having a laser projection unit by means of which a laser beam can be generated, a deflection device by means of which an emission angle of the laser beam can be adjusted and at least one photosensitive detection device by means of which light radiation can be detected, wherein the laser projection system contains a plurality of lighting devices by means of which light detectable by the detection device can be generated, and wherein the lighting devices each have a control unit by means of which the lighting devices can be activated and deactivated;Arranging the lighting devices on the workpiece and / or in a fixed spatial relationship to the workpiece such that a light generated by the lighting devices can be detected by the detection device; activating one or more lighting devices such that the activated lighting devices emit light that can be detected by the detection device; detecting the light emitted by at least some of the activated lighting devices by means of the detection device; determining a spatial relationship between the workpiece and the laser projection unit, wherein the spatial position of the lighting devices detected by the detection device is evaluated to determine the spatial relationship between the workpiece and the laser projection unit; generating a laser projection based on the determined spatial relationship between the workpiece and the laser projection unit.
[0009] Detection devices of the type according to the invention are increasingly being designed using imaging components, such as machine vision cameras. The tasks of these imaging components include, for example, determining the location of the laser projection system relative to an object or workpiece onto which a laser image is to be projected and monitoring its temporal and spatial positioning integrity in the coordinate system of the projection surface. Such imaging systems can also be used for other tasks, such as quality assurance and documentation in the production process.
[0010] As previously mentioned, the positioning of a laser projection system with a laser projection device and imaging components is typically achieved using retroreflective or coded target markers, which are illuminated either by the laser of the laser projection system or by another illumination source such as spotlights or ambient light. In all cases, light from a source falls onto the target markers, with a portion of the incident light being reflected back into the laser projection system by the target markers and detected there by photosensitive sensors such as photodiode arrays, CCD, or CMOS sensors for further processing.
[0011] A significant disadvantage, particularly in the detection of optical reflections from targets by imaging components in laser projection systems, is the occurrence of additional light reflections that are not caused by the targets. Sources of these interfering reflections are very diverse, e.g., reflective surfaces of support systems of the projection surface or reflective workpieces or optical reflections from targets on spatially adjacent projection surfaces. These interfering reflections cannot always be completely suppressed even with suitable image processing measures and lead to an impairment of the positioning functionality. Furthermore, the retroreflectors typically used in laser projection systems have the disadvantage that the generated reflections cannot be directly assigned to a projection surface or a position on the projection surface.This is particularly problematic when using projection surfaces where the retroreflectors are arranged symmetrically or when using several projection surfaces with the same arrangement of retroreflectors at the same time.
[0012] The method according to the invention makes it possible to specifically activate or deactivate certain lighting devices acting as target markers. Unlike conventional methods, in which all target markers are detected using a single detection device, the method according to the invention allows selected target marker lighting devices to be activated or deactivated. This enables flexible adaptation of the location of the laser projection system or the target object in different application situations. For example, by specifically activating or deactivating certain target markers, stray light sources can be detected and taken into account in order to obtain a high-quality image captured by the detection device and thus improve further processing for determining the spatial relationship between the workpiece and the laser projection unit.
[0013] The spatial relationship between the workpiece and the laser projection unit can be determined, for example, using triangulation. This requires capturing the target markers and illumination devices with an image capture device from two different positions. For example, a capture device with at least two cameras, such as a stereo camera, can be used. Determining the spatial relationship using a two-camera system is known as stereo photogrammetry.
[0014] Alternatively or additionally, the spatial relationship between the workpiece and the laser projection unit can be determined using only one camera. When using a single-camera system or only one camera in a stereo camera system, a process called association is performed. This involves assigning calibration data for the marker positions in the world coordinate system to the image data. The calibration data can, for example, be generated from a CAD model of the workpiece using the predefined positions of target markers relative to the workpiece. With the help of at least three target markers arranged on an object, the pose of the object with its world coordinate system can be calculated in the camera coordinate system. The assignment of 2D image points and 3D marker positions in the world coordinate system can be determined iteratively using a perspective-n-point (PnP) method.A key idea of the PnP method is to hypothetically assign the 3D marker positions to the 2D pixels in the camera's image plane for at least three target markers, usually four in practice, and to determine the object's pose for this hypothetical assignment. For this pose determined in this way, it can then be determined whether the other 3D marker positions, assuming the pose in the projection into the camera's image plane, lead to 2D pixels that match the captured pixels in the camera image. A comparison is therefore made in the camera image to determine whether the captured pixels of the target markers match the marker positions projected onto the camera image assuming the hypothetical assignment. The deviation between the expected 2D image positions and the image positions present in the camera image is then a measure of the quality of the hypothetical assignment.This process can be repeated for a series of hypothetical assignments, and finally, the assignment showing the smallest deviation can be selected. Once the pose of the workpiece has been determined, the laser projector located in the camera coordinate system or the laser projector's deflection device can be controlled such that the laser beam reaches a desired point in relation to the workpiece. For fine adjustment, the laser beam can be directed at one or more specific target markers and a reflection of the laser light can be detected. With targeted control and detection of the expected reflection, small deviations in the determined spatial position of the workpiece can be detected and taken into account for correcting the spatial relationship.
[0015] To accurately determine the spatial relationship between the workpiece and the laser projection unit, the laser projection system should include at least three lighting devices.
[0016] The method according to the invention allows for the avoidance of disruptive reflections that can be caused by external lighting, since potentially disruptive reflection sources, such as reflective surfaces of surrounding objects or the workpiece itself, are not illuminated by additional lighting. The consideration or identification of disruptive reflections can be further supported if, in addition to recording an ambient image in which the light emitted by activated lighting devices is recorded, the detection device also records an ambient image without activated targets. By comparing images and the resulting differential image analysis, further disruptive reflections can be detected and taken into account.If there are additional projection surfaces on the same workpiece or on other workpieces in the vicinity of the projection surface to be viewed, interfering reflections from such adjacent projection surfaces can be suppressed by assigning different control addresses to the target marker lighting devices of the adjacent projection surfaces. Thus, only the target markers of the selected projection surface are actively illuminated via the address control. In a preferred embodiment, self-luminous electrical light sources are used as the lighting devices, in particular light sources that comprise LED and / or OLED lamps. Depending on the design of the laser projection system and / or the application, lamps that emit adapted light at a specific light wavelength or in a specific light wave range can be used.The wavelength of the emitted light can, for example, be matched to the characteristics of the receiving sensors used in the laser projection system, e.g. IR light, visible light, UV light.
[0017] To control the lighting devices, one embodiment envisages using lighting devices that can each be activated and / or deactivated by means of a control signal that can be transmitted wirelessly and / or wired to the control unit of the respective lighting device. Alternatively or additionally, it can be provided that the lighting devices can be activated and / or deactivated without external control signals. For example, it can be provided that the lighting devices can be activated and / or deactivated manually, for example by means of a switch on the housing of the lighting devices. It is also conceivable for the lighting devices to exchange data with one another, for example to synchronize lighting sequences or to exchange control signals.
[0018] Further preferably, it can be provided that lighting devices are used which each emit light after activation until deactivation, wherein activated lighting devices can be brought into a first activity state in which the emitted light is emitted with constant intensity and / or can be brought into a second activity state in which the emitted light is emitted with intensity modulation, in particular in a pulsed manner. The target marks can therefore be switched ON in an electrically controlled manner and switched OFF either after a fixed programmed duration or in a controlled manner. In the ON phase, the light can be emitted either with constant intensity or in an intensity-modulated mode (pulse mode).
[0019] In one embodiment, it is conceivable to use lighting devices that can be deactivated after activation by means of a control signal transmitted wirelessly and / or wired to the control unit of the respective lighting device, and / or that remain activated for a predetermined period of time after activation, with the control unit automatically deactivating the respective lighting device after the expiration of the period. Thus, for example, repeating sequences can be provided for regular target marker detection, which only needs to be triggered by an activation command.
[0020] Preferably, it is contemplated that lighting devices are used that can be activated and / or deactivated by means of a control signal wirelessly transmitted from a transmitting device of the laser projection system to the control unit of the respective lighting device. The wirelessly transmitted control signal can, in particular, be an optically transmitted signal, such as a signal transmitted by infrared radiation and / or a signal transmitted by means of a radio protocol, such as a radio protocol according to an industry standard, such as Bluetooth, Zigbee, Wi-Fi, Z-Wave, or the like. The transmitting device can, for example, be part of a control device of the laser projector or the detection device.
[0021] In order to activate or deactivate a specific number or one or more different groups of lighting devices, one embodiment of the invention can provide for the use of lighting devices that can be controlled individually and / or in groups by means of a control signal. It can therefore be provided that each individual target marker can be addressed electronically with a control command. Furthermore, several target markers can be combined to form a target marker group with the same address. In particular when using several laser projectors in a laser projection system or when using several laser projection systems that project onto a common workpiece or onto different workpieces, the respectively assigned target marker lighting devices can be activated or deactivated in an individually configurable manner for each laser projector or each laser projection system.This allows the spatial relationship between the workpiece and a laser projector to be determined by detecting the relevant target markers without interference from unrelated target markers. Further flexibility in the use of individually controllable target marker illumination devices can be achieved by operating individual illumination devices within a group in pulsed mode. The target markers operating in pulsed mode can be identified by the laser projection system based on their programmed individual light modulation sequence. This facilitates the algorithmic evaluation of the local target marker arrangement.This configuration is particularly advantageous for laser projection systems with multiple detection devices, for example in the case of larger or more complex projection environments, because each detection device can ignore other target markers that have also been detected for further evaluation or take them into account differently when identifying the target markers assigned to it.
[0022] The individual or group-based controllability of target marker lighting devices can, in particular, also improve the spatial positioning of projection surfaces with a symmetrical arrangement of the target markers. With a symmetrical arrangement of the target markers, the rotational position of the workpiece in space may not be clearly recognized when the target markers are detected. To avoid this problem, a group of target markers can contain one or more target markers that respond not only to a specific group address, but also to an individual target marker address. This enables the coding of individual target marker lighting devices.
[0023] Finally, it is conceivable that all target marker lighting devices could be addressed using a common address (broadcast address) to process specific control commands. This broadcast address is the same for all target marker lighting devices; all target marker lighting devices are addressed simultaneously.
[0024] It is further contemplated that, to determine the spatial relationship between the workpiece and the laser projection unit, the distribution of the detected lighting devices in an imaging plane is evaluated, in particular in an imaging plane of the detection device. For the precise determination of the position of the workpiece in space, one embodiment may provide for the determination of the spatial relationship between the workpiece and the laser projection unit to be carried out using computer-aided pattern recognition, wherein the pattern recognition is carried out in particular by means of a computer-implemented method that operates on the basis of algorithms based on artificial intelligence.
[0025] According to the invention, there is also provided a laser projection system which is set up and designed to carry out the method described above.
[0026] In one embodiment, it can be provided that the laser projection system comprises a transmitting device by means of which control signals can be transmitted wirelessly, in particular by means of optical radiation, to the control units of the lighting devices.
[0027] It can further preferably be provided that the laser projection system comprises a receiving device by means of which data signals can be received, wherein the receiving device is in particular set up and designed to receive the data signals wirelessly, in particular by means of optical radiation.
[0028] Furthermore, it can be provided that at least one of the lighting devices has a retroreflector. A retroreflector or retroreflector is a device which reflects incident electromagnetic waves - in the light or invisible part of the spectrum - largely in the direction from which they came, largely independent of the direction of incidence with regard to the orientation of the device. A retroreflector is particularly useful when, after an initial determination of the spatial relationship between the target marker lighting devices or the workpiece and the laser projection unit, a fine-tuning of the spatial relationship is to be carried out. For a fine-tuning of the spatial relationship, the laser beam of the laser projection unit can be directed onto the target marker and, with detection of a reflected laser beam, for example by means of the detection device ora laser sensor, thus verifying or correcting the assumed spatial orientation of the workpiece and laser projection unit. For optical identification of a lighting device, at least one of the lighting devices can have a surface detectable by the detection device with a coded marking.
[0029] Finally, according to the invention, a lighting device with the features relating to the lighting device is one of the previously described embodiments of the method according to the invention or of the laser projection system according to the invention.
[0030] The invention is described in more detail below with reference to figures.
[0031] Fig. 1 is a schematic representation of a laser projection system according to the invention, and
[0032] Fig. 2 is a schematic representation of a transmission / reception sequence of a laser projection system according to the invention.
[0033] Figure 1 shows a laser projection system 10 with a laser projection unit 11. The laser projection unit 11 comprises a laser generation unit 12 that generates a laser beam 14. The laser beam 14 is directed by a deflection device 16 such that the laser beam 14 sweeps a predeterminable trajectory in space and thus generates a laser image 18 on an object, such as a workpiece 22. A control device 20 can control the deflection device 16 such that a predefined navigation angle is controlled. The deflection device 16 can be configured to direct the laser beam 14 into any navigation angle located within a beam range 24 of the laser projection unit 11.
[0034] A plurality of lighting devices 15 are positioned in a fixed spatial relationship to the workpiece 22. The lighting devices 15 can be arranged directly on the workpiece 22, or at another location or object that is immovable relative to the workpiece. For example, the lighting devices 15 can be arranged on the body of a frame, a transport carriage, a mold, or the like, which is fixed in location and position relative to the workpiece 22. A detection device 13 is arranged in the region of the workpiece 22 such that light emitted by the lighting devices 15 can be detected by the detection device 13. A control signal 26 (cf. Fig. 2) is transmitted to the lighting devices 15 by means of a transmitting device 28 arranged on the detection device 13 in the example shown.The control signal 26 is received at the lighting devices 15, and depending on the addressing contained in the control signal 26, a single lighting device 15 or a group of lighting devices 15 can be activated. A control signal 26 can contain other or additional control commands as an alternative or in addition to an activation command. For example, a control signal 26 can contain a deactivation command and / or programming commands with which a configuration of addressed lighting devices can be changed.
[0035] Figure 2 shows, by way of example and schematically, a temporal sequence for activating lighting devices B using control signals A and detecting the light C emitted by the lighting devices. To activate lighting devices 15, the laser projection system 10 emits a control signal 26, for example, using the control device 20. The control signal 26 can, for example, be an infrared signal emitted by an IR transmitter in the direction of the lighting device 15 to be activated. The control signal 26 can—as indicated here—be a pulse-width modulated signal in order to transmit a specific address code and a command for activating the addressed lighting devices 15. The control signal 26 is received by the lighting devices 15 and evaluated by the control units of the lighting devices 15.Lighting devices 15 addressed by the control signal are activated and emit light that can be detected by the detection device 13 of the laser projection system 10. The emitted light can preferably be light in the infrared range. For example, the light of the lighting device 15 can be generated using infrared LEDs (IR LEDs). In the present example, the lighting device 15 is configured such that the light is emitted for a predetermined time, for example, 30 ms. After 30 ms, the lighting device 15 is automatically deactivated.
[0036] Simultaneously with the activation of the addressed lighting devices 15, image capture is initiated by the detection device 13 of the laser projection system. The detection device 13 can be configured, for example, as a camera, with the camera's exposure time being a preset period of 30 ms, for example. Thus, the light emitted by the activated lighting devices 15 coincides with the camera's exposure time. In this case, deactivation of the lighting devices 15 also terminates the camera's exposure and image capture.
[0037] To detect and account for unwanted interfering reflections, a second camera image can be captured, as shown in Fig. 2. The second image can be taken, for example, approximately 20 ms after the previous image. The second image is taken without the lighting devices activated. Interfering reflections, which may arise, for example, from the activation of the addressed lighting devices, can be detected using image comparison and difference image analysis. This cycle can be repeated multiple times or continuously, for example, every 100 ms.
[0038] The two camera images created in Fig. 2 in the cycle shown as an example – once with activated lighting devices and once without activated lighting devices – can be further processed, for example, in a control device of the laser projection system, whereby the position of the lighting devices is extracted from the images in order to ultimately calculate the spatial relationship between the lighting devices and the laser projection unit, or the orientation of the workpiece and laser projection unit in space. Since the method according to the invention does not require the use of an additional light source to illuminate the target markers, interfering reflections are largely minimized from the outset. Therefore, a second image acquisition and a difference image analysis are not absolutely necessary. Nevertheless, the method can include these steps in order to detect and take into account any further interfering reflections. LIST OF REFERENCE SYMBOLS
[0039] 10 Laser projection system
[0040] 11 Laser projection unit
[0041] 12 Laser generation unit
[0042] 13 Recording device
[0043] 14 Laser beam
[0044] 15 Lighting device
[0045] 16 Deflection device
[0046] 18 Laser image
[0047] 20 Control device
[0048] 22 Workpiece
[0049] 24 From beam area
[0050] 26 Control signal
[0051] 28 Transmitting device
[0052] 30 activation time
[0053] 32 exposure time
[0054] A temporal course of control signals
[0055] B Time course of activation of lighting devices
[0056] C temporal course of a detection by means of the detection device
Claims
CLAIMS 1. A method for projecting a laser image (18) onto a workpiece (22), comprising the steps: Providing a laser projection system (10), comprising a laser projection unit (11) by means of which a laser beam (14) can be generated, a deflection device (16) by means of which an emission angle of the laser beam (14) can be adjusted and at least one photosensitive detection device (13) can be detected by means of the light radiation, wherein the laser projection system (10) contains a plurality of lighting devices (15) by means of which light detectable by the detection device (13) can be generated, and wherein the lighting devices (15) each have a control unit by means of which the lighting devices (15) can be activated and deactivated; Arranging the lighting devices (15) on the workpiece (22) and / or in a fixed spatial relationship to the workpiece (22) such that a light generated by means of the lighting devices (15) can be detected by means of the detection device (13); Activating one or more lighting devices (15) so that the activated lighting devices (15) emit a light that can be detected by the detection device (13); detecting the light emitted by at least some of the activated lighting devices (15) by means of the detection device (13); Determining a spatial relationship between the workpiece (22) and the laser projection unit (11), wherein the spatial position of the lighting devices (15) detected by means of the detection device (13) is evaluated for determining the spatial relationship between the workpiece (22) and the laser projection unit (11); Generating a laser projection based on the determined spatial relationship between the workpiece (22) and the laser projection unit (11).
2. Method according to claim 1, characterized in that self-luminous electrical light sources are used as lighting devices (15), in particular light sources comprising LED and / or OLED lighting means.
3. Method according to claim 1 or 2, characterized in that lighting devices (15) are used which can each be activated and / or deactivated by means of a control signal (26) which can be transmitted wirelessly and / or by wire to the control unit of the respective lighting device (15).
4. Method according to one of the preceding claims 1 to 3, characterized in that lighting devices (15) are used which each emit light after activation until deactivation, wherein activated lighting devices (15) can be brought into a first activity state in which the emitted light is emitted with constant intensity and / or can be brought into a second activity state in which the emitted light is emitted in an intensity-modulated, in particular pulsating, manner.
5. Method according to one of the preceding claims 1 to 4, characterized in that lighting devices (15) are used which can be deactivated after activation by means of a control signal (26) which can be transmitted wirelessly and / or by wire to the control unit of the respective lighting device (15) and / or which remain activated for a predetermined period of time after activation, wherein after the expiry of the period of time the control unit automatically deactivates the respective lighting device (15).
6. Method according to one of the preceding claims 1 to 5, characterized in that lighting devices (15) are used which can be activated and / or deactivated by means of a control signal (26) transmitted wirelessly from a transmitting device (28) of the laser projection system (10) to the control unit of the respective lighting device (15), wherein the wirelessly transmitted control signal (26) can in particular be an optically transmitted signal, such as a signal transmitted by means of infrared radiation and / or a signal transmitted by means of a radio protocol, such as a radio protocol according to an industry standard, such as Bluetooth, Zigbee, Wi-Fi, Z-Wave or the like.
7. Method according to one of the preceding claims 1 to 6, characterized in that lighting devices (15) are used which can be controlled individually and / or in groups by means of a control signal (26).
8. Method according to one of the preceding claims 1 to 7, characterized in that for determining the spatial relationship between the workpiece (22) and the laser projection unit (11), the distribution of the detected lighting devices (15) in an imaging plane is evaluated, in particular in an imaging plane of the detection device (13).
9. Method according to one of the preceding claims 1 to 8, characterized in that the determination of the spatial relationship between the workpiece (22) and the laser projection unit (11) is carried out using a computer-aided evaluated pattern recognition, wherein the pattern recognition is carried out in particular by means of a computer-implemented method which operates on the basis of algorithms based on artificial intelligence.
10. Laser projection system which is arranged and designed to carry out the method according to one of claims 1 to 9.
11. Laser projection system according to claim 10, characterized in that the laser projection system (10) comprises a transmitting device (28) by means of which control signals (26) can be transmitted wirelessly, in particular by means of optical radiation, to the control units of the lighting devices (15).
12. Laser projection system according to claim 11, characterized in that the laser projection system (10) comprises a receiving device by means of which data signals can be received, wherein the receiving device is in particular set up and designed to receive the data signals wirelessly, in particular by means of optical radiation.
13. Laser projection system according to one of claims 10 to 12, characterized in that at least one of the lighting devices (15) has a retroreflector.
14. Laser projection system according to one of claims 10 to 13, characterized in that at least one of the lighting devices (15) has a surface detectable by the detection device (13) with a coded marking.
15. Lighting device with the features relating to the lighting device according to one of claims 1 to 14.
Citation Information
Patent Citations
Systems and methods for dynamic projection mapping for animated figures
US20220323874A1
Image auto alignment process
WO2019162662A2