Method for projecting a laser image pattern

A laser projector and camera system dynamically tracks and corrects for motion to ensure precise laser image projection on dynamic surfaces, addressing pattern disruptions and achieving sub-millimeter accuracy for applications like aircraft assembly.

DE102014001045B4Active Publication Date: 2024-11-28VIRTEK VISION INT INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
DE102014001045
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Priority Date
2013-01-28
Filing Date
2014-01-27
Publication Date
2024-11-28
Estimated Expiration
2034-01-27

AI Technical Summary

Technical Problem

Existing laser projection methods struggle to accurately project images on dynamic surfaces due to vibration and environmental changes, leading to noticeable pattern disruptions and inability to correct for rapid movements such as those caused by wind or crane loading.

Method used

A method using a laser projector and high-resolution camera system that continuously tracks and compensates for relative motion by determining the position of the projector and camera relative to fixed targets, adjusting the laser beam path in real-time to maintain pattern accuracy.

Benefits of technology

Achieves precise laser image projection with errors less than 0.005 cm (0.002 in.) under dynamic conditions, enabling accurate assembly tasks like aircraft carbon fiber layup by predicting and correcting for movement and acceleration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

Method for projecting a laser image pattern onto a work surface (24) and continuously compensating for any relative movement between the laser projector (20, 26) and the work surface (24), comprising a laser projector and camera system (56) including at least one laser projector with a source of a laser beam and rotating mirrors (60, 62) which deflect the laser beam onto the work surface (24), at least one camera (28) which is mounted relative to the laser projector (20, 26), a plurality of retroreflective targets at fixed locations relative to the work surface (24), and a computer (40) which communicates with the laser projector (20, 26) and the camera (28), comprising the method: Directing a flashing light source onto the retroreflective targets; Determining the position of the camera (28) in three dimensions relative to the work surface (24) by locating the illuminated retroreflective targets (22, 30) in the camera image; Determining a position of the laser projector (20, 26) relative to the working surface (24) in three dimensions using the fixed position of the laser projector (20, 26) relative to the camera (28); and Correcting the pattern projected by the laser projector by predicting a next location of the laser projector (20, 26) relative to the working surface (24) based on at least two continuously tracked previous locations of the laser projector (20, 26) relative to the working surface (24) to compensate for a calculated position of the laser projector (20, 26) relative to the working surface (24).
Need to check novelty before this filing date? Find Prior Art

Description

AREA

[0001] This application relates to a method for accurately projecting a laser image onto a target or work surface and continuously compensating for relative dynamic movement between the laser projector and the surface of the target. BACKGROUND

[0002] Currently, laser projectors are used to arrange components on a work surface, such as the arrangement of components of a roof truss, as disclosed, for example, in my US patent US 5,646,859 A. At least four targets, typically retroreflective targets, are mounted relative to the work surface, and a laser projector periodically scans the targets to calibrate the relative position between the work surface and the laser projector. This is currently accomplished by measuring the target coordinates relative to the coordinate system of the tool and then locating the position of the projector relative to the tool through a resection process (calculating the position of the projector where the known laser radiation passes to the target through the known 3-D target coordinates).Once the relative position of a laser projector is determined, the required deflection angle of the laser projector scanning mirrors can be calculated to trace the laser spot through the desired template pattern onto the known three-dimensional profile of the tool or work surface. The template pattern is projected as a series of vector movements with the laser sequentially traced through the features of the pattern. If the trace through the pattern features occurs with sufficient frequency, the pattern can be displayed without visible flicker artifacts.

[0003] The precise location of the retroreflective calibration targets is determined by scanning the laser spot in a grid pattern across the target's surface. Locating the position of four or more targets allows calibration from the relative projector position. In the traditional process, a host computer calculates the projector location and required scanning vector mirror movements to scan through the defined 3D template pattern, then transmits the resulting vector display list of two-axis scanning mirror movements to an embedded computer in the projector, which continuously traces the desired scanning sequence.Periodically, the scanning sequence is stopped and a target is located to check for variations in the projected pattern location due to a change in the position of the projector relative to the target or tool surface, or to compensate for other factors, such as drift due to ambient temperature variations. If a variation is detected, the targets are relocated and a new scanning sequence is calculated and transmitted to the projector.

[0004] However, scanning of target positions with the projector must be performed sequentially, and at best, several targets can be scanned per second. As a result, tests for projection drift can only be performed intermittently, and correction of the projector pattern results in a noticeable interruption in the projected pattern. While this traditional method can effectively compensate for slowly changing environmental conditions or temporary events, such as deliberate repositioning of the target or tool surface, it cannot correct for more dynamic changes, such as vibration in a laser projector mounted on a ceiling due to the movement of loading cranes or building movement in response to air pressure variations caused by wind gusts or the opening of a loading dock door.

[0005] The prior art includes proposals for compensating for motion of a projector relative to a target surface using a camera to obtain the pose of the system relative to the surface of a target. Using a camera to perform tracking of the projector has two important advantages, namely (i) modern cameras can operate at high speed, locating targets hundreds of times per second, and (ii) target detection is independent of projection and does not disturb the dynamic motion of the projected image. Generally, however, the projector disclosed in the prior art is a video projector (i.e., displaying raster images at a fixed update rate, such as 30 or 60 frames per second). In contrast, the method of this invention corrects for a dynamically displayed vector image produced by a scanning laser. While the entire laser pattern can only be updated more slowly (e.g.,B. 40 times per second), the individual points being recorded are updated more than 10,000 times per second as discussed below. An exception is disclosed in published PCT application WO 2012 / 136345 A2 by Keitler, et al., which states that the disclosed dynamic tracking method can be used for either video or laser projectors; However, the disclosed method relies on uniquely configured targets, which have not been proven to be industrially applicable, particularly for laser projectors. Further proposals are described in US 2005 / 0 082 262 A1, US 2011 / 0 050 872 A1, US 2004 / 0 201 823 A1 and DE 10 2011 015 987 A1. SUMMARY OF THE INVENTION

[0006] The subject matter of the present invention is defined by independent claim 1. Preferred embodiments emerge from the subclaims. A method for accurately projecting a laser image onto a target or work surface and continuously compensating for relative, dynamic movement between the laser projector and the work surface includes a laser projector and camera system having at least one laser projector with a source of a laser beam and rotating mirrors that redirect the laser beam onto the work surface, at least one high-resolution camera mounted relative to the laser projector, targets at fixed locations relative to the work surface, and a computer that communicates with the laser projector and the camera. The method of this invention then includes determining a position of the camera in three dimensions relative to the work surface by locating the targets in the camera image.The method then includes determining a position of the laser projector relative to the work surface using the fixed position of the laser projector relative to the camera. In a preferred embodiment, the method of this invention then includes using the computer to continuously adjust the rotation of the laser projector mirrors in response to dynamic movement of the laser projector relative to the work surface, as determined by the camera, continuously correcting the location of the laser beam on the work surface.In a preferred embodiment, the method of this invention includes using the computer to continuously track at least two previous locations of the laser projector relative to the work surface in three dimensions, and predicting the next location of the laser projector, thereby compensating for motion of the laser projector relative to the work surface; that is, the relative velocity of the laser projector relative to the work surface. The acceleration (i.e., changes in velocity) of the laser projector relative to the work surface in three dimensions can also be determined by continuously tracking at least three previous locations of the laser projector relative to the work surface, then predicting the next location of the laser projector to compensate for acceleration of the laser projector relative to the work surface.

[0007] The method of this invention may include determining the position of the camera relative to the laser projector by first scanning the targets with the laser projector and then locating the targets with the camera when the laser projector is stable relative to the targets. The targets may be retroreflective targets, and the system may include a light flashing arrangement, such as an LED strobe arrangement, if the method includes directing the light flashes toward the retroreflective targets and locating the illuminated retroreflective targets with the camera to very accurately locate the retroreflective targets and the surface of the targets relative to the laser projector and camera system. As stated above, the targets are at fixed locations relative to the work surface and, in this embodiment, emit or reflect a light source.The method of this invention may also include determining the position of the laser projector relative to the work surface by using the computer to convert three-dimensional pattern coordinates into project vector specular definition coordinates in the laser projector when the corrected pattern is drawn onto the work surface. The method of this invention may also include directly observing an element of the laser projector with the camera, such as a spot or series of spots projected by the laser projector, and calculating the position of the camera relative to the laser projector with the computer.

[0008] As will be understood from the above description of the method of this invention, the method of this invention is particularly suitable for accurate alignment of components, such as the alignment of aircraft carbon fiber designs, which requires an accuracy of at least 0.50 mm (0.0197 in.) under dynamic conditions. As stated above, laser projection updates the individual points recorded in the pattern more than 10,000 times per second. By curving within each pattern rather than between patterns, it is thus possible with the method of this invention to correct with much greater accuracy (e.g., if the projection needs to be corrected for movement of 50 cm / second, a 50 Hz correction produces a 1 cm error in the pattern, while 10,000 Hz corrects to 0.005 cm (or approximately 0.002 in.), which is an imperceptible error and can be used for precise alignment tasks such as aircraft carbon fiber layup).'Bending' is correction for 3D perspective, stretching for two dimensions. Dynamic, local correction of the recorded laser projector vector pattern is an important feature of this invention.

[0009] As outlined above, dynamic tracking is an important feature of this invention; that is, the method of this invention adapts the relative motion of the laser projector and the work surface to the measurement described as a function of time and then predicts where the object will be at the precise moment in the future when the correction is actually applied to the projected display (e.g., to correct for processing delays of a few milliseconds). This allows for extremely natural movements where the applied forces are reasonably continuous. However, if the relative motion strikes a fixed object, it will stop immediately, and the projected pattern will be reflected past the position for a few thousandths of a second.

[0010] Other advantages and appreciable features of the method for accurately projecting a laser image pattern onto a work surface of this invention will be further understood from the following description of the preferred embodiments of this invention. As will be understood, however, this method is not limited to the described embodiments except as set forth in the appended claims. SHORT DESCRIPTION OF THE DRAWINGS

[0011] Other advantages of the present invention will be readily apparent as the same becomes better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, in which Fig. is a schematic view of the prior art industrial method for projecting a laser stencil onto a work surface; Fig. is a schematic view of an improved method for accurately projecting a laser image; Fig. is a schematic illustration of a further improvement to the method of projecting a laser image; Fig. the embodiment of the method described in Fig. shown, further illustrates; Fig. illustrates the method of this invention including a preferred embodiment of a laser projector; and Fig. is an end view of an embodiment of the camera illustrating another embodiment of the method of this invention. DESCRIPTION OF THE EMBODIMENTS

[0012] As set forth above in the background and in my US Patent No. 5,646,859, current industrial arrangements of laser projection systems include a laser projector 20 and a plurality (at least four) targets (typically retroreflective targets) 22 at fixed locations relative to the workpiece or work surface 24, as shown in Fig. shown. The laser projector 20 periodically scans the targets 22 to locate the position of the laser projector relative to the tool through a resection process. The laser projector scanning mirrors can be calculated to record a laser spot through the desired template pattern onto the known three-dimensional profile of the tool. However, this method has several disadvantages, including that the projection of a template onto the tool must be interrupted to confirm the relative position of the laser projector and the tool surface, and it cannot correct for more dynamic changes or movement of the laser projector relative to the target surface.

[0013] An embodiment of the laser projection system of this invention, which is shown in Fig. , includes a laser projector 26, a high-resolution or high-definition camera 28 which is mounted relative to the laser projector 26 or integrated into the laser projector housing, as shown. Targets 30 are mounted relative to the workpiece 32. In this embodiment, the laser projector 26 and camera are used to determine the relative position between the fixed camera 28 location and the laser projector. First, the system locates the targets 30 using the laser projector 26 and camera 28 to determine the relative correspondence between the fixed camera location and the projector-system coordinate origin. This is accomplished by scanning the targets when the system is stable, or by compensating for the scanned position of the laser target by observing the relative position between the cameras during the time period between each target being scanned.Once the relative correspondence between the laser and the camera has been established, the relative position of the laser projector 26 relative to the work tool 32 can be calculated based solely on the camera position without any disruption to the displayed laser projection. Modifying the laser projection position based on the updated camera position allows evaluation of a corrected projection list to maintain a fixed location for the optical template on the tool surface regardless of movement of the laser projector 26 or work tool 32. The preferred embodiment of the laser projection system shown in FIG. Fig. provides two methods for updating the associated vector display list to allow integration into existing laser projection systems and to provide optimized performance with an improved projector designed to directly support the correction process.

[0014] Fig. illustrates a laser and camera projection system that uses dynamic compensation to enhance an existing laser projection setup. In this configuration, an external camera 36 is physically connected to an existing laser projector 34. The camera 36 has an integrated LED flash 38 (discussed below with respect to Fig. ) using a high-speed communication link (for example, an industrial camera model acA2500-14gm, which can be obtained from Basler AG, communicates via Gigabyte Ethernet to provide rapid transmission of high-speed, high-resolution images positioned at a distance from a host computer). Detected locations 42 can then be used to calculate the relative position of the laser projector 34. The host computer 40 can then calculate a relative sampling vector offset based on the 3-D extent of the currently projected template. The vector offset can be sent to the projector 34 and dynamically applied when recording the currently stored vector display list to correct the movement of the projector relative to the target surface or tool (not shown). The system, which is described in Fig. However, the approach shown is not optimal because the simple transformation from the vector display does not allow for complex dynamic movement relative to the tool and can only approximate the required correction to the displayed template. For example, components of the template that are significantly closer to the projector will require a larger correction in the vector angle for accurate compensation.

[0015] The improved projector design, which is Fig. allows for more flexible performance. The embodiment of the projection system shown in Fig. shown includes a laser projector 44 and a high-resolution camera 46. As shown, the vector display list 48 is augmented by entering the source 3-D coordinate of the displayed template. The embedded processor then dynamically applies the currently evaluated projector transform 50 to convert the vector display list 48 into a corrected vector display list 52 once the vector pattern from the laser spot is recorded, to provide a fully corrected template projection 54 in the presence of any relative motion. Image processing and computation of the relative projection can occur in a host computer, or preferably, the camera 46 and image processing software to locate the retroreflective targets and evaluate the camera pose are integrated directly into the projector.

[0016] To calculate the vector representing the rotation angles of the two scanning mirrors of the laser projector (60 and 62 in Fig. ), the position of the laser projector 44 relative to the work surface onto which the laser is projecting is first determined. Once the position of a projector relative to the work surface is determined, the computer calculates the necessary rotation of the scanning mirrors to change the projector position to the desired point at which the laser is to project onto the surface, as discussed further below. Once the camera 46 is located relative to the work surface by observing the targets as discussed above, the location of the laser projector 44 relative to the camera is determined, either by the known mechanical relationship or by scanning, which requires the position of the laser projector relative to the surfaces that have been calculated. Once the position of the laser projector relative to the surface is calculated, the position is used to determine the required mirror rotation vectors.

[0017] The host computer observes changes in the calculated vectors from the initially calculated vectors and outputs a correction for the mirror rotation angles, which shifts the projected pattern into a 2-D delta, and a second more advanced example, the correction is updated value from the 3-D coordinates, as described further below.

[0018] Fig. illustrates a laser projector and camera assembly 56 suitable for the method of accurately projecting a laser image pattern onto a work surface of this invention. The assembly includes a laser or laser source 58, rotating mirrors 60, 62, each mounted on a rotating galvanometer motor 64, 66. The laser beam 68 is redirected by the rotating galvanometer mirrors 60, 62, which deflect the laser beam onto a cylindrical workpiece 72, recording a pattern 70 on the workpiece. The workpiece includes at least four targets 74, preferably retroreflective targets, at predetermined locations on the cylindrical workpiece 72. As will be understood, the work surface 72 can have any shape and is not limited to cylindrical.The assembly further includes a high-definition camera 78, and the targets 74 are illuminated by a light source 76, preferably an LED light source or LED strobe array 76. A processing board 80 detects the target locations and the camera in the image from the camera 78 to calculate the relative position of the projector with respect to the target 72 and determines the modified mirror deflections necessary to record the pattern at a consistent location on the part regardless of any movement of the projector relative to the surface of the target due to movement of the target 72, the projector, or both. Thus, the camera 78 continuously updates the location of the assembly 56 relative to the workpiece 72 without interruption from the laser projection.In a preferred embodiment, the computer continuously adjusts the rotation of the laser projector mirrors 60 and 62 by controlling the rotary galvanometer motors 64, 66 in response to dynamic movement of the laser projector relative to the work surface, as determined by the camera 78. The computer can be used to continuously track at least two previous locations of the laser projector 78 relative to the work surface 72 and predict the next location of the laser projector to compensate for relative movement of the laser projector and the work surface 72. The method of this invention can also be used to compensate for acceleration (ieChanges in velocity) of the laser projector 58 relative to the work surface 72 by continuously tracking at least three previous locations of the laser projector 58 relative to the work surface in three dimensions, and to predict the next location of the laser projector and compensate for movement of the laser projector relative to the work surface. The method of this invention can also be used to track the acceleration (change in velocity) of the laser projector or laser projector and camera assembly 56 relative to the work surface. As will be understood by those skilled in the art, the laser projector and camera assembly 56 is typically mounted on an overhead crane, girder, or gantry and is thus subject to erratic movement due to wind or earth movement. Similarly, the workpiece 72 may be movable relative to the laser projector.In certain applications where accuracy is critical, such as aircraft carbon fiber layup used in the construction of certain aircraft, the error in laser projection must be less than 0.5 mm or 0.01968 in. This accuracy can be achieved with the method of this invention, even under dynamic conditions.

[0019] Fig. is an end view from camera 78, which is in Fig. (and Fig. ), wherein the camera 78 is surrounded by an LED strobe array 76. As stated above, the LED strobe array 76 illuminates the retroreflective targets 74 in Fig. seen by the camera 78, to improve the accuracy of measuring or determining the relative position of the camera and laser projector relative to the work surface.

[0020] As will be understood by those skilled in the art, various modifications may be made to the embodiments of the method for projecting a laser image pattern onto a work surface of this invention within the scope of the appended claims. For example only, two or more cameras may be utilized in the method of this invention, and the cameras may be mounted relative to the laser projector by any means, including placing the camera within the housing of the laser projector, as shown in Fig. shown, or spacing the camera from the laser projector with a holder or beam. Having described the method of this invention, the method is now claimed as follows.

Claims

[1] A method for projecting a laser image pattern onto a work surface (24) and continuously compensating for any relative movement between the laser projector (20, 26) and the work surface (24), comprising a laser projector and camera system (56) including at least one laser projector with a source of a laser beam and rotating mirrors (60, 62) which redirect the laser beam onto the work surface (24), at least one camera (28) which is mounted relative to the laser projector (20, 26), a plurality of retroreflective targets at fixed locations relative to the work surface (24), and a computer (40) which communicates with the laser projector (20, 26) and the camera (28), the method comprising: Directing a flashing light source onto the retroreflective targets; Determining a position of the camera (28) in three dimensions relative to the work surface (24) by locating the illuminated retroreflective targets (22, 30) in the camera image; Determining a position of the laser projector (20, 26) relative to the work surface (24) in three dimensions using the fixed position of the laser projector (20, 26) relative to the camera (28); and Correcting the pattern projected by the laser projector by predicting a next location of the laser projector (20, 26) relative to the work surface (24) based on at least two continuously tracked previous locations of the laser projector (20, 26) relative to the work surface (24) to compensate for a calculated position of the laser projector (20, 26) relative to the work surface (24). [2] A method as defined in claim 1, wherein the camera (28) includes an LED strobe array (76) and the method includes directing the LED strobe array (76) toward the work surface (24), imaging the illuminated retroreflective targets (22, 30) with the camera (28), and accurately determining the position of the laser projector (20, 26) relative to the work surface (24) in three dimensions by locating the illuminated retroreflective targets within the camera image. [3] A method as defined in claim 1, wherein the method includes using the computer (40) to continuously adjust the rotation of the laser projector mirrors (60, 62) in response to any relative movement of the laser projector (20, 26) relative to the work surface (24) as determined by the camera (28), continuously correcting the location of the laser beam on the work surface (24). [4] A method as defined in claim 3, wherein the computer (40) continuously tracks the at least two previous locations of the laser projector (20, 26) relative to the work surface (24) in three dimensions and predicts the next location of the laser projector (20, 26) to compensate for the movement of the laser projector (20, 26) relative to the work surface (24).

Citation Information

Patent Citations

  • System and method for the visual representation of information on real objects

    DE102011015987A1

  • Context aware projector

    US20040201823A1

  • Laser projection systems and methods

    US20050082262A1

  • Augmented reality industrial overline systems and methods

    US20110050872A1

  • Method and apparatus for defining a template for assembling a structure

    US5646859A