Accurate registration of images for assembly verification

Abstract

The present disclosure relates to precise registration of images for assembly verification. A method and apparatus for verifying the position and orientation of a component attached to a work surface to manufacture a workpiece is disclosed. A controller is programmed with design data for a manufacturing step. Multiple wing panel layers are continuously stacked on the work surface. An imaging system capable of redirecting a field of view along the work surface is provided. A light projector is provided for projecting a reference feature onto the work surface. The light projector projects the reference feature to a known position within the field of view of the imaging device relative to the component applied to the work surface. The imaging device generates an image of the reference feature projected by the light projector and the component applied to the work surface. The controller compares the position of the component on the work surface with the design data to verify the accurate placement of the component.

Description

[0001] Existing Application

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 792,969, filed on January 16, 2019, the contents of which are incorporated herein by reference. Technical Field

[0003] The present invention generally relates to methods and apparatus for inspecting a part to verify an assembly process. More particularly, the present invention relates to providing accurate registration of a sequence of images of a part after a piece has been secured to the part to provide assembly verification. Background Art

[0004] Certain types of assemblies require verification of the placement of the component on a substrate or work surface, such as, for example, a mandrel. Automated verification may be difficult to achieve using conventional imaging systems. For example, when creating a composite part, many layers of tape or plies are sequentially placed on a work surface. A typical component may be a ply or tape member that includes a fiber material, such as, for example, carbon fiber, impregnated with a polymer material. The ply layers are applied to the mandrel, presenting a substantially indistinguishable appearance between the layers, as each ply is identical except for the shape and orientation of the fiber material.

[0005] To reduce the need for manual inspection, it is desirable to automate the inspection process, for example by using inspection cameras to measure component position and fiber orientation. However, the tools used to create the parts can be very large, while the features of the components are very small (e.g., a 20-foot tool with material composed of carbon fiber has a diameter of only a few thousandths of an inch). Therefore, any inspection camera or equipment does not typically have the ability to directly resolve the features of individual components to verify their placement. Attempts to redirect the camera view using galvanometer mirrors have not proven satisfactory, and the cost is beyond practical limitations.

[0006] During assembly, each component is typically placed either by an automated mechanical fiber tape laying machine or manually under the guidance of a projected laser pattern that acts as an optical template to align the components in the assembly. However, for either type of placement, the position of each component and the orientation of the fibers within the material must be confirmed to ensure the strength and structural integrity of the manufactured component. Currently, analysis of placement position and fiber orientation is accomplished through manual inspection, comparing the placement of each component to a projected laser template. Unfortunately, this inspection method is slow and prone to error because the layered components are nearly indistinguishable.

[0007] To reduce the need for manual inspection, it is desirable to automate the inspection process by, for example, measuring component position and fiber orientation using an inspection camera. However, the tools used to create the parts can often be very large, while the features of the components being analyzed are very small (e.g., a 20 foot tool having a material comprised of carbon fibers may be only a few thousandths of an inch in diameter). Therefore, any inspection camera or device typically does not have the ability to directly resolve the features of individual components to verify proper placement and orientation. Analysis of small elements on large work surfaces has proven difficult to achieve, in part because the camera cannot quickly locate and focus on small elements. Additional problems arise when the work surface is three-dimensional. Efforts to overcome these difficulties, even when galvanometer mirrors were introduced, have been unsatisfactory and the costs have exceeded practical limitations.

[0008] Automated analysis of small parts made from dark fiber materials has proven to be difficult to achieve due to the specular absorption and reflection of ambient light. Therefore, it is desirable to develop an imaging system that overcomes these problems and provides reliable and accurate automated inspection, especially during the continuous component layer-by-layer manufacturing process. Summary of the invention

[0009] A method and apparatus for verifying the position and orientation of a component attached to a work surface to continuously manufacture a workpiece is disclosed. A controller can be programmed with design data to continuously manufacture the workpiece. In a non-limiting embodiment, a plurality of wing panels are continuously stacked on a mandrel or a work surface. An imaging system is provided that is capable of redirecting a field of view along the work surface. A light projector for projecting a reference feature to a known position on the work surface is provided. It is contemplated that the light projector is a laser projector or other light source capable of projecting a reference feature onto a mandrel, a work surface, or a wing panel layer that has been applied to the work surface. In an exemplary embodiment, the component is applied to a predetermined position identified on the work surface. After application, the light projector projects the reference feature to a known position relative to the component applied to the work surface within the field of view of the imaging device. The imaging device generates an image of the reference feature projected by the light projector and the component applied to the work surface. The image is then notified to the controller by signaling. The controller compares the position of the component on the work surface as aligned with the reference feature with the design data to verify the accurate placement of the component on the work surface.

[0010] Modern manufacturing using, for example, wing panel layups on a mandrel requires precise placement of each wing panel. This process has been widely used in the manufacture of composite aerospace components and even high performance land vehicle components. Misalignment of even a single wing panel can result in catastrophic failure of the component. As such, automated inspection is critical to eliminating human error, but has proven difficult to achieve for widespread use. The apparatus of the present invention overcomes the problems associated with earlier automated inspection systems, particularly when large layup components require inspection of very small areas. Coordinated aiming of the imaging system with the light projector allows for rapid inspection of any large component. This and other inventive features, which will be described in further detail below, enable rapid and highly accurate inspections, thereby overcoming the above-mentioned problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other advantages of the present invention will become more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, in which:

[0012] Figure 1 A schematic diagram of an inspection system of the present invention is shown;

[0013] Figure 2 A schematic diagram showing an inspection system having a light projector scanning and locating a target; and

[0014] Figure 3 A close up view of a reference feature surrounding a projection of a component placed on a work surface is shown. DETAILED DESCRIPTION

[0015] refer to Figure 1, the inspection system of the present invention is generally shown at 10. The system 10 includes an imaging system 12 and a light projector 14. In one embodiment, the imaging system 12 is a camera 16 that can be moved by a pan / tilt device 18. The pan / tilt device 18 moves the field of view 20 of the imaging system around the work surface 22. The camera 16 also includes a zoom lens 24 or equivalent to enhance the clarity of the image by focusing the field of view 20 of the camera 16 on the work surface 22. The zoom lens 23 is particularly useful when image clarity on the three-dimensional work surface 22 is required. As an alternative to the zoom lens 24 or in combination with the zoom lens 24 is to use a camera 20 with a variable focal length sensor array, as disclosed in U.S. Patent No. 9,442,075, the contents of which are incorporated herein by reference for the sake of brevity of this application. With this device, a limited image range or frequency band can be achieved with high accuracy, which is most useful when verifying the exact position of smaller components. Thus, it should be appreciated that the imaging system 12 utilizing the pan / tilt device 18 and the zoom lens 24 enables the camera 20 to quickly move the field of view 20 around the surface while maintaining optimal focus and image clarity, even if the work surface 22 is three-dimensional. The zoom lens 24 enables more precise focus in the direction of arrow 25, as will be further explained below.

[0016] In one embodiment, the light projector 14 takes the form of a laser projector, such as, for example, a 532 nm green laser or equivalent, hereinafter referred to as "projector 14". The projector 14 projects a light beam 26 to first and second scanning mirrors 28, 30, which reflect the light beam 26 toward the work surface 22. The first scanning mirror 28 is pivotally moved by a first electric motor (galvo-motor) 32, and the second scanning mirror 30 is pivotally moved by a second electric motor 34. The first scanning mirror 28 and the second scanning mirror 30 are cooperatively driven together by the first and second electric motors 32, 34, respectively, to scan the light beam 26 along the work surface 22 under the direction of a controller 36. Thereby, the controller 36 is electrically interconnected to the first and second electric motors 32, 34 through circuitry 37, as will be further understood below.

[0017] The movement of the pan / tilt device 18 is coordinated with the movement of the first and second electric motors 32, 34 and directed by the controller 36. The controller 36 directs the pan / tilt device 18 to direct the field of view 20 of the camera 16 to the location where the laser beam 26 is directed by the first and second electric motors 32, 34. The precise orientation of the camera 16 and the optical characteristics of the zoom lens 24 are evaluated and updated by evaluating one or more reference features 38 generated by the laser beam 26 on the work surface 22. Thus, the projector 14 scans the reference features 38 to the location directed by the scanning mirrors 28, 30.

[0018] In one embodiment, the reference feature 38 projected into the field of view 20 of the camera 16 is an arbitrary laser spot. The controller 36 electronically directs the electric motors 32, 34 to orient the scanning mirrors 28, 30 to direct the laser beam 26 to an area of ​​interest, such as, for example, the location of a component (e.g., a wing panel) 40 that has been attached to the working surface 22. At this point, the camera 16 only needs to have sufficient resolution to focus on the component 40 that has been attached to the working surface 22, identify the periphery 42 of the component 38 and the orientation of the fibers 44 disposed in the component 38. As used within the scope of the present application, the working surface 22 includes a mandrel, a surface of an assembled component, a layered component of a wing panel, and combinations or equivalents thereof.

[0019] In another embodiment, the location of the reference features 38 projected onto the work surface 22 by the projector 14 is the same as the template projected onto the work surface, as disclosed in U.S. Pat. No. 9,200,899, the contents of which are incorporated herein by reference. Figure 2 As shown, the projector 14 projects the laser beam 26 onto the reflective target 46. In one embodiment, the closest coordinates are provided to the projector 14 or the controller 36 in the three-dimensional coordinate system of the reflective target 46. Alternative methods of providing the reflective target 46 to the proximity position of the laser projector are within the scope of the present invention. As described in U.S. Patent No. 9,200,899, the laser beam 26 is reflected back to the sensor, or any reference feature is reflected back to the camera 16, so that the position of the projector 14 aligned with the work surface 22 can be accurately calculated. Once accurate alignment has been determined, the projector 14 can project the reference feature 38 to any location on the work surface 22, from which position the imaging system 12 aligns its position relative to the work surface 22, and additionally aligns its position relative to the member 40.

[0020] As described above, the laser template 48 can be scanned by the projector 14 or other laser projector onto the work surface 22 to identify to the operator the location where the component 40 is to be fitted to the work surface 22. While the template 48 is being scanned, the camera 16 images the component 40 after placement and signals the image to the controller 36. The controller 36 verifies that the component 40 is properly aligned within the template 48 and further compares the alignment of the component 40 to the CAD location. Alternatively, after the component 40 is placed on the work surface 22 at the location indicated by the template 48, the projector 14 terminates scanning the template 48 and the camera 16 provides sufficient image resolution to the controller 36 so that the periphery or edge 42 of the component 40 when placed can be compared with the CAD data to ensure accurate placement. In addition, the combination of edge detection 42 and template 48 comparison is associated with the CAD data to verify accurate placement.

[0021] Controller 36 directs movement of field of view 20 of camera 16 through a sequence of assembly steps of the layered composite coordinated with features 38 scanned by projector 14. Thus, pan / tilt device 18 directs field of view 20 of camera 16 based on detection of reference features 38 on work surface 22 by camera 16. In this embodiment, reference features 38 are arbitrary laser dots projected onto work surface 22 by projector 14. After registering the position of camera's field of view 20 to the placed component 40, the camera captures an image and the image is signaled to controller 36 to verify the correct orientation and placement of component 40. This verification process is repeated such that camera 16 generates a sequence of images of each component 40 placed on work surface 22 as the assembly process proceeds.

[0022] As described above, the camera 16 includes a zoom lens 24 or other focusing device that cooperates with the pan / tilt device 18 to adjust to focus on the newly assembled component 40, even when the field of view of the camera 40 spans a three-dimensional surface. The camera 16 includes an optical sensor 48, which in some imaging systems 12 can signal an image to the controller 36, thereby enabling the controller 36 to identify image shifts so that the focus of the camera 16 can be adjusted. However, when the position of the camera 16 is accurately registered relative to the work surface 22 by sensing the reference features 38 projected into the field of view of the camera 16, such image shifts can be eliminated.

[0023] In one embodiment, the reference feature 38 is projected to a known location on the work surface 22. Since the laser projector 14 has been accurately positioned relative to the work surface 22 by scanning the target 46 or other positioning methods such as disclosed in U.S. Patent No. 9,200,899, no matter which arbitrary location the laser projector 14 projects the reference feature 38, it is a known location. Therefore, the reference feature 38 is projected to known coordinates relative to the coordinate system defined by the work surface 22. Therefore, as long as the reference target 38 is within the field of view 20 of the camera 16, the controller 36 can accurately position the image generated by the camera relative to the work surface 22 by imaging the reference point in the work surface coordinate system. In this way, the camera 16, including the pan / tilt device 18 and the zoom lens 24, is calibrated to the work surface 22. Calibration allows the controller 36 to eliminate imaging errors caused by inaccurate positioning of the pan / tilt / zoom device 18 or inaccuracies in the lens 24.

[0024] It will be appreciated by those skilled in the art that the positioning sequence eliminates any need to physically position the camera 16 relative to a known feature, such as, for example, an edge, opening, fiducial, etc. of the work surface 22. Any projection of a laser dot or reference feature 38 within the field of view 20 of the camera 16 is sufficient to accurately position the camera 16 image within the coordinate system defined by the work surface 22. Repeating this sequence allows the pan / tilt device 18 and zoom lens 24 to be accurately calibrated relative to the projector 14, even when the camera 16 is not mechanically fixed to the projector 14.

[0025] Computer 36 uses imaged reference features 38 to accurately quantify the measured deviation of the positioned component 40 from the theoretical position of the component as specified by the CAD of the assembled component 40. When the registration of camera 16 using projected template 48 to quantify the component 40 provides acceptably accurate results, the accuracy can be improved by also projecting reference features 38 within field of view 20 of camera 16. Figure 3 Contrary to the solid lines shown, the projected template 48 may be modified to incorporate the template laser dots 50. In this manner, the computer 36 may both identify the orientation of the component 40 relative to the template and accurately register the position of the work surface 22 and the component from the template laser dots 50. Alternatively, once the component 40 has been placed, the template laser dots 50 may be replaced with spaced apart templates to establish accurate registration of the camera 16 image relative to the work surface 22.

[0026] In an alternative embodiment, the laser template 48 can be used to register the positions of the work surface 22 and the component 40. However, the combination of the laser template 48 and the laser dots (features 38) further improves image accuracy. Using both the laser template 48 and the laser dots (features 38) for registration provides further improved correction for image variations due to optical outliers that occur between the optical sensor 50, the zoom feature of the lens 24, and the movement of the camera 16 through the tilt / pan device 18.

[0027] The present invention has been described in an illustrative manner, and it should be understood that the terms that have been used are intended to have the nature of descriptive rather than limiting words. Obviously, many modifications and variations can be made to the present invention in accordance with the above teachings. Therefore, it should be understood that in this specification, the figure numerals are only for convenience and are not limiting in any way, and the present invention can be implemented in a manner different from the specific description. Therefore, the present invention can be practiced in a manner different from the specific description within the scope of the intended claims.

Claims

1. A method for verifying the position and orientation of a component attached to a working surface for manufacturing a workpiece, comprising the steps of: providing a controller capable of being programmed with design data for a manufactured workpiece; Providing an imaging system capable of redirecting a field of view along a work surface; providing a light projector for projecting a reference feature onto a known location on the work surface; applying the member to a predetermined location identified on the work surface; The light projector projects a reference feature into a field of view of the imaging device at a known location relative to a component applied to the work surface; The imaging device generates an image of the reference feature and the component applied to the work surface projected by the light projector and signals the image to the controller; The controller compares the position of the component on the work surface as registered with the reference features to the design data to verify accurate placement of the component on the work surface.

2. A method as claimed in claim 1, wherein the step of providing a light projector is further defined by providing a scanning mirror assembly, wherein the scanning mirror assembly is used to redirect light generated by the light projector to a predetermined position on a work surface, thereby generating a reference feature on the work surface.

3. The method of claim 2 further comprising the step of coordinating movement of the scanning mirror assembly with redirection of the imaging device so as to focus the field of view of the imaging device toward a feature applied to a work surface.

4. The method of claim 1 further comprising the step of the imaging device generating an image of the orientation of the component and the controller comparing the image of the orientation of the component to design data programmed in the controller.

5. The method of claim 1, wherein projecting the reference feature to a known position relative to the component applied to the work surface is further defined by projecting the reference feature to a periphery of the component applied to the work surface.

6. The method of claim 1, wherein the step of providing a light projector is further defined by providing a laser projector.

7. The method of claim 1 wherein the step of providing an imaging system is further defined by providing a camera capable of panning and tilting to provide movement of the camera coordinated with the location of a reference feature.

8. The method of claim 1 further comprising the step of adjusting the focus of the imaging device in response to changes in a three-dimensional contour of a work surface within a field of view of the imaging device.

9. The method of claim 1 further comprising the step of securing a positioning target for reflecting light from the light projector to a device light sensor to determine a relative position of the light projector with respect to a work surface.

10. The method of claim 1 further comprising the step of generating an image of the orientation of fibers arranged in the component by the imaging device and signaling the image to the controller so that the controller can compare the orientation of the fibers in the image with the orientation of the fibers in the design data to analyze the orientation of the fibers arranged in the component.

11. The method of claim 1, wherein the step of projecting a reference feature onto the work surface is further defined by projecting a plurality of laser spots onto the work surface.

12. The method of claim 11, wherein the step of projecting a plurality of laser spots onto the work surface is further defined by projecting a plurality of laser spots onto arbitrary locations on the work surface.

13. A laser projector and inspection system, comprising: Controller; a laser source for projecting reference features onto the work surface; a galvanometer assembly including a first scanning mirror and a first electric motor and a second scanning mirror and a second electric motor; a camera including a pan and / or tilt device and a zoom lens for directing the field of view to the work surface; as well as The controller coordinates movement of the first and second electric motors with movement of the pan and / or tilt device and the zoom lens of the camera so that the laser source projects a reference feature into the field of view of the camera, thereby enabling the controller to calculate the position of the camera relative to a work surface based on an image of the reference feature generated by the camera.

14. The system of claim 13, wherein the position of the laser source relative to the work surface is determined by scanning a reflective target disposed at a known position on the work surface.

15. The system of claim 13, wherein the camera is adapted to generate an image of a component applied to a work surface to signal the image to the controller, and the controller is programmed with design data for comparing the position of the component in the image with the position of the component in the design data.

16. The system of claim 13, wherein the controller is programmed to direct the first electric motor and the second electric motor to a position to scan a photo template onto a work surface to provide a position for placing a component on the work surface.

17. The system of claim 13, wherein the pan and / or tilt device and the zoom lens are movable relative to a coordinate system defined by a work surface to generate a high resolution image of a camera view of the work surface.

18. The system of claim 17, wherein the pan and / or tilt device and the zoom lens are adapted to generate a high-resolution focus based on scanning a reference feature projected by a laser source onto the work surface to an arbitrary position on the work surface.

Images