Fluorescence detection of impurities

Abstract

The present disclosure relates to fluorescent detection of impurities. A method for detecting the presence of impurities in a board layer is disclosed. A source of the impurity is marked with a fluorescent indicator. The source of the impurity is separated from the board layer. An illumination source is provided for illuminating the board layer at a wavelength different from that fluorescing from the fluorescent indicator, so that the board layer reflects light of the different wavelength. A sensor detects the illumination of the board layer and the fluorescent indicator included in the impurity disposed on the board layer. A difference in light reflected from the board layer and the fluorescence of the indicator disposed in the impurity is detected, thereby identifying the presence of the impurity in the board layer.

Description

[0001] Prior application

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 653,890, filed on April 6, 2018, the contents of which are incorporated herein by reference. Technical Field

[0003] The present application generally relates to the detection of foreign matter adhering to an assembly during the assembly process. More specifically, the present application relates to an improved method of detecting foreign matter using fluorescence incorporated into a known source of foreign matter for improving the ability to distinguish foreign matter from an assembly. Background Art

[0004] In many high-tech manufacturing processes, the inadvertent inclusion of impurities that contaminate the part or workpiece being manufactured can render the part unusable due to the possibility of catastrophic failure. For example, when constructing a composite part, multiple layers of plies made of a composite material (such as, for example, a polymer impregnated with fibers) are applied to a mandrel for forming the composite part. Typically, a woven fiber material (including, but not limited to, carbon fibers and equivalents) is saturated with an epoxy or similar resin to form a sheet of plies. The plies are formed on a backing paper and are pre-cut into the desired shape to form the assembly before delivery to the assembly facility.

[0005] Before stacking each ply to form a composite part, the backing paper is separated from the assembly. After removal, the multiple layers of the ply are covered on a mandrel to form the composite part. A known source of contamination is small pieces or particles of backing paper that neither loosen from the sheet nor fall onto the sheet after removal. This may cause pieces of backing paper to become trapped between the layers of the ply. This contamination may cause catastrophic failure of the composite part. Once trapped between the layers of the ply, it becomes difficult to remove the contaminant. After assembly, thermal or ultrasonic methods are sometimes used to test the composite part. However, detecting such contaminants after assembly usually requires the composite part to be scrapped at great expense.

[0006] Attempts have been made to identify contaminants from known sources by analyzing images collected from cameras viewing an assembly of composite parts. One such example is disclosed in U.S. Patent No. 6,064,429, in which the disclosed assembly uses a color camera to view the work surface and attempts to distinguish between contaminants and the work surface based on differences in color. When identified, the location of the contaminant is indicated to the operator. Due to the difficulty in distinguishing between the typically flat black composite plies and even colored contaminants, this system has not proven to be effective.

[0007] A further attempt to identify contaminants on a work surface is disclosed in U.S. Patent Application Publication No. 20160061746, which attempts to use a color classifier to compare scenes with contaminants and scenes without contaminants. This method also fails to adequately distinguish between contaminants and a flat black composite background.

[0008] An additional disadvantage of these methods is that adequate contrast between the contaminant material and the material making up the ply cannot be achieved. Many ply materials used during assembly include polymers impregnated with carbon fiber, fiberglass, Kevlar, and other composites. Modifications to material formulations and impregnation amounts based solely on manufacturing variability may also make it impossible to distinguish between ply material and contaminants. In addition, ambient light is not taken into account. The detection of color depends on the color balance of ambient light in the environment on adjacent reflective sources. The color of adjacent materials (including operator clothing, reflective targets, etc.) may trigger false contaminant alarms.

[0009] The trade-off between the detection of even very small contaminant particles and the adverse reduction in production throughput due to false detection of contaminants has proven difficult to achieve. No camera-based system has proven adequate to ensure that contaminants are not trapped between layers of a ply. Reliance solely on the visible light spectrum to distinguish between contaminants and composite materials has rendered many of these systems inoperable. Therefore, it would be desirable to develop a method that identifies contaminants from known sources (such as, for example, liner paper, gloves, and related items) prior to mating layers of a ply with a workpiece while providing highly accurate results. Summary of the invention

[0010] A method of detecting the presence of impurities in a board layer is disclosed. A source of impurities is marked with a fluorescent indicator. The source of impurities is separated from the board layer. An illumination source is provided for illuminating the board layer so that the board layer reflects light of a wavelength different from that fluoresced by the fluorescent indicator. A sensor is provided for detecting the illumination of the board layer and the fluorescent indicator included in the impurities disposed on the board layer. A difference in light reflected from the board layer and the fluorescence of the indicator disposed in the impurities is identified for finding and identifying the presence of impurities in the board layer.

[0011] The invention of the present application detects fluorescent materials with high accuracy due to the high contrast between the fluorescence of the mold and the flat black, even matte composite material when illuminated at the desired wavelength. The fluorescent mold can be added to the backing paper, gloves, or any other material known to cause unwanted contaminants to be trapped between the layers of the ply. Once a piece of ply is ready to be applied to the mandrel or the layers of the pre-existing ply, the back side of the ply is illuminated and scanned by the camera sensor. Illumination of even very small pieces of contaminants, for example, treated with an orange fluorescent material and illuminated by a light source with the desired wavelength enables the camera sensor to easily detect the contaminants. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Other advantages of the present invention will be readily appreciated as they become better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings, in which:

[0013] Figure 1 shows a comparison of the illumination of the fluorescent material comparing an orange fluorescent material to a non-fluorescent material (both of which are illuminated by ambient light);

[0014] Figure 2 shows a comparison of illumination of fluorescent and non-fluorescent materials with a green laser while exposed to ambient light;

[0015] Figure 3 shows a comparison of illumination of fluorescent and non-fluorescent materials with a green laser with ambient light subtracted;

[0016] Figure 4 A schematic diagram showing a first embodiment of the invention of the present application, the schematic diagram showing a general illumination system;

[0017] Figure 5 A schematic diagram showing a second embodiment of the invention of the present application, the schematic diagram showing a universal illumination system combined with a laser projector;

[0018] Figure 6 A schematic diagram showing a third embodiment of the invention of the present application using only a laser projector; and

[0019] Figure 7 A schematic diagram showing a third embodiment of the invention of the present application using a laser beam to contour contaminant material. DETAILED DESCRIPTION

[0020] The invention of the present application overcomes existing foreign object detection methods by stimulating a fluorescent mold or equivalent that has been added to a known source of contaminants or impurities, such as, for example, liner paper, work gloves, protective work clothing, etc. Unlike traditional colored materials that absorb some portion of the ambient light, fluorescent materials absorb incidental light and emit incidental radiation of a wavelength different from the wavelength of the illuminating light. For example, Figure 1 As best represented in FIG. 1 , the orange fluorescent material absorbs ambient light at wavelengths below about 575 nm and re-emits longer wavelength fluorescence with peak fluorescence at about 606 nm. The peak fluorescence at 606 nm provides the ability to better distinguish between fluorescent and non-fluorescent objects.

[0021] Additional differentiation benefits are achieved when the fluorescent material is illuminated by an illumination source having a wavelength outside the fluorescence wavelength band. For example, a fluorescent material, such as an orange fluorescent material, is illuminated with a laser projecting in the green spectrum or a light emitting diode (LED) illuminating in a non-orange spectrum. Illuminating the fluorescent material with an illumination source that produces light outside the fluorescence band of the fluorescent material provides the ability to separate the illumination light from the fluorescence of the fluorescent material. Figure 2 As shown in , a green laser (or green LED source) used to illuminate an orange fluorescent material provides a clearly distinguished emission response. The green laser provides a peak at about 535nm, while the orange fluorescence, when illuminated by a green source, has a peak at about 606nm. The fluorescence peak at 606nm provides an advantageous distinguishing characteristic, which enables improved sensor detection of contaminants in a sheet of board material by distinguishing light reflected from non-fluorescent materials.

[0022] Further accuracy in detecting contaminants is achieved by filtering the ambient illumination. When the ambient illumination is attenuated, the fluorescence of the fluorescent material becomes even more pronounced. In this way, even very small pieces of contaminants or contaminant particles treated with the fluorescent mold can be reliably detected. Figure 3 As best represented in , the removal of ambient light provides a much clearer separation between the excitation light source (green laser or LED) and the corresponding responsive fluorescence than previously thought possible.

[0023] Now refer to Figure 4 , a first embodiment of the detection assembly of the present invention is shown schematically at 10. Assembly 10 includes a camera 12 having a lens 14 and a shutter 16. A light sensor 15 (not shown) is included in camera 12. In one embodiment, sensor 15 is a CMOS sensor. In an alternative embodiment, sensor 15 is a CCD sensor. Other types of sensors 15 are also within the scope of the present invention. An illumination source 18 illuminates a workpiece, in this example, a laminated composite ply 20. Multiple illumination sources can be used to excite fluorescent materials. In one embodiment, the illumination source is a laser having a wavelength different from the excitation wavelength of the fluorescent mold used to mark the contaminant material 22. Although a laser-based illumination source 18 is discussed in more detail below in this document, it should be understood that the invention of the present application includes alternative illumination sources, including but not limited to, LED illumination, and even a combination of illumination sources. Accuracy is significantly improved when the illumination is outside the wavelength of the fluorescence spectrum.

[0024] The illumination source 18 is electrically linked to the camera 12 either via a computer 24 or directly. The camera lens 14, shutter 16, and image capture are synchronized with the illumination system 18 to interleave the capture of images. The interleaved image capture of images with and without the illumination system exciting the fluorescent material 22 allows the computer 24 to subtract any excitation of the fluorescent material 22 by ambient light. This provides isolation of the excitation of the fluorescent material 22 by the illumination system 18. In one embodiment, the interleaving is performed at a rate that is imperceptible to the operator. Thus, image capture occurs at the same rate as the illumination pulses or flashes of the illumination source 18 (in this embodiment, envisioned as light emitting diode (LED) flashes or pulses). In addition, the emission signal (fluorescence) from multiple image captures is averaged to improve the sensitivity and reliability of contaminant detection.

[0025] In one embodiment, the camera 12 includes a "rolling" shutter 16 having an illumination strobe rate that is a fraction of the image capture rate of the sensor 15 to generate an illumination band with the captured image. By using a reduced duty cycle for the strobe illumination of the illumination system 18, the power of the strobe illumination is much greater than the brightness of the illumination perceived by the operator. Therefore, when the illumination occurs at a much higher rate than the entire image capture, when the strobe flash is activated, the illumination occurs in a "band". During the moments when the strobe occurs, the illumination is substantial when compared to the ambient illumination, but the operator perceives only the average illumination power so that the illumination does not appear too bright. This phenomenon is particularly true when the illumination band only includes half or one-quarter of the total image capture period. This strategy improves operator viewing comfort without reducing the detection sensitivity of the sensor 15 and camera 12.

[0026] As explained above, it is desirable to isolate the fluorescence emission upon illumination. To aid in the isolation, a monochrome camera (or cameras) is included by using filters to isolate the wavelength of the excitation energy from the fluorescence emitted from the fluorescent material at a selected fluorescence emission wavelength. In an alternative embodiment, selection of compatible wavelengths provides the ability to separate the fluorescence wavelength from the illumination (excitation) wavelength using a filter grid of a standard color camera. For example, a green laser producing excitation light having a wavelength of 520 nm can easily filter out orange fluorescence emission having a wavelength of 606 nm.

[0027] In an alternative embodiment, the assembly 10 operates continuously until no further excitation of the fluorescent material is detected and the computer and / or operator determines that all contaminants have been separated from the slab 20. The camera 12 includes the ability to detect a signal from the operator initiating a detection sequence through a hand gesture, a physical switch, an interface on a handheld tablet or other remote control device, including but not limited to augmented reality goggles (not shown). The illumination system 18 is powered on and illumination of the slab 20 is continued until the contaminants 22 are no longer detected by fluorescence. While fluorescence is detected, the camera 12 sensor 15 continues to send a view of the slab 20 to a monitor 23, such as, for example, a fixed monitor, a handheld device, a smart phone, a template, or even augmented reality goggles. The operator can also override the illumination when the contaminants detected by the camera 12 sensor 15 are considered to be trace amounts acceptable.

[0028] Alternative embodiments are Figure 5 100. The alternative embodiment 100 integrates an optical stack laser projection system 110 similar to the system disclosed in U.S. Patent No. 9,200,899, the contents of which are incorporated herein by reference. The projection system 110 includes a laser projector 112 and a photogrammetry assembly 114. The optical stack system 110 projects a laser template on an assembly tool or workpiece to provide an operator with the location where each sequential piece of the ply will be placed. As disclosed in the 9,200,899 patent, the photogrammetry assembly 114 locates the assembly tool or workpiece and signals the computer 24 where to project the laser template based on a computer-aided design (CAD) model. The photogrammetry assembly 114 includes one or more photogrammetry cameras 116, which can also detect fluorescence when the contaminant is illuminated. However, it should also be understood that the optical stack system 110 remains separate from the illumination system relying on a separate camera 116. In this embodiment, the photogrammetry component 114 communicates with the laser projector 112 via the computer 24 to coordinate laser illumination with shutter 118 speed, image capture, and the like.

[0029] Figure 7 An additional advantage of the illustrated alternative embodiment is that the laser projection system 110 identifies the exact location where the part or ply will be placed. Thus, the computer 24 accurately directs the camera 124 where to search for contaminants when searching for contaminants on the mandrel or work surface. This provides the computer 24 with the ability to ignore contaminants that are not located in critical areas (such as, for example, grading areas). The ability to distinguish between critical contaminants and irrelevant contaminants provides additional manufacturing efficiency. Additionally, when scanning with the laser beam 111 to illuminate the fluorescent material, processing time is reduced by directing the laser scan to the exact area of ​​interest.

[0030] The laser projector 112 cooperates with the photogrammetry assembly 114 and the inspection assembly 120. The inspection assembly 120 includes a secondary illumination system 122 that illuminates the ply 20 either simultaneously with the laser projector 112, intermittently with the laser projector 112, or before and after the laser projector 112 illuminates. In one example, the secondary illumination system 122 illuminates the ply 20 before an operator initiates an inspection sequence. In this way, the computer 24 is provided with an indication of where contaminants may be on the ply 20 by detecting fluorescence through the photogrammetry assembly 114. Thus, the laser projector 112 can begin its projection toward the location on the ply 20 where contaminants are initially detected.

[0031] Still further, the detection system 120 includes a secondary camera 124 in a manner similar to that described above. The secondary camera 124 either signals the computer 24 of the preliminary location of the contaminant 24 or detects the exact location of the contaminant 24 by detecting fluorescence produced by the laser projector 112, the secondary illuminator 122, or a combination thereof. In this manner, the secondary camera includes a lens 126 and a shutter 128 that coordinate with the illumination to selectively allow light to reach the sensor 115 in a manner similar to that described above. Although the shutter 128 is shown as a mechanical shutter for exemplary purposes only, it should be understood that the shutter 128 can also be an electronic shutter that can quickly discern light transmission to the sensor 15 at a rate comparable to the flashes or pulses from the illumination source.

[0032] A further embodiment of the invention of the present application is Figure 6 and 7 200. The second alternative embodiment 200 relies solely on an optical stack laser projection system 210 similar to the system disclosed in U.S. Patent No. 9,200,899. The projection system 210 includes a laser projector 212 and a photogrammetry assembly 214. As described above, the optical stack system 210 projects a laser template on an assembly tool or workpiece to provide an operator with the location where each sequential piece of the ply will be placed. The photogrammetry assembly 114 locates the assembly tool or workpiece and signals a computer 224 where to project the laser template based on a computer-aided design (CAD) model. The photogrammetry assembly 114 includes one or more photogrammetry cameras 216 and sensors 226, which can also detect fluorescence when the contaminant is illuminated.

[0033] The laser projector 212 projects a green laser beam 211 toward the location of the slab as described above. In one embodiment, the laser beam 211 includes a linear focus that covers a larger surface area with each pass. As also explained above, the excitation of the fluorescent material by the green laser beam 211 enables the camera 216 of the photogrammetric assembly 214 to distinguish between the contaminant 22 and the slab 20. In addition, the photogrammetric assembly 214 identifies the location of the contaminant 22 by triangulation of the fluorescence in a manner similar to identifying the location of a reflector (not shown) as further described in U.S. Patent No. 9,200,899. The photogrammetric assembly then signals the computer 23 to notify the location of the contaminant 24.

[0034] Once the contaminant 22 has been located, the laser projector 212 delineates the location of the contaminant 22 by scanning a ring or box 218 around the contaminant 24. The laser projector 212 also optionally continues to illuminate the contaminant so that the photogrammetric component 214 can continue to monitor for fluorescence. Once the operator removes the contaminant, the photogrammetric component 214 no longer detects fluorescence, and the laser projector 212 ceases to scan the box 218 around the area where the fluorescence was initially detected. The photogrammetric camera 216 includes a sensor 226 that is adapted to selectively filter ambient light to allow for the detection of fluorescence. Figure 3 Fluorescence detection is achieved as shown in .

[0035] The steps of the method for detecting contaminants are now described. The following description is merely exemplary and is not intended to be limited to the various uses of the components 10, 100, 200 of the present invention. The operator first determines the orientation of a sheet of ply 20 so that the backing material faces the detection component 10, 100, 200. The detection component 10, 100, 200 optionally scans the backing material to scan the bar code (not shown) on the backing paper to verify that the correct ply 20 has been selected in relation to the pre-programmed assembly step sequence. After verification, the operator removes the backing material. The operator then initiates the detection sequence in the manner described above, and the detection system begins to verify that all materials have been separated from the ply 20. If any amount of fluorescence is detected, the computer 24 sends a signal to the monitor to notify the operator of the location of the identified contaminant. The monitor generates an enhanced image of the ply 20 when interacting with the components 10, 100, 200, as described in more detail in the co-pending U.S. patent application No. 15 / 058,867, the contents of which are incorporated herein by reference. The assembly 10, 100, 200 continues the detection sequence until no further fluorescence is detected. Once no further fluorescence is detected, a process log programmed into the computer 24 or signaled to a cooperating computer records that the sheet layer 20 corresponding to the detected bar code is free of contaminants. The record in the process log assists in preventing operator error.

[0036] When the inspection assembly 120 is integrated with the optical stack laser projection system 110, the assembly 120 and the system 110 are aligned to a common coordinate system established by the photogrammetry assembly 114, 214, or equivalent. Thus, the inspection system 120 can be programmed to verify that no impurities or contaminants 22 are present prior to placement by illuminating the stack prior to placing a ply 20. Inspection is accomplished throughout the field of view of the secondary camera 124 or photogrammetry system 114, 214. Detection of contaminants 22 in non-critical locations within the field of view is optionally attenuated while ensuring that no contaminants remain in the area where a ply 20 will be placed. Further, the optical template system 110 is capable of isolating the location of a detected contaminant 22 by projecting it by the laser projector 112 onto the relevant area of ​​the stack.

[0037] The present invention has been described in an illustrative manner, and it is to be understood that the terms used are intended to be of a verbal or descriptive nature, rather than limiting. It is apparent that many modifications and variations of the present invention are possible in light of the above teachings. It is therefore to be understood that within this specification, reference numerals are merely for convenience, and are not limiting in any manner, and that the present invention may be implemented in ways other than those specifically described. Therefore, the present invention may be implemented in ways other than those specifically described within the scope of the intended claims.

Claims

1. A method for detecting the presence of impurities in a plate layer, comprising the following steps: treating the impurity source with a fluorescent indicator having a peak fluorescent wavelength disposed within a fluorescent wavelength band; separating the impurity source from the plate layer; providing an illumination source including a peak illumination wavelength outside the wavelength band of the fluorescent indicator for illuminating the sheet layer, whereby the sheet layer reflects light at a wavelength outside the fluorescent wavelength band when illuminated by the illumination source; providing a sensor for detecting a fluorescent indicator included in an impurity disposed on the ply and illumination of the ply; as well as The board layer is illuminated using the illumination source and a difference between light reflected from the board layer and fluorescence of an indicator disposed in the impurity is identified, thereby identifying the presence of the impurity in the board layer.

2. The method according to claim 1, further comprising: The step of filtering light from said illumination source from said sensor.

3. The method according to claim 1, wherein: The step of providing an illumination source is further defined by providing one of a green laser and a light emitting diode.

4. The method according to claim 1, wherein: The step of identifying a difference between light reflected from the ply and fluorescence of an indicator disposed in the impurity is further defined by detecting fluorescence having a wavelength between 590 nm and 625 nm.

5. The method according to claim 1, wherein: The step of identifying a difference between light reflected from the ply and fluorescence of an indicator disposed in the impurity is further defined by detecting fluorescence having a peak wavelength of approximately 606 nm.

6. The method according to claim 1, further comprising: The step of interleaving the image of the panel without illumination by the light source with the image of the panel with illumination by the light source for subtracting illumination by ambient light.

7. The method according to claim 1, further comprising: The step of providing a monitor for receiving signals from said sensor to generate an image identifying the location of fluorescent contaminants on said ply.

8. The method according to claim 1, wherein: The step of illuminating the slab is further defined by the laser projector continuously illuminating the slab until the fluorescent contaminants are separated.

9. The method according to claim 1, further comprising: A step to record verification that all fluorescent contaminants have been separated from the laminate.

10. The method according to claim 1, further comprising: The step of scanning the illumination source across the ply for a marker to identify the location of a contaminant.

11. The method according to claim 1, further comprising: The step of providing an optical stack projection system for locating a region of interest on the stack.

12. The method of claim 11 further comprising the steps of identifying an area of ​​interest and providing a laser projector for scanning illumination as directed by the optical stack projection system and signaling a camera to detect the location of the illumination to identify contaminants.

13. The method according to claim 12, further comprising: The step of distinguishing between regions of interest that are critical for detecting contaminants and irrelevant regions where contaminants are not critical.

Images