Off-axis tracking focusing device and method based on structured light

Abstract

The invention provides an off-axis tracking focusing device and method based on structured light. The device comprises an area-array camera, a first telecentric lens, a first reflector, an LED light source, a collecting lens, a structure template, a second telecentric lens, a second reflector and a detection head. Wherein the area-array camera, the first telecentric lens and the first reflector are sequentially and vertically arranged from top to bottom to form a side light path; the LED light source, the collecting lens, the structure template, the second telecentric lens and the second reflecting mirror are sequentially and vertically arranged from top to bottom to form a light path on the other side; and the light path on one side and the light path on the other side are symmetrically arranged relative to the position of the detection head. According to the invention, the requirement of higher detection precision is met, the influence of the process is avoided, and focusing offset cannot occur at an uneven position.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of optical detection, and particularly relates to an off-axis tracking focusing device and method based on structured light. BACKGROUND

[0002] With the gradual improvement of the resolution of LCD screens, OLED screens and Micro LED screens, the detection precision of the detection equipment is also required to be improved. With the improvement of the detection precision of the detection equipment, the depth of field of the detection head is also reduced. When the flatness of the product during the detection process cannot reach the depth of field range of the detection head, the detection head is required to have tracking focusing capability. In order to improve the detection effect, the detection head generally adopts a line scanning lens, and only an off-axis tracking focusing device can be used.

[0003] As shown in Figure 1 The existing off-axis tracking focusing device uses laser triangulation to detect the height variation of the product relative to the detection head and feed back to the controller to control the corresponding movement of the detection head. However, the existing technology has the disadvantages that the precision cannot meet the requirement of higher detection precision, and is easily disturbed by the process, and the focusing offset will occur at the uneven position. SUMMARY

[0004] The purpose of the present application is achieved by the following technical solutions.

[0005] Specifically, the present application provides an off-axis tracking focusing device based on structured light, comprising: a surface array camera, a first telecentric lens, a first reflector, an LED light source, a condenser lens, a structure template, a second telecentric lens, a second reflector and a detection head; wherein, the surface array camera, the first telecentric lens and the first reflector are vertically arranged in order from top to bottom to form one side light path; and the LED light source, the condenser lens, the structure template, the second telecentric lens and the second reflector are vertically arranged in order from top to bottom to form another side light path. The one side light path and the other side light path are symmetrically arranged relative to the position of the detection head.

[0006] Further, the detection head comprises an imaging lens and a TDI camera.

[0007] Further, the light emitted by the LED light source passes through the condenser lens, the structure template, the second telecentric lens and the second reflector in sequence, and then is irradiated to the surface of the glass to be measured. After being reflected by the surface of the glass to be measured, the light passes through the first reflector and the first telecentric lens in sequence, and then is irradiated to the surface array camera. The image of the structure template is obtained by imaging through the surface array camera.

[0008] Further, the magnification of the two telecentric lenses is 2 times.

[0009] Further, the structure template is a pair of lines with certain rules and non-transparency on the glass.

[0010] Further, the rules are encoding cycles.

[0011] Further, the device further comprises a high-frame-rate real-time automatic focusing system based on FPGA connected with the area array camera, and the system comprises: an FPGA chip and an MCU chip; wherein the FPGA chip analyzes and calculates the image, calculates the offset and sends it to the MCU chip, and the MCU controls the Z-axis motor to adjust the height of the detection head and the off-axis tracking focusing device; The FPGA chip, the MCU chip and the CMOS chip of the area array camera are located on the same PCB board.

[0012] Further, based on the active vision triangulation method and the absolute encoding grating, the absolute position information of the grating is decoded by projecting the grating pattern with interval encoding to the measured object surface, using the linear translation effect of the image on the imaging surface when defocusing, and real-time image acquisition and sub-pixel edge positioning technology. By comparing the current absolute position with the reference position calibrated when focusing clearly, the defocusing amount and direction without cumulative error are calculated, and converted into a pulse signal to drive the servo motor to complete the lens position correction.

[0013] Further, the edge positioning technology uses a two-stage strategy of coarse positioning combined with fine positioning: first, the comparator locks the approximate position at the pixel level, and then the sub-pixel offset is calculated in a small range through quadratic fitting.

[0014] Further, the FPGA chip comprises the following modules: (1) Column integral module: in the FWFT mode, the control logic determines whether the column data is received according to the data flow counter; during the receiving process, the accumulation operation is triggered in real time to realize the integration while receiving; (2) Real-time derivative module: the derivative is divided into serial mode and parallel mode; (3) Coarse positioning module: when three derivative results are obtained, the extreme point is calculated, and the output result of coarse positioning is the integer column index, i.e. the preliminary position of the grating fringe; (4) Fine positioning module: the quadratic fitting operation is connected with the coarse positioning result through the pipeline mode, and the output result of fine positioning is the accurate position of the grating fringe edge; (5) Key data calculation and storage module: the sub-pixel precision edge position sequence from the fine positioning module is received, and the absolute encoding information of the grating is calculated according to the sequence; the center position sequence and the center distance sequence calculated are written into the block memory of the FPGA chip as the reference template; (6) Defocus detection module: pipeline parallel structure, synchronous operation with the previous integral, derivative, positioning module.

[0015] The advantage of the present application is to solve the requirement of higher detection accuracy, not affected by the process, and the uneven position will not appear focus shift. BRIEF DESCRIPTION OF DRAWINGS

[0016] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the present application. Moreover, like reference numerals designate like parts throughout the several views in the drawings. In the drawings: Figure 1 The existing off-axis tracking focusing device is shown.

[0017] Figure 2 The off-axis tracking focusing device based on structured light according to the embodiment of the present application is shown.

[0018] Figure 3 The control principle diagram of the off-axis tracking focusing device based on structured light according to the embodiment of the present application is shown.

[0019] Figure 4 The grating edge recognition diagram according to the embodiment of the present application is shown.

[0020] Figure 5 The grating edge recognition result diagram according to the embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments of the present disclosure are shown. It is to be understood that the present disclosure can be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0022] The key point of the present application is to solve the process influence and uneven influence by using structured light imaging, and to measure the structured light offset by using the structured light imaging offset when the object distance changes, so as to improve the accuracy. The off-axis tracking focusing device and method based on structured light are protected.

[0023] As shown in Figure 2 The off-axis tracking focusing device based on structured light of the present application comprises: a surface array camera, a first telecentric lens, a first mirror (located on the left side), an LED light source, a condenser lens, a structure template, a second telecentric lens, a second mirror (located on the right side), a detection head (includingFigure 2 The detection head consists of an imaging lens and a TDI camera. The area array camera, first telecentric lens, and first reflector are arranged vertically from top to bottom, forming the left optical path; the LED light source, condenser lens, structural template, second telecentric lens, and second reflector are arranged vertically from top to bottom, forming the right optical path. The central detection head includes an imaging lens and a TDI camera. The left and right optical paths are symmetrically arranged relative to the detection head, ensuring that the light emitted from the LED light source passes sequentially through the condenser lens, structural template, second telecentric lens, and second reflector before illuminating the surface of the glass under test. After reflection from the glass surface, the light enters the left optical path, then passes sequentially through the first reflector and first telecentric lens before reaching the area array camera. The area array camera then obtains an image of the structural template.

[0024] The light source in this invention can be more than just LED; other types of light sources can also be considered. The magnification of the two telecentric lenses is approximately 2x. If the magnification is too high, the working distance will be insufficient; if the magnification is too low, the resolution will be insufficient.

[0025] The structural template in this invention is generally made by engraving a certain regularity of opaque line pairs on the glass. For example, in a specific implementation case, three types of line pairs, 20um, 40um, and 80um, are etched onto the glass in a coded manner, such as 111 (20um line pair x1, 40um line pair x1, 80um line pair x1), 211, 311, 411, 121, 221, etc.

[0026] like Figure 2 As shown, this invention employs an off-axis structured light imaging design. The structured light source assembly (light source + structure) on the right illuminates the object surface through an illumination lens and a reflector, while the structured light image on the object surface is imaged onto an area array camera through a reflector and an imaging lens on the left. When the relative height between the object surface and the structure (the device of this invention, the entire structured light off-axis tracking and focusing device) changes, the structured light image will shift.

[0027] like Figure 3 As shown, this invention employs machine vision and control design. After the area scan camera captures the image, it is connected to a high frame rate real-time automatic focusing system based on FPGA. The FPGA chip analyzes and calculates the image, informing the MCU chip of the offset. The MCU then controls the Z-axis motor to adjust the height of the detection head and the off-axis structured light imaging device. The FPGA, MCU, and CMOS chip are all on the same PCB board. Furthermore, the FPGA can directly control the Z-axis motor, eliminating the need for the MCU. The CMOS chip is... Figure 2 The image sensor of the internal camera is located there. The Z-axis motor is situated within... Figure 2 On the back, drive Figure 2The whole up and down movement. A Z-axis motor can simultaneously adjust the height of the imaging lens in the detection head and the height of the structured light imaging device, and the off-axis structured light imaging device specifically refers to the whole structured light off-axis focusing device.

[0028] The following detailedly describes the FPGA-based high-frame-rate real-time automatic focusing system implementation of the application. I. System overall scheme

[0029] (1) Core principle The system is based on active vision triangulation and absolute encoding grating technology. By projecting a grating pattern with a unique interval code onto the surface of the measured object, the linear translation effect of the image on the imaging plane when defocusing is used. The FPGA acquires images in real time and decodes the absolute position information of the grating using sub-pixel edge positioning technology; by comparing the current absolute position with the reference position calibrated when in sharp focus, the defocusing amount and direction without cumulative error are directly calculated, and converted into high-precision pulse signals to drive the servo motor (i.e. Z-axis motor) to complete lens position correction, thereby constructing a high-speed, high-precision optical closed-loop focusing control system.

[0030] (2) System workflow 1. Projection: The coded grating is projected onto the surface of the object to be measured.

[0031] 2. Acquisition: The black and white high-speed camera acquires the deformed grating image.

[0032] 3. Processing: FPGA processes the image in real time (vertical integration, edge detection), and decodes the current grating phase information.

[0033] 4. Calculation: Compare the decoding result with the reference code of the sharp focus position (Make0), and calculate the defocusing distance and direction.

[0034] 5. Control: Convert the defocusing amount into a servo motor control instruction to drive the lens to move, forming a closed-loop control.

[0035] (3) System architecture and component selection Assembly Model Description Master FPGA Xilinx Artix-7 series (XC7A100T) Rich in logic resources, supports high-speed I / O, and has a large number of Block RAMs built-in for image caching. Key: Supports Vivado simulation tool chain, easy for algorithm verification. Image sensor Global shutter black and white CMOS camera Resolution: Supports 1280x1024, and through ROI (region of interest) cropping, the resolution is 1280x16 to achieve ≥1K fps frame rate. Interface: MIPI. Actuator Panasonic MINAS A6 series servo motor and driver High responsiveness, high precision. Supports pulse / direction control mode, easy for FPGA direct control. Optical components LED + coded grating sheet, industrial lens Form a triangulation optical path. Debugging interface Gigabit Ethernet on the FPGA development board Used to send image data stream to PC for visual verification. (4) Key parameters Parameter Value Description Camera horizontal resolution 1280 pixels Image 1280*1024@240fps→1280*16@2500fps Pixel size pixel 4 μm Single pixel corresponds to the size of the square 4 μm The basic scale of decoding. The image moves 1 pixel, and the grating physically moves 4 μm. Optical system YZ ratio 4 : 1 Object defocus ΔZ = 1 μm → grating image offset ΔY = 4 μm. This is the cornerstone of precision and range calculation. Camera depth of field (Z axis) ±200 μm The Z axis range that the system needs to focus. Grating needs to cover Y axis offset ±800 μm ΔY = ΔZ * 4 = ±200 μm * 4 = ±800 μm. Camera field of view (FOV) 5120 μm 1280 pixels * 4 μm / pixel = 5120 μm. Minimum total width of grating 6720 μm FOV + II. Detailed design and implementation steps

[0036] (I): Algorithm implementation steps Let the two-dimensional image be I (x,y), where the image width is w , and the height is h。

[0037] Vertical integration: summing the collected 2D image along the vertical direction (column direction) to get a 1D signal : , This can compress the vertical fringe information of the grating to the X-axis, improving the accuracy and speed of grating edge recognition.

[0038] Edge recognition: first use the central difference method to get the first derivative : , Edge positioning uses a two-stage strategy of "coarse positioning + fine positioning": because the extreme points present sharp pulse characteristics in the first derivative signal, first use a simple comparator to quickly lock the approximate position at the pixel level (coarse positioning), and then accurately calculate the sub-pixel offset in this small range through quadratic fitting (fine positioning), which balances the speed and accuracy requirements by ensuring the real-time performance of pixel full-image search and achieving high-precision measurement.

[0039] Coarse positioning: use local analysis method to judge the extreme points of the first derivative : , This method can effectively detect flat or sharp peak extremes and is more suitable for discrete image data.

[0040] Fine positioning: near the pixel-level extreme point found by coarse positioning , use quadratic function to fit the first derivative signal , and obtain the edge position with sub-pixel accuracy by solving the extreme point of the fitted curve.

[0041] Assume that the first derivative signal near the extreme point satisfies the quadratic function: , where x is the continuous coordinate.

[0042] Use , , the derivative values of the three points to fit: , Solve the above equation set to get the sub-pixel offset of the extreme point position (parabola vertex): , The edge position with sub-pixel accuracy is: , Threshold suppression noise: In order to remove the small fluctuations caused by noise, only the extreme value points with large amplitude are retained: , Where T It can be a certain proportion of the maximum absolute value of the derivative, for example: , If , the extreme value point is ignored.

[0043] Experimental verification: As shown in Figure 4 , using a grating image with dirt, test whether the algorithm can accurately identify all edges. As shown in Figure 5 , the grating edge recognition result. Calculate the center position and spacing of adjacent edges to confirm that it meets the design of the equidistant grating.

[0044] (II) FPGA algorithm implementation steps Key issues: After PC algorithm verification, the system's out-of-focus detection algorithm has been proven to have good feasibility and precision. However, due to the extremely high image acquisition frame rate (>2000 fps), traditional PC serial processing cannot meet the real-time requirements. To achieve high-speed parallel processing and real-time output, the algorithm needs to be deployed on the FPGA platform for hardware-level real-time processing.

[0045] Overall implementation idea: The FPGA system completes the whole process of image input, column item integration, derivation, edge detection, and precise positioning through a pipeline parallel structure. The design uses a modular structure, with each sub-module running independently and in parallel under a 200 MHz clock (MIPI data clock), achieving nanosecond-level cascade operation, and ultimately achieving the goal of real-time out-of-focus measurement.

[0046] Data structure and input characteristics: Data format: 4 bytes (32 bits) of data per clock, Image width: 1280 pixels, Image height: 16 rows, Pixel clock frequency: 200 MHz, Under these conditions, when 320 groups of data are accumulated, the acquisition of a column of image data is completed (16 rows × 4 bytes = 64 bytes × 20 clocks = 320 bytes). Therefore, real-time synchronous processing of column data is required in the FPGA.

[0047] Module design and flow: (1)Column integral module: using FIFO module to work in First Word Fall Through (FWFT) mode. In FWFT mode, the first data can be taken immediately when needed, ensuring that the integral logic has no waiting delay. The control logic determines whether the column data has been received according to the data flow counter (320 data). The accumulation operation is triggered in real time during the receiving process to realize the integration while receiving.

[0048] (2) Real-time derivative module: the derivative can be divided into "derivative after integration (serial mode) and real-time derivative (parallel mode)".

[0049] The design adopts a real-time derivative scheme: when it is detected that the integral results of the first, second and third columns have been completed, the derivative can be immediately started. In this way, under a 200 MHz clock, integration and derivation are almost synchronized. The typical delay is only 2-4 clock periods (about 10-20 ns).

[0050] (3) Coarse positioning module: the core of coarse positioning is to judge the extreme point of column derivative. The extreme point can be obtained when the three derivative results are obtained, and the hardware logic can be realized by only a simple comparator. The output result of the coarse positioning is an integer column index, that is, the preliminary position of the grating fringe.

[0051] (4) Fine positioning module: in FPGA implementation, the quadratic fitting operation and the coarse positioning result are connected through a pipeline, and the delay can be completed in only one clock period. The output result of the fine positioning is the accurate position of the grating fringe edge.

[0052] (5) Key data calculation and storage module: this module receives the sub-pixel accuracy edge position sequence from the fine positioning module, and calculates the absolute encoding information of the grating according to the sequence. The center position sequence and the center distance sequence calculated are written into the block memory (Block RAM) of the FPGA as a reference template. The template uniquely defines the absolute encoding characteristics of the grating in the current field of view, and is a reference for subsequent real-time matching.

[0053] (6) Defocus detection module: the defocus detection also adopts a pipeline parallel structure and is synchronized with the previous integral, derivative and positioning modules.

[0054] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical range disclosed by the present application can be easily thought by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A structured light based off-axis tracking focusing device, characterized in that, Comprise: A plane array camera, a first telecentric lens, a first mirror, an LED light source, a condenser, a structure template, a second telecentric lens, a second mirror, a detection head; wherein, The plane array camera, the first telecentric lens and the first mirror are arranged vertically from top to bottom in sequence to form one side light path; the LED light source, the condenser, the structure template, the second telecentric lens and the second mirror are arranged vertically from top to bottom in sequence to form another side light path; The one side light path and the another side light path are arranged symmetrically relative to the position of the detection head.

2. The off-axis tracking focusing device based on structured light according to claim 1, wherein The detection head comprises an imaging lens and a TDI camera.

3. The off-axis tracking focusing device based on structured light according to claim 1 or 2, wherein The light emitted by the LED light source passes through the condenser, the structure template, the second telecentric lens and the second mirror in sequence, and then is reflected by the surface of the glass to be measured, and then passes through the first mirror and the first telecentric lens in sequence to be incident on the plane array camera, and the image of the structure template is obtained by imaging through the plane array camera.

4. The off-axis tracking focusing device based on structured light according to claim 1 or 2, wherein The magnification of the two telecentric lenses is 2 times.

5. The off-axis tracking focusing device based on structured light according to claim 1 or 2, wherein The structure template is a pair of lines with certain regularity that are opaque on the glass.

6. The off-axis tracking focusing device based on structured light according to claim 5, wherein The regularity is a coding cycle.

7. The off-axis tracking focusing device based on structured light according to claim 1 or 2, wherein The device further comprises a high-frame-rate real-time automatic focusing system based on FPGA connected with the plane array camera, and the system comprises: An FPGA chip and an MCU chip; wherein the FPGA chip analyzes and calculates the image, calculates the offset and sends it to the MCU chip, and the MCU controls the Z-axis motor to adjust the height of the detection head and the off-axis tracking focusing device; The FPGA chip, the MCU chip and the CMOS chip of the plane array camera are located on the same PCB board.

8. The off-axis tracking focusing device based on structured light according to claim 7, wherein The FPGA chip comprises the following modules: (1) Column integral module: in the FWFT mode, the control logic judges whether the column data is received according to the data flow counter; in the receiving process, the cumulative operation is triggered in real time to realize the integration while receiving; (2) Real-time derivative module: the derivative is divided into serial mode and parallel mode; (3) Coarse positioning module: the extreme point is obtained when three derivative results are obtained, and the output result of coarse positioning is the integer column index, that is, the preliminary position of the grating fringe; (4) Fine positioning module: the quadratic fitting operation is connected with the coarse positioning result through the pipeline mode, and the output result of fine positioning is the accurate position of the grating fringe edge. (5) Key data calculation and storage module: receiving the sub-pixel precision edge position sequence from the fine positioning module, and calculating the absolute encoding information of the grating according to the sequence; taking the center position sequence and the center distance sequence calculated as the reference template, and writing them into the block memory of the FPGA chip; (6) Defocus detection module: using a pipeline parallel structure, and synchronously running with the previous integral, derivative and positioning modules.

9. A structured light based off-axis tracking focusing method using the device of any one of claims 1-8, characterized in that, based on active vision triangulation and absolute encoding grating, by projecting a grating pattern with interval encoding to the measured object surface, using the linear translation effect of the image on the imaging plane when defocusing, real-time image acquisition and absolute position information decoding of the grating using sub-pixel edge positioning technology; by comparing the current absolute position with the reference position calibrated when in focus, the defocus amount and direction without cumulative error are calculated, and converted into a pulse signal to drive the servo motor to complete the lens position correction.

10. The structured light based off-axis tracking focusing method according to claim 9, characterized in that, the edge positioning technology uses a two-stage strategy of coarse positioning combined with fine positioning: first, locking the approximate position at the pixel level using a comparator, and then calculating the sub-pixel offset in a small range through quadratic fitting.

Images