A multi-camera auto focus detection system and method of detection

Abstract

The application provides a multi-camera automatic focusing detection system and a detection method, and belongs to the field of machine vision detection, and comprises a detection light machine, a displacement meter, a Z-axis compensation planning module and a servo control module, wherein the detection light machine and the displacement meter each have multiple sets, the number of the displacement meters is less than the number of the detection light machines, the displacement meter is connected with the Z-axis compensation planning module, which is used for detecting the height of the surface of a product to be detected in the Z direction and feeding back the height information to the Z-axis compensation planning module, the Z-axis compensation planning module is connected with the servo control module, and the servo control module can control the detection light machine to adjust the height in the Z direction according to the expected adjustment after receiving the Z-direction height adjustment information provided by the Z-axis compensation planning module, so that automatic focusing is realized. The application also provides a multi-camera automatic focusing detection method. The application can solve the problems that the automatic focusing speed is not fast enough and the detection efficiency is not high enough when the multi-camera combination is used for detecting large-size products.

Description

Technical Field

[0001] This invention belongs to the field of machine vision inspection, and more specifically, relates to a multi-camera autofocus inspection system and inspection method. Background Technology

[0002] In precision vision optical measurement systems, the depth of field of the optomechanical system is typically only a few micrometers to tens of micrometers. Affected by factors such as stage flatness and product warpage, it is prone to image blurring. Current technical solutions usually address this issue by incorporating autofocus. The larger the product being inspected, the more likely warpage will cause image blurring. Furthermore, in practical applications, to shorten production time in the inspection of large-size products, combinations of two or more cameras are often used. This involves multi-camera autofocus inspection devices and methods; how to quickly achieve multi-camera autofocus and perform inspection is a pressing issue that requires attention. Summary of the Invention

[0003] In view of the shortcomings of the prior art, the purpose of this invention is to provide a multi-camera autofocus detection system and detection method, which aims to solve the problems of insufficient autofocus speed and low detection efficiency in the detection of large-size products using multiple camera combinations.

[0004] To achieve the above objectives, the present invention provides a multi-camera autofocus detection system, which includes a detection optical engine, a displacement meter, a Z-axis compensation planning module, and a servo control module. Multiple sets of the detection optical engine and displacement meters are provided, with the number of displacement meters less than the number of detection optical engines. The displacement meters are connected to the Z-axis compensation planning module, which detects the height of the surface of the product under inspection in the Z-axis direction and feeds this height information back to the Z-axis compensation planning module. The Z-axis compensation planning module is connected to the servo control module. After receiving the Z-axis height adjustment information provided by the Z-axis compensation planning module, the servo control module can control the detection optical engine to adjust the Z-axis height as expected, thereby achieving autofocus.

[0005] In the above invention, each inspection optical machine has its own corresponding displacement gauge. One displacement gauge corresponds to one or more inspection optical machines. For example, in areas where the height fluctuation of the product to be inspected is small, one displacement gauge can be set up, and the height information measured by this displacement gauge is shared with multiple inspection optical machines. The displacement gauge is fixed in the Z-axis direction, but it can move along the X and Y directions. The displacement gauge is connected to the Z-axis compensation planning module, which is connected to the servo control module. The servo control module controls the inspection optical machine to adjust the Z-axis height as expected.

[0006] Furthermore, it also includes a Z-axis displacement axis, with each detection optical machine correspondingly set on an independent Z-axis displacement axis, which is connected to a servo control module.

[0007] In the above invention, the servo control module is connected to the Z-axis displacement axis, and each detection optical engine is correspondingly set on an independent Z-axis displacement axis. The servo control module controls the Z-axis displacement axis to move in the Z direction, thereby realizing the fine adjustment of the detection optical engine along the Z direction and achieving automatic focusing.

[0008] Furthermore, at least two sets of inspection optical machines adjust the Z-axis height based on the height information obtained from one set of displacement gauges. The at least two sets of inspection optical machines and the at least one set of displacement gauges are suspended above the center of the product to be inspected. The displacement gauges are laser displacement detectors. Inspection optical machines and displacement gauges are suspended at the edges of the product to be inspected, with a one-to-one correspondence between the inspection optical machines and displacement gauges at the edges. One set of inspection optical machines adjusts the Z-axis height based on the height information obtained from one set of displacement gauges.

[0009] In the above inventive concept, for large-sized products to be inspected, such as G4.5 generation lines and above, products with a length × width of 730mm × 920mm can be called large-sized products. Generally, their corners have large warping, while the center has small warping. That is, the height fluctuation at the corners of the product to be inspected is large, while the height fluctuation at the center is small. One displacement meter can be used on both sides for one set of inspection optical machines, and one laser displacement meter probe can be used in the middle for two sets of inspection optical machines. This design is practical and can significantly reduce the number of ranging probes. Using fewer ranging probes can assist more cameras in automatic and fast focusing, saving the number of probes, reducing system costs, and simplifying the structure of the device or system.

[0010] Furthermore, both the servo control module and the Z-axis compensation planning module are PLC modules. The detection optical machine has, for example, four to ten sets, and the displacement meter has, for example, three to eight sets. The displacement meter is fixed in the Z-axis, and the detection optical machine can move along the X, Y, and Z directions.

[0011] The above invention provides the integration scale of multi-camera autofocus detection, but does not limit its specific integration scale. In actual engineering practice, the number, structure and type of integration can be flexibly adjusted according to engineering needs.

[0012] According to a second aspect of the present invention, a multi-camera autofocus detection method is also provided. First, the distance between each set of displacement gauges and the corresponding detection optical engine located at the initial position in the X and Y directions is individually calibrated, and the initial height between each set of displacement gauges and the sample to be inspected is measured. The X-direction distance ΔX, Y-direction distance ΔY, and Z-direction displacement axis height Z0 that each set of detection optical engines needs to adjust to achieve clear focus are obtained. ΔX, ΔY, and Z0 are used as teaching values. Then, multiple detection optical engines scan the sample to be inspected back and forth along the X-axis. Multiple displacement gauges continuously collect the surface height information of the product to be inspected in real time and feed it back to their respective detection optical engines. The height of the Z-direction displacement axis is adjusted according to their respective teaching values ​​and the height of the product to be inspected collected in real time to compensate for the height fluctuation of the surface of the product to be inspected, thereby achieving autofocus.

[0013] Furthermore, the displacement gauge is fixed in the Z direction, while the detection optical mechanism reciprocates along the X direction, which includes the following more detailed steps:

[0014] S1: The distances ΔX and ΔY between the calibrated displacement gauge and the corresponding detection optical mechanism along the X and Y directions, respectively.

[0015] S2: The displacement gauge measures the initial height H0 of the product under inspection.

[0016] S3: The height Z0 of the Z-axis displacement axis during calibration and testing of the optical engine when it is in sharp focus.

[0017] S4: Write ΔX, ΔY, H0, and Z0 as teaching values ​​into the Z-axis compensation planning module.

[0018] S5: Repeat steps S1 to S5 above to obtain the teaching values ​​of all displacement gauges and the corresponding detection optical mechanisms.

[0019] S6: Multiple inspection optical engines reciprocate along the X-axis, and multiple displacement gauges detect the height Hi of the sample to be inspected below them in real time. The Z-axis compensation planning module controls the height changes of different Z-axis displacement axes through the servo control module according to different teaching values ​​and the corresponding height Hi of the sample to be inspected, so as to compensate for the height fluctuations of the surface of the product to be inspected and achieve automatic focusing.

[0020] Furthermore, step S1 specifically includes:

[0021] S1: Select a calibration plate with a Mark mark and fix it on the stage. First, move the calibration plate under the displacement meter probe and observe that the light spot of the displacement meter probe is aligned with the Mark mark. Record the X and Y axis coordinates as (x1, y1). Then, move the calibration plate under the detection optical machine corresponding to the displacement meter so that the Mark mark on the calibration plate is imaged in the center of the field of view of the detection optical machine. Record the X and Y axis coordinates at this time as (x2, y2). The distances between the displacement meter probe and the detection optical machine along the X and Y axes are: ΔX = x1 - x2, ΔY = y1 - y2.

[0022] Furthermore, step S6 specifically involves:

[0023] The product to be inspected is placed on the stage. The inspection optical machine reciprocates along the X-axis, or the stage drives the sample to be inspected to reciprocate along the X-axis. When the inspection optical machine moves a distance ΔX along the X-axis, the displacement sensor probe continuously collects the height Hi of the product to be inspected and inputs it into the Z-axis compensation planning module. Based on the collected height Hi, the Z-axis motion compensation height Zi is obtained, Zi = Z0 + H0 - Hi. The Z-axis height adjustment trajectory is output to the servo control module. The Z-axis displacement axis is controlled by the servo control module to drive the Z-axis displacement axis and carry the corresponding inspection optical machine to the corresponding height Zi, realizing real-time automatic focusing during the inspection process.

[0024] In summary, compared with the prior art, the above-described technical solutions conceived by this invention have the following advantages:

[0025] Beneficial effects:

[0026] In this invention application, for large-sized samples, multiple cameras are used for combined detection, and multiple sets of detection optical engines and multiple sets of displacement gauges are deployed. The number of displacement gauges is less than the number of detection optical engines. In areas with large height fluctuations, one set of displacement gauges corresponds to one set of detection optical engines, while in areas with small height fluctuations, one set of displacement gauges corresponds to multiple sets of detection optical engines. This can achieve simultaneous detection by multiple cameras while reducing the number of displacement gauge probes. It can not only greatly shorten the detection time of large-sized products, but also simplify the system and save costs.

[0027] Furthermore, each inspection optical machine in the multi-camera setup corresponds to a set of displacement gauges. Based on the height detected by each displacement gauge, the height of the corresponding inspection optical machine in the Z-axis is adjusted to achieve high-speed autofocus, significantly reducing the focusing time of the various cameras distributed above the surface of large-sized products. Compared to the previous method of first scanning the product under inspection with displacement gauges to obtain overall height fluctuation data, and then scanning the product again with the inspection optical machine for focusing and imaging, this invention only requires multiple inspection optical machines and displacement gauges to perform a single scan simultaneously, improving the inspection time by approximately 40% or more. This significantly meets the needs of high-speed automated industrial inspection. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the structural components of the multi-camera autofocus detection system provided in an embodiment of the present invention.

[0029] Figure 2 This is a flowchart illustrating the multi-camera autofocus detection method provided in an embodiment of the present invention.

[0030] In this context, the same reference numerals throughout the accompanying drawings denote the same structure, module, or component, specifically:

[0031] 1-Stage, 2-Detection optical engine, 3-Displacement meter, 4-Z-axis compensation planning module, 5-Servo control module, 6-Z-direction displacement axis. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] In the description of this invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the appendix. Figure 1 The orientations or positional relationships shown are for the purpose of facilitating and simplifying the description of the present invention, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the present invention.

[0035] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "configuration" should be interpreted broadly. For example, they can refer to a fixed connection or configuration, a detachable connection or configuration, or an integral connection or configuration. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0036] Figure 1This is a schematic diagram of the component structure of the multi-camera autofocus detection system provided in this embodiment of the invention. As shown in the diagram, it includes a detection optical engine 2, a displacement meter 3, a Z-axis compensation planning module 4, a servo control module 5, and a Z-axis displacement axis 6. Multiple sets of both the detection optical engine 2 and the displacement meter 3 are available. In this embodiment, there are four sets of detection optical engines and three sets of displacement meters 3, with the number of displacement meters 3 being less than the number of detection optical engines 2. The displacement meter 3 is connected to the Z-axis compensation planning module 4, and it is used to detect the height of the surface of the product under inspection in the Z-axis direction and feed the height information back to the Z-axis compensation planning module 4. The Z-axis compensation planning module 4 is connected to the servo control module 5, and the output of the servo control module 5 is connected to the Z-axis displacement axis 6. Each detection optical engine 2 is independently mounted on one Z-axis displacement axis 6. After receiving the Z-axis height adjustment information provided by the Z-axis compensation planning module 4, the servo control module 5 can control the Z-axis displacement axis 6 to make fine adjustments up and down. The detection optical engine 2 adjusts the Z-axis height as expected, achieving autofocus. The above solution proposes a multi-camera combination method and realizes autofocus in the system, achieving a multi-camera detection system with autofocus function.

[0037] In one embodiment of the present invention, the system is used to inspect large-size display screens (panels). Through long-term engineering practice, it has been found that the height fluctuations vary across different areas of large-size panels, with differences between the corners and the center. Based on this finding, the number of displacement meters and inspection optical machines has been cleverly designed. Previously, one set of displacement meters provided data for one set of inspection optical machines. Now, in the central area with smaller fluctuations, one set of displacement meters provides height data to at least two sets of inspection optical machines simultaneously. That is, at least two sets of inspection optical machines 2 adjust the Z-axis height based on the height information obtained from one set of displacement meters 3. The at least two sets of inspection optical machines 2 and the at least one set of displacement meters 3 are suspended above the center of the product under inspection. Inspection optical machines 2 and displacement meters 3 are suspended at the edges of the product under inspection, with a one-to-one correspondence between the edge-mounted inspection optical machines 2 and displacement meters 3. One set of inspection optical machines 2 adjusts the Z-axis height based on the height information obtained from one set of displacement meters 3. In a specific embodiment of the present invention, two sets of displacement gauges and two sets of detection optical machines are positioned at the edge of the panel to be inspected, while two sets of detection optical machines and one set of displacement gauges are positioned at the center of the product to be inspected. The displacement gauges and detection optical machines, working together, can move as a whole along the X and Y directions. After moving to a set position, the displacement gauges measure the height and provide feedback to the detection optical machine, which then adjusts the height in the Z direction.

[0038] In another embodiment of the present invention, both the servo control module 5 and the Z-axis compensation planning module 4 are PLC modules. In actual engineering practice, they can also be other control modules that are available on the market. The detection optical machine 4 has four to ten sets, and the displacement meter 3 has three to eight sets. The number of integrated detection optical machines and displacement meters is determined according to the size of the product to be inspected and can be flexibly selected. The displacement meter 3 is a laser displacement detector, which is fixed in the Z-axis and can detect a fixed height. The detection optical machine 4 can move along the X, Y, and Z directions, and can be moved as needed to the area requiring focusing and imaging.

[0039] The multi-camera autofocus inspection system described above enables multi-camera autofocus inspection. The general method is as follows: First, the distance between each set of displacement gauges and its corresponding inspection optical engine at the initial position is individually calibrated in the X and Y directions. The initial height between each set of displacement gauges and the sample under inspection is measured. The X-direction distance ΔX, Y-direction distance ΔY, and Z-direction displacement axis height Z0 required for each inspection optical engine to achieve clear focus are obtained. ΔX, ΔY, and Z0 are used as teaching values. Next, multiple inspection optical engines scan the sample under inspection back and forth along the X-axis. Multiple displacement gauges continuously collect the surface height information of the product under inspection in real time and feed it back to their respective inspection optical engines. Based on their respective teaching values ​​and the real-time collected height of the product under inspection, the Z-direction displacement axis height is adjusted to compensate for height fluctuations on the surface of the product under inspection, thus achieving autofocus.

[0040] In one embodiment of the present invention, the detection optical engine includes a camera, a lens, and a light source.

[0041] Figure 2 This is a flowchart illustrating the multi-camera autofocus detection method provided in an embodiment of the present invention. As shown in the figure, it includes the following steps, wherein the displacement meter is fixed in the Z direction, and the detection optical mechanism can reciprocate along the X direction, specifically:

[0042] S1: The distances ΔX and ΔY between the calibrated displacement gauge and the corresponding detection optical mechanism along the X and Y directions, respectively.

[0043] S2: The displacement gauge measures the initial height H0 of the product under inspection.

[0044] S3: The height Z0 of the Z-axis displacement axis during calibration and testing of the optical engine when it is in sharp focus.

[0045] S4: Write ΔX, ΔY, H0, and Z0 as teaching values ​​into the Z-axis compensation planning module.

[0046] S5: Repeat steps S1 to S5 above to obtain the teaching values ​​of all displacement gauges and the corresponding detection optical mechanisms.

[0047] S6: Multiple inspection optical engines reciprocate along the X-axis, and multiple displacement gauges detect the height Hi of the sample to be inspected below them in real time. The Z-axis compensation planning module controls the height changes of different Z-axis displacement axes through the servo control module according to different teaching values ​​and the corresponding height Hi of the sample to be inspected, so as to compensate for the height fluctuations of the surface of the product to be inspected and achieve automatic focusing.

[0048] In one embodiment of the present invention, step S1 is specifically as follows: S1: Select a calibration plate with a Mark mark and fix it on the stage. First, move the calibration plate below the displacement meter probe and observe that the light spot of the displacement meter probe is aligned with the Mark mark. Record the X and Y axis coordinates as (x1, y1). Then, move the calibration plate below the detection optical machine corresponding to the displacement meter so that the Mark mark on the calibration plate is imaged in the center of the field of view of the detection optical machine. Record the X and Y axis coordinates (x2, y2) at this time. Then, the distances between the displacement meter probe and the detection optical machine along the X and Y axes are respectively: ΔX = x1 - x2, ΔY = y1 - y2. Step S6 is as follows: The product to be inspected is placed on the stage, and the inspection optical machine reciprocates along the X-axis or the stage drives the sample to be inspected to reciprocate along the X-axis. When the inspection optical machine moves a distance ΔX along the X-axis, the probe of the displacement gauge continuously collects the height Hi of the product to be inspected and inputs it into the Z-axis compensation planning module. Based on the collected height Hi, the Z-axis motion compensation height Zi is obtained, Zi = Z0 + H0 - Hi, and the Z-axis height adjustment trajectory is output to the servo control module. The Z-axis displacement axis is controlled by the servo control module to drive the Z-axis displacement axis to carry the corresponding inspection optical machine to the corresponding height Zi, thereby realizing real-time automatic focusing in the inspection process.

[0049] This invention employs a multi-camera combination for the inspection of large-size products, significantly reducing inspection time in practical applications. A laser displacement meter measures the height fluctuation of the inspected object, and a servo control module controls the height changes of the inspection optical engine, achieving high-speed, real-time automatic focusing during inspection. This solves the problem of image blurring caused by factors such as product warping. Furthermore, using one laser displacement meter probe paired with multiple inspection optical engines reduces the number of laser displacement meter probes required, lowering system costs. This invention achieves multi-camera automatic focusing and can be applied to high-speed, high-precision inspection of large-size products.

[0050] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-camera autofocus detection system, characterized in that, It includes a detection optical engine (2), a displacement meter (3), a Z-axis compensation planning module (4), and a servo control module (5), among which, Multiple sets of both the detection optical machine (2) and the displacement meter (3) are available. The number of displacement meters (3) is less than the number of detection optical machines (2). One set of displacement meters corresponds to one or more sets of detection optical machines. In the central area of ​​the product to be inspected, at least two sets of inspection optical machines (2) adjust the height in the Z direction based on the height information obtained by a set of displacement gauges (3). The detection optical instrument (2) and displacement meter (3) located at the edge of the product to be inspected are in a one-to-one correspondence. The displacement gauge (3) is connected to the Z-axis compensation planning module (4), which is used to detect the height of the surface of the product under inspection in the Z direction and feed the height information back to the Z-axis compensation planning module (4). The Z-axis compensation planning module (4) is connected to the servo control module (5). After receiving the Z-axis height adjustment information provided by the Z-axis compensation planning module (4), the servo control module (5) can control the detection optical engine (2) to adjust the Z-axis height as expected, so as to achieve automatic focusing. The displacement meter (3) is fixed in the Z direction, while the detection optical machine (2) can move in the X, Y and Z directions.

2. The multi-camera autofocus detection system as described in claim 1, characterized in that, It also includes a Z-axis displacement axis (6), and each detection optical machine (2) is set on an independent Z-axis displacement axis (6), which is connected to the servo control module (5).

3. The multi-camera autofocus detection system as described in claim 2, characterized in that, The displacement meter (3) is a laser displacement meter.

4. A multi-camera autofocus detection system as described in any one of claims 1-3, characterized in that, Both the servo control module (5) and the Z-axis compensation planning module (4) are PLC modules.

5. A method for multi-camera autofocus detection using the multi-camera autofocus detection system as described in any one of claims 1-4, characterized in that, First, the distance between each set of displacement gauges and the corresponding inspection optical unit located at the initial position in the X and Y directions is individually calibrated, and the initial height between each set of displacement gauges and the sample to be inspected is measured. The X-axis distance, Y-axis distance, and Z-axis displacement axis height that each set of inspection optical units needs to adjust to achieve clear focus are obtained. The X-axis distance, Y-axis distance, and Z-axis displacement axis height that need to be adjusted are used as teaching values. Then, multiple inspection optical units scan the sample to be inspected back and forth along the X-axis. Multiple displacement gauges continuously collect the surface height information of the sample to be inspected in real time and feed it back to their respective inspection optical units. Based on their respective teaching values ​​and the real-time height of the sample to be inspected, the Z-axis displacement axis height is adjusted to compensate for the height fluctuation of the sample to be inspected and achieve automatic focusing.

6. The multi-camera autofocus detection method as described in claim 5, characterized in that, The displacement gauge is fixed in the Z-axis, while the detection optical mechanism reciprocates along the X-axis. This process includes the following steps: S1: The distance between the calibrated displacement gauge and the corresponding detection optical mechanism along the X and Y directions, respectively. S2: The displacement gauge measures the initial height of the product to be inspected. S3: The height of the Z-axis displacement axis during calibration and detection of sharp focusing of the optical engine. S4: Write the spacing, the initial height, and the height of the Z-axis displacement axis as teaching values ​​into the Z-axis compensation planning module. S5: Repeat steps S1 to S5 in sequence to obtain the teaching values ​​of all displacement gauges and the corresponding detection optical mechanisms. S6: Multiple inspection optical machines reciprocate along the X-axis, and multiple displacement gauges detect the height of the product surface below them in real time. The Z-axis compensation planning module controls the height changes of different Z-axis displacement axes through the servo control module according to different teaching values ​​and the corresponding product surface height to compensate for the height fluctuations of the product surface and achieve automatic focusing.

7. The multi-camera autofocus detection method as described in claim 6, characterized in that, Step S1 is as follows: S1: Select a calibration plate with markings and fix it on the stage. First, move the calibration plate under the displacement meter and observe that the light spot of the displacement meter probe is aligned with the mark. Record the X and Y axis coordinates. Then, move the calibration plate under the detection optical machine corresponding to the displacement meter so that the mark on the calibration plate is imaged in the center of the field of view of the detection optical machine. Record the X and Y axis coordinates at this time. The distance between the displacement meter and the detection optical machine along the X and Y axes are the difference between the two X axis coordinates and the difference between the two Y axis coordinates, respectively.

8. The multi-camera autofocus detection method as described in claim 7, characterized in that, Step S6 is as follows: The product to be inspected is placed on an external stage. The inspection optical machine reciprocates along the X-axis or the stage drives the sample to be inspected to reciprocate along the X-axis. When the inspection optical machine moves a set distance along the X-axis, the displacement sensor probe continuously collects the height of the product to be inspected and inputs it into the Z-axis compensation planning module. The Z-axis motion compensation height is obtained based on the collected height, and the Z-axis height adjustment trajectory is output to the servo control module. The Z-axis displacement axis is driven by the servo control module and thus carries the corresponding inspection optical machine to the corresponding height, realizing real-time automatic focusing in the inspection process.

Images