Image processing method and detection system for optical detection of an object to be detected with a smooth surface

Abstract

The present invention discloses an image processing method and a detection system for optical detection of an object to be detected with a smooth surface. The method is based on a stereoscopic optical method to capture and synthesize each image of the object to be detected under different lighting directions, and after capturing the image and before synthesizing the image, an image data pre-processing step is performed to perform nonlinear adjustment on the grayscale value of each pixel, so that the grayscale change ratio of multiple pixels whose grayscale values ​​before adjustment are less than the grayscale threshold can be set relative to the grayscale change ratio of the remaining pixels whose grayscale values ​​before adjustment are not less than the grayscale threshold. Thus, the detection system based on the stereoscopic optical method disclosed by the present invention can be applied to the detection of objects to be detected with smooth surfaces after the corresponding image processing, and is not limited to uniform parallel irradiation light.

Description

Technical Field

[0001] The present invention relates to an image processing method for optical detection and a detection system thereof, and more specifically, to an image processing method for optical detection of an object to be detected with a smooth surface and a detection system thereof. Background Art

[0002] Photometric Stereo Method (PSM) is an algorithm for reconstructing the surface information of an object, which is derived from the optical projection modeling method. A single camera is used to obtain multiple images of the object to be measured at the same shooting angle. The object to be measured is illuminated one by one by light from different lighting directions, and then the algorithm is used to superimpose these images to produce a synthetic image.

[0003] Traditionally, the stereo optical method uses the perfect diffusion model in optics to solve the gradient vectors on the surface of the object to be tested, and then obtains a three-dimensional model after integrating the vector field, thereby also obtaining the light intensity distribution on the surface of the object to be tested.

[0004] The stereo optical method is based on the light intensity value as the calculation basis of the overall model. Since it is not necessary to calculate all the parameter values ​​that will affect the light intensity (such as the reflection coefficient) during the solution process, good results can be obtained, so the stereo optical method is widely used.

[0005] However, since the stereo optical method requires the use of a perfect diffusion model in optics to solve, its application is limited to objects with rough surfaces and must be used with a uniform parallel light source. As a result, it cannot be used for testing objects with smooth surfaces or easily reflective properties and under non-collimated light sources. Summary of the invention

[0006] One of the objectives of the present invention is to make the stereoscopic optical method applicable to the detection of objects to be detected with smooth surfaces, and not limited to the condition of uniform parallel irradiation light.

[0007] Another object of the present invention is to make it easy to detect tiny protrusions or depressions on a metal object to be tested.

[0008] Another object of the present invention is to use a smaller detection system configuration space while improving the accuracy of defect detection of an object to be detected with a smooth surface.

[0009] In order to achieve the above-mentioned and other purposes, the present invention proposes an image processing method for optical detection of an object to be tested having a smooth surface, which comprises: based on different illumination directions of the object to be tested, sequentially obtaining original images corresponding to the illumination directions, wherein the number of the original images is at least 3; an image data pre-processing step, performing nonlinear adjustment on a pre-adjusted grayscale value of each pixel in each of the original images to generate a corresponding post-adjusted grayscale value, wherein when a grayscale threshold is less than an inversion threshold, the pre-adjusted grayscale value is adjusted. The grayscale change ratios of the multiple pixels whose grayscale values ​​are less than the grayscale threshold are all greater than the grayscale change ratios of the remaining pixels whose grayscale values ​​are not less than the grayscale threshold before adjustment. When the grayscale threshold is greater than the inversion threshold, the grayscale change ratios of the multiple pixels whose grayscale values ​​are less than the grayscale threshold before adjustment are all smaller than the grayscale change ratios of the remaining pixels whose grayscale values ​​are not less than the grayscale threshold before adjustment; and a synthesis step, based on a stereoscopic optical method, the original images processed by the image data pre-processing step are synthesized into a synthetic image for subsequent defect detection.

[0010] In one embodiment of the present invention, the above-mentioned synthesis step may further include an image data post-processing step, based on the grayscale distribution data of the synthesized image, the gradient values ​​in the corresponding gradient distribution data that are less than the first gradient threshold or greater than the second gradient threshold are all adjusted to 0, and the grayscale values ​​​​of 0 to 255 are distributed between the first gradient threshold and the second gradient threshold to generate an image to be analyzed with a new grayscale distribution for subsequent defect detection.

[0011] In one embodiment of the present invention, the absolute values ​​of the first gradient threshold and the second gradient threshold may be the same.

[0012] In one embodiment of the present invention, the first gradient threshold may be -0.5, and the second gradient threshold may be 0.5.

[0013] In one embodiment of the present invention, the grayscale value of the new grayscale distribution corresponding to the gradient value 0 may be 128.

[0014] In one embodiment of the present invention, the surface roughness Ra value of the tested area of ​​the object to be tested is less than or equal to 1.6 (μm).

[0015] In order to achieve the above and other purposes, the present invention also proposes a detection system using the aforementioned image processing method, which includes: a stage for placing an object to be detected, an imaging device disposed above the stage, a light source module disposed around the imaging device, and a control host coupled to the imaging device and the light source module. The light source module includes a plurality of light source devices, and the control host executes the aforementioned image processing method.

[0016] In one embodiment of the present invention, the control host can be used to execute a defect judgment program, and a pixel with a grayscale value greater than a convex defect threshold on the image to be analyzed is defined as a convex defect, and a pixel with a grayscale value less than a concave defect threshold on the image to be analyzed is defined as a concave defect.

[0017] In one embodiment of the present invention, the angle between the light irradiation direction of each light source device and the optical axis of the imaging device may be 0-30 degrees.

[0018] Thus, in the embodiment disclosed in the present invention, the pre-processing step of nonlinear conversion of the input image data allows the computing model based on the stereoscopic optical method to correctly process the image data from the object to be tested with a smooth surface, improve the problem of uneven brightness in the image, and make it applicable to the object to be tested that is more prone to reflection. In addition, the post-processing step of limiting and normalizing the synthesized image can also improve the problem of inconsistent image data intervals, so that tiny defects can be correctly detected.

[0019] The detection system based on the stereoscopic optical method disclosed in the present invention can be applied to the detection of objects to be detected with smooth surfaces after corresponding image processing, and is not limited to the use of uniform parallel irradiation light. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of an image processing method according to an embodiment of the present invention;

[0021] Figure 2 is a flow chart of an image processing method according to another embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of a detection system in one embodiment of the present invention; and

[0023] Figure 4 for Figure 3 Schematic diagram of the detection system from a top-down perspective. DETAILED DESCRIPTION

[0024] In order to fully understand the purpose, features and effects of the present invention, the present invention is now described in detail with the help of the following specific embodiments and the accompanying drawings, as follows:

[0025] In this document, the terms "a" or "an" are used to describe steps, components, structures, devices, modules, systems, parts or regions, etc. This is only for the convenience of explanation and to provide a general meaning for the scope of the present invention. Therefore, unless it is obvious that it is otherwise intended, such description should be understood to include one or at least one, and the singular also includes the plural.

[0026] In this document, the terms "include, include, have" or any other similar terms described herein are not limited to the elements listed herein, but may include other elements that are not explicitly listed but are generally inherent to the steps, components, structures, devices, modules, systems, parts or regions.

[0027] In this document, the words "first" or "second" and the like are used to distinguish or refer to the same or similar steps, procedures or operation data, and do not necessarily imply the temporal order of these steps, procedures or operation data. It should be understood that in some cases or configurations, the ordinal words can be used interchangeably without affecting the implementation of the present invention.

[0028] In the embodiment disclosed in the present invention, the object to be tested is imaged more than twice, and the lighting conditions during each image acquisition are different, so as to obtain multiple original images that can be calculated according to the stereo optical method. However, since the stereo optical method is more suitable for the object to be tested with a surface similar to that of Lambert, when the surface contour of the object to be tested is relatively smooth and prone to reflection, the original image obtained will be difficult to directly apply to the calculation model of the stereo optical method. Therefore, in order to use the advantages of the stereo optical method, in the embodiment disclosed in the present invention, the original image obtained is pre-processed so that the image grayscale on the original image can be applied to the calculation model of the stereo optical method, and then after the stereo optical method is used for image synthesis, the image data can be further normalized based on the defect detection requirements of the object to be tested with a relatively smooth surface, thereby generating an image to be analyzed, so as to enhance the consistency of the image data interval, and then make the defect area on the image to be analyzed easily identifiable, thereby improving the accuracy of the back-end defect detection.

[0029] The degree of profile change of the surface of the object to be measured can be presented by means of surface roughness (measured according to the specification of ISO 13565). For example, it can be represented by the arithmetic mean roughness Ra value. When the Ra value is less than or equal to 1.6 (μm), it is difficult to directly apply it to the calculation model of the stereoscopic optical method due to the low degree of diffusion. The pre-processing disclosed in the embodiment of this case can be used to solve this problem.

[0030] Please refer to Figure 1 , is a flow chart of an image processing method in one embodiment of the present invention. First, step S100 is performed to obtain each original image under different lighting directions; then step S200 is performed to pre-process the image data (non-linear adjustment); then step S300 is performed to synthesize the image based on the stereoscopic optical method, so that each original image is synthesized into a synthetic image for subsequent defect detection.

[0031] In step S100, the inspected surface of the object to be inspected is imaged under illumination from different illumination directions. For example, illumination from three different illumination directions is sequentially used to obtain original images corresponding to each illumination direction. A preferred implementation is to use illumination from more than three different illumination directions to correspondingly capture more than three original images.

[0032] In step S200, nonlinear adjustment is to process the grayscale value of the image. That is, a pre-adjustment grayscale value G1 of each pixel in each original image is nonlinearly adjusted, and then a post-adjustment grayscale value G2 is generated accordingly. The ratio of the pre-adjustment grayscale value to the post-adjustment grayscale value G2 / G1 is the grayscale change ratio.

[0033] The nonlinear adjustment in step S200 is to use a grayscale threshold to distinguish the difference between the values ​​of the two regions. A grayscale value G1 before adjustment corresponds to the grayscale performance of a pixel, and the grayscale values ​​of a pixel before and after adjustment have a ratio G2 / G1. The grayscale threshold uses the grayscale value G1 before adjustment as the numerical position for distinction, and the grayscale change ratio is selected to distinguish the conditions for two regions. For a smooth surface of the object to be tested, for example, the grayscale threshold (grayscale value G1 before adjustment) can be selected between 50 and 60, and in other cases it may be set at 170 to 180. The grayscale threshold can be defined according to the unacceptable degree of the defect point and the average grayscale value of the defect point and its surroundings. For example, if the unacceptable degree of the defect point and the average grayscale value of its surroundings are set to 40, the grayscale threshold can be set to a position greater than 40 (for example: 50); in addition, for another example, if the unacceptable degree of the defect point and the average grayscale value of its surroundings are set to 180, the grayscale threshold can be set to a position less than 180 (for example: 170).

[0034] In the implementation method of the dissimilarity adjustment of the values ​​of the two regions, the numerical region of the grayscale value G1 before adjustment that is less than the grayscale threshold is called the first region, and the numerical region of the grayscale value G1 before adjustment that is greater than the grayscale threshold is called the second region. When the grayscale threshold is less than an inversion threshold, the nonlinear adjustment refers to the adjustment rule that makes the grayscale change ratio G2 / G1 of the first region greater than the grayscale change ratio G2 / G1 of the second region; on the contrary, when the grayscale threshold is greater than the inversion threshold, the nonlinear adjustment refers to the adjustment rule that makes the grayscale change ratio G2 / G1 of the first region less than the grayscale change ratio G2 / G1 of the second region. That is, based on this adjustment rule, the object to be tested with a smooth surface can be applied to the calculation model of the stereoscopic optical method. The original image adjusted in step S200 can be synthesized by the subsequent synthesis step. The inversion threshold can define the boundary position of the two opposite nonlinear adjustment methods mentioned above, so as to determine which adjustment rule of the nonlinear adjustment method a grayscale threshold should correspond to. The reversal threshold can be set to any value between 70 and 80.

[0035] In step S300, a synthetic image is generated based on the computing model of the stereoscopic optical method. In this step, the surface normal vector of each input original image is calculated as a whole, and based on this, the orthogonal vector perpendicular to the surface normal vector is further calculated, and a depth image is obtained through the integration of the vector field. This depth image is a synthetic image that can be used for subsequent defect detection after synthesis. Among them, the computing model of the stereoscopic optical method is a well-known computing processing technology, which is based on the light intensity value as the calculation basis of the overall model, and its computing details are not repeated here.

[0036] Please refer to Figure 2 , is a flow chart of an image processing method in another embodiment of the present invention. Figure 1 The embodiment further comprises an image data post-processing step S400. Step S400 is to normalize the image data and further generate an image to be analyzed to enhance the consistency of the image data interval and improve the accuracy of subsequent defect judgment.

[0037] Step S400 may include: S410 a step of converting gray value distribution into gradient value distribution, S420 a step of adjusting the gradient value, and S430 a step of converting the gradient value distribution back into gray value distribution.

[0038] Step S410 is to convert the grayscale distribution data corresponding to each pixel of the synthetic image into a gradient value. The gradient value refers to the degree of change of the grayscale values ​​of adjacent pixels. Therefore, this step is to calculate the degree of change of the grayscale values ​​of each pixel arranged in sequence according to the sorting direction of the row or column on the image data, that is, to calculate the degree of change with the gradient value of the previous sorting in the sequence, and then obtain the gradient distribution data in the sorting direction. The value of the gradient distribution data will be between -1 and 1.

[0039] Step S420 refers to selecting the gradient values ​​less than the first gradient threshold or greater than the second gradient threshold in the gradient distribution data, and setting these selected gradient values ​​to 0. This step is a limited normalization process. Adding a limited process to the gradient distribution data can further help highlight the defects. The first gradient threshold and the second gradient threshold can be two thresholds that are equal to the same value after taking the absolute value. For example, for the measured area with an arithmetic mean roughness Ra value less than or equal to 1.6 (μm) on the surface of the object to be measured, the first gradient threshold can be selected as -0.5, and the second gradient threshold can be selected as 0.5. Further, it can even be applicable to the measured area with an arithmetic mean roughness Ra value less than or equal to 0.8 (μm).

[0040] Step S430 is to assign grayscale values ​​of 0 to 255 to the adjusted gradient distribution data so that it can be converted back to grayscale distribution data. This step is equivalent to the reverse process of step S410.

[0041] Gray values ​​of 0 to 255 are distributed between the first gradient threshold and the second gradient threshold to generate an image to be analyzed with a new gray distribution for subsequent defect detection. Since the gradient values ​​in the range outside the first gradient threshold and the second gradient threshold are all 0, step S430 is equivalent to distributing the gray values ​​of 0 to 255 between the first gradient threshold and the second gradient threshold to generate an image to be analyzed with a new gray distribution. The new gray value corresponding to the gradient value 0 can be set to 128 as the basis for conversion.

[0042] Please also refer to Figure 3 and Figure 4 , Figure 3 A schematic diagram of a detection system in one embodiment of the present invention; Figure 4 for Figure 3The schematic diagram of the top view of the detection system can show the relative positions of the light source devices 31 and between the light source device 31 and the imaging device 20. The detection system includes: a carrier 10, an imaging device 20, a light source module 30, and a control host 40. The carrier 10 can be used to place the object 50 to be detected. The imaging device 20 is arranged above the carrier 10. The light source module 30 has a plurality of light source devices 31 arranged around the imaging device 20. The control host 40 is coupled to the imaging device 20 and the light source module 30 to control the imaging device 20 and the light source module 30 and execute the aforementioned image processing method.

[0043] Under the harsh conditions of some machines, the space available for configuring the detection system may be quite limited, especially the imaging device 20 and the light source module 30 must be compactly arranged together and must be located above the carrier 10. In fact, the imaging device 20 must be located directly above the carrier 10, and the light source module 30 must be adjacent to the imaging device 20 to further reduce the occupied space.

[0044] Therefore, under these narrow space conditions, it is not possible to configure the image capturing device 20 and the light source module 30 as generally used for smooth surface detection, and there is no way to respectively configure the image capturing device 20 and the light source module 30 on two opposite sides of the stage 10 or even change their relative positions so that the reflected light can easily enter the image capturing device 20. Accordingly, the application of the known stereo photometry method and the image processing method disclosed in the present invention to adjust the relevant image data can solve these technical problems, and make it easy to detect the tiny defects of the protrusions or depressions on the metal material to be tested, and also improve the accuracy of defect detection of the object to be tested with a smooth surface.

[0045] The control host 40 can further execute a defect judgment program. When the grayscale value of the pixels on the image to be analyzed is greater than a convex defect threshold, these pixels can be defined as convex defects. In addition, when the grayscale value of the pixels on the image to be analyzed is less than a concave defect threshold, these pixels can be defined as concave defects. Based on the grayscale value performance of each pixel after adjustment on the image, and with the help of the setting of the defect threshold, the defect position and condition can be judged on the smoother surface of the object to be tested. In general, the convex defect threshold of the smoother surface of the object to be tested can be set to any value between 155 and 160, and the concave defect threshold can be set to any value between 95 and 100. In other cases, the grayscale value performance of each pixel after adjustment on the image can also be used to set the concave defect threshold and the convex defect threshold accordingly according to the degree of concavity and convexity to be detected (more stringent or looser conditions). Therefore, even when the angle θ between the average light irradiation direction of each light source device 31 and the optical axis of the imaging device 20 is only within 30 degrees, or even between 0 and 10 degrees, the image processing method disclosed in the present invention can still be used to adjust the relevant image data and highlight the concave or convex defect area.

[0046] In summary, the detection system based on the stereoscopic optical method disclosed in the present invention can be applied to the surface defect detection of the object to be tested with a smooth surface after the corresponding image processing, and is not limited to uniform parallel irradiation light, thereby solving the problem that the stereoscopic optical method is limited to the application object having a rough surface or a non-reflective surface.

[0047] The present invention discloses preferred embodiments above, but those skilled in the art should understand that the embodiments are only used to describe the present invention and should not be interpreted as limiting the scope of the present invention. It should be noted that all changes and substitutions equivalent to the embodiments should be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be based on the scope of the patent application.

[0048] Reference numerals

[0049] 10. Stage

[0050] 20 Imaging device

[0051] 30 Light source module

[0052] 31 Light source device

[0053] 40 Control Host

[0054] 50 Objects to be tested

[0055] S100~S400 steps

[0056] Steps S410 to S430

[0057] θ Angle.

Claims

1. A method for image processing of an object to be tested having a smooth surface during optical detection, comprising: Based on different illumination directions of the object to be tested, original images corresponding to the illumination directions are sequentially obtained, and the number of the original images is at least 3; An image data pre-processing step is to perform nonlinear adjustment on a pre-adjustment grayscale value of each pixel in each original image to generate a corresponding post-adjustment grayscale value, wherein: When a grayscale threshold is less than an inversion threshold, the grayscale change ratios of the multiple pixels whose grayscale values ​​before adjustment are less than the grayscale threshold are all greater than the grayscale change ratios of the remaining pixels whose grayscale values ​​before adjustment are not less than the grayscale threshold; when the grayscale threshold is greater than the inversion threshold, the grayscale change ratios of the multiple pixels whose grayscale values ​​before adjustment are less than the grayscale threshold are all less than the grayscale change ratios of the remaining pixels whose grayscale values ​​before adjustment are not less than the grayscale threshold; and A synthesis step is performed to synthesize the original images processed by the image data pre-processing step into a synthetic image for subsequent defect detection based on a stereoscopic optical method.

2. The image processing method according to claim 1, wherein: After the synthesis step, an image data post-processing step is further included. Based on the grayscale distribution data of the synthesized image, the gradient values ​​in the corresponding gradient distribution data that are less than the first gradient threshold or greater than the second gradient threshold are all adjusted to 0, and the grayscale values ​​​​of 0 to 255 are distributed between the first gradient threshold and the second gradient threshold to generate an image to be analyzed with a new grayscale distribution for subsequent defect detection.

3. The image processing method according to claim 2, wherein: The absolute value of the first gradient threshold is the same as that of the second gradient threshold.

4. The image processing method according to claim 2, wherein: The first gradient threshold is -0.5, and the second gradient threshold is 0.

5.

5. The image processing method according to claim 4, wherein: The grayscale value of the new grayscale distribution corresponding to the gradient value 0 is 128.

6. The image processing method according to any one of claims 1 to 5, wherein: The surface roughness Ra value of the tested area of ​​the object to be tested is less than or equal to 1.6 μm.

7. A detection system using the image processing method according to any one of claims 1 to 6, comprising: A carrier for placing an object to be tested; An imaging device, disposed above the stage; a light source module, comprising a plurality of light source devices arranged around the imaging device; and A control host is coupled to the imaging device and the light source module, and the control host executes the image processing method according to any one of claims 1 to 6.

8. The detection system according to claim 7, wherein: The angle between the light irradiation direction of each light source device and the optical axis of the imaging device is 0 to 30 degrees.

9. A detection system using the image processing method according to claim 5, comprising: A carrier for placing an object to be tested; An imaging device, disposed above the stage; a light source module, comprising a plurality of light source devices arranged around the imaging device; and A control host is coupled to the imaging device and the light source module, and the control host executes the image processing method as claimed in claim 5, wherein: The control host is further used to execute a defect judgment program to define a pixel on the image to be analyzed whose grayscale value is greater than a convex defect threshold as a convex defect, and to define a pixel on the image to be analyzed whose grayscale value is less than a concave defect threshold as a concave defect.

10. The detection system according to claim 9, wherein: The angle between the light irradiation direction of each light source device and the optical axis of the imaging device is 0 to 30 degrees.

Images