A method and system for detecting the appearance quality of a substrate

Abstract

The present invention discloses a method and system for detecting the appearance quality of a substrate. The method includes the following steps: S1: Preset the illuminance received by the substrate to be 70 lux - 100 lux, and adjust the power of the input LED lamp; S2: The industrial camera is installed at a position symmetric to the installation position of the LED lamp with the projection of the central axis of the substrate onto the installation plane of the LED lamp as the symmetry line; S3: Collect the substrate image, and preprocess the substrate image based on photometric stereology; S4: Based on the HALCON machine vision algorithm, design image enhancement algorithms for the defective part and the stained part of the substrate respectively, which are used to detect the preprocessed substrate image to determine the surface defects and oil stains of the substrate. The advantages of the present invention are as follows: The designed inclined light source can detect conventional defects and oil stains simultaneously; The image generated by processing based on photometric stereology has low noise and rich image details; The algorithms for the surface defects and oil stains of the substrate avoid missed detection.

Description

Technical Field

[0001] The present invention relates to the technical field of ceramic substrate defect detection, and specifically to a method and system for detecting the appearance quality of a substrate. Background Art

[0002] After the ceramic substrate is processed, it is necessary to inspect its appearance quality. Defects such as chipping, burrs, defects, scratches, contamination, cracks, etc. on the substrate surface need to be detected to prevent them from being stored in the warehouse. Customers have high requirements for cracks in the substrate. During their use, after surface treatment processes such as pickling and alkali washing, tiny cracks will be exposed, which we call "hidden cracks". As an insulating material, if there are hidden cracks in the ceramic substrate, it will cause breakdown under a certain voltage, damaging the electronic components fixed on the substrate, which is not allowed by customers.

[0003] Currently, the inspection of ceramic substrates is carried out by the naked eye of workers and mechanical detection. However, some tiny apparent defects are difficult to detect by conventional naked-eye observation methods. It is necessary to take pictures with a camera with a certain magnification and then identify them by a computer through a comparison program by mechanical detection. For example, the "A Ceramic Substrate Surface Detection Projector" proposed in the published patent document (CN215768193U) proposes a method for observing substrate surface defects based on vision.

[0004] From the installation structure of the image acquisition device 4 in the "A Ceramic Substrate Surface Detection Projector", it can be seen that this method directly acquires the substrate surface image at a vertical angle, and there are still the following technical problems to be solved: Obvious defects such as chipping, burrs, defects, scratches, etc. can generally be directly detected through photo comparison. However, for some oil stain defects, the LED light source must be irradiated at a certain inclination angle and then the substrate is observed from the opposite direction of the light source to be detected. After the LED light source is tilted, the light intensity of the light irradiated on the back-illuminated object will change to a certain extent, thus having a certain impact on the detection effect. For this, a system with an LED light with a certain inclination angle and capable of adaptively adjusting the light source intensity needs to be designed to detect oil stain defects. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; for this purpose, the present invention proposes a method and system for detecting the appearance quality of a substrate, which is used to solve the technical problem that in the prior art, when only considering the detection of obvious defects, no inclined-angle light source is designed to fully detect oil stain defects.

[0006] To achieve the above object, the present invention provides a method for detecting the appearance quality of a substrate, including the following steps:

[0007] S1: Preset the illuminance received by the substrate to be 70 lux - 100 lux, and adjust the power input to the LED lamps according to the LED lamp parameters, the installation position of the LED lamps feedback by the position sensor, and the substrate area;

[0008] S2: The industrial camera is installed at a position symmetric to the projection of the substrate central axis onto the LED lamp installation plane, with respect to the symmetry line passing through the installation position of the LED lamps;

[0009] S3: Collect the substrate image and preprocess the substrate image based on photometric stereo;

[0010] S4: Based on the HALCON machine vision algorithm, design image enhancement algorithms for the defective part and the stained part of the substrate respectively, to detect the preprocessed substrate image and determine the surface defects and oil stains on the substrate;

[0011] To ensure the imaging quality of the industrial camera, preset the illuminance received by the substrate, and adjust the power of the LED lamps according to this illuminance to make the lighting environment reach an appropriate state.

[0012] Furthermore: The specific method of S1 is as follows:

[0013] The installation position of the position sensor is the same as the installation position of the LED lamps;

[0014] There is an inclination angle between the installation position of the LED lamps and the substrate installation position. Specifically: In the space coordinate system (x, y, z) with the center of the substrate as the origin and the substrate plane as the (x, y) two-dimensional plane, the inclination angle between the central coordinates of the LED lamp installation position and the central coordinates of the substrate is 45° - 75°;

[0015] The LED lamp parameters include: the number of LED lamps N, the space utilization factor CU, the maintenance factor K, and the luminous efficiency η;

[0016] The adjustment formula for the power of the LED lamps is as follows:

[0017]

[0018] Among them, E is the preset illuminance received by the substrate, which is 70 lux - 100 lux;

[0019] A is the substrate area;

[0020] θ is the inclination angle between the central coordinates of the LED lamp installation position and the central coordinates of the substrate, which is specifically observed by the position sensor.

[0021] Furthermore: The specific method of S3 is:

[0022] Collect the substrate image, determine the installation position of the industrial camera, and obtain the unit direction vectors (L1 to L4) from the four vertices of the substrate image to the industrial camera and the unit normal vectors (N1 to N4) corresponding to the four vertices T ;

[0023] Construct a pixel matrix to calculate the gradients of the four vertices in the x and y directions respectively. According to the mean values of the gradients of the four vertices, enlarge the pixels of the substrate image and perform a difference operation with the pixel values of the substrate image to obtain a low-noise acquired image, that is, the difference image;

[0024] Combine the ostu maximum inter-class variance method to binarize the difference image.

[0025] Furthermore: The specific method for calculating the gradients of the four vertices in the x and y directions is as follows:

[0026] The unit direction vectors (L1 to L4) from the four vertices to the industrial camera and the unit normal vectors (N1 to N4) corresponding to the four vertices T are respectively substituted into the pixel matrix according to the one-to-one correspondence rule. The pixel matrix is as follows:

[0027]

[0028] where k is from 1 to 4, representing the numbers of the four vertices on the substrate image;

[0029] L xk 、L yk 、L zk represent the unit direction vector L k , and the unit vectors of k ∈ [1, 4] in the x, y, and z directions;

[0030] N xk 、N yk 、N zk represent the unit normal vector N k , and the unit normal vectors of k ∈ [1, 4] in the x, y, and z directions;

[0031] Thus, the unit normal vectors (N1 to N4) corresponding to the four vertices can be obtained T , which are specifically shown by the following formula:

[0032]

[0033] where ξ is the substrate surface reflectivity, taking 92% - 98%

[0034] I is the light source intensity reflected from the substrate surface, which is shown by the following formula:

[0035] I = I0 cosθ V ;

[0036] Wherein, I0 is the light source intensity, i.e., the luminous intensity of the LED lamp;

[0037] θ is the inclination angle between the central coordinate of the installation position of the LED lamp and the central coordinate of the substrate;

[0038] V is the direction vector of the light ray incident on the center of the substrate irradiated by the LED lamp;

[0039] Finally, partial derivatives are taken for the four vertices respectively to obtain the gradients X and Y of the four vertices in the x and y directions respectively;

[0040]

[0041] Furthermore: The specific method of the Otsu maximum inter-class variance method is as follows:

[0042] The threshold t is adaptively selected. The gray values of the difference image are bounded by the threshold t. The part greater than t is set as the background class of the difference image; the part less than or equal to t is set as the target class; combining the gray levels, the number of pixels, and the gray probabilities of the background class and the target class, the background mean and the target mean corresponding to the background class and the target class are respectively obtained; the inter-class variance is solved from the background mean and the target mean;

[0043] Take the maximum value in the inter-class variance as the maximum separation degree between the target class and the background class, which is the separation degree in the HALCON machine vision algorithm.

[0044] Furthermore: The step S4 includes:

[0045] Image enhancement, specifically: enhancement for the defective part of the substrate and enhancement for the stained part of the substrate;

[0046] For the defective part of the substrate: Design a method of performing differential after two mean filters to extract defects. First, use a small template and a large template to perform mean filtering on the preprocessed substrate image to suppress noise, and then perform differential on the processed image to enhance the contrast between the scratched area and other areas;

[0047] The small template is designed to be 3*3 pixels in size, and the large template is designed to be 250*250 pixels in size;

[0048] The mean filtering process needs to process the original pixels of the preprocessed substrate image. Specifically: Replace the original pixels with the mean value of the currently selected template;

[0049] For the part of the substrate stain: Median filtering is used to reduce isolated noise and combined with features such as connected components to identify the stain area; Median filtering means that a sliding window with an odd number of points slides on the image, sorts the gray values in the window and assigns the median value to the center point; The sliding window is a circular window with a radius of 10*10 pixel size;

[0050] Among them, since the substrate defect is an obvious breakage, it is brighter than other areas and has a larger gray value. However, there is a lot of noise in the image and the gray value is also large, forming interference. Therefore, to suppress the influence of noise, two mean filtering operations are designed; The stain area is relatively dark in the image and has a relatively low gray value. Therefore, the detection method is different from that of the defect area, and there are many isolated noise points in the stain image. Therefore, median filtering is used.

[0051] The present invention also provides a system for detecting the appearance quality of a substrate, including: a terminal device;

[0052] The terminal device includes: a memory, a processor, and a program stored in the memory and executable on the processor;

[0053] The terminal device is electrically connected to a PLC programmable logic controller and an industrial camera;

[0054] The PLC programmable logic controller is electrically connected to an LED lamp to adjust the brightness of the LED lamp;

[0055] When the processor executes the program, it implements the methods and steps described in any one of the above.

[0056] The beneficial effects of the present invention are:

[0057] 1. It can adaptively adjust the power of the LED lamp so that the illuminance of the substrate is not affected by the adjustment of the position of the LED lamp, ensuring that the imaging quality of the industrial camera is not affected.

[0058] 2. The collected substrate image is first processed based on photometric stereology to reduce its noise, and also combines the advantages of bright and dark field images to generate an image with low noise and rich image details.

[0059] 3. HALCON machine vision algorithms for substrate surface defects and oil stains are designed respectively to avoid the situation of missing detection of one of them during the analysis process. Description of the Drawings

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention and the prior art, the following will briefly introduce the attached drawings required for the description of the embodiments and the prior art. Obviously, the attached drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, without creative efforts, other attached drawings can also be obtained based on these attached drawings.

[0061] Figure 1 is the system flow chart of the present invention;

[0062] Figure 2 is the system schematic diagram of the present invention. Specific Embodiments

[0063] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the following will describe and explain the present application in combination with the attached drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. Based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0064] Obviously, the attached drawings in the following description are only some examples or embodiments of the present application. For those of ordinary skill in the art, without creative efforts, the present application can also be applied to other similar scenarios based on these attached drawings.

[0065] In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for those of ordinary skill in the art related to the content disclosed in the present application, some design, manufacturing or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be understood as the content disclosed in the present application being insufficient.

[0066] If there is no special description, all embodiments and optional embodiments of the present application can be combined with each other to form a new technical solution.

[0067] Please refer to Figure 1 , the system flow chart of the present invention, which includes the following steps:

[0068] S1: Preset the illuminance received by the substrate to be 70 lux - 100 lux, and adjust the power input to the LED lamps according to the LED lamp parameters, the installation position of the LED lamps feedback by the position sensor, and the substrate area;

[0069] S2: The industrial camera is installed at: the position symmetrical to the installation position of the LED lamps with the projection of the central axis of the substrate onto the LED lamp installation plane as the symmetry line;

[0070] S3: Collect the substrate image and preprocess the substrate image based on photometric stereology;

[0071] S4: Based on the HALCON machine vision algorithm, design image enhancement algorithms for the defective part and the stained part of the substrate respectively, which are used to detect the preprocessed substrate image to determine the surface defects and oil stains of the substrate.

[0072] Specifically:

[0073] In S1, to ensure the imaging quality, the illuminance of the substrate is set to 70 lux - 100 lux in the program. Since the LED lamp needs to be set in an inclined state, to ensure that the illuminance of the substrate is within the set range, the illuminance intensity (power) of the LED lamp needs to be adjusted accordingly according to the installation angle, and the adjustment is based on the following rules: The installation position of the position sensor is the same as the installation position of the LED lamp. Establish a spatial coordinate system (x, y, z) with the center of the substrate as the origin and the substrate plane as the two-dimensional plane (x, y). The inclination angle between the center coordinates of the LED lamp installation position and the center coordinates of the substrate is between 45° and 75°;

[0074] The LED lamp parameters include: the number of LED lamps N, the space utilization factor CU, the maintenance factor K, and the luminous efficiency η;

[0075] The adjustment formula for the power of the LED lamp is as follows:

[0076]

[0077] Among them, E is the preset illuminance of the substrate, which is 70 lux - 100 lux;

[0078] A is the area of the substrate;

[0079] θ is the inclination angle between the center coordinates of the LED lamp installation position and the center coordinates of the substrate, which is observed by the position sensor.

[0080] In S2, to ensure the imaging quality, the installation position of the industrial camera also needs to be symmetrically installed with the position of the LED lamp (light source).

[0081] In S3, the position sensor determines the installation position of the industrial camera, and the direction unit vectors (L1 - L4) from the four vertices of the collected substrate image to the industrial camera and the corresponding unit normal vectors (N1 - N4) T Are respectively brought into the pixel matrix according to the one-to-one correspondence rule, and the gradients of the four vertices in the x and y directions are respectively obtained. The pixels of the substrate image are amplified according to the mean value of the gradients of the four vertices, and the difference is processed with the pixel values of the substrate image to obtain a low-noise collected image, that is, the difference image;

[0082] Furthermore, the pixel matrix is as follows:

[0083]

[0084] where k is from 1 to 4, representing the numbers of the four vertices on the substrate image;

[0085] L xk 、L yk 、L zk represent the direction unit vector L k , and the unit vectors of k ∈ [1, 4] in the x, y, and z directions;

[0086] N xk 、N yk 、N zk represent the unit normal vector N k , and the unit normal vectors of k ∈ [1, 4] in the x, y, and z directions;

[0087] Thus, the unit normal vectors (N1 to N4) corresponding to the four vertices can be obtained T , specifically shown by the following formula:

[0088]

[0089] where ξ is the reflectivity of the substrate surface, taking 92% - 98%

[0090] I is the light source intensity reflected from the substrate surface, shown by the following formula:

[0091] I = I0 cosθ V ;

[0092] where I0 is the light source intensity, that is, the light intensity of the LED lamp, obtained by looking up the table according to the LED lamp power P determined in S1;

[0093] θ is the tilt angle between the central coordinate of the LED lamp installation position and the central coordinate of the substrate;

[0094] V is the direction vector of the light ray incident on the center of the substrate by the LED lamp;

[0095] Finally, by taking partial derivatives of the four vertices respectively, the gradients X and Y of the four vertices in the x and y directions can be obtained;

[0096]

[0097] Then, combined with the Otsu maximum inter-class variance method, the difference image is binarized. The process of binarizing the difference image uses the traditional HALCON machine vision algorithm. The method selected in the specified threshold step of the traditional HALCON machine vision algorithm is the Otsu maximum inter-class variance method. The rule is as follows: The threshold t is adaptively selected. The gray values of the difference image are bounded by the threshold t. The part greater than t is set as the background class of the difference image; the part less than or equal to t is set as the target class. Combining the gray levels, the number of pixels, and the gray probabilities of the background class and the target class, the background mean and the target mean corresponding to the background class and the target class are respectively calculated. The inter-class variance is solved from the background mean and the target mean. The maximum value in the inter-class variance is taken as the maximum separation degree between the target class and the background class, which is the separation degree in the HALCON machine vision algorithm.

[0098] After preprocessing, the substrate images are respectively subjected to substrate defect analysis and substrate stain analysis. For this purpose, two different image enhancement rules are designed:

[0099] I. For the substrate defect part: Since the substrate defect is an obvious breakage, it is brighter and has a larger gray value compared to other areas. However, there is a lot of noise in the image and the gray value is also large, forming interference. Therefore, to suppress the influence of noise, a method of performing differential after two mean filters is designed to extract defects. The preprocessed substrate image is successively subjected to mean filtering with a small template and a large template to suppress noise, and then the processed image is differentiated to enhance the contrast between the scratch area and other areas. The small template is designed to be 3*3 pixels in size, and the large template is designed to be 250*250 pixels in size. The mean filtering process needs to process the original pixels of the preprocessed substrate image. Specifically: the mean value of the currently selected template replaces the original pixels;

[0100] II. For the substrate stain part: Since the stain area is relatively dark and has a relatively low gray value in the image, the detection method is different from that of the defect area. Moreover, there are many isolated noise points in the stain image. Therefore, median filtering is used to reduce the isolated noise and combined with features such as connected components to identify the stain area. Median filtering means that a sliding window containing an odd number of points slides on the image, the gray values in the window are sorted, and the median value is assigned to the center point. The sliding window is a circular window with a radius of 10*10 pixels in size.

[0101] After the above two different enhancements, it is directly observed by the HALCON machine vision algorithm to respectively determine whether there are defects or stains.

[0102] The advantages of the present invention are as follows: By designing an inclined light source, it is possible to detect both conventional defects and oil stains simultaneously, and the light source is designed as an adjustable light source to ensure imaging quality; the substrate images collected are first processed based on photometric stereology to reduce noise, and the advantages of bright and dark field images are combined to generate images with low noise and full image details; HALCON machine vision algorithms for substrate surface defects and oil stains are designed respectively to avoid the situation of missing inspection of either one during the analysis process.

[0103] The above has given a very detailed description of the application of one or more embodiments of the present invention. However, the content described is only a specific example of the present invention and cannot be considered as defining the scope of implementation of the present invention. Any other methods and changes proposed based on the content of the present invention should fall within the scope of patent protection of the present invention.

Claims

1. A method for detecting the appearance quality of a substrate, characterized in that: The steps include: S1: The illumination of the substrate is preset to be 70lux-100lux, and the surface reflectivity of the substrate is 92%-98%. The power input to the LED lamp is adjusted according to the parameters of the LED lamp, the installation position of the LED lamp fed back by the position sensor, and the substrate area. The installation position of the position sensor is consistent with the installation position of the LED lamp; There is an inclination angle between the LED lamp installation position and the substrate installation position, specifically: in a spatial coordinate system (x, y, z) with the center of the substrate as the origin and the substrate plane as the (x, y) two-dimensional plane, the inclination angle between the center coordinate of the LED lamp installation position and the center coordinate of the substrate is 45° to 75°; The LED lamp parameters include: the number of LED lamps N, the space utilization coefficient CU, the maintenance coefficient K and the luminous efficiency η; The adjustment formula of the power of the LED lamp is as follows: ； Wherein, E is the preset illumination intensity of the substrate; A is the substrate area; θ is the inclination angle between the center coordinate of the LED lamp installation position and the center coordinate of the substrate, which is specifically observed by the position sensor; S2: The industrial camera is installed at a position symmetrical to the installation position of the LED lamp, with the projection of the central axis of the substrate onto the installation plane of the LED lamp as the symmetry line; S3: collecting substrate images and preprocessing the substrate images based on photometric stereo; S4: Based on the HALCON machine vision algorithm, image enhancement algorithms are designed for the defective part of the substrate and the stain part of the substrate respectively, which are used to detect the pre-processed substrate image and determine the surface defects and oil stains of the substrate.

2. A method for detecting the appearance quality of a substrate according to claim 1, characterized in that: The specific method of S3 is: Collect the substrate image, determine the installation position of the industrial camera, and obtain the direction unit vectors (L1~L4) from the four vertices of the substrate image to the industrial camera and the unit normal vectors corresponding to the four vertices ; Construct a pixel matrix to obtain the gradients of the four vertices in the x and y directions respectively, amplify the pixels of the substrate image according to the mean value of the gradients of the four vertices, and perform a difference process with the pixel value of the substrate image to obtain a low-noise acquisition image, namely, a difference image; Combined with the ostu maximum inter-class variance method, the difference image is binarized.

3. A method for detecting the appearance quality of a substrate according to claim 2, characterized in that: The specific method for obtaining the gradients of the four vertices in the x and y directions is: The direction unit vectors (L1~L4) of the four vertices to the industrial camera and the unit normal vectors corresponding to the four vertices Substitute the pixel matrix into the pixel matrix according to the one-to-one correspondence rule, and the pixel matrix is ​​as follows: ； Wherein, k is 1 to 4, indicating the numbers of the four vertices on the substrate image; Lxk, Lyk, Lzk represent the direction unit vector Lk, k∈[1,4] in the x, y, z directions; Nxk, Nyk, Nzk represent the unit normal vector Nk, k∈[1,4] in the x, y, z directions; From this, we can get the unit normal vectors corresponding to the four vertices , which is specifically shown by the following formula: ； Where, ξ is the reflectivity of the substrate surface; I is the intensity of the light source reflected from the substrate surface, as shown in the following formula: I=I0cosθ V ; Among them, I0 is the light source intensity, that is, the light intensity of the LED lamp; θ is the inclination angle between the center coordinate of the LED lamp installation position and the center coordinate of the substrate; V is the direction vector of the light from the LED lamp incident on the center of the substrate; Finally, partial derivatives are taken for the four vertices to obtain the gradients X and Y of the four vertices in the x and y directions respectively. ； 。 4. A method for detecting the appearance quality of a substrate according to claim 2, characterized in that: The specific method of the ostu maximum inter-class variance method is: Adaptively select the threshold t, set the gray value of the difference image with the threshold t as the boundary, set the part greater than t as the background class of the difference image; set the part less than or equal to t as the target class; combine the gray level, number of pixels and gray probability of the background class and the target class to calculate the background mean and target mean corresponding to the background class and the target class respectively; solve the inter-class variance from the background mean and the target mean; The maximum value of the inter-class variance is taken as the maximum separation between the target class and the background class, which is the separation in the HALCON machine vision algorithm.

5. The method for detecting the appearance quality of a substrate according to claim 1, characterized in that: The S4, the image enhancement algorithm, specifically includes: enhancement for defective parts of the substrate and enhancement for stains on the substrate; For the defects of the substrate: a method of performing two mean filters followed by difference is designed to extract the defects. The pre-processed substrate image is subjected to mean filtering using a small template and then a large template to suppress noise. The processed image is then differentiated to enhance the contrast between the scratch area and other areas. The small template is designed as 3 3 pixels in size, the large template is designed to be 250 250 pixels in size; The mean filtering process requires processing the original pixels of the preprocessed substrate image, specifically: replacing the original pixels with the mean of the currently selected template; For the substrate stains: use median filtering to reduce isolated noise and combine connected domain features to identify the stain area; Median filtering uses a sliding window with an odd number of points to slide on the image, sort the grayscale values ​​in the window and assign its median to the center point; The sliding window is a circular window with a radius of 10 10 pixels in size.

6. A system for detecting the appearance quality of a substrate, characterized in that: include: A terminal device; The terminal device comprises: a memory, a processor, and a program stored in the memory and executable on the processor; The terminal device is electrically connected to a PLC programmable logic controller and an industrial camera; The PLC programmable logic controller is electrically connected to the LED lamp to adjust the brightness of the LED lamp; When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

Images