An AOI defect detection method and system based on industrial vision

Abstract

The present application relates to the technical field of image processing, more particularly, the present application relates to an AOI defect detection method and system based on industrial vision, comprising: acquiring a gray scale image of an electronic screen to be measured; determining the physical period of pixel arrangement of the electronic screen; constructing a local period difference degree, which includes calculating the gradient difference value between any pixel point in the gray scale image and its adjacent pixel point along the physical period. The present application quantifies the degree of dirtiness of the image by calculating the local period difference degree and constructing a period confusion factor combined with statistical characteristics. On this basis, the ADMM acceleration coefficient is dynamically generated: a conservative step size is used for low-quality images to prevent algorithm divergence, and an aggressive step size is used for high-quality images to improve efficiency. This method effectively solves the problem that the traditional fixed step size strategy cannot balance detection speed and accuracy, significantly improving the robustness of the AOI system.

Description

Technical Field

[0001] This invention relates to the field of image processing technology. More specifically, this invention relates to an AOI defect detection method and system based on industrial vision. Background Technology

[0002] With the rapid development of display technology, the requirements for surface defect detection in the manufacturing process of electronic screens (such as OLED and LCD panels) are becoming increasingly stringent. Automated Optical Inspection (AOI) systems based on industrial vision are widely used to detect various defects (such as murmurs and dot defects) in screen production. Since electronic screen pixels typically exhibit a highly periodic arrangement, Tensor Robust Principal Component Analysis (TRPCA) has become one of the mainstream processing methods for images with complex periodic texture backgrounds. This is because it can utilize the low-rank and sparsity of image data to decompose the image into a low-rank background tensor and a sparse defect tensor. This algorithm typically employs an alternating direction multiplier method (ADMM) solver for iterative solution to separate potential defect regions.

[0003] However, traditional TRPCA algorithms typically employ a fixed step size strategy to update penalty parameters when using the ADMM solver for iterative solutions. This fixed parameter update method ignores the quality differences between the screen images being tested. In real-world inspection scenarios, the background complexity and noise levels of screen images vary across different batches or processes. For high-quality images with strong low-rank properties, a fixed step size often leads to slow algorithm convergence, severely hindering detection efficiency. Conversely, for images with complex background textures or process noise, a fixed step size is too aggressive, easily causing the algorithm to fail to converge or even diverge, and readily misjudging complex background textures as defects, thus failing to balance detection speed and accuracy. Summary of the Invention

[0004] This invention provides an AOI defect detection method and system based on industrial vision, aiming to solve the problem that fixed parameter update methods in related technologies ignore the quality differences between the images of the electronic screen under test.

[0005] In a first aspect, the present invention provides an AOI defect detection method based on industrial vision, comprising: acquiring a grayscale image of an electronic screen to be tested; determining the physical period of the pixel arrangement of the electronic screen; constructing a local periodicity difference, the calculation of which includes acquiring the gradient difference between any pixel in the grayscale image and its neighboring pixels along the physical period, and acquiring the gradient standard deviation in the neighborhood of the arbitrary pixel, wherein the local periodicity difference is proportional to the product of the gradient difference and the gradient standard deviation; performing threshold segmentation on the local periodicity difference to obtain a first subset and a second subset, and performing a weighted summation of the number of pixels in the first subset and the second subset to obtain the periodicity disorder factor; constructing an ADMM acceleration coefficient, wherein the ADMM acceleration coefficient is inversely proportional to the periodicity disorder factor; inputting the grayscale image into a tensor robust principal component analysis algorithm model, and using an ADMM solver to iteratively solve the algorithm model, wherein, during the iterative solution process, the penalty parameter of the ADMM solver is updated according to the ADMM acceleration coefficient; and acquiring defects in the electronic screen image based on the solution result of the algorithm model. Compared to the fixed step size strategy commonly used in existing technologies, this method can automatically increase the step size to improve detection efficiency for high-quality, low-rank images while ensuring algorithm convergence, and automatically decrease the step size for conservative iteration to prevent algorithm divergence or misjudgment for images with complex textures or high noise. Thus, it balances the speed and accuracy of defect detection for images of different quality in the same system, effectively solving the problem that traditional algorithms are difficult to adapt to the changing image quality in industrial settings.

[0006] Furthermore, the formula for calculating the local periodicity difference is as follows: In the formula, Represents the coordinates of the image on the electronic screen. Local periodicity at the location; and These represent the coordinates of the image on the electronic screen. and coordinates The gradient value at that point; Indicates the physical period of pixel arrangement; Represented in pixels The standard deviation of the gradient within the 8-neighborhood of the center; This represents the global mean of the gradient across the entire image. Calculating local periodic differences using a specific formula not only utilizes the gradient difference between adjacent physical period pixels to characterize the consistency changes in microstructure, but also introduces the gradient standard deviation within the pixel neighborhood as a weighting adjustment factor. This effectively amplifies the difference values ​​in areas that neither conform to periodic patterns nor possess extremely complex textures, thus more sensitively capturing periodic damage caused by process errors, optical distortion, or minor defects. This provides a high signal-to-noise ratio quantitative basis for accurately assessing the overall image's disorder level.

[0007] Furthermore, the method for determining the physical period includes: acquiring the grayscale sequence of any column of pixels in the grayscale image, calculating the autocorrelation function of the grayscale sequence, and determining the step size corresponding to the peak value of the autocorrelation function as the physical period. Compared to manually preset fixed periods or simple geometric measurements, this method can automatically and accurately calculate the actual physical period step size based on the statistical characteristics of the image data itself. This effectively overcomes the small errors in pixel spacing caused by slight changes in shooting distance, angle, or different screen batches, ensuring the accuracy of subsequent gradient difference calculations based on the period.

[0008] Furthermore, the local periodicity difference is segmented using a threshold method, wherein the threshold segmentation method can adaptively determine the segmentation threshold using the Otsu method.

[0009] Furthermore, the first weight ranges from 0.35 to 0.45, and the second weight ranges from 0.55 to 0.65. By assigning a higher weight to the first subset, the calculation results focus on reflecting the degree of severe damage or contamination in the image, while also taking into account the overall background noise level, thus obtaining a precise quantitative index that can comprehensively characterize the overall image's level of dirtiness and the difficulty of algorithm processing.

[0010] Furthermore, the formula for calculating the ADMM acceleration factor is as follows: In the formula, This represents the speedup factor ultimately used for algorithm iteration; Indicates the basic growth rate; This is the gain adjustment constant; The periodic disorder factor for electronic screen images; To prevent the smoothing constant from being zero in the denominator, an inverse proportional function model is used to construct the ADMM acceleration coefficient. A direct nonlinear mapping relationship is established between the image periodic disorder factor and the algorithm iteration step size. This ensures that when the image quality is poor (large disorder factor), the algorithm automatically reduces the acceleration coefficient to guarantee convergence stability, while significantly increasing the acceleration coefficient to shorten the computation time when the image quality is good. Furthermore, by introducing a smoothing constant and a basic growth rate, the case of a zero denominator is prevented, and the basic convergence performance of the algorithm is guaranteed.

[0011] Furthermore, the penalty parameters of the ADMM solver are updated by multiplying the penalty parameters of the current iteration of the ADMM solver by the ADMM acceleration coefficient to obtain the penalty parameters for the next iteration. By directly using the ADMM acceleration coefficient, calculated in real-time based on image quality, as a multiplier to update the penalty parameters of the next iteration, dynamic adaptive adjustment of the step size is achieved during the solution process. This allows the TRPCA algorithm to appropriately plan the optimization path for the quality status of each specific image, avoiding the phenomenon of misclassifying the background as a defect or the iteration failing to converge that easily occurs when dealing with complex textured backgrounds using fixed parameter update methods.

[0012] Furthermore, before acquiring the grayscale image of the electronic screen under test, the method further includes: scanning and imaging the electronic screen under test using a linear CCD camera to obtain a color image; performing bilateral filtering or guided filtering on the color image; cropping the filtered color image to remove its non-display areas; and converting the cropped color image into a grayscale image. Compared to directly processing the original image, this invention can effectively remove electromagnetic interference and Gaussian noise commonly found in industrial environments while fully preserving the micron-level RGB sub-pixel texture details of the screen. Combined with the cropping of non-display areas, this significantly improves the signal-to-noise ratio of the input data and reduces unnecessary computation, laying a high-quality data foundation for subsequent high-precision tensor decomposition.

[0013] Furthermore, the process of obtaining defects in the electronic screen image includes: based on the final sparse tensor output by the algorithm model, performing maximum value projection on each channel of the sparse tensor to obtain a two-dimensional defect saliency map. An adaptive threshold segmentation method is used to binarize the defect saliency map to obtain a mask of potential defects, thereby obtaining the defects in the electronic screen image. By fusing the multi-channel sparse tensors into a two-dimensional defect saliency map through maximum value projection and combining it with adaptive threshold segmentation to generate a mask, the process can effectively filter out residual background noise from the sparse tensor results, accurately locate and extract the true defect morphology, and achieve an automated closed loop from data processing results to visualized defect output.

[0014] In a second aspect, an AOI defect detection system based on industrial vision is also provided, including a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the AOI defect detection method based on industrial vision described in any of the above embodiments.

[0015] Beneficial Effects: Addressing the issue of inconsistent texture complexity and quality in electronic screen images, this invention quantifies the degree of image scragginess by calculating local periodicity differences and combining them with statistical characteristics to construct a periodic disorder factor. Based on this, ADMM acceleration coefficients are dynamically generated: a conservative step size is used for low-quality images to prevent algorithm divergence, while an aggressive step size is used for high-quality images to improve efficiency. This method effectively solves the problem that traditional fixed-step-size strategies cannot balance detection speed and accuracy, significantly improving the robustness of the AOI system. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating a defect detection flowchart according to an embodiment of the present invention; Figure 2 This is a schematic illustration of an electronic screen image to be detected according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the periodic difference diagram according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating significant defects according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating defect detection according to an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] like Figures 1 to 5 As shown, S101: Image acquisition and preprocessing.

[0019] In this embodiment, a high-precision image acquisition environment is first established by illuminating the electronic screen to be inspected (e.g., an OLED or LCD panel) and placing it on a precision inspection platform. To avoid parallax and distortion caused by traditional industrial lenses, a high-resolution linear CCD camera combined with dual telecentric lenses is preferably used for scanning imaging. As the conveyor belt moves at a constant speed, the linear camera scans the screen line by line to acquire a complete high-resolution color image of the electronic screen containing the R, G, and B channels.

[0020] Considering the electromagnetic interference and light source fluctuations of the backlight module in industrial environments, the original image often carries Gaussian noise. In order to remove noise without destroying the micron-level texture details of the screen (such as the RGB subpixel arrangement), this step preferably uses bilateral filtering or guided filtering to process the image.

[0021] For example, after denoising is completed, a preset masking algorithm is used to identify the effective display area in the image, and the non-display areas (such as the border glue area and FPC ribbon cable area) are cropped and removed, leaving only the image of the test area containing the pixel array. Then, the image is converted into a grayscale image using a weighted average method to reduce the data dimension of subsequent tensor calculations.

[0022] S102: Construct local periodicity differences.

[0023] The pixels of an electronic screen exhibit a highly periodic arrangement at the microscopic level. To quantify the deviation of the actual image texture from the ideal periodic texture, this step constructs a local periodicity difference.

[0024] First, determine the physical period of the pixel arrangement on the electronic screen. Obtain any column of pixel grayscale sequence from the preprocessed image and calculate the autocorrelation function of that sequence. Specifically, using... The integers within the range are the step size ( (Given the image height), find the step size corresponding to the peak value of the autocorrelation function; this step size is the physical period of that column. Calculate the mode of the physical periods of all columns in the entire image and determine it as the physical period of the pixel arrangement on the electronic screen. .

[0025] Subsequently, the Sobel operator or the Prewitt operator is used to calculate the gradient maps of the image in the horizontal and vertical directions, respectively, in order to eliminate the influence of low-frequency background brightness unevenness and focus on high-frequency textures.

[0026] Based on the above parameters, a local periodicity difference is constructed. The calculation formula is based on: using the gradient difference between adjacent periodic pixels to represent structural differences, and using the dispersion of the local gradient as a weighting factor to amplify the difference value in complex texture regions. The specific formula is as follows: In the formula, Represents the coordinates of the image on the electronic screen. Local periodicity at the location; and These represent the coordinates of the image on the electronic screen. and coordinates The gradient value at that point; Indicates the physical period of pixel arrangement; Represented in pixels The standard deviation of the gradient within the 8-neighborhood of the center; This represents the global mean of the gradient across the entire image.

[0027] From this formula, we can see that when When the value increases, it indicates a deterioration in the structural consistency between adjacent periods; while when the term... An increase in the value indicates significant local texture fluctuations in the region, such as the presence of Moire patterns or complex film particles. The product of these two factors effectively amplifies the local periodicity differences in areas that neither conform to periodicity nor possess highly complex textures. Therefore, when manufacturing errors in electronic screens are large, or when optical distortion causes texture deformation due to different locations, the unique characteristic of the data is a decrease in periodic consistency. The greater the local periodicity difference calculated by the corresponding formula, the more accurate the subsequent algorithm's assessment of processing difficulty.

[0028] S103: Construct a periodic disorder factor.

[0029] To further distinguish between random disturbances caused by sensor thermal noise and structural disorder caused by poor manufacturing processes, this step calculates the global periodic disorder factor based on the local periodicity difference.

[0030] First, the feature map formed by the local periodicity differences of each pixel obtained in the above steps is denoted as the periodicity difference map. This map is used as the input to the Otsu thresholding method, and the Otsu method is used to adaptively calculate the segmentation threshold. Pixels with values ​​greater than the threshold are defined as the first subset; pixels with values ​​less than or equal to the threshold are defined as the second subset. Based on the statistical characteristics of these two subsets, a periodicity disorder factor is constructed, as shown in the following formula: ;In the formula, The periodic disorder factor represents the image on the electronic screen. and Let these represent the sums of the local periodicity differences of all pixels in the first and second subsets, respectively. and Let represent the mean of the local periodicity difference of all pixels in the first subset and the second subset, respectively. and These represent the first weight and the second weight, respectively. The value range of the first weight is 0.35 to 0.45, and the value range of the second weight is 0.55 to 0.65. In this embodiment, the first weight and the second weight are 0.4 and 0.6, respectively, and can be selected according to the actual situation.

[0031] In the above formula, the first term This term focuses on reflecting the total amount of damaged areas in an image; it increases significantly as the number of highly disordered pixels increases. The second term... This factor focuses on reflecting the intensity of the damage. Even if the damaged area is small, if the local periodicity of the area is extremely high (such as severe mura), this factor will increase significantly. Through weighted summation, the periodicity disorder factor can comprehensively quantify the overall messiness of the image. The smaller the value of the periodicity disorder factor, the cleaner and more regular the screen background; the larger the value of the periodicity disorder factor, the more noisy and complex the screen background.

[0032] S104: Construct: ADMM acceleration factor.

[0033] To address the slow convergence of simple samples and the tendency to diverge on complex samples caused by the fixed step size in the traditional TRPCA algorithm, this embodiment establishes an ADMM acceleration strategy based on adaptive image quality adjustment. The reason for this is that when the TRPCA algorithm uses the ADMM solver for iteration, the core lies in the penalty parameter. The update strategy is as follows. Existing algorithms typically use a fixed speedup factor. However, for electronic screen detection, if the image quality is good and the low-rank property is strong, a significantly faster update strategy can be adopted. A higher value can aggressively accelerate convergence, thus significantly reducing the time; conversely, if the image quality is poor, a smaller value must be used. The values ​​are conservatively iterated to prevent misclassifying the background as a defect. Therefore, it is necessary to construct an ADMM acceleration coefficient that can comprehensively reflect the image processing difficulty and directly guide the solver step size.

[0034] The construction method involves using an inverse proportional function model to map the degree of image disorder, i.e., the periodic disorder factor, to the iteration step size multiplier of the algorithm. Specifically, this involves constructing the acceleration coefficient. The formula is as follows: In the formula, This represents the speedup factor ultimately used for algorithm iteration; This represents the basic growth rate, typically set to 1.05, to ensure the algorithm's most basic convergence. This is a gain adjustment constant used to control the acceleration sensitivity. It is set according to the camera bit depth; for example, it can be 0.5 for an 8-bit image. The periodic disorder factor for electronic screen images; To prevent the smoothing constant from having a denominator of zero, it is set to 1 in this embodiment.

[0035] S105: Defect detection based on improved algorithm.

[0036] The acceleration coefficient obtained using the above steps An improvement is made to the ADMM solver in the traditional TRPCA algorithm. This is achieved by constructing the three-dimensional tensor of the image. Then, initialize the low-rank tensor. sparse tensors and Lagrange multipliers After entering the iteration loop, the penalty parameter is updated. In the steps, the traditional fixed parameters are abandoned, and they are substituted into the formula. .in, This refers to the specific acceleration coefficient calculated for the current image under test. During the iteration process, the low-rank background is updated using Tensor Singular Value Decomposition (t-SVD). Update sparse foreground using soft threshold operator Convergence condition: When the reconstruction error... When the number of iterations falls below a preset threshold or reaches the maximum number of iterations, the iteration stops, and the final sparse tensor is output. .

[0037] Based on the sparse tensor output by the improved algorithm Defect detection is achieved. The sparse tensor contains all anomalous signals separated from the background, but may still contain a small amount of residual noise. Maximum projection is performed on each channel of the sparse tensor to obtain a two-dimensional defect saliency map. An adaptive thresholding method (such as the Otsu algorithm) is used to binarize the defect saliency map, obtaining a mask of potential defects, thus completing the automated quality inspection of electronic screens.

[0038] The present invention also provides an AOI defect detection system based on industrial vision. The system includes a processor and a memory, the memory storing computer program instructions. When the processor executes the computer program instructions, it implements the AOI defect detection method based on industrial vision according to the first aspect of the present invention.

[0039] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and therefore will not be described in detail here.

[0040] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions stored or otherwise maintained on such a computer-readable medium.

[0041] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. An AOI defect detection method based on industrial vision, characterized in that, include: Acquire a grayscale image of the electronic screen under test; Determine the physical period of the pixel arrangement of the electronic screen; construct the local period difference, the calculation of which includes obtaining the gradient difference between any pixel in the grayscale image and its neighboring pixels along the physical period, and obtaining the gradient standard deviation in the neighborhood of any pixel. The local period difference is proportional to the product of the gradient difference and the gradient standard deviation. The local periodicity difference is divided by a threshold to obtain a first subset and a second subset. The sum of the sum of the local periodicity differences in the first subset divided by the mean and the sum of the sum of the local periodicity differences in the second subset divided by the mean are used as the periodic disorder factor. An ADMM acceleration coefficient is constructed, which is inversely proportional to the periodic disorder factor; a grayscale image is input into a tensor robust principal component analysis algorithm model, and the ADMM solver is used to iteratively solve the algorithm model. During the iterative solution process, the penalty parameter of the ADMM solver is updated according to the ADMM acceleration coefficient; based on the solution results of the algorithm model, defects in the electronic screen image are obtained. The formula for calculating the local periodicity difference is: ； In the formula, Represents the coordinates of the image on the electronic screen. Local periodicity at the location; and These represent the coordinates of the image on the electronic screen. and coordinates The gradient value at that point; Indicates the physical period of pixel arrangement; Represented in pixels The standard deviation of the gradient within the 8-neighborhood of the center; This represents the global mean of the gradient across the entire image.

2. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, Methods for determining physical periods include: Obtain the grayscale sequence of any column of pixels in the grayscale image, calculate the autocorrelation function of the grayscale sequence, and determine the step size corresponding to the peak value of the autocorrelation function as the physical period.

3. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, The local periodicity difference is segmented by a threshold, wherein the threshold segmentation method uses the Otsu method to adaptively determine the segmentation threshold.

4. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, The first weight has a value range of 0.35 to 0.45, and the second weight has a value range of 0.55 to 0.

65.

5. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, The formula for calculating the ADMM acceleration coefficient is: In the formula, This represents the speedup factor ultimately used for algorithm iteration; Indicates the basic growth rate; This is the gain adjustment constant; The periodic disorder factor for electronic screen images; To prevent smoothing constants with denominators of zero.

6. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, Update the penalty parameters of the ADMM solver, including: The penalty parameter for the current iteration of the ADMM solver is multiplied by the ADMM acceleration coefficient to obtain the penalty parameter for the next iteration.

7. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, Before acquiring the grayscale image of the electronic screen under test, the method further includes: A linear CCD camera is used to scan and image the electronic screen under test to obtain a color image; the color image is then subjected to bilateral filtering or guided filtering; the filtered color image is cropped to remove its non-display areas; and the cropped color image is converted into a grayscale image.

8. The AOI defect detection method based on industrial vision according to claim 1, characterized in that, Defects in electronic screen images include: Based on the algorithm model, the final sparse tensor is output. The maximum value projection is performed on each channel of the sparse tensor to obtain a two-dimensional defect saliency map. The defect saliency map is binarized using an adaptive threshold segmentation method to obtain a mask of potential defects, thereby obtaining the defects in the electronic screen image.

9. An AOI defect detection system based on industrial vision, comprising a processor and a memory, characterized in that, The memory stores a computer program, and the processor executes the computer program to implement the AOI defect detection method based on industrial vision as described in any one of claims 1-8.

Images