An AOI optical detection method and system based on deep learning

Abstract

The present invention relates to the field of image analysis technology, specifically to an AOI optical detection method and system based on deep learning, comprising the following steps: acquiring transmission, reflection, and scattering imaging data, calculating edge direction change rates, setting gradient change thresholds to screen data points, normalizing texture gradients and spectral reflectance, and establishing a multi-view optical feature data matrix. In the present invention, based on multi-view optical feature data, the edge direction change rate calculation is combined to improve data screening accuracy, the feature contribution is dynamically adjusted by differential calculation of the change rate and redundancy analysis to reduce invalid feature interference, the illumination intensity gradient calculation is combined with the brightness change curve to screen stable pixels to improve illumination adaptability, the gradient direction consistency analysis is combined with the inflection point density calculation to optimize the defect edge feature description and enhance the ability to identify subtle defects, and the classification is combined with the weight adjustment coefficient and the gradient direction matching parameter to establish a weighted feature matching mechanism to optimize classification accuracy and improve the stability of the detection results.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to an AOI optical detection method and system based on deep learning. Background Art

[0002] The field of image analysis technology encompasses the acquisition, processing, analysis, and recognition of image data, encompassing multiple technical areas such as computer vision, pattern recognition, and deep learning. Core aspects of this technology include image preprocessing, feature extraction, target detection, target classification, and segmentation. By constructing mathematical models or neural network structures to analyze images, accurate recognition and analysis of targets within the image can be achieved. This technology is widely used in a variety of fields, including industrial inspection, medical imaging analysis, security monitoring, and autonomous driving. Automated Optical Inspection (AOI) technology in the industrial inspection field can be used to detect defects in precision manufacturing processes for circuit boards, semiconductor chips, and display screens, improving product quality and reducing production costs.

[0003] Among them, the AOI optical inspection method based on deep learning refers to the use of deep neural networks to process and analyze the optical images of the inspected object to achieve an automated way of defect recognition and classification. This method covers technical matters such as image data acquisition, image preprocessing, deep learning model training and defect classification. First, the optical image of the object to be inspected is acquired through a high-resolution camera, and preprocessed using filtering, contrast enhancement and other methods to improve the clarity and usability of the image. Subsequently, a convolutional neural network (CNN) is constructed to extract local and global features of the image, and the model is optimized through training and validation sets to improve the ability to recognize defects. Finally, a fully connected layer or a feature-based classifier is used to classify the inspection target to determine whether there are defects and output the inspection results.

[0004] In terms of feature data processing, the existing technology's fixed weight parameters fail to adapt to complex defect changes, resulting in subtle defects being easily overlooked, affecting detection accuracy. Feature redundancy analysis is insufficient, and some low-contribution features interfere with classification, increasing the probability of false detection. Lighting adaptability relies on a single light source and fails to fully utilize information about lighting changes, resulting in unstable detection in different environments. The edge detection method does not fully consider the gradient change trend, and the defect boundary is easily distorted under noise interference, reducing recognition accuracy. Classification optimization does not screen low-confidence samples, affecting classification reliability and increasing the risk of misjudgment. In precision manufacturing inspection scenarios, affected by these factors, the defect detection rate decreases, quality control becomes more difficult, and production costs increase. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose an AOI optical detection method and system based on deep learning.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solution: an AOI optical detection method based on deep learning, comprising the following steps:

[0007] S1: Acquire transmission, reflection, and scattering imaging data, calculate the edge direction change rate, set the gradient change threshold to filter data points, normalize the texture gradient and spectral reflectance, and establish a multi-view optical feature data matrix;

[0008] S2: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple features to the classification, use the difference to calculate the contribution change rate, screen the features with consistent change trends, calculate the mutual information value to analyze the redundancy, set the initial weight, and adjust the weight parameter in real time according to the redundancy and feature contribution change trend to obtain the feature weight adjustment coefficient;

[0009] S3: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, calculate the light intensity gradient between adjacent light sources, fit the brightness change curve to select stable pixels, and establish a light gradient direction feature matrix;

[0010] S4: Call the illumination gradient direction feature matrix, calculate the gradient direction consistency through defect edge detection in AOI optical inspection, screen the stable area, calculate the gradient change inflection point density, establish a feature curve database, and obtain the gradient direction matching parameters.

[0011] As a further solution of the present invention, the multi-view optical feature data matrix includes normalized texture gradient, normalized spectral reflectance, and filtered data points; the feature weight adjustment coefficient includes the feature contribution change rate, mutual information value, and real-time adjusted weight parameters; the illumination gradient direction feature matrix includes stable pixel points, illumination intensity gradient between adjacent light sources, and brightness change curve fitting parameters; the gradient direction matching parameters include gradient direction consistency, stable area, and gradient change inflection point density.

[0012] As a further solution of the present invention, the steps of acquiring the multi-view optical characteristic data matrix are specifically as follows:

[0013] S101: Acquire transmission, reflection, and scattering imaging data, calculate the optical characteristic matrix of the data points, and calculate the edge direction gradient change rate of each imaging data using the formula:

[0014] ;

[0015] Obtaining the optical gradient change rate;

[0016] in, represents the rate of change of optical gradient, Representative The edge gradient of the data point, represents the direction vector of the data point, represents the total amount of imaging data;

[0017] S102: calling the optical gradient change rate, setting a gradient change threshold, screening data points, calculating the difference between the gradient change rate of the data point and the threshold, removing points with a difference lower than the set threshold, calling the remaining data points, and obtaining a screened data point set;

[0018] S103: calling the filtered data point set, normalizing the texture gradient and spectral reflectance, performing a standardized conversion on the spectral reflectance values of the data points, and establishing a multi-view optical feature data matrix.

[0019] As a further solution of the present invention, the step of obtaining the feature weight adjustment coefficient is specifically as follows:

[0020] S201: Based on the multi-view optical feature data matrix, calling multi-angle imaging in AOI optical inspection, calculating the contribution gradients of multiple optical features to classification, using differential calculation to obtain the contribution change rate, screening features with consistent change trends, and obtaining feature change consistency screening results;

[0021] S202: Based on the feature change consistency screening result, calculate the mutual information value of multiple screening features, analyze feature redundancy, set initial weights, and obtain feature redundancy calculation results;

[0022] S203: Based on the feature redundancy calculation result, and in accordance with the change trend of redundancy and feature contribution, the weight parameter is adjusted in real time, using the formula:

[0023] ;

[0024] The calculation obtains the adjustment range of multiple feature weights, and combines them with the initial weights to obtain the feature weight adjustment coefficient;

[0025] in, represents the feature weight adjustment coefficient, represents the initial weight, represents the adjustment step size factor, Representative The contribution gradient of each feature, Represents the mean of all feature contribution gradients, Representative The redundancy of features, Representative The mutual information value of features, represents the redundancy balance factor, Represents the number of features after filtering.

[0026] As a further solution of the present invention, the step of obtaining the illumination gradient directional feature matrix is specifically as follows:

[0027] S301: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, obtain pixel brightness data under multiple light source directions, calculate the light intensity gradient between adjacent light sources, establish a light gradient change data set, and obtain a light gradient distribution value;

[0028] S302: Calling the illumination gradient distribution value, analyzing the variation trend of the illumination intensity gradient between adjacent light sources for multiple pixels, fitting a brightness variation curve, calculating the brightness variation amplitude of the pixels, and screening pixels whose brightness variation amplitude is within a target range to obtain a stable pixel set;

[0029] S303: Based on the change of the illumination gradient direction of the stable pixel set, the illumination gradient direction feature vector of multiple stable pixels is calculated using the formula:

[0030] ;

[0031] Obtain the element values of the illumination gradient directional feature matrix through calculation, and arrange all stable pixels into a matrix to establish the illumination gradient directional feature matrix;

[0032] in, Represents the element value of the illumination gradient direction feature matrix, Representative The brightness change of a stable pixel, Representative The change in the illumination direction angle of a stable pixel point, Representative The illumination time interval of a stable pixel point, Represents the total number of stable pixels.

[0033] As a further solution of the present invention, the step of obtaining the gradient direction matching parameter is specifically as follows:

[0034] S401: Based on the illumination gradient direction feature matrix, for the defect area in the AOI optical inspection, the gradient direction information of all pixels in the area is obtained, the degree of change of the gradient directions of adjacent pixels is calculated, and the gradient direction consistency distribution value of multiple inspection areas is calculated for all pixels in the area;

[0035] S402: Based on the gradient direction consistency distribution value, the region with stable changes is screened to obtain the gradient direction change trend within the stable region, and the change rate of the gradient direction of the pixel points in the stable region is calculated. The distribution of inflection points is detected, and the gradient change inflection point density distribution values of all stable regions are calculated.

[0036] S403: Based on the gradient change inflection point density distribution value and the gradient direction feature, a characteristic curve of the gradient change trend is established, and matching parameters of the characteristic curve are calculated using the formula:

[0037] ;

[0038] Calculate and obtain the gradient direction matching parameters;

[0039] in, Represents the gradient direction matching parameter value, Representative The density value of the gradient change inflection point in the stable area, Representative The gradient direction consistency distribution value of the stable region, represents the matching adjustment coefficient, Representative The gradient direction change trend within a stable region, Represents the number of feature points involved in the matching calculation.

[0040] As a further embodiment of the present invention, the method further comprises:

[0041] S5: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, a weighted feature matching mechanism is constructed, a classification confidence threshold is set, low-confidence samples are screened, and illumination gradient feature map conversion data is called to obtain classification optimization results;

[0042] The classification optimization results include weighted feature matching mechanism, classification confidence threshold, and illumination gradient feature map conversion data.

[0043] As a further solution of the present invention, the steps of obtaining the classification optimization results are specifically as follows:

[0044] S501: Calculating the matching degree between feature points based on the feature weight adjustment coefficient and the gradient direction matching parameter, calculating the matching error value according to the matching degree, setting an error threshold, screening feature point matching, and eliminating feature points whose matching error values exceed the error threshold to obtain a screened matching feature point set;

[0045] S502: Call the screened matching feature point set and calculate the classification confidence of the feature point based on the classification confidence threshold using the formula:

[0046] ;

[0047] Calculate the confidence distribution of feature points, filter out samples with confidence lower than the threshold, and obtain the confidence-filtered feature point set;

[0048] in, Represents the calculated confidence value of feature point classification, represents the frequency of the eigenvalue, represents the gradient direction matching parameter, Represents the weight adjustment coefficient corresponding to multiple feature points, Represents the total number of feature points, Represent the maximum and minimum values of the weight coefficient respectively, is the stability constant;

[0049] S503: calling the confidence screening feature point set, performing illumination gradient feature mapping conversion, calculating the mapping transformation value of the feature point gradient direction, and updating the classification result to obtain the classification optimization result.

[0050] A deep learning-based AOI optical inspection system, wherein the deep learning-based AOI optical inspection system is used to execute the above-mentioned deep learning-based AOI optical inspection method, and the system comprises:

[0051] The optical feature extraction module acquires transmission, reflection, and scattering imaging data, calculates the edge direction change rate, sets the gradient change threshold to filter data points, normalizes texture gradient and spectral reflectance, calls multi-view optical imaging data, calculates the feature difference ratio of each view data, and establishes a multi-view optical feature data matrix;

[0052] The feature contribution analysis module calls multi-angle imaging data based on the multi-view optical feature data matrix, calculates the contribution gradient of multiple features to classification, calculates the contribution change rate using difference, screens features with consistent change trends, calculates mutual information values to analyze redundancy, sets initial weights, adjusts weight parameters, and obtains feature weight adjustment coefficients;

[0053] The illumination gradient construction module sets multi-directional light source imaging based on the multi-view optical feature data matrix, calculates the light intensity gradient between adjacent light sources, fits the brightness change curve to select stable pixel points, and establishes an illumination gradient direction feature matrix;

[0054] The defect edge detection module calls the illumination gradient direction feature matrix, calculates the gradient direction consistency, screens the stable area, calculates the gradient change inflection point density, establishes a feature curve database, and obtains the gradient direction matching parameters;

[0055] The weighted feature matching module constructs a weighted feature matching mechanism based on the feature weight adjustment coefficient and the gradient direction matching parameter, sets the classification confidence threshold, filters low-confidence samples, calls the illumination gradient feature map conversion data, and obtains the classification optimization result.

[0056] Compared with the prior art, the advantages and positive effects of the present invention are:

[0057] In the present invention, based on multi-view optical feature data, combined with edge direction change rate calculation and gradient normalization, the accuracy of data screening is improved and feature stability is ensured. The feature contribution is calculated by differential change rate and combined with redundancy analysis to dynamically adjust the weight parameters to reduce invalid feature interference. The light intensity gradient calculation is combined with the brightness change curve to screen stable pixels, thereby improving the adaptability of detection to different lighting conditions. The gradient direction consistency analysis is combined with the inflection point density calculation to optimize the defect edge feature description and enhance the recognition ability of subtle defects. During the classification process, the weight adjustment coefficient and the gradient direction matching parameter are combined to establish a weighted feature matching mechanism, and the classification accuracy is optimized by screening low-confidence samples to improve the stability of the detection results. The improvement of data screening, feature extraction, light adaptability and classification optimization enables the detection system to have higher accuracy in complex environments, reduce the influence of interference factors, and improve the defect detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0059] Figure 2 Flowchart of the steps for obtaining the multi-view optical characteristic data matrix of the present invention;

[0060] Figure 3 Flowchart of the steps for obtaining the feature weight adjustment coefficient of the present invention;

[0061] Figure 4 Flowchart of the steps for obtaining the illumination gradient direction feature matrix of the present invention;

[0062] Figure 5 Flowchart of the steps for obtaining the gradient direction matching parameters of the present invention;

[0063] Figure 6 Flowchart of the steps for obtaining the classification optimization results of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0066] Example 1

[0067] See also Figure 1 The present invention provides a technical solution: an AOI optical detection method based on deep learning, comprising the following steps:

[0068] S1: Acquire transmission, reflection, and scattering imaging data, calculate the edge direction change rate, set the gradient change threshold to filter data points, normalize the texture gradient and spectral reflectance, and establish a multi-view optical feature data matrix;

[0069] S2: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple features to the classification, use the difference to calculate the contribution change rate, screen the features with consistent change trends, calculate the mutual information value to analyze the redundancy, set the initial weight, and adjust the weight parameters in real time according to the redundancy and feature contribution change trend to obtain the feature weight adjustment coefficient;

[0070] S3: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, calculate the light intensity gradient between adjacent light sources, fit the brightness change curve to select stable pixels, and establish the light gradient direction feature matrix;

[0071] S4: Call the illumination gradient direction feature matrix, detect the defect edge in AOI optical inspection, calculate the gradient direction consistency, screen the stable area, calculate the gradient change inflection point density, establish a characteristic curve database, and obtain the gradient direction matching parameters;

[0072] S5: Based on the feature weight adjustment coefficient and gradient direction matching parameters, a weighted feature matching mechanism is constructed, the classification confidence threshold is set, low-confidence samples are filtered, the illumination gradient feature map conversion data is called, and the classification optimization results are obtained.

[0073] The multi-view optical feature data matrix includes normalized texture gradient, normalized spectral reflectance, and filtered data points. The feature weight adjustment coefficient includes the feature contribution change rate, mutual information value, and real-time adjustment weight parameters. The illumination gradient direction feature matrix includes stable pixel points, illumination intensity gradient between adjacent light sources, and brightness change curve fitting parameters. The gradient direction matching parameters include gradient direction consistency, stable area, and gradient change inflection point density. The classification optimization results include weighted feature matching mechanism, classification confidence threshold, and illumination gradient feature mapping conversion data.

[0074] See also Figure 2 , the specific steps for obtaining the multi-view optical feature data matrix are:

[0075] S101: Acquire transmission, reflection, and scattering imaging data, calculate the optical characteristic matrix of the data points, and calculate the edge direction gradient change rate of each imaging data using the formula:

[0076] ;

[0077] Obtaining the optical gradient change rate;

[0078] in, represents the rate of change of optical gradient, Representative The edge gradient of the data point, represents the direction vector of the data point, represents the total amount of imaging data;

[0079] For example, in the detection of optical thin films, the light source is transmitted through the material and received by the sensor to form a transmission data matrix. Reflection imaging is used to obtain surface features. For example, in metal surface detection, the light source is illuminated at a specific angle and the reflected light intensity is measured. Scattering imaging mainly targets the microstructural changes of the material. For example, in the detection of turbid media, the scattered light distribution is collected from multiple angles to construct scattering feature data. After obtaining images of each imaging mode, the optical feature matrix of each data point is calculated. During the calculation process, the optical parameters of each pixel are first extracted, including light intensity value, color information, polarization state, etc., and then the edge gradient is calculated through image processing technology. The calculation of the edge gradient is based on the change of pixel grayscale value. For example, the Sobel operator is used to calculate the gradient amplitude and direction of the pixel point to define a certain pixel point. The gradient change rate is:

[0080] ;

[0081] in, and Represents the gradient components of the image in the horizontal and vertical directions respectively, and calculates the gradient direction of each pixel , and use the gradient change rate formula:

[0082] ;

[0083] Set the total number of data points , assuming that the edge gradient values of some pixels are as follows:

[0084] ;

[0085] Assuming the direction vector for:

[0086] ;

[0087] Calculate for each data point :

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] Calculate the rate of change of the optical gradient:

[0094] ;

[0095] The results show that the average optical gradient change rate of the data points is 2.128, which represents the degree of change in the overall edge direction of the image and is closely related to the edge clarity under different optical modes during the actual imaging process. The larger the value, the greater the degree of change in the edge area of the image. In subsequent steps, this value can be used to screen data points with significant edge changes to eliminate invalid or redundant data.

[0096] S102: calling the optical gradient change rate, setting the gradient change threshold, screening the data points, calculating the difference between the gradient change rate of the data points and the threshold, eliminating the points whose difference is lower than the set threshold, calling the remaining data points, and obtaining the screened data point set;

[0097] Need to set the gradient change threshold The threshold value can be set based on empirical statistics or experimental measurements. For example, for industrial detection applications, the lower limit of the 90% confidence interval can be set as the screening threshold by analyzing the optical gradient distribution of normal samples. Assuming that 1000 sample points are obtained, the calculated If the mean is 2.0 and the standard deviation is 0.5, the threshold can be set as follows:

[0098] ;

[0099] For each data point, the gradient change rate is compared with the set threshold:

[0100] ;

[0101] ;

[0102] ;

[0103] ;

[0104] ;

[0105] If the gradient change rate of a data point is less than 1.25, the point is eliminated. For example, if there is a data point , then the point is removed, and finally the set of data points that meet the requirements is retained. This result shows that the data points below the set threshold are successfully eliminated, ensuring that the subsequent data set only contains data points with significant gradient change rates, thereby improving the reliability and availability of the data.

[0106] S103: calling the filtered data point set, normalizing the texture gradient and spectral reflectance, performing a standardized conversion on the spectral reflectance values of the data points, and establishing a multi-view optical feature data matrix.

[0107] To standardize different optical feature data, the normalized texture gradient can be normalized using maximum and minimum normalization:

[0108] ;

[0109] Assume that the gradient value range after screening is:

[0110] ;

[0111] Among them, the maximum value , minimum , calculate the normalized value:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] Finally, the multi-view optical feature data matrix is constructed, as shown in Table 1.

[0118] Table 1 Multi-view optical characteristic data matrix

[0119]

[0120] The results show that through normalization processing, the gradients and spectral reflectances of different data points are converted to the same dimension, ensuring the comparability of different feature data in subsequent analysis, while improving the applicability and stability of the multi-view optical feature data matrix in different computing tasks.

[0121] See also Figure 3 , the steps for obtaining the feature weight adjustment coefficient are as follows:

[0122] S201: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple optical features to the classification, use differential calculation to obtain the contribution change rate, screen the features with consistent change trends, and obtain the feature change consistency screening results;

[0123] First, call the multi-angle imaging in AOI optical inspection to collect the optical features of the same target object at different angles, set the angle step θ, for example, select 0°, 15°, 30°, 45°, 60°, 75°, and 90° for data collection, and the optical feature data corresponding to each angle is recorded as , these data include gray value, edge sharpness, reflectivity, etc., calculate the contribution gradient of these features to the classification, that is, calculate the feature change rate at different angles by taking derivatives. The specific calculation method is as follows: Features , calculate its gradient :

[0124] ;

[0125] Set angle interval , use finite differences to calculate the feature contribution gradient:

[0126] ;

[0127] For example, if the values of a feature at 30° and 45° are 85 and 90 respectively, then its contribution gradient is:

[0128] ;

[0129] Similarly, the contribution gradients of all features at different angles are calculated, and then their changing trends are analyzed. By screening features with consistent changing trends, features with unstable or excessively fluctuating changing trends are eliminated. The judgment method uses the standard deviation Filter and set thresholds , that is, filter out those that meet: ;

[0130] For example, if the contribution gradients of a feature are , calculate its standard deviation as:

[0131] ;

[0132] because , then this feature meets the screening conditions and is included in the feature change consistency screening results.

[0133] The results show that the contribution trend of this feature is relatively stable at different angles, which meets the screening criteria and can be used as an effective feature for subsequent calculation of feature weights. However, the excluded features may have large changes in contribution at different angles and cannot provide stable classification information.

[0134] S202: Based on the feature change consistency screening results, calculate the mutual information values of multiple screening features, analyze feature redundancy, set initial weights, and obtain feature redundancy calculation results;

[0135] Based on the joint probability distribution and marginal probability distribution , mutual information The calculation formula is as follows:

[0136] ;

[0137] Hypothetical characteristics and The values are and , the joint probability matrix is as follows:

[0138] Calculate the mutual information value:

[0139] ;

[0140] If the mutual information mean of a feature and other features is greater than the set threshold , then it is determined that the feature redundancy is too high, for example:

[0141] ;

[0142] Therefore, the feature redundancy is high and its weight needs to be reduced to finally obtain the feature redundancy calculation result.

[0143] This result shows that the information redundancy between this feature and other features is high, and the information it carries may overlap with other features. Reducing its weight can help reduce redundancy and improve the final classification performance.

[0144] S203: Based on the result of feature redundancy calculation, and in accordance with the changing trend of redundancy and feature contribution, the weight parameter is adjusted in real time using the formula:

[0145] ;

[0146] The calculation obtains the adjustment range of multiple feature weights, and combines them with the initial weights to obtain the feature weight adjustment coefficient;

[0147] in, represents the feature weight adjustment coefficient, represents the initial weight, represents the adjustment step size factor, Representative The contribution gradient of each feature, Represents the mean of all feature contribution gradients, Representative The redundancy of features, Representative The mutual information value of features, represents the redundancy balance factor, Represents the number of features after filtering.

[0148] Set the adjustment step factor , redundancy balance factor , through the formula:

[0149] ;

[0150] Calculate the weight adjustment, assuming the characteristics Initial weight , feature contribution gradient mean , , mutual information value , redundancy , substitute into the calculation: ;

[0151] This result shows that the characteristics The final weight of increases slightly, indicating that the contribution of this feature is high and the redundancy is relatively low. In the final feature selection, its influence is correspondingly improved, while other features with high redundancy will receive smaller weight adjustments or weight reductions, thereby achieving reasonable optimization of feature weights.

[0152] See also Figure 4 , the specific steps for obtaining the illumination gradient directional feature matrix are:

[0153] S301: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, obtain pixel brightness data under multiple light source directions, calculate the light intensity gradient between adjacent light sources, establish a light gradient change data set, and obtain the light gradient distribution value;

[0154] First, determine the multi-directional light source for imaging, where the angle range of the light source is set to 0° to 180°, and set the light source in six directions at intervals of 30°, namely 0°, 30°, 60°, 90°, 120°, and 150°. Image the target object respectively and obtain the pixel brightness data under different light source directions. For example, in the 0° direction, the brightness measurement value of a pixel point is 120, in the 30° direction the brightness is 135, and in the 60° direction the brightness is 150. And so on. Store the data of all pixel points in the matrix. Then, by calculating the brightness change in the adjacent light source directions, use the brightness gradient calculation formula Calculate the brightness gradient of each pixel under different light source angles. For example, between 30° and 0°, the brightness gradient is Between 60° and 30°, the brightness gradient is , and calculate the complete brightness gradient data set in sequence. This data set is used to describe the trend of pixel brightness changes under different light source angles, and then obtain the overall illumination gradient distribution value, specifically the brightness gradient mean of each pixel. For example, the brightness gradient mean of the pixels in a certain area is calculated as , this value is used as the illumination gradient distribution value of the area, and finally the illumination gradient change dataset is obtained to ensure that the subsequent steps can be analyzed based on the illumination gradient characteristics.

[0155] S302: Calling the illumination gradient distribution value, analyzing the variation trend of the illumination intensity gradient between adjacent light sources for multiple pixels, fitting the brightness variation curve, calculating the brightness variation amplitude of the pixels, and screening the pixels whose brightness variation amplitude is within the target range to obtain a stable pixel set;

[0156] For a certain pixel, the brightness changes from 0° to 150° in the order of 120, 135, 150, 170, 195, and 225. The brightness change curve can be processed by the curve fitting method. The brightness change curve is fitted using the least squares method to calculate the brightness change amplitude. , if the maximum brightness value of the pixel is 225 and the minimum brightness value is 120, then the brightness change range is Next, set the screening threshold of the brightness change amplitude. Assuming that the target range is set between 80 and 120, the brightness change amplitude of the pixel meets the requirements and is screened as a stable pixel. If the brightness change amplitude of a certain pixel exceeds the range, for example, the change amplitude is 60 or 130, the stable pixel set is excluded. After traversing all pixels, all stable pixel sets that meet the screening conditions are obtained and stored in the stable pixel matrix to ensure the stability of subsequent calculations.

[0157] S303: Based on the change of the illumination gradient direction of the stable pixel set, the illumination gradient direction feature vector of multiple stable pixels is calculated using the formula:

[0158] ;

[0159] Obtain the element values of the illumination gradient directional feature matrix through calculation, and arrange all stable pixels into a matrix to establish the illumination gradient directional feature matrix;

[0160] in, Represents the element value of the illumination gradient direction feature matrix, Representative The brightness change of a stable pixel, Representative The change in the illumination direction angle of a stable pixel point, Representative The illumination time interval of a stable pixel point, Represents the total number of stable pixels.

[0161] Assume that the brightness changes of a stable pixel under six light source angles are 15, 15, 20, 25, and 30 respectively, and the illumination direction angle change is 30°. Calculate the illumination gradient directional eigenvalue using the formula:

[0162] ;

[0163] Set the sampling interval It is 0.1 seconds, and the specific calculation is as follows:

[0164] ;

[0165] Further calculation yields:

[0166] ;

[0167] ;

[0168] ;

[0169] The calculation result is the illumination gradient directional eigenvalue of the pixel point. Finally, the calculation results of all stable pixels are arranged into a matrix to establish the illumination gradient directional feature matrix.

[0170] Table 2 lists the datasets for the example calculations:

[0171] Table 2 Light gradient calculation data

[0172]

[0173] As shown in Table 2, the calculation results of the illumination gradient directional eigenvalues of different pixels are different. By arranging the calculation results of all stable pixels in a matrix, a complete illumination gradient directional feature matrix is finally obtained.

[0174] See also Figure 5 , the specific steps for obtaining the gradient direction matching parameters are:

[0175] S401: Based on the illumination gradient direction feature matrix, for the defect area in the AOI optical inspection, the gradient direction information of all pixels in the area is obtained, the degree of change in the gradient direction of adjacent pixels is calculated, and the gradient direction consistency distribution value of multiple inspection areas is calculated for all pixels in the area;

[0176] First, inside the defect area, calculate the gradient direction of all pixels point by point and obtain the gradient value of each pixel , the gradient direction can be calculated by the Sobel operator, such as the coordinates of a pixel point are , then its horizontal and vertical gradients can be expressed as , and thus the gradient direction is calculated as Next, select adjacent pixel pairs within the region , calculate their gradient direction changes , and record the gradient direction changes of all adjacent pixels. Then, the gradient direction change data of all pixels in the entire area are counted, and the gradient direction consistency distribution value of each detection area is calculated. , which can be obtained by calculating the mean and variance of the gradient direction of all pixels in the region. For example, let the region contain pixel points, the mean gradient direction is , then the consistency distribution value can be defined as If the value is low, it means that the gradient direction in the region is relatively consistent. If the value is high, it means that the gradient direction distribution in the region is relatively discrete. For example, in a detection area, if the gradient direction of all pixels is Between, then May be low (such as about 2°), and if the gradient direction is distributed in If the value is between , the value is higher (such as above 20°). After the gradient direction consistency distribution value of multiple detection areas is finally obtained, the value will be used for screening and feature matching in subsequent steps.

[0177] S402: Based on the gradient direction consistency distribution value, the stable change area is screened to obtain the gradient direction change trend within the stable area, and the change rate of the pixel gradient direction in the stable area is calculated. The inflection point distribution is detected and the gradient change inflection point density distribution value of all stable areas is calculated;

[0178] Based on the gradient direction consistency distribution value calculated above , in the detection area, the stable changing area is screened by setting the threshold of the consistency distribution value , only keep area, for example, if , then only the area with a consistency distribution value less than 5° is retained as the stable area. Then, in the selected stable area, the gradient direction change trend of the pixel point is calculated. The specific method is to perform linear fitting on the gradient direction of all pixels in the region to obtain the change trend curve ,in It represents the rate of change of the gradient direction. This value can be calculated by the least squares method. For example, if the gradient direction of several pixels in the area is , the corresponding horizontal coordinate position is , then the gradient direction change rate The formula Calculated, for example, if the gradient directions of the 5 pixels in the area are , corresponding to the position , then the gradient direction change rate is calculated For each pixel point, the gradient direction change rate of the pixel points in the stable area is statistically analyzed, and the inflection point distribution is detected, that is, on the change trend curve, the position where the gradient direction change rate has an extreme value is found, and the gradient change inflection point density distribution value is calculated. , which can be expressed as the number of inflection points per unit area. For example, if 10 inflection points are detected in a stable region with an area of 100 pixels, then Inflection points / pixel points, and finally obtain the gradient change inflection point density distribution value of all stable areas .

[0179] S403: Based on the density distribution value of the gradient change inflection point and the gradient direction characteristics, a characteristic curve of the gradient change trend is established, and the matching parameters of the characteristic curve are calculated using the formula:

[0180] ;

[0181] Calculate and obtain the gradient direction matching parameters;

[0182] in, Represents the gradient direction matching parameter value, Representative The density value of the gradient change inflection point in the stable area, Representative The gradient direction consistency distribution value of the stable region, represents the matching adjustment coefficient, Representative The gradient direction change trend within a stable region, Represents the number of feature points involved in the matching calculation.

[0183] Based on the gradient change inflection point density distribution value obtained by the above calculation , combined with the gradient direction consistency distribution value , establish the characteristic curve of the gradient change trend, which can be expressed as ,in and Determine by fitting, then calculate the matching parameters of the characteristic curve , using the formula:

[0184] ;

[0185] in, Representative The density value of the gradient change inflection point in the stable area, Representative The gradient direction consistency distribution value of the stable region, To match the adjustment coefficient, set it to 0.5. Representative The gradient direction change trend in a stable region is , assuming that the data of three stable regions are as follows:

[0186]

[0187] Then calculate:

[0188] ;

[0189] ;

[0190] Calculated , and the result is used to obtain the gradient direction matching parameters.

[0191] See also Figure 6 , the specific steps for obtaining classification optimization results are:

[0192] S501: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, the matching degree between the feature points is calculated, the matching error value is calculated according to the matching degree, and an error threshold is set to screen the feature point matching, and the feature points whose matching error values exceed the error threshold are eliminated to obtain a screened matching feature point set;

[0193] First, obtain the gradient direction information of the feature points and the initial weight adjustment coefficient of the feature points, and set the feature point set. , for each feature point , calculate the gradient direction vector respectively And the corresponding weight adjustment coefficient , when calculating the matching degree, for two feature points Gradient direction matching parameter Calculation is performed, which can be obtained through cosine similarity: ;

[0194] Then, the matching error value is calculated based on the matching degree of the feature points. It can be calculated from the deviation between the gradient direction matching parameter of the feature point and the weight adjustment coefficient: ;

[0195] Set matching error threshold ,like , then the feature point pairs whose matching error value exceeds the error threshold are eliminated , and finally obtain the filtered matching feature point set .

[0196] S502: Calling the filter matching feature point set, and calculating the classification confidence of the feature points based on the classification confidence threshold, using the formula:

[0197] ;

[0198] Calculate the confidence distribution of feature points, filter out samples with confidence lower than the threshold, and obtain the confidence-filtered feature point set;

[0199] in, Represents the calculated confidence value of feature point classification, represents the frequency of the eigenvalue, represents the gradient direction matching parameter, Represents the weight adjustment coefficient corresponding to multiple feature points, Represents the total number of feature points, Represent the maximum and minimum values of the weight coefficient respectively, is the stability constant;

[0200] The classification confidence of the feature points is calculated based on the classification confidence threshold. When the frequency of feature points is counted , gradient direction matching parameters and weight adjustment coefficient , the classification confidence is calculated according to the following formula:

[0201] ;

[0202] Setting the classification confidence threshold ,like , then filter out the samples with lower confidence and remove them. Assuming the total number of feature points , , for feature points , and its corresponding parameters are as follows:

[0203] Table 3:

[0204] Feature point confidence calculation parameter table

[0205]

[0206] calculate

[0207] ;

[0208] Then calculate the normalization factor

[0209] ;

[0210] Finally calculate the confidence value:

[0211] ;

[0212] Setting confidence thresholds ,because , so the sample is eliminated, and finally the confidence screening feature point set is obtained.

[0213] S503: Call the confidence filter feature point set, perform illumination gradient feature mapping conversion, calculate the mapping transformation value of the feature point gradient direction, and update the classification result to obtain the classification optimization result.

[0214] In the illumination gradient feature mapping process, we first need to obtain the illumination intensity of each feature point and its corresponding gradient direction , set the conversion factor , calculate the gradient direction mapping transformation value of the feature point through the following operation :

[0215] ;

[0216] Assumptions , for feature points ,like , , then calculate:

[0217] ;

[0218] The mapping transformation values of other feature points are calculated in sequence, and the classification results are updated. The final classification optimization results are obtained through statistical analysis of the feature space after feature point classification, completing the entire process of feature point screening and classification optimization.

[0219] An AOI optical inspection system based on deep learning is used to execute the above-mentioned AOI optical inspection method based on deep learning. The system includes:

[0220] The optical feature extraction module acquires transmission, reflection, and scattering imaging data, calculates the edge direction change rate, sets the gradient change threshold to filter data points, normalizes texture gradient and spectral reflectance, calls multi-view optical imaging data, calculates the feature difference ratio of each view data, and establishes a multi-view optical feature data matrix;

[0221] The feature contribution analysis module is based on the multi-view optical feature data matrix, calls multi-angle imaging data, calculates the contribution gradient of multiple features to classification, uses differential calculation to calculate the contribution change rate, screens features with consistent change trends, calculates mutual information values to analyze redundancy, sets initial weights, adjusts weight parameters, and obtains feature weight adjustment coefficients;

[0222] The illumination gradient construction module is based on the multi-view optical feature data matrix, sets multi-directional light source imaging, calculates the light intensity gradient between adjacent light sources, fits the brightness change curve to select stable pixels, and establishes the illumination gradient direction feature matrix;

[0223] The defect edge detection module calls the illumination gradient direction feature matrix, calculates the gradient direction consistency, screens the stable area, calculates the density of gradient change inflection points, establishes a feature curve database, and obtains the gradient direction matching parameters;

[0224] The weighted feature matching module constructs a weighted feature matching mechanism based on the feature weight adjustment coefficient and the gradient direction matching parameter, sets the classification confidence threshold, filters low-confidence samples, calls the illumination gradient feature map to convert data, and obtains the classification optimization results.

[0225] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. An AOI optical detection method based on deep learning, characterized in that: The following steps are involved: S1: Acquire transmission, reflection, and scattering imaging data, calculate the edge direction change rate, set the gradient change threshold to filter data points, normalize the texture gradient and spectral reflectance, and establish a multi-view optical feature data matrix; S2: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple features to the classification, use the difference to calculate the contribution change rate, screen the features with consistent change trends, calculate the mutual information value to analyze the redundancy, set the initial weight, and adjust the weight parameter in real time according to the redundancy and feature contribution change trend to obtain the feature weight adjustment coefficient; The steps for obtaining the feature weight adjustment coefficient are specifically as follows: S201: Based on the multi-view optical feature data matrix, calling multi-angle imaging in AOI optical inspection, calculating the contribution gradients of multiple optical features to classification, using difference to calculate the contribution change rate, screening features with consistent change trends, and obtaining feature change consistency screening results; S202: Based on the feature change consistency screening result, calculate the mutual information value of multiple screening features, analyze feature redundancy, set initial weights, and obtain feature redundancy calculation results; S203: Based on the feature redundancy calculation result, the weight parameter is adjusted in real time according to the feature redundancy and feature contribution gradient change trend, using the formula: ； The calculation obtains the adjustment range of multiple feature weights, and combines them with the initial weights to obtain the feature weight adjustment coefficient; in, represents the feature weight adjustment coefficient, represents the initial weight, represents the adjustment step size factor, Representative The contribution gradient of each feature, Represents the mean of all feature contribution gradients, Representative The redundancy of features, Representative The mutual information value of features, represents the redundancy balance factor, Represents the number of features after screening; S3: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, calculate the light intensity gradient between adjacent light sources, fit the brightness change curve to select stable pixels, and establish a light gradient direction feature matrix; S4: calling the illumination gradient direction feature matrix, calculating the gradient direction consistency through defect edge detection in AOI optical inspection, screening the stable area, calculating the gradient change inflection point density, establishing a feature curve database, and obtaining the gradient direction matching parameters; S5: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, a weighted feature matching mechanism is constructed, a classification confidence threshold is set, low-confidence samples are screened, and illumination gradient feature map conversion data is called to obtain classification optimization results; The classification optimization results include weighted feature matching mechanism, classification confidence threshold, and illumination gradient feature map conversion data.

2. The AOI optical detection method based on deep learning according to claim 1, characterized in that: The multi-view optical feature data matrix includes normalized texture gradient, normalized spectral reflectance, and filtered data points; the feature weight adjustment coefficient includes mutual information value and real-time adjusted weight parameters; the illumination gradient direction feature matrix includes stable pixel points, illumination intensity gradient between adjacent light sources, and brightness change curve fitting parameters; the gradient direction matching parameters include gradient direction consistency, stable area, and gradient change inflection point density.

3. The AOI optical detection method based on deep learning according to claim 2, characterized in that: The steps for obtaining the multi-view optical characteristic data matrix are specifically as follows: S101: Acquire transmission, reflection, and scattering imaging data, calculate the optical characteristic matrix of the data points, and calculate the edge direction gradient change rate of each imaging data to obtain the optical gradient change rate; S102: calling the optical gradient change rate, setting a gradient change threshold, screening data points, calculating the difference between the gradient change rate of the data point and the threshold, removing points with a difference lower than the set threshold, calling the remaining data points, and obtaining a screened data point set; S103: calling the filtered data point set, normalizing the texture gradient and spectral reflectance, performing a standardized conversion on the spectral reflectance values of the data points, and establishing a multi-view optical feature data matrix.

4. The AOI optical detection method based on deep learning according to claim 1, characterized in that: The steps for obtaining the illumination gradient directional feature matrix are specifically as follows: S301: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, obtain pixel brightness data under multiple light source directions, calculate the light intensity gradient between adjacent light sources, establish a light gradient change data set, and obtain a light gradient distribution value; S302: Calling the illumination gradient distribution value, analyzing the variation trend of the illumination intensity gradient between adjacent light sources for multiple pixels, fitting a brightness variation curve, calculating the brightness variation amplitude of the pixels, and screening pixels whose brightness variation amplitude is within a target range to obtain a stable pixel set; S303: Based on the change of the illumination gradient direction of the stable pixel set, the illumination gradient direction feature matrix of multiple stable pixel points is calculated using the formula: ； Obtain the element values of the illumination gradient directional feature matrix through calculation, and arrange all stable pixels into a matrix to establish the illumination gradient directional feature matrix; in, Represents the element value of the illumination gradient direction feature matrix, Representative The brightness change of a stable pixel, Representative The change in the illumination direction angle of a stable pixel point, Representative The illumination time interval of a stable pixel point, Represents the total number of stable pixels.

5. The AOI optical detection method based on deep learning according to claim 4, characterized in that: The steps for obtaining the gradient direction matching parameters are specifically as follows: S401: Based on the illumination gradient direction feature matrix, for the defect area in the AOI optical inspection, the gradient direction information of all pixels in the area is obtained, the degree of change of the gradient directions of adjacent pixels is calculated, and the gradient direction consistency distribution value of multiple inspection areas is calculated for all pixels in the area; S402: Based on the gradient direction consistency distribution value, screen the stable change area, obtain the gradient direction change trend within the stable area, calculate the change rate of the gradient direction of the pixel point in the stable area, detect the inflection point distribution, and calculate the gradient change inflection point density distribution value of all stable areas; S403: Based on the gradient change inflection point density distribution value and in combination with the gradient direction feature, a characteristic curve of the gradient change trend is established, matching parameters of the characteristic curve are calculated, and gradient direction matching parameters are obtained by operation.

6. The AOI optical detection method based on deep learning according to claim 5, characterized in that: The steps for obtaining the classification optimization results are specifically as follows: S501: Calculating the matching degree between feature points based on the feature weight adjustment coefficient and the gradient direction matching parameter, calculating the matching error value according to the matching degree, setting an error threshold, screening feature point matching, and eliminating feature points whose matching error values exceed the error threshold to obtain a screened matching feature point set; S502: Call the screened matching feature point set and calculate the classification confidence of the feature point based on the classification confidence threshold using the formula: ； Calculate the confidence distribution of feature points, filter out samples with confidence lower than the threshold, and obtain the confidence-filtered feature point set; in, Represents the calculated confidence value of feature point classification, represents the frequency of the eigenvalue, represents the gradient direction matching parameter, Represents the weight adjustment coefficient corresponding to multiple feature points, Represents the total number of feature points, Represent the maximum and minimum values of the weight adjustment coefficient, respectively. is the stability constant, is the index of the feature point; S503: calling the confidence screening feature point set, performing illumination gradient feature mapping conversion, calculating the mapping transformation value of the feature point gradient direction, and updating the classification result to obtain the classification optimization result.

7. An AOI optical inspection system based on deep learning, characterized in that: The deep learning-based AOI optical inspection method according to any one of claims 1 to 6, wherein the system comprises: The optical feature extraction module acquires transmission, reflection, and scattering imaging data, calculates the edge direction change rate, sets the gradient change threshold to filter data points, normalizes texture gradient and spectral reflectance, calls multi-view optical imaging data, calculates the feature difference ratio of each view data, and establishes a multi-view optical feature data matrix; The feature contribution analysis module calls multi-angle imaging data based on the multi-view optical feature data matrix, calculates the contribution gradient of multiple features to classification, calculates the contribution change rate using difference, screens features with consistent change trends, calculates mutual information values to analyze redundancy, sets initial weights, adjusts weight parameters, and obtains feature weight adjustment coefficients; The illumination gradient construction module sets multi-directional light source imaging based on the multi-view optical feature data matrix, calculates the light intensity gradient between adjacent light sources, fits the brightness change curve to select stable pixel points, and establishes an illumination gradient direction feature matrix; The defect edge detection module calls the illumination gradient direction feature matrix, calculates the gradient direction consistency, screens the stable area, calculates the gradient change inflection point density, establishes a feature curve database, and obtains the gradient direction matching parameters; The weighted feature matching module constructs a weighted feature matching mechanism based on the feature weight adjustment coefficient and the gradient direction matching parameter, sets the classification confidence threshold, filters low-confidence samples, calls the illumination gradient feature map conversion data, and obtains the classification optimization result.

Images