AOI optical detection method and system based on deep learning

Abstract

The invention relates to the technical field of image analysis, in particular to an AOI optical detection method and system based on deep learning, and the method comprises the following steps: obtaining transmission, reflection and scattering imaging data, calculating an edge direction change rate, setting a gradient change threshold, screening data points, and normalizing texture gradient and spectral reflectivity. And establishing a multi-view optical characteristic data matrix. According to the method, based on multi-view optical feature data, the data screening accuracy is improved by combining edge direction change rate calculation, the feature contribution degree dynamically adjusts weight parameters through difference calculation of the change rate and redundancy analysis to reduce invalid feature interference, illumination intensity gradient calculation is combined with a brightness change curve to screen stable pixels, and the illumination adaptability is improved; gradient direction consistency analysis is combined with inflection point density calculation, defect edge feature description is optimized, the fine defect recognition capability is enhanced, classification is combined with a weight adjustment coefficient and gradient direction matching parameters, a weighted feature matching mechanism is established, classification precision is optimized, and the stability of a detection result is improved.

Description

Technical Field

[0001] The present invention relates to the field of image analysis technology, and in particular to an AOI optical detection method and system based on deep learning. Background Art

[0002] The field of image analysis technology includes the core links of acquiring, processing, analyzing and identifying image data, involving multiple technical directions such as computer vision, pattern recognition, and deep learning. The core content of this technical field includes image preprocessing, feature extraction, target detection, target classification and segmentation, etc. By building a mathematical model or a neural network structure to analyze the image, the accurate recognition and analysis of the target in the image can be achieved. This technology is widely used in industrial inspection, medical image analysis, security monitoring, automatic driving and other fields. Among them, the automatic optical inspection AOI technology in the field of industrial inspection can be used to detect defects in precision manufacturing processes such as circuit boards, semiconductor chips, and display screens, improve product quality and reduce production costs.

[0003] Among them, the AOI optical inspection method based on deep learning refers to the use of deep neural networks to process and analyze the optical images of the inspected object to achieve an automated way of defect recognition and classification. This method covers technical matters such as image data acquisition, image preprocessing, deep learning model training and defect classification. First, the optical image of the object to be tested is acquired through a high-resolution camera, and preprocessed by filtering, contrast enhancement and other methods to improve the clarity and usability of the image. Subsequently, a convolutional neural network CNN is constructed to extract local and global features of the image, and the model is optimized through training sets and validation sets to improve the ability to recognize defects. Finally, a fully connected layer or a feature-based classifier is used to classify the inspection target to determine whether there are defects and output the inspection results.

[0004] In terms of feature data processing, the fixed weight parameters of the existing technology fail to adapt to complex defect changes, resulting in minor defects being easily overlooked, affecting detection accuracy. Feature redundancy analysis is insufficient, and some low-contribution features interfere with classification, increasing the probability of false detection. Light adaptability relies on a single light source and fails to fully utilize information on light changes, resulting in unstable detection in different environments. The edge detection method does not fully consider the gradient change trend, and the defect boundary is easily distorted under noise interference, reducing recognition accuracy. Classification optimization does not screen low-confidence samples, affecting classification reliability and increasing the risk of misjudgment. In precision manufacturing inspection scenarios, affected by these factors, the defect detection rate decreases, quality control becomes more difficult, and production costs rise. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose an AOI optical detection method and system based on deep learning.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: an AOI optical detection method based on deep learning, comprising the following steps:

[0007] S1: Acquire transmission, reflection, and scattering imaging data, calculate the edge direction change rate, set the gradient change threshold to filter data points, normalize the texture gradient and spectral reflectance, and establish a multi-view optical feature data matrix;

[0008] S2: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple features to the classification, use the difference to calculate the contribution change rate, screen the features with the same change trend, calculate the mutual information value to analyze the redundancy, set the initial weight, adjust the weight parameter in real time according to the redundancy and feature contribution change trend, and obtain the feature weight adjustment coefficient;

[0009] S3: Based on the multi-view optical feature data matrix, multi-directional light source imaging is set, the light intensity gradient between adjacent light sources is calculated, the brightness change curve is fitted to select stable pixel points, and a light gradient direction feature matrix is ​​established;

[0010] S4: calling the illumination gradient direction feature matrix, calculating the gradient direction consistency through defect edge detection in AOI optical inspection, screening the stable area, calculating the gradient change inflection point density, establishing a feature curve database, and obtaining the gradient direction matching parameters.

[0011] As a further solution of the present invention, the multi-view optical feature data matrix includes normalized texture gradient, normalized spectral reflectance, and screened data points; the feature weight adjustment coefficient includes feature contribution change rate, mutual information value, and real-time adjusted weight parameters; the illumination gradient direction feature matrix includes stable pixel points, illumination intensity gradient between adjacent light sources, and brightness change curve fitting parameters; the gradient direction matching parameters include gradient direction consistency, stable area, and gradient change inflection point density.

[0012] As a further solution of the present invention, the steps of acquiring the multi-view optical characteristic data matrix are specifically as follows:

[0013] S101: Acquire transmission, reflection, and scattering imaging data, calculate the optical feature matrix of the data points, and calculate the edge direction gradient change rate of each imaging data using the formula:

[0014] ;

[0015] Obtaining the rate of change of optical gradient;

[0016] in, represents the rate of change of the optical gradient, Representative The edge gradient of the data point, represents the direction vector of the data point, represents the total amount of imaging data;

[0017] S102: calling the optical gradient change rate, setting a gradient change threshold, screening data points, calculating the difference between the gradient change rate of the data point and the threshold, removing points whose difference is lower than the set threshold, calling the remaining data points, and obtaining a screened data point set;

[0018] S103: calling the filtered data point set, normalizing the texture gradient and the spectral reflectance, performing a standardized conversion on the spectral reflectance values ​​of the data points, and establishing a multi-view optical feature data matrix.

[0019] As a further solution of the present invention, the step of obtaining the feature weight adjustment coefficient is specifically as follows:

[0020] S201: Based on the multi-view optical feature data matrix, calling the multi-angle imaging in the AOI optical inspection, calculating the contribution gradient of multiple optical features to the classification, using differential calculation to obtain the contribution change rate, screening the features with consistent change trends, and obtaining the feature change consistency screening results;

[0021] S202: Based on the feature change consistency screening result, calculate the mutual information values ​​of multiple screening features, analyze feature redundancy, set initial weights, and obtain feature redundancy calculation results;

[0022] S203: Based on the feature redundancy calculation result, according to the change trend of redundancy and feature contribution, the weight parameter is adjusted in real time, using the formula:

[0023] ;

[0024] The calculation is used to obtain the adjustment range of multiple feature weights, and combined with the initial weights, the feature weight adjustment coefficient is obtained;

[0025] in, represents the feature weight adjustment coefficient, represents the initial weight, represents the adjustment step size factor, Representative The contribution gradient of the feature, Represents the mean of all feature contribution gradients, Representative The redundancy of features, Representative The mutual information value of features, represents the redundancy balance factor, Represents the number of features after filtering.

[0026] As a further solution of the present invention, the step of obtaining the illumination gradient directional feature matrix is ​​specifically as follows:

[0027] S301: Based on the multi-view optical feature data matrix, multi-directional light source imaging is set, pixel brightness data under multiple light source directions is obtained, the light intensity gradient between adjacent light sources is calculated, a light gradient change data set is established, and a light gradient distribution value is obtained;

[0028] S302: calling the illumination gradient distribution value, analyzing the variation trend of the illumination intensity gradient between adjacent light sources of multiple pixel points, fitting the brightness variation curve, calculating the brightness variation amplitude of the pixel points, and screening the pixel points whose brightness variation amplitude is within the target range to obtain a stable pixel point set;

[0029] S303: Based on the change of the illumination gradient direction of the stable pixel set, the illumination gradient direction feature vector of multiple stable pixels is calculated using the formula:

[0030] ;

[0031] Obtain the element values ​​of the illumination gradient direction feature matrix through calculation, and arrange all stable pixels into a matrix to establish the illumination gradient direction feature matrix;

[0032] in, Represents the element value of the illumination gradient direction feature matrix, Representative The brightness change of a stable pixel, Representative The change in the illumination direction angle of a stable pixel point, Representative The illumination time interval of a stable pixel point, Represents the total number of stable pixels.

[0033] As a further solution of the present invention, the step of obtaining the gradient direction matching parameter is specifically as follows:

[0034] S401: Based on the illumination gradient direction feature matrix, for the defective area in the AOI optical inspection, the gradient direction information of all pixels in the area is obtained, the degree of change of the gradient directions of adjacent pixels is calculated, and the gradient direction consistency distribution value of the multi-inspection area is calculated for all pixels in the area;

[0035] S402: Based on the gradient direction consistency distribution value, the region with stable changes is screened to obtain the gradient direction change trend inside the stable region, and at the same time, the change rate of the gradient direction of the pixel point in the stable region is calculated, and the inflection point distribution is detected to calculate the gradient change inflection point density distribution value of all stable regions;

[0036] S403: Based on the density distribution value of the gradient change inflection point and in combination with the gradient direction feature, a characteristic curve of the gradient change trend is established, and matching parameters of the characteristic curve are calculated using the formula:

[0037] ;

[0038] Calculate and obtain the gradient direction matching parameters;

[0039] in, Represents the gradient direction matching parameter value, Representative The density value of the gradient change inflection point in the stable area, Representative The gradient direction consistency distribution value of the stable region, represents the matching adjustment factor, Representative The gradient direction change trend in a stable region, Represents the number of feature points involved in the matching calculation.

[0040] As a further embodiment of the present invention, the method further comprises:

[0041] S5: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, a weighted feature matching mechanism is constructed, a classification confidence threshold is set, low confidence samples are screened, and illumination gradient feature mapping conversion data is called to obtain classification optimization results;

[0042] The classification optimization results include weighted feature matching mechanism, classification confidence threshold, and illumination gradient feature mapping conversion data.

[0043] As a further solution of the present invention, the step of obtaining the classification optimization result is specifically:

[0044] S501: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, the matching degree between feature points is calculated, the matching error value is calculated according to the matching degree, and an error threshold is set to screen the feature point matching, and the feature points whose matching error values ​​exceed the error threshold are eliminated to obtain a screened matching feature point set;

[0045] S502: calling the screening matching feature point set, and calculating the classification confidence of the feature point based on the classification confidence threshold, using the formula:

[0046] ;

[0047] The confidence distribution of feature points is obtained by calculation, and samples with confidence lower than the threshold are filtered to obtain a confidence filtered feature point set;

[0048] in, Represents the calculated value of feature point classification confidence. represents the frequency of the eigenvalue, represents the gradient direction matching parameter, Represents the weight adjustment coefficient corresponding to multiple feature points, Represents the total number of feature points, , Represent the maximum and minimum values ​​of the weight coefficient respectively, is the stability constant;

[0049] S503: calling the confidence screening feature point set, performing illumination gradient feature mapping conversion, calculating the mapping transformation value of the feature point gradient direction, and updating the classification result to obtain the classification optimization result.

[0050] An AOI optical inspection system based on deep learning, the AOI optical inspection system based on deep learning is used to execute the above-mentioned AOI optical inspection method based on deep learning, and the system comprises:

[0051] The optical feature extraction module obtains transmission, reflection, and scattering imaging data, calculates the edge direction change rate, sets the gradient change threshold to filter data points, normalizes texture gradient and spectral reflectance, calls multi-view optical imaging data, calculates the feature difference ratio of each view data, and establishes a multi-view optical feature data matrix;

[0052] The feature contribution analysis module calls multi-angle imaging data based on the multi-view optical feature matrix, calculates the contribution gradient of multiple features to classification, uses difference to calculate the contribution change rate, selects features with consistent change trends, calculates mutual information value analysis redundancy, sets initial weights, adjusts weight parameters, and obtains feature weight adjustment coefficients;

[0053] The illumination gradient construction module sets multi-directional light source imaging based on the multi-view optical feature matrix, calculates the light intensity gradient between adjacent light sources, fits the brightness change curve to select stable pixel points, and establishes an illumination gradient direction feature matrix;

[0054] The defect edge detection module calls the illumination gradient direction feature matrix, calculates the gradient direction consistency, screens the stable area, calculates the gradient change inflection point density, establishes a feature curve database, and obtains the gradient direction matching parameters;

[0055] The weighted feature matching module constructs a weighted feature matching mechanism based on the feature weight adjustment coefficient and the gradient direction matching parameter, sets a classification confidence threshold, filters low-confidence samples, calls the illumination gradient feature mapping conversion data, and obtains classification optimization results.

[0056] Compared with the prior art, the advantages and positive effects of the present invention are:

[0057] In the present invention, based on multi-view optical feature data, combined with edge direction change rate calculation and gradient normalization, the accuracy of data screening is improved to ensure feature stability. The feature contribution is calculated by differential change rate and combined with redundancy analysis to dynamically adjust the weight parameters to reduce invalid feature interference. The light intensity gradient calculation is combined with the brightness change curve to screen stable pixels to improve the adaptability of detection to different lighting conditions. The gradient direction consistency analysis is combined with the inflection point density calculation to optimize the defect edge feature description and enhance the recognition ability of subtle defects. During the classification process, a weighted feature matching mechanism is established by combining the weight adjustment coefficient with the gradient direction matching parameter, and the classification accuracy is optimized by screening low-confidence samples to improve the stability of the detection results. The improvement of data screening, feature extraction, light adaptability and classification optimization enables the detection system to have higher accuracy in complex environments, reduce the influence of interference factors, and improve the defect detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0059] Figure 2 Flow chart of the steps of obtaining the multi-viewing angle optical characteristic data matrix of the present invention;

[0060] Figure 3 This is a flow chart of the steps for obtaining the feature weight adjustment coefficient of the present invention;

[0061] Figure 4 Flow chart of the steps of obtaining the illumination gradient direction feature matrix of the present invention;

[0062] Figure 5 A flow chart of the steps for obtaining the gradient direction matching parameters of the present invention;

[0063] Figure 6 The present invention is a flowchart of the steps for obtaining the classification optimization results. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0065] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0066] Embodiment 1

[0067] See also Figure 1 The present invention provides a technical solution: an AOI optical detection method based on deep learning, comprising the following steps:

[0068] S1: Acquire transmission, reflection, and scattering imaging data, calculate the edge direction change rate, set the gradient change threshold to filter data points, normalize the texture gradient and spectral reflectance, and establish a multi-view optical feature data matrix;

[0069] S2: Based on the multi-view optical feature data matrix, call the multi-angle imaging in AOI optical inspection, calculate the contribution gradient of multiple features to classification, use difference to calculate the contribution change rate, screen the features with consistent change trends, calculate the mutual information value to analyze the redundancy, set the initial weight, adjust the weight parameter in real time according to the redundancy and feature contribution change trend, and obtain the feature weight adjustment coefficient;

[0070] S3: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, calculate the light intensity gradient between adjacent light sources, fit the brightness change curve to select stable pixels, and establish the light gradient direction feature matrix;

[0071] S4: Call the illumination gradient direction feature matrix, calculate the gradient direction consistency through defect edge detection in AOI optical inspection, screen the stable area, calculate the gradient change inflection point density, establish the feature curve database, and obtain the gradient direction matching parameters;

[0072] S5: Based on the feature weight adjustment coefficient and gradient direction matching parameter, a weighted feature matching mechanism is constructed, the classification confidence threshold is set, low-confidence samples are screened, the illumination gradient feature map conversion data is called, and the classification optimization result is obtained.

[0073] The multi-view optical feature data matrix includes normalized texture gradient, normalized spectral reflectance, and screened data points. The feature weight adjustment coefficient includes the feature contribution change rate, mutual information value, and real-time adjusted weight parameters. The illumination gradient direction feature matrix includes stable pixel points, illumination intensity gradient between adjacent light sources, and brightness change curve fitting parameters. The gradient direction matching parameters include gradient direction consistency, stable area, and gradient change inflection point density. The classification optimization results include weighted feature matching mechanism, classification confidence threshold, and illumination gradient feature mapping conversion data.

[0074] See also Figure 2 , the steps for obtaining the multi-view optical feature data matrix are as follows:

[0075] S101: Acquire transmission, reflection, and scattering imaging data, calculate the optical feature matrix of the data points, and calculate the edge direction gradient change rate of each imaging data using the formula:

[0076] ;

[0077] Obtaining the optical gradient change rate;

[0078] in, represents the rate of change of the optical gradient, Representative The edge gradient of the data point, represents the direction vector of the data point, Represents the total amount of imaging data;

[0079] For example, in the detection of optical films, the light source is transmitted through the material and received by the sensor to form a transmission data matrix. Reflection imaging is used to obtain surface features. For example, in metal surface detection, the light source is illuminated at a specific angle and the reflected light intensity is measured. Scattering imaging mainly targets the microstructural changes of the material. For example, in the detection of turbid media, the scattered light distribution is collected from multiple angles to construct scattering feature data. After obtaining the images of each imaging mode, the optical feature matrix of each data point is calculated. During the calculation process, the optical parameters of each pixel are first extracted, including light intensity value, color information, polarization state, etc., and then the edge gradient is calculated through image processing technology. The edge gradient is calculated based on the change of pixel grayscale value. For example, the Sobel operator is used to calculate the gradient amplitude and direction of the pixel point to define a certain pixel point. The gradient change rate is:

[0080] ;

[0081] in, and Respectively represent the gradient components of the image in the horizontal and vertical directions, and calculate the gradient direction of each pixel , and use the gradient change rate formula:

[0082] ;

[0083] Set the total number of data points , assuming that the edge gradient values ​​of some pixels are as follows:

[0084] ;

[0085] Assuming the direction vector for:

[0086] ;

[0087] Calculate for each data point :

[0088] ;

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] Calculate the rate of change of the optical gradient:

[0094] ;

[0095] The results show that the average optical gradient change rate of the data points is 2.128, which represents the degree of change in the overall edge direction of the image and is closely related to the edge clarity under different optical modes in the actual imaging process. The larger the value, the greater the degree of change in the edge area of ​​the image. In subsequent steps, this value can be used to screen data points with significant edge changes to eliminate invalid or redundant data.

[0096] S102: calling the optical gradient change rate, setting the gradient change threshold, screening data points, calculating the difference between the gradient change rate of the data point and the threshold, removing the points whose difference is lower than the set threshold, calling the remaining data points, and obtaining a screened data point set;

[0097] Need to set the gradient change threshold The threshold value can be set based on empirical statistics or experimental measurements. For example, for industrial detection applications, the lower limit of the 90% confidence interval can be set as the screening threshold by analyzing the optical gradient distribution of normal samples. Assuming that 1000 sample points are obtained, the calculated The mean is 2.0 and the standard deviation is 0.5, so the threshold can be set:

[0098] ;

[0099] For each data point, the gradient change rate is compared with the set threshold:

[0100] ,reserve;

[0101] ,reserve;

[0102] ,reserve;

[0103] ,reserve;

[0104] ,reserve;

[0105] If the gradient change rate of a data point is less than 1.25, the point is removed. For example, if there is a data point , then the point is removed, and finally the set of data points that meet the requirements is retained. This result shows that the data points below the set threshold are successfully removed, ensuring that the subsequent data set only contains data points with significant gradient change rates, so as to improve the reliability and availability of the data.

[0106] S103: calling the filtered data point set, normalizing the texture gradient and the spectral reflectance, performing a standardized transformation on the spectral reflectance values ​​of the data points, and establishing a multi-view optical feature data matrix.

[0107] To standardize different optical feature data, the normalized texture gradient can be normalized by maximum and minimum normalization:

[0108] ;

[0109] Assume that the gradient value range after screening is:

[0110] ;

[0111] Among them, the maximum value , minimum , calculate the normalized value:

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] Finally, the multi-view optical feature data matrix is ​​constructed, as shown in Table 1.

[0118] Table 1 Multi-view optical characteristic data matrix

[0119]

[0120] The results show that through normalization processing, the gradients and spectral reflectances of different data points are converted to the same dimension, ensuring the comparability of different feature data in subsequent analysis, while improving the applicability and stability of the multi-view optical feature data matrix in different computing tasks.

[0121] See also Figure 3 , the specific steps for obtaining the feature weight adjustment coefficient are:

[0122] S201: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple optical features to the classification, use differential calculation to obtain the contribution change rate, screen the features with consistent change trends, and obtain the feature change consistency screening results;

[0123] First, call the multi-angle imaging in AOI optical inspection to collect the optical features of the same target object at different angles, set the angle step θ, for example, select 0°, 15°, 30°, 45°, 60°, 75°, 90° for data collection, and the optical feature data corresponding to each angle is recorded as , these data include gray value, edge sharpness, reflectivity, etc., calculate the contribution gradient of these features to classification, that is, calculate the feature change rate at different angles by derivation. The specific calculation method is as follows: Features , calculate its gradient :

[0124] ;

[0125] Set angle interval , use finite differences to calculate the feature contribution gradient:

[0126] ;

[0127] For example, if the values ​​of a feature at 30° and 45° are 85 and 90 respectively, then its contribution gradient is:

[0128] ;

[0129] Similarly, the contribution gradients of all features at different angles are calculated, and then their changing trends are analyzed. By screening features with consistent changing trends, features with unstable changing trends or excessive fluctuations are eliminated. The judgment method uses the standard deviation Filter and set thresholds , that is, filter out those that satisfy:

[0130] ;

[0131] For example, if the contribution gradients of a feature are , and calculate its standard deviation as:

[0132] ;

[0133] because , then this feature meets the screening conditions and is included in the feature change consistency screening results.

[0134] The result shows that the contribution trend of this feature is relatively stable at different angles, which meets the screening criteria and can be used as an effective feature for subsequent calculation of feature weights. However, the excluded features may have large changes in contribution at different angles and cannot provide stable classification information.

[0135] S202: Based on the feature change consistency screening results, calculate the mutual information values ​​of multiple screening features, analyze the feature redundancy, set the initial weight, and obtain the feature redundancy calculation result;

[0136] Based on the joint probability distribution and marginal probability distribution , mutual information The calculation formula is as follows:

[0137] ;

[0138] Hypothesized Features and The values ​​are and , the joint probability matrix is ​​as follows:

[0139]

[0140] Calculate the mutual information value:

[0141] ;

[0142] After all mutual information values ​​are calculated, set the feature redundancy measurement index , defined as the average mutual information value between this feature and all other features:

[0143] ;

[0144] If the mean mutual information between a feature and other features is greater than the set threshold , then the feature redundancy is judged to be too high, for example:

[0145] ;

[0146] Therefore, the feature redundancy is high and its weight needs to be reduced to finally obtain the feature redundancy calculation result.

[0147] This result shows that the information redundancy between this feature and other features is high, and the information it carries may overlap with other features. Reducing its weight can help reduce redundancy and improve the final classification performance.

[0148] S203: Based on the feature redundancy calculation result, according to the change trend of redundancy and feature contribution, the weight parameter is adjusted in real time, using the formula: ; The calculation is used to obtain the adjustment range of multiple feature weights, and combined with the initial weights, the feature weight adjustment coefficient is obtained; in, represents the feature weight adjustment coefficient, represents the initial weight, represents the adjustment step size factor, Representative The contribution gradient of the feature, Represents the mean of all feature contribution gradients, Representative The redundancy of features, Representative The mutual information value of features, represents the redundancy balance factor, Represents the number of features after filtering.

[0149] Set the adjustment step factor , redundancy balance factor , through the formula: ;

[0150] Calculate the weight adjustment, assuming that the characteristics Initial weight , feature contribution gradient mean , , mutual information value , redundancy , substitute into the calculation:

[0151] ;

[0152] This result shows that the characteristics The final weight of has increased slightly, indicating that the contribution of this feature is high and the redundancy is relatively low. In the final feature selection, its influence has been correspondingly improved, while other features with high redundancy will receive smaller weight adjustments or weight reductions, thereby achieving reasonable optimization of feature weights.

[0153] See also Figure 4 , the specific steps for obtaining the illumination gradient directional feature matrix are:

[0154] S301: Based on the multi-view optical feature data matrix, set multi-directional light source imaging, obtain pixel brightness data under multiple light source directions, calculate the light intensity gradient between adjacent light sources, establish a light gradient change data set, and obtain the light gradient distribution value;

[0155] First, determine the multi-directional light source for imaging, where the angle range of the light source is set to 0° to 180°, and set the light source in six directions at intervals of 30°, namely 0°, 30°, 60°, 90°, 120°, and 150°. Image the target object respectively and obtain the pixel brightness data under different light source directions. For example, in the 0° direction, the brightness measurement value of a certain pixel point is 120, the brightness in the 30° direction is 135, and the brightness in the 60° direction is 150, and so on. Store the data of all pixel points in the matrix. Then, by calculating the brightness change in the adjacent light source directions, use the brightness gradient calculation formula Calculate the brightness gradient of each pixel at different light source angles. For example, between 30° and 0°, the brightness gradient is , between 60° and 30°, the brightness gradient is , and calculate the complete brightness gradient data set in sequence. This data set is used to describe the trend of pixel brightness changes under different light source angles, and then obtain the overall illumination gradient distribution value, specifically the brightness gradient mean of each pixel. For example, the brightness gradient mean of the pixels in a certain area is calculated as , this value is used as the illumination gradient distribution value of the area, and finally the illumination gradient change data set is obtained to ensure that the subsequent steps can be analyzed based on the illumination gradient characteristics.

[0156] S302: calling the illumination gradient distribution value, analyzing the variation trend of the illumination intensity gradient between adjacent light sources of multiple pixel points, fitting the brightness variation curve, calculating the brightness variation amplitude of the pixel points, and screening the pixel points whose brightness variation amplitude is within the target range to obtain a stable pixel point set;

[0157] For a certain pixel point, the brightness changes from 0° to 150° in the direction of the light source are 120, 135, 150, 170, 195, and 225, respectively. The brightness change curve can be processed by the curve fitting method. The brightness change curve is fitted by the least squares method to calculate the brightness change amplitude. , if the maximum brightness value of the pixel is 225 and the minimum brightness value is 120, then the brightness change range is Next, set the screening threshold of the brightness change amplitude. Assuming that the target range is set between 80 and 120, the brightness change amplitude of the pixel meets the requirements and is screened as a stable pixel. If the brightness change amplitude of a pixel exceeds the range, for example, the change amplitude is 60 or 130, the stable pixel set is excluded. After traversing all pixels, all stable pixel sets that meet the screening conditions are obtained and stored in the stable pixel matrix to ensure the stability of subsequent calculations.

[0158] S303: Based on the change of the illumination gradient direction of the stable pixel set, the illumination gradient direction feature vector of multiple stable pixels is calculated using the formula:

[0159] ;

[0160] Obtain the element values ​​of the illumination gradient direction feature matrix through calculation, and arrange all stable pixels into a matrix to establish the illumination gradient direction feature matrix;

[0161] in, Represents the element value of the illumination gradient direction feature matrix, Representative The brightness change of a stable pixel, Representative The change in the illumination direction angle of a stable pixel point, Representative The illumination time interval of a stable pixel point, Represents the total number of stable pixels.

[0162] The brightness changes of a stable pixel under six light source angles are set to 15, 15, 20, 25, and 30 respectively, and the illumination direction angle change is 30°. The illumination gradient directional eigenvalue is calculated using the formula:

[0163] ;

[0164] Set the sampling interval is 0.1 seconds, and the specific calculation is as follows:

[0165] ;

[0166] Further calculation yields:

[0167] ;

[0168] ;

[0169] ;

[0170] The calculation result is the illumination gradient directional eigenvalue of the pixel. Finally, the calculation results of all stable pixels are arranged into a matrix to establish the illumination gradient directional feature matrix.

[0171] Table 2 lists the datasets for the example calculations:

[0172] Table 2 Light gradient calculation data

[0173]

[0174] As shown in Table 2, the calculation results of the illumination gradient directional eigenvalues ​​of different pixels are different. By arranging the calculation results of all stable pixels in a matrix, a complete illumination gradient directional feature matrix is ​​finally obtained.

[0175] See also Figure 5 , the specific steps for obtaining the gradient direction matching parameters are:

[0176] S401: Based on the illumination gradient direction feature matrix, for the defective area in the AOI optical inspection, the gradient direction information of all pixels in the area is obtained, the degree of change of the gradient directions of adjacent pixels is calculated, and the gradient direction consistency distribution value of the multi-inspection area is calculated for all pixels in the area;

[0177] First, inside the defect area, the gradient direction of all pixels is calculated point by point, and the gradient value of each pixel is obtained. , the gradient direction can be calculated by the Sobel operator, such as the coordinates of a pixel point are , then its horizontal and vertical gradients can be expressed as and , so the gradient direction is calculated as Next, select adjacent pixel pairs within the region , calculate their gradient direction changes , and record the gradient direction changes of all adjacent pixels. Then, the gradient direction change data of all pixels in the entire area are counted to calculate the gradient direction consistency distribution value of each detection area. , which can be obtained by calculating the mean and variance of the gradient direction of all pixels in the region. For example, suppose the region contains pixel points, the mean gradient direction is , then the consistency distribution value can be defined as If the value is low, it means that the gradient direction in the region is relatively consistent. If the value is high, it means that the gradient direction distribution in the region is relatively discrete. For example, in a detection area, if the gradient direction of all pixels is Between may be low (such as about 2°), and if the gradient direction is distributed in If the value is between , the value is higher (such as above 20°). After the gradient direction consistency distribution value of multiple detection areas is finally obtained, the value will be used for screening and feature matching in subsequent steps.

[0178] S402: Based on the gradient direction consistency distribution value, the region with stable changes is screened to obtain the gradient direction change trend inside the stable region, and at the same time, the change rate of the gradient direction of the pixel point in the stable region is calculated, and the distribution of the inflection points is detected to calculate the gradient change inflection point density distribution value of all stable regions;

[0179] Based on the gradient direction consistency distribution value calculated above , in the detection area, the stable changing area is screened by setting the threshold of the consistency distribution value , only keep area, for example, if , only the area with a consistency distribution value less than 5° is retained as the stable area. Then, in the selected stable area, the gradient direction change trend of the pixel point is calculated. The specific method is to perform linear fitting on the gradient direction of all pixels in the region to obtain the change trend curve ,in It represents the rate of change of the gradient direction. This value can be calculated by the least squares method. For example, if the gradient direction of several pixels in the region is , the corresponding horizontal coordinate position is , then the gradient direction change rate The formula Calculated, for example, if the gradient directions of the 5 pixels in the area are , corresponding to the position , then the gradient direction change rate is calculated For each pixel point, then, the gradient direction change rate of the pixel points in the stable area is counted, and the inflection point distribution is detected, that is, on the change trend curve, the position where the gradient direction change rate has an extreme value is found, and the density distribution value of the gradient change inflection point is calculated , which can be expressed as the number of inflection points per unit area. For example, if 10 inflection points are detected in a stable region, and the area is 100 pixels, then Inflection points / pixels, and finally obtain the gradient change inflection point density distribution value of all stable areas .

[0180] S403: Based on the density distribution value of the gradient change inflection point and the gradient direction characteristics, a characteristic curve of the gradient change trend is established, and the matching parameters of the characteristic curve are calculated using the formula:

[0181] ;

[0182] Calculate and obtain the gradient direction matching parameters;

[0183] in, Represents the gradient direction matching parameter value, Representative The density value of the gradient change inflection point in the stable area, Representative The gradient direction consistency distribution value of the stable region, represents the matching adjustment factor, Representative The gradient direction change trend in a stable region, Represents the number of feature points involved in the matching calculation.

[0184] Based on the gradient change inflection point density distribution value obtained by the above calculation , combined with the gradient direction consistency distribution value , establish the characteristic curve of the gradient change trend, which can be expressed as ,in and Determine by fitting, then calculate the matching parameters of the characteristic curve , using the formula:

[0185] ;

[0186] in, Representative The density value of the gradient change inflection point in the stable area, Representative The gradient direction consistency distribution value of the stable region, To match the adjustment coefficient, set it to 0.5. Representative The gradient direction change trend in a stable region is , assuming that the data of three stable regions are as follows:

[0187]

[0188] Then calculate:

[0189] ;

[0190] ;

[0191] Calculated , and the result is used to obtain the gradient direction matching parameters.

[0192] See also Figure 6 , the specific steps for obtaining classification optimization results are:

[0193] S501: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, the matching degree between the feature points is calculated, the matching error value is calculated according to the matching degree, and the error threshold is set to screen the feature point matching, and the feature points whose matching error values ​​exceed the error threshold are eliminated to obtain a screened matching feature point set;

[0194] First, obtain the gradient direction information of the feature points and the initial weight adjustment coefficient of the feature points, and set the feature point set. , for each feature point , respectively calculate its gradient direction vector And the corresponding weight adjustment coefficient , when calculating the matching degree, for two feature points Gradient direction matching parameters The calculation method can be obtained through cosine similarity:

[0195] ;

[0196] Next, the matching error value is calculated based on the matching degree of the feature points. It can be calculated from the deviation between the gradient direction matching parameter of the feature point and the weight adjustment coefficient:

[0197] ;

[0198] Set the matching error threshold ,like , then the feature point pairs whose matching error value exceeds the error threshold are eliminated , and finally obtain the set of filtered matching feature points .

[0199] S502: Calling to filter the matching feature point set, based on the classification confidence threshold, calculates the classification confidence of the feature point, using the formula: ;

[0200] The confidence distribution of feature points is obtained by calculation, and samples with confidence lower than the threshold are filtered to obtain a confidence filtered feature point set; in, Represents the calculated value of feature point classification confidence. represents the frequency of the eigenvalue, represents the gradient direction matching parameter, Represents the weight adjustment coefficient corresponding to multiple feature points, Represents the total number of feature points, , Represent the maximum and minimum values ​​of the weight coefficient respectively, is the stability constant;

[0201] The classification confidence of the feature point is calculated based on the classification confidence threshold. When the frequency of feature points is counted , Gradient direction matching parameters and weight adjustment coefficient , the classification confidence is calculated according to the following formula:

[0202] ;

[0203] Setting the classification confidence threshold ,like , then filter out the samples with lower confidence and remove them. Assuming the total number of feature points , , , , for feature points , and its corresponding parameters are as follows:

[0204] Table 3: Feature point confidence calculation parameter table

[0205]

[0206] calculate

[0207] ;

[0208] Then calculate the normalization factor

[0209] ;

[0210] Finally calculate the confidence value:

[0211] ;

[0212] Setting the confidence threshold ,because , so the sample is eliminated, and finally the confidence filtered feature point set is obtained.

[0213] S503: calling the confidence filter feature point set, performing illumination gradient feature mapping conversion, calculating the mapping transformation value of the feature point gradient direction, and updating the classification result to obtain the classification optimization result.

[0214] In the illumination gradient feature mapping process, we first need to obtain the illumination intensity of each feature point. and its corresponding gradient direction , set the conversion factor , the gradient direction mapping transformation value of the feature point is calculated by the following operation :

[0215] ;

[0216] Assumptions , for feature points ,like , , then calculate:

[0217] ;

[0218] The mapping transformation values ​​of other feature points are calculated in turn, and the classification results are updated. The final classification optimization results are obtained through statistical analysis of the feature space after feature point classification, completing the entire process of feature point screening and classification optimization.

[0219] An AOI optical inspection system based on deep learning, the AOI optical inspection system based on deep learning is used to execute the above-mentioned AOI optical inspection method based on deep learning, and the system includes:

[0220] The optical feature extraction module obtains transmission, reflection, and scattering imaging data, calculates the edge direction change rate, sets the gradient change threshold to filter data points, normalizes texture gradient and spectral reflectance, calls multi-view optical imaging data, calculates the feature difference ratio of each view data, and establishes a multi-view optical feature data matrix;

[0221] The feature contribution analysis module is based on the multi-view optical feature matrix, calls multi-angle imaging data, calculates the contribution gradient of multiple features to classification, uses difference to calculate the contribution change rate, selects features with consistent change trends, calculates mutual information value to analyze redundancy, sets initial weights, adjusts weight parameters, and obtains feature weight adjustment coefficients;

[0222] The illumination gradient construction module is based on the multi-view optical feature matrix, sets multi-directional light source imaging, calculates the light intensity gradient between adjacent light sources, fits the brightness change curve to select stable pixels, and establishes the illumination gradient direction feature matrix;

[0223] The defect edge detection module calls the illumination gradient direction feature matrix, calculates the gradient direction consistency, screens the stable area, calculates the gradient change inflection point density, establishes the feature curve database, and obtains the gradient direction matching parameters;

[0224] The weighted feature matching module constructs a weighted feature matching mechanism based on the feature weight adjustment coefficient and the gradient direction matching parameter, sets the classification confidence threshold, filters low-confidence samples, calls the illumination gradient feature map to convert data, and obtains the classification optimization results.

[0225] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. An AOI optical detection method based on deep learning, characterized in that: The following steps are involved: S1: Acquire transmission, reflection, and scattering imaging data, calculate the edge direction change rate, set the gradient change threshold to filter data points, normalize the texture gradient and spectral reflectance, and establish a multi-view optical feature data matrix; S2: Based on the multi-view optical feature data matrix, call the multi-angle imaging in the AOI optical inspection, calculate the contribution gradient of multiple features to the classification, use the difference to calculate the contribution change rate, screen the features with the same change trend, calculate the mutual information value to analyze the redundancy, set the initial weight, adjust the weight parameter in real time according to the redundancy and feature contribution change trend, and obtain the feature weight adjustment coefficient; S3: Based on the multi-view optical feature data matrix, multi-directional light source imaging is set, the light intensity gradient between adjacent light sources is calculated, the brightness change curve is fitted to select stable pixel points, and a light gradient direction feature matrix is ​​established; S4: calling the illumination gradient direction feature matrix, calculating the gradient direction consistency through defect edge detection in AOI optical inspection, screening the stable area, calculating the gradient change inflection point density, establishing a feature curve database, and obtaining the gradient direction matching parameters.

2. The AOI optical detection method based on deep learning according to claim 1, characterized in that: The multi-view optical feature data matrix includes normalized texture gradient, normalized spectral reflectance, and screened data points. The feature weight adjustment coefficient includes feature contribution change rate, mutual information value, and real-time adjusted weight parameters. The illumination gradient direction feature matrix includes stable pixel points, illumination intensity gradient between adjacent light sources, and brightness change curve fitting parameters. The gradient direction matching parameters include gradient direction consistency, stable area, and gradient change inflection point density.

3. The AOI optical detection method based on deep learning according to claim 2, characterized in that: The steps of acquiring the multi-view optical feature data matrix are specifically as follows: S101: Acquire transmission, reflection, and scattering imaging data, calculate the optical characteristic matrix of the data points, and calculate the edge direction gradient change rate of each imaging data to obtain the optical gradient change rate; S102: calling the optical gradient change rate, setting a gradient change threshold, screening data points, calculating the difference between the gradient change rate of the data point and the threshold, removing points whose difference is lower than the set threshold, calling the remaining data points, and obtaining a screened data point set; S103: calling the filtered data point set, normalizing the texture gradient and the spectral reflectance, performing a standardized conversion on the spectral reflectance values ​​of the data points, and establishing a multi-view optical feature data matrix.

4. The AOI optical detection method based on deep learning according to claim 3, characterized in that: The steps for obtaining the feature weight adjustment coefficient are specifically as follows: S201: Based on the multi-view optical feature data matrix, calling the multi-angle imaging in the AOI optical inspection, calculating the contribution gradient of multiple optical features to the classification, using differential calculation to obtain the contribution change rate, screening the features with consistent change trends, and obtaining the feature change consistency screening results; S202: Based on the feature change consistency screening result, calculate the mutual information values ​​of multiple screening features, analyze feature redundancy, set initial weights, and obtain feature redundancy calculation results; S203: Based on the feature redundancy calculation result, according to the change trend of redundancy and feature contribution, the weight parameter is adjusted in real time, using the formula: ； The calculation is used to obtain the adjustment range of multiple feature weights, and combined with the initial weights, the feature weight adjustment coefficient is obtained; in, represents the feature weight adjustment coefficient, represents the initial weight, represents the adjustment step size factor, Representative The contribution gradient of the feature, Represents the mean of all feature contribution gradients, Representative The redundancy of features, Representative The mutual information value of features, represents the redundancy balance factor, Represents the number of features after filtering.

5. The AOI optical detection method based on deep learning according to claim 4, characterized in that: The steps for obtaining the illumination gradient directional feature matrix are specifically as follows: S301: Based on the multi-view optical feature data matrix, multi-directional light source imaging is set, pixel brightness data under multiple light source directions is obtained, the light intensity gradient between adjacent light sources is calculated, a light gradient change data set is established, and a light gradient distribution value is obtained; S302: calling the illumination gradient distribution value, analyzing the variation trend of the illumination intensity gradient between adjacent light sources of multiple pixel points, fitting the brightness variation curve, calculating the brightness variation amplitude of the pixel points, and screening the pixel points whose brightness variation amplitude is within the target range to obtain a stable pixel point set; S303: Based on the change of the illumination gradient direction of the stable pixel set, the illumination gradient direction feature vector of multiple stable pixels is calculated using the formula: ； Obtain the element values ​​of the illumination gradient direction feature matrix through calculation, and arrange all stable pixels into a matrix to establish the illumination gradient direction feature matrix; in, Represents the element value of the illumination gradient direction feature matrix, Representative The brightness change of a stable pixel, Representative The change in the illumination direction angle of a stable pixel point, Representative The illumination time interval of a stable pixel point, Represents the total number of stable pixels.

6. The AOI optical detection method based on deep learning according to claim 5, characterized in that: The steps for obtaining the gradient direction matching parameters are specifically as follows: S401: Based on the illumination gradient direction feature matrix, for the defective area in the AOI optical inspection, the gradient direction information of all pixels in the area is obtained, the degree of change of the gradient directions of adjacent pixels is calculated, and the gradient direction consistency distribution value of the multi-inspection area is calculated for all pixels in the area; S402: Based on the gradient direction consistency distribution value, the region with stable changes is screened to obtain the gradient direction change trend inside the stable region, and at the same time, the change rate of the gradient direction of the pixel point in the stable region is calculated, and the inflection point distribution is detected to calculate the gradient change inflection point density distribution value of all stable regions; S403: Based on the density distribution value of the gradient change inflection point and in combination with the gradient direction feature, a characteristic curve of the gradient change trend is established, matching parameters of the characteristic curve are calculated, and gradient direction matching parameters are obtained by operation.

7. The AOI optical detection method based on deep learning according to claim 6, characterized in that: The method further comprises: S5: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, a weighted feature matching mechanism is constructed, a classification confidence threshold is set, low confidence samples are screened, and illumination gradient feature mapping conversion data is called to obtain classification optimization results; The classification optimization results include weighted feature matching mechanism, classification confidence threshold, and illumination gradient feature mapping conversion data.

8. The AOI optical detection method based on deep learning according to claim 7, characterized in that: The steps for obtaining the classification optimization results are specifically as follows: S501: Based on the feature weight adjustment coefficient and the gradient direction matching parameter, the matching degree between feature points is calculated, the matching error value is calculated according to the matching degree, and an error threshold is set to screen the feature point matching, and the feature points whose matching error values ​​exceed the error threshold are eliminated to obtain a screened matching feature point set; S502: calling the screening matching feature point set, and calculating the classification confidence of the feature point based on the classification confidence threshold, using the formula: ； The confidence distribution of feature points is obtained by calculation, and samples with confidence lower than the threshold are filtered to obtain a confidence filtered feature point set; in, Represents the calculated confidence value of feature point classification, represents the frequency of the eigenvalue, represents the gradient direction matching parameter, Represents the weight adjustment coefficient corresponding to multiple feature points, Represents the total number of feature points, , Represent the maximum and minimum values ​​of the weight coefficient respectively, is the stability constant; S503: calling the confidence screening feature point set, performing illumination gradient feature mapping conversion, calculating the mapping transformation value of the feature point gradient direction, and updating the classification result to obtain the classification optimization result.

9. An AOI optical inspection system based on deep learning, characterized in that: According to the deep learning-based AOI optical inspection method according to any one of claims 1 to 8, the system comprises: The optical feature extraction module obtains transmission, reflection, and scattering imaging data, calculates the edge direction change rate, sets the gradient change threshold to filter data points, normalizes texture gradient and spectral reflectance, calls multi-view optical imaging data, calculates the feature difference ratio of each view data, and establishes a multi-view optical feature data matrix; The feature contribution analysis module calls multi-angle imaging data based on the multi-view optical feature matrix, calculates the contribution gradient of multiple features to classification, uses difference to calculate the contribution change rate, selects features with consistent change trends, calculates mutual information value analysis redundancy, sets initial weights, adjusts weight parameters, and obtains feature weight adjustment coefficients; The illumination gradient construction module sets multi-directional light source imaging based on the multi-view optical feature matrix, calculates the light intensity gradient between adjacent light sources, fits the brightness change curve to select stable pixel points, and establishes an illumination gradient direction feature matrix; The defect edge detection module calls the illumination gradient direction feature matrix, calculates the gradient direction consistency, screens the stable area, calculates the gradient change inflection point density, establishes a feature curve database, and obtains the gradient direction matching parameters; The weighted feature matching module constructs a weighted feature matching mechanism based on the feature weight adjustment coefficient and the gradient direction matching parameter, sets a classification confidence threshold, filters low-confidence samples, calls the illumination gradient feature mapping conversion data, and obtains classification optimization results.

Images