A method and system for detecting the surface quality of an electroplated finished product
By acquiring information on electroplated finished products and process parameters, adapting image acquisition parameters, extracting multi-dimensional optical features, and fusing process information, the problems of light interference and insufficient accuracy in traditional detection are solved, achieving more efficient quality inspection of electroplated finished products.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU YUNHUI HARDWARE ELECTROPLATING CO LTD
- Filing Date
- 2026-05-28
- Publication Date
- 2026-07-07
AI Technical Summary
Traditional electroplating finished product quality inspection is easily affected by ambient light, resulting in insufficient accuracy and failing to effectively consider the correlation between electroplating process parameters and defect formation.
By acquiring information about the electroplated finished product, combining it with actual process parameters, adapting image acquisition parameters, extracting multi-dimensional optical features, and fusing process parameters with visual features to generate process perception fusion features, the system then uses a preset defect detection model for detection.
It improves the accuracy and robustness of surface quality inspection of electroplated finished products, effectively distinguishes different defects, and reduces false detections and missed detections.
Smart Images
Figure CN122345618A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of machine vision inspection technology, and in particular to a method and system for inspecting the surface quality of electroplated finished products. Background Technology
[0002] Electroplating, as an important surface treatment process, is widely used in the automotive, electronics, and hardware industries. The surface quality of electroplated products directly affects their corrosion resistance, aesthetics, and service life. Therefore, efficient and accurate quality testing of electroplated products is of great significance.
[0003] Traditional quality inspection of electroplated finished products mainly relies on manual visual inspection or general machine vision systems. However, due to the strong specular reflection characteristics of electroplated surfaces, traditional visual inspection methods are easily affected by ambient light, leading to image overexposure or information loss. This results in high inspection difficulty and poor stability. Furthermore, traditional inspection methods only rely on image information for defect identification and do not consider the direct correlation between actual process parameters and defects, which can easily lead to insufficient inspection accuracy. Summary of the Invention
[0004] The purpose of this application is to provide a method and system for detecting the surface quality of electroplated finished products. By adapting to the optical properties of the electroplated surface and integrating prior knowledge of the process, the accuracy and robustness of the detection of the surface quality of electroplated finished products can be improved.
[0005] In a first aspect, this application provides a method for detecting the surface quality of electroplated finished products, including: Obtain information about the electroplated finished product to be tested, including finished product attribute information and actual electroplating process parameters; Based on the finished product attribute information and actual electroplating process parameters, image acquisition parameters are determined through a preset parameter mapping table. The image acquisition parameters include lighting parameters and camera parameters. Based on the image acquisition parameters, the electroplated finished product under test is image acquired to obtain the surface image of the electroplated finished product. Multidimensional optical features are extracted from the surface image of the electroplated finished product. These multidimensional optical features include specular reflection features, diffuse reflection features, color features, and surface texture features. By fusing actual electroplating process parameters with multi-dimensional optical features, process-aware fusion features are generated. Based on process perception fusion features, the surface quality inspection results of electroplated finished products are obtained through a preset defect detection model.
[0006] The above technical solution extracts specular reflection components, diffuse reflection features, color difference, and texture features from the strong specular reflection characteristics of the electroplated surface. The electroplating process parameters are used as prior guiding information and fused with visual features to determine defects. This effectively distinguishes defects with similar causes but different properties, and effectively solves the problem that traditional visual inspection is easily affected by reflection interference and has limited detection accuracy on electroplated surfaces.
[0007] Optionally, the finished product attribute information includes electroplating type, geometry, and surface condition. The step of determining image acquisition parameters based on the finished product attribute information and actual electroplating process parameters through a preset parameter mapping table includes: Based on the finished product attribute information, a corresponding reference imaging template is matched from a preset reference parameter mapping table. The reference imaging template includes reference illumination parameters and reference camera parameters. Based on the electroplating type and actual electroplating process parameters, a potential defect distribution is generated through a preset process defect classification model. The potential defect distribution includes defect types and their corresponding occurrence probabilities. Based on the distribution of potential defects, the detectability score of each defect type is evaluated using preset historical detection data. Calculate the demand adjustment degree for each defect type based on the detectability score and the probability of occurrence; Based on the required adjustment degree for each defect type, parameter adjustment amounts are generated through a preset defect-imaging parameter mapping library; The reference imaging template is adjusted by parameter adjustment to generate image acquisition parameters.
[0008] Optionally, the step of generating parameter adjustment amounts based on the required adjustment degree for each defect type, through a preset defect-imaging parameter mapping library, includes: By using a pre-defined defect-imaging parameter mapping library, the adjustment direction and unit adjustment range of each imaging parameter for each defect type are obtained. For each imaging parameter, the total adjustment amount of the imaging parameter is obtained by multiplying the adjustment requirement of each defect type by the corresponding unit adjustment range and summing the results. The total adjustment of all imaging parameters is combined to form the parameter adjustment amount.
[0009] Optionally, after acquiring an image of the electroplated finished product based on image acquisition parameters, the process includes: For the surface image of the electroplated finished product, quality assessment indicators are extracted, including brightness uniformity index, sharpness index and reflection suppression index; The quality assessment indicators are weighted and integrated using preset weighting coefficients to calculate a comprehensive quality score. If the overall quality score is lower than the preset quality threshold, the image acquisition parameters will be adjusted and a new image of the electroplated finished product quality will be acquired.
[0010] Optionally, the electroplated finished product surface image includes a first grayscale image acquired in orthogonal polarization mode and a second color image acquired in non-polarization mode. Extracting multidimensional optical features from the electroplated finished product surface image includes: Specular reflection suppression analysis was performed on the first grayscale image to extract diffuse reflection and specular reflection features; The second color image is converted to a uniform color space and compared with a preset standard color swatch to extract color features. A gray-level co-occurrence matrix is constructed for the first gray-level image, and surface texture features are extracted based on the gray-level co-occurrence matrix.
[0011] Optionally, the process of fusing actual electroplating process parameters with multi-dimensional optical features to generate process-aware fusion features includes: Calculate the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters, and generate a process deviation vector, which includes the process deviation components corresponding to each process parameter. Based on the preset process-defect sensitivity mapping table, determine the defect type and defect-sensitive optical feature dimension corresponding to each process deviation component; Based on the absolute value of the process deviation component corresponding to each defect type, the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type is determined. The enhancement weights are applied to the corresponding feature dimensions in the multi-dimensional optical features to perform weighted enhancement of the optical features and generate process-aware fusion features.
[0012] Optionally, the step of calculating the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters, and generating a process deviation vector, includes: Based on the actual electroplating process parameters and the preset standard process parameters, the relative deviation between the actual value and the standard value of each process parameter is calculated. Based on the relative deviation of each process parameter, a weighted relative deviation is generated by pre-setting directional weighting coefficients; The weighted relative deviations of each process parameter are combined to generate a process deviation vector.
[0013] Optionally, determining the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type based on the absolute value of the process deviation component corresponding to each defect type includes: For each defect type, the initial enhancement weight of the defect type is calculated based on the absolute value of the process deviation component corresponding to that defect type. When the same optical feature dimension corresponds to multiple defect types, determine whether the enhancement direction of each defect type is the same for that optical feature dimension; If the enhancement directions are the same, the weighted sum or maximum value of the initial enhancement weights for each defect type is taken as the final enhancement weight for that optical feature dimension. If the enhancement directions are different, the enhancement direction of the defect type with the largest initial enhancement weight shall be used to determine the final enhancement weight of that dimension.
[0014] Secondly, this application provides a surface quality inspection system for electroplated finished products, comprising: The data acquisition module (101) is used to acquire information on the electroplated finished product to be tested, including finished product attribute information and actual electroplating process parameters; The image acquisition parameter determination module (102) is used to determine the image acquisition parameters based on the finished product attribute information and the actual electroplating process parameters through a preset parameter mapping table. The image acquisition parameters include lighting parameters and camera parameters. The surface image acquisition module (103) is used to acquire images of the electroplated finished product under test based on image acquisition parameters, and obtain surface images of the electroplated finished product. The feature extraction and fusion module (104) is used to extract multi-dimensional optical features from the surface image of the electroplated finished product. The multi-dimensional optical features include specular reflection features, diffuse reflection features, color features and surface texture features. The actual electroplating process parameters are fused with the multi-dimensional optical features to generate process-aware fusion features. The detection result generation module (105) is used to obtain the surface quality detection results of electroplated finished products based on process perception fusion features and through a preset defect detection model.
[0015] Thirdly, this application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed as described above for a method of detecting the surface quality of electroplated finished products.
[0016] In summary, this application first matches a reference imaging template based on the attribute information of the electroplated finished product, predicts potential defects based on process parameters, and generates parameter adjustment amounts, thereby achieving automatic adaptation of image acquisition parameters. This solves the problems of reflection interference and loss of details caused by fixed parameters in traditional detection. In addition, through a comprehensive quality scoring and re-sampling mechanism, it ensures that the image quality entering the feature extraction stage meets the requirements, which can avoid false detections and missed detections caused by image quality issues to a certain extent. Furthermore, by using actual electroplating process parameters as prior guiding information and deeply integrating them with multi-dimensional optical features, the detection results are not only based on visual information but also incorporate process knowledge, which can significantly improve detection accuracy. Attached Figure Description
[0017] Figure 1This is a flowchart of a method for detecting the surface quality of electroplated finished products provided in an embodiment of this application; Figure 2 This is a flowchart provided in this application embodiment, which determines image acquisition parameters based on finished product attribute information and actual electroplating process parameters through a preset parameter mapping table; Figure 3 This is a flowchart of extracting multidimensional optical features from an image of an electroplated finished product, provided in an embodiment of this application. Figure 4 This is a flowchart provided in the embodiments of this application, which shows how actual electroplating process parameters are fused with multi-dimensional optical features to generate process-aware fusion features; Figure 5 This is a schematic diagram of a surface quality inspection system for electroplated finished products provided in an embodiment of this application. Detailed Implementation
[0018] The following is in conjunction with the appendix Figure 1 -Appendix Figure 5 This application will be described in further detail below.
[0019] This application provides a method for inspecting the surface quality of electroplated finished products. See [link to relevant documentation]. Figure 1 This includes the following steps: S100: Obtain information about the electroplated finished product to be tested.
[0020] S200: Based on the finished product attribute information and actual electroplating process parameters, the image acquisition parameters are determined through a preset parameter mapping table.
[0021] S300: Based on image acquisition parameters, the electroplated finished product under test is image acquired to obtain an image of the surface of the electroplated finished product.
[0022] S400. Extract multi-dimensional optical features from the surface image of the electroplated finished product. The multi-dimensional optical features include specular reflection features, diffuse reflection features, color features, and surface texture features.
[0023] S500 integrates actual electroplating process parameters with multi-dimensional optical features to generate process-aware fusion features.
[0024] S600: Based on process perception fusion features, it obtains the surface quality inspection results of electroplated finished products through a preset defect detection model.
[0025] In this embodiment, electroplating cost information is first obtained. The electroplated finished product information includes finished product attribute information and actual electroplating process parameters. Finished product attribute information includes electroplating type, geometry, and surface condition. Electroplating types include, for example, chromium plating, zinc plating, nickel plating, and copper plating; geometry includes planar and curved surfaces; and surface conditions include high gloss, matte, and frosted finishes.
[0026] Actual electroplating process parameters include current density, plating solution temperature, additive concentration, pH value, and electroplating time. These parameters can be obtained in real time through communication with the electroplating production line control system.
[0027] Then, images can be acquired on the electroplated finished product to obtain surface images. Considering the reflective properties of the electroplated surface, fixed imaging parameters are difficult to capture enough effective image information. Furthermore, different defects have different sensitivities to imaging parameters. For example, scorch defects are sensitive to side lighting (enhancing shadows) and high dynamic range (avoiding overexposure of highlights); pinhole defects are sensitive to high resolution and polarized light; and color difference defects are sensitive to multispectral / true color reproduction and uniform diffuse light.
[0028] Therefore, in this embodiment of the application, in order to match the imaging parameters with the optical characteristics and defect risks of the electroplated finished product, the image acquisition parameters are determined based on the finished product attribute information and the actual electroplating process parameters.
[0029] The image acquisition parameters include illumination parameters and camera parameters. Illumination parameters include light source brightness, illumination angle, and polarization mode; camera parameters include exposure time and gain.
[0030] Specifically, see Figure 2 Based on the finished product attribute information and actual electroplating process parameters, the image acquisition parameters are determined through a preset parameter mapping table, including the following steps: S210. Based on the finished product attribute information, match the corresponding reference imaging template from the preset reference parameter mapping table.
[0031] S220. Based on the electroplating type and actual electroplating process parameters, a potential defect distribution is generated through a preset process defect classification model. The potential defect distribution includes the defect type and its corresponding probability of occurrence.
[0032] S230. Based on the distribution of potential defects, the detectability score of each defect type is evaluated using preset historical detection data.
[0033] S240. Calculate the demand adjustment degree for each defect type based on the detectability score and the probability of occurrence.
[0034] S250. Based on the required adjustment degree for each defect type, the parameter adjustment amount is generated through a preset defect-imaging parameter mapping library.
[0035] S260. Adjust the reference imaging template by adjusting the parameter amount to generate image acquisition parameters.
[0036] First, based on the finished product attribute information, a corresponding benchmark imaging template is matched from a preset benchmark parameter mapping table. This preset benchmark parameter mapping table refers to a pre-established imaging parameter database indexed by combinations of finished product attribute information. Specifically, for each combination of electroplating type, geometry, and surface condition, a large number of representative electroplating samples are collected. Under standard imaging conditions, a parameter optimization algorithm is used to obtain the optimal combination of imaging parameters for image quality. After statistical optimization, a benchmark imaging template corresponding to that attribute combination is formed. The benchmark imaging template includes benchmark illumination parameters and benchmark camera parameters.
[0037] Then, based on the electroplating type and actual electroplating process parameters, a potential defect distribution is generated through a preset process defect classification model. The potential defect distribution is used to characterize the types and probabilities of defects that may occur in the electroplated finished product during the electroplating process. The preset process defect classification model refers to a machine learning model that is pre-trained according to the electroplating type and is used to establish the mapping relationship between electroplating process parameters and electroplating defect types. The input of the model is the actual electroplating process parameter vector, and the output is the probability vector of each defect type, which is used to characterize the types and probabilities of defects that may occur in the electroplated finished product under given process conditions.
[0038] Next, based on the distribution of potential defects, the detectability score of the benchmark imaging template for each defect type is evaluated by using preset historical detection data. The so-called detectability score refers to the expected ability to accurately identify and detect a specific type of defect under given imaging parameters (i.e., the benchmark imaging template here).
[0039] By pre-setting historical detection data, historical detection records with similar parameters to the current benchmark imaging template can be filtered out for various defect types in the potential defect distribution. The actual detection rate of this defect type in historical detection records is statistically analyzed and used as the detectability score for this defect type. .
[0040] Based on detectability score and probability of occurrence It can calculate the demand adjustment degree for each defect type. The demand adjustment degree refers to the urgency of adjusting the imaging parameters based on the baseline imaging template for a specific defect type. The demand adjustment degree is denoted as... , can be represented as .
[0041] Therefore, only when defects are prone to occur ( Large), and at the same time, it is easy to miss detection ( Hour, The higher the required adjustment degree, the more the imaging parameters need to be adjusted to improve the detection capability for this type of defect; the lower the required adjustment degree, the better the baseline imaging template can meet the detection requirements of this defect.
[0042] Finally, based on the required adjustment degree for each defect type, the parameter adjustment amount is generated through a preset defect-imaging parameter mapping library. This preset defect-imaging parameter mapping library records the adjustment direction and unit adjustment range for each imaging parameter for each defect type. The unit adjustment range is the specific value that the imaging parameter needs to be adjusted when the adjustment requirement degree for a certain defect type is 1.0.
[0043] Specifically, based on the required adjustment degree for each defect type, parameter adjustment amounts are generated using a pre-set defect-imaging parameter mapping library, including the following steps: S251. By using a preset defect-imaging parameter mapping library, obtain the adjustment direction and unit adjustment range of each imaging parameter for each defect type.
[0044] S252. For each imaging parameter, multiply the adjustment requirement of each defect type by the corresponding unit adjustment range and sum them to obtain the total adjustment amount of the imaging parameter.
[0045] S253. Combine the total adjustment of all imaging parameters to form the parameter adjustment amount.
[0046] First, by using a preset defect-imaging parameter mapping library, the adjustment direction and unit adjustment range of each imaging parameter for each defect type can be obtained. That is, the unit adjustment range is a signed value, with a positive sign indicating that the parameter is increased and a negative sign indicating that the parameter is decreased.
[0047] Then, for each imaging parameter, the adjustment requirement for each defect type is multiplied by the corresponding unit adjustment magnitude, and the results are summed to obtain the total adjustment amount for that imaging parameter, i.e., for the th... Total adjustment of each imaging parameter , can be represented as: in, For the first The degree of demand adjustment for each type of defect For the first The first type of defect is related to the second. The unit adjustment range of each imaging parameter, This represents the number of defect types included in the potential defect distribution.
[0048] When the total adjustment is positive, this parameter increases from the original baseline value; when the total adjustment is negative, this parameter decreases from the original baseline value; when the algebraic sum is zero or close to zero, this parameter remains unchanged.
[0049] Finally, combining the total adjustments of all imaging parameters yields the parameter adjustment amount. , This represents the number of imaging parameters.
[0050] Once the parameter adjustment amount is determined, the reference imaging template can be adjusted using the parameter adjustment amount to generate image acquisition parameters.
[0051] Once the image acquisition parameters are determined, images of the electroplated finished product under test can be acquired based on these parameters to obtain an image of the surface of the electroplated finished product.
[0052] Considering that electroplated surfaces have strong specular reflection characteristics, the image quality is easily affected by factors such as illumination angle deviation, polarization mode failure, focus deviation, light source attenuation, and ambient light interference. Although the image acquisition parameters are determined, the actual acquired image quality may not meet the standards due to uncontrollable factors such as equipment status fluctuations and environmental changes. If low-quality images are directly input into the detection, it will easily affect the detection results.
[0053] Therefore, after obtaining the surface image of the electroplated product, it is necessary to conduct a quality assessment. If the quality is qualified, it can be input into the subsequent inspection process. If the quality is not qualified, the image acquisition parameters can be adjusted to ensure that the image quality entering the subsequent inspection process meets the requirements.
[0054] Therefore, in this embodiment of the application, after acquiring an image of the electroplated finished product based on the image acquisition parameters and obtaining the surface image of the electroplated finished product, the following steps are included: S310. Extract quality assessment indicators from the surface image of the electroplated finished product.
[0055] S320. For quality assessment indicators, a weighted fusion is performed using preset weighting coefficients to calculate a comprehensive quality score.
[0056] S330. If the overall quality score is lower than the preset quality threshold, adjust the image acquisition parameters and reacquire the quality image of the electroplated finished product.
[0057] First, quality assessment indicators are extracted from the surface image of the electroplated finished product. These indicators include brightness uniformity, sharpness, and reflection suppression.
[0058] Brightness uniformity is used to evaluate the consistency of brightness across different areas of an image, reflecting whether the lighting system uniformly illuminates the surface of the electroplated product. The image can be divided into M×N equal-sized rectangular blocks (e.g., M=8, N=8). For each block, the average grayscale value of all pixels within that block is calculated, and then the overall average grayscale value of all blocks is calculated. and standard deviation Finally, based on the overall mean and standard deviation, the brightness uniformity index U can be determined, which can be expressed as: ,in U is a very small constant (e.g., 0.01) used to prevent division by zero errors. The value of U ranges from [0,1]. The closer the value is to 1, the more uniform the brightness.
[0059] Sharpness metrics are used to evaluate the edge sharpness of an image, reflecting diagonal accuracy. They are obtained by applying a Laplacian filter to the image to obtain a gradient image, and then calculating the variance of the gradient image. The sharpness index S can be determined based on the variance, and can be expressed as: ,in The maximum reference value for image clarity is set. The value of S ranges from [0,1]. The closer the value is to 1, the clearer the image.
[0060] The reflection suppression index is used to assess the degree to which specular reflection is suppressed, reflecting the effectiveness of polarized illumination or illumination angle adjustment. It can be achieved by setting a pre-defined highlight threshold, then counting the pixel data in the image whose grayscale value exceeds the highlight threshold, and calculating the proportion of highlight pixels. Based on the proportion of highlight pixels, the reflection suppression index G can be determined, which can be expressed as: The value of G ranges from [0,1], and the closer the value is to 1, the better the reflection suppression effect.
[0061] After determining the quality assessment indicators, these indicators can be weighted and integrated using preset weighting coefficients to calculate the comprehensive quality score. The comprehensive quality score, Q, can be expressed as: in, , and These are the weighting coefficients for brightness uniformity, sharpness, and glare suppression, respectively. .
[0062] Finally, the overall quality score is compared with the preset quality threshold, which is the critical value of the overall quality score used to determine whether the surface image of the electroplated finished product meets the requirements for subsequent feature extraction and defect detection. This threshold can be obtained through historical data statistics.
[0063] If the overall quality score is higher than the preset quality threshold, it means that the surface image quality of the electroplated finished product meets the requirements, and subsequent testing can continue.
[0064] If the overall quality score is lower than the preset quality threshold, it indicates that the surface image quality of the electroplated product does not meet the requirements. In this case, it is necessary to adjust the image acquisition parameters and then re-acquire the quality image of the electroplated product. For example, when the brightness uniformity index is lower than the target value, adjust the illumination angle or switch the light source type; when the sharpness index is lower than the target value, adjust the camera focus position or aperture size; when the reflection suppression index is lower than the target value, enable or enhance the orthogonal polarization mode.
[0065] After acquiring the surface image of the electroplated product, multidimensional optical features are extracted from it. These features include specular reflection, diffuse reflection, color, and surface texture characteristics. It's important to note that the surface image of the electroplated product here includes a first grayscale image acquired in orthogonal polarization mode and a second color image acquired in non-polarization mode.
[0066] Specifically, see Figure 2 Extracting multidimensional optical features from the surface image of the electroplated finished product includes the following steps: S410. Perform specular reflection suppression analysis on the first grayscale image to extract diffuse reflection features and specular reflection features.
[0067] S420. Convert the second color image to a uniform color space, compare it with a preset standard color swatch, and extract color features.
[0068] S430. Construct a gray-level co-occurrence matrix for the first gray-level image, and extract surface texture features based on the gray-level co-occurrence matrix.
[0069] First, specular reflection suppression analysis is performed on the first grayscale image to extract diffuse reflection features and specular reflection features.
[0070] Because the reflected light from the electroplated surface consists of two parts: specular reflection and diffuse reflection, the specular reflection and diffuse reflection components can be separated by performing specular reflection suppression analysis on the first grayscale image. The so-called specular reflection suppression analysis is an image processing process that separates or suppresses the specular reflection component of the electroplated surface and extracts the diffuse reflection component through the principle of polarization optics.
[0071] The diffuse reflection component reflects the material properties of the electroplated surface. Extracting the intensity distribution characteristics of the diffuse reflection component, including the gray-level mean, gray-level variance, and local variance distribution, yields the diffuse reflection characteristics.
[0072] The specular reflection component reflects the gloss information of the electroplated surface. The uniformity statistical characteristics of the specular reflection component are extracted, including the mean specular reflection intensity, the standard deviation of specular reflection intensity, and the proportion of high gloss areas, which are the specular reflection characteristics.
[0073] Then, the second color image is converted to a uniform color space and compared with a preset standard color swatch to extract color features.
[0074] The second color image is converted to the CIELAB color space. In the CIELAB space, the Euclidean distance between two color points is basically consistent with the color difference perceived by the human eye. Then, by comparing it with a preset standard color chart, a color difference distribution map can be generated. The preset standard color chart is a physical color chart or digital color chart with known standard color values, which is used as a reference benchmark for measuring the color of electroplated finished products.
[0075] The value of each pixel in the color difference distribution map represents the degree of color deviation between that location and the standard color swatch. By extracting the distribution statistics of the color difference distribution map, such as the mean, variance, and extreme values, we obtain the color features.
[0076] Finally, a gray-level co-occurrence matrix (GLCM) is constructed for the first grayscale image. Based on the GLCM, surface texture features can be extracted. The GLCM is a statistical texture analysis method that describes texture features by statistically analyzing the spatial distribution relationships of pixel pairs at different gray levels in an image. Contrast, energy, entropy, and correlation parameters are calculated based on the GLCM, which constitute the surface texture features.
[0077] After acquiring multi-dimensional optical features, the actual electroplating process parameters are fused with the multi-dimensional optical features to generate process-aware fusion features. That is, the actual electroplating process parameters are used as prior guiding information to guide and enhance the expression of visual features, so that the defect detection model can not only see the visual appearance of defects, but also understand the process root cause of defects.
[0078] Specifically, see Figure 4 The process involves fusing actual electroplating process parameters with multi-dimensional optical features to generate process-aware fusion features, including the following steps: S510. Calculate the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters, and generate a process deviation vector, wherein the process deviation vector includes the process deviation components corresponding to each process parameter.
[0079] S520. Based on the preset process-defect sensitivity mapping table, determine the defect type and defect-sensitive optical feature dimension corresponding to each process deviation component.
[0080] S530. Based on the absolute value of the process deviation component corresponding to each defect type, determine the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type.
[0081] S540. Apply the enhancement weights to the corresponding feature dimensions in the multi-dimensional optical features to perform weighted enhancement of the optical features and generate process-aware fusion features.
[0082] First, the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters is calculated, and a process deviation vector is generated. This vector includes the process deviation components corresponding to each process parameter. The preset standard process parameters are benchmark values determined during the electroplating process design phase and verified through mass production to stably produce qualified products. When the actual process parameters deviate from the standard process parameters, the probability of various defects in the electroplated layer increases significantly.
[0083] Specifically, the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters is calculated, and a process deviation vector is generated, including the following steps: S511. Based on the actual electroplating process parameters and the preset standard process parameters, calculate the relative deviation between the actual value and the standard value of each process parameter.
[0084] S512. Based on the relative deviation of each process parameter, a weighted relative deviation is generated by using a preset directional weighting coefficient.
[0085] S513. Combine the weighted relative deviations of each process parameter to generate a process deviation vector.
[0086] Considering that the engineering significance of deviations in various process parameters is different, for example, a deviation of 0.5 in current density may be serious, while a deviation of 5℃ in temperature may not have much impact. In addition, in the electroplating process, when a parameter deviates from the standard value, being too high or too low may lead to different types of defects, or have different degrees of impact on the same defect. For example, a high current density is likely to cause scorching defects, while a low current density is likely to cause an increase in porosity.
[0087] Therefore, on the one hand, it is necessary to convert the deviation between the actual value and the standard value of the process parameter into a dimensionless relative deviation. On the other hand, it is necessary to weight the positive and negative deviations of the same process parameter to the different degrees of influence on the formation of electroplating defects, and finally form a weighted relative deviation.
[0088] Therefore, the first step is to calculate the relative deviation between the actual value and the standard value of each process parameter, that is, to convert the deviation between the actual value and the standard value of the process parameter into a dimensionless deviation vector.
[0089] Record the actual values of the electroplating process parameters. The corresponding standard value is The relative deviation can then be expressed as .
[0090] Based on the relative deviations of various electroplating process parameters, a weighted relative deviation can be generated using preset directional weighting coefficients. These preset directional weighting coefficients are quantitative parameters used to characterize the different degrees of influence of positive and negative deviations of the same process parameter on the formation of electroplating defects. They can be determined by the electroplating process mechanism. For example, the positive deviation of current density has a significantly greater impact on scorching defects than the negative deviation; therefore, the positive weighting coefficient for current density is greater than the negative weighting coefficient.
[0091] Let the preset direction weighting coefficient be... ,but Therefore, the weighted relative deviation of each process parameter can be expressed as: .
[0092] By combining the weighted relative deviations of all process parameters, a process deviation vector can be generated. , This refers to the number of electroplating process parameters.
[0093] Then, based on the preset process-defect sensitivity mapping table, the defect type and defect-sensitive optical feature dimension corresponding to each process deviation component are determined. The defect-sensitive optical feature dimension refers to the feature component that is highly correlated with the detection of a specific type of defect among the extracted multidimensional optical features. The preset process-defect sensitivity mapping table records the correspondence between each process parameter deviation and the defect type, as well as the correspondence between each defect type and the sensitive optical feature dimension.
[0094] After determining each process deviation component, the corresponding defect type, and the defect-sensitive optical feature dimension, the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type can be determined based on the absolute value of the process deviation component corresponding to each defect type.
[0095] Specifically, based on the absolute value of the process deviation component corresponding to each defect type, the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type is determined, including: S531. For each defect type, calculate the initial enhancement weight of the defect type based on the absolute value of the process deviation component corresponding to that defect type.
[0096] S532. When the same optical feature dimension corresponds to multiple defect types, determine whether the enhancement direction of each defect type on the optical feature dimension is the same.
[0097] S533. If the enhancement directions are the same, the weighted sum or maximum value of the initial enhancement weights of each defect type shall be taken as the final enhancement weight of that optical feature dimension.
[0098] S534. If the enhancement directions are different, the enhancement direction of the defect type with the largest initial enhancement weight shall be used as the main factor to determine the final enhancement weight of that dimension.
[0099] For each defect type, the initial reinforcement weight is calculated based on the absolute value of the corresponding process deviation component. The initial reinforcement weight is positively correlated with the absolute value of the process deviation component; the larger the deviation, the higher the reinforcement requirement. For example, for defect type... The corresponding process deviation component is Then the initial enhancement weight for this defect type is .
[0100] Then, for each optical feature dimension, a set of all defect types sensitive to that dimension can be determined. When the same optical feature dimension corresponds to multiple defect types, it is determined whether the enhancement direction of each defect type to that optical feature dimension is the same.
[0101] If each defect type enhances the same optical feature dimension in the same direction, then the weighted sum or maximum value of the initial enhancement weights of each defect type is taken as the final enhancement weight of the optical feature dimension.
[0102] If the enhancement directions are different, the enhancement direction of the defect type with the largest initial enhancement weight shall be used to determine the final enhancement weight of that dimension.
[0103] Finally, the enhancement weights are applied to the corresponding feature dimensions in the multi-dimensional optical features to perform weighted enhancement of the optical features, thereby generating process-aware fusion features. The optical feature dimensions are then denoted as follows: The final enhancement weight is After weighting and enhancing the corresponding feature dimensions in the multidimensional optical features, the enhanced optical feature dimensions can be expressed as: in, This is the preset maximum enhancement coefficient, for example, a value of 0.5.
[0104] In this way, the final generated process-aware fusion features can dynamically enhance the corresponding defect-sensitive optical feature dimensions according to the degree of deviation between process parameters and standard values. This makes the model pay more attention to the defect areas that may be caused by process anomalies, thereby increasing the expression of defect features and helping to improve the accuracy of electroplating surface defect detection.
[0105] Finally, based on the process-aware fusion features, the surface quality inspection results of the electroplated finished product can be obtained by using a preset defect detection model. The preset defect detection model is a defect detection model trained by commonly used target detection models, such as YOLO and Mask-R-CNN, with each type of defect as a category.
[0106] By inputting process-aware fusion features into a preset defect detection model, defect detection results can be output. These results include defect type, defect location, and defect confidence level. Based on each defect detection result, along with process parameter deviation information associated with the defect detection results, the surface quality inspection results of the electroplated product can be generated, which facilitates the indication of the process causes of defects and the direction of process adjustment.
[0107] This application also provides a surface quality inspection system for electroplated finished products, see [link to relevant documentation]. Figure 5 The system includes: a data acquisition module 101, an image acquisition parameter determination module 102, a surface image acquisition module 103, a feature extraction and fusion module 104, and a detection result generation module 105.
[0108] The data acquisition module 101 is used to acquire information about the electroplated finished product to be tested.
[0109] The image acquisition parameter determination module 102 is used to determine the image acquisition parameters based on the information of the electroplated finished product to be tested, through a preset parameter mapping table. The image acquisition parameters include lighting parameters and camera parameters.
[0110] The surface image acquisition module 103 is used to acquire images of the electroplated finished product under test based on image acquisition parameters, and obtain surface images of the electroplated finished product.
[0111] The feature extraction and fusion module 104 is used to extract multi-dimensional optical features from the surface image of the electroplated finished product, fuse the actual electroplating process parameters with the multi-dimensional optical features, and generate process-aware fusion features.
[0112] The test result generation module 105 is used to obtain the surface quality test results of the electroplated finished product based on the process perception fusion features and through a preset defect detection model.
[0113] In this embodiment of the application, the data acquisition module 101 is specifically used to acquire information about the electroplated finished product to be tested, wherein the electroplated finished product information includes finished product attribute information and actual electroplating process parameters.
[0114] The image acquisition parameter determination module 102 is specifically used to determine the image acquisition parameters based on the finished product attribute information and actual electroplating process parameters generated by the data acquisition module 101, through a preset parameter mapping table. The image acquisition parameters include lighting parameters and camera parameters.
[0115] The surface image acquisition module 103 is specifically used to acquire images of the electroplated finished product under test based on the image acquisition parameters generated by the image acquisition parameter determination module 102, and obtain the surface image of the electroplated finished product.
[0116] The feature extraction and fusion module 104 is specifically used to extract multi-dimensional optical features from the electroplated finished product surface image acquired by the surface image acquisition module 103. The multi-dimensional optical features include specular reflection features, diffuse reflection features, color features, and surface texture features. The actual electroplating process parameters are fused with the multi-dimensional optical features to generate process-aware fusion features.
[0117] The detection result generation module 105 is specifically used to obtain the surface quality detection results of the electroplated finished product based on the process-aware fusion features generated by the feature extraction and fusion module 104 and through a preset defect detection model.
[0118] This application also provides a computer-readable storage medium storing a computer program that can be loaded by a processor and executed by any of the above-described methods for detecting the surface quality of electroplated finished products.
[0119] The embodiments described in this application are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the principles of this application should be included within the scope of protection of this application.
Claims
1. A method for inspecting the surface quality of electroplated finished products, characterized in that, include: Obtain information about the electroplated finished product to be tested, including finished product attribute information and actual electroplating process parameters; Based on the finished product attribute information and actual electroplating process parameters, image acquisition parameters are determined through a preset parameter mapping table. The image acquisition parameters include lighting parameters and camera parameters. Based on the image acquisition parameters, the electroplated finished product under test is image acquired to obtain the surface image of the electroplated finished product. Multidimensional optical features are extracted from the surface image of the electroplated finished product. These multidimensional optical features include specular reflection features, diffuse reflection features, color features, and surface texture features. By fusing actual electroplating process parameters with multi-dimensional optical features, process-aware fusion features are generated. Based on process perception fusion features, the surface quality inspection results of electroplated finished products are obtained through a preset defect detection model.
2. The method for detecting the surface quality of electroplated finished products according to claim 1, characterized in that, The finished product attribute information includes electroplating type, geometry, and surface condition. Based on the finished product attribute information and actual electroplating process parameters, the image acquisition parameters are determined through a preset parameter mapping table, including: Based on the finished product attribute information, a corresponding reference imaging template is matched from a preset reference parameter mapping table. The reference imaging template includes reference illumination parameters and reference camera parameters. Based on the electroplating type and actual electroplating process parameters, a potential defect distribution is generated through a preset process defect classification model. The potential defect distribution includes defect types and their corresponding occurrence probabilities. Based on the distribution of potential defects, the detectability score of each defect type is evaluated using preset historical detection data. Calculate the demand adjustment degree for each defect type based on the detectability score and the probability of occurrence; Based on the required adjustment degree for each defect type, parameter adjustment amounts are generated through a preset defect-imaging parameter mapping library; The reference imaging template is adjusted by parameter adjustment to generate image acquisition parameters.
3. The method for detecting the surface quality of electroplated finished products according to claim 2, characterized in that, The adjustment based on the required degree of each defect type, through a preset defect-imaging parameter mapping library, generates parameter adjustment amounts, including: By using a pre-defined defect-imaging parameter mapping library, the adjustment direction and unit adjustment magnitude of each imaging parameter for each defect type are obtained. For each imaging parameter, the total adjustment amount of the imaging parameter is obtained by multiplying the adjustment requirement of each defect type by the corresponding unit adjustment range and summing the results. The total adjustment of all imaging parameters is combined to form the parameter adjustment amount.
4. The method for detecting the surface quality of electroplated finished products according to claim 1, characterized in that, The process of acquiring an image of the electroplated product based on image acquisition parameters, and obtaining an image of the surface of the electroplated product, includes: For the surface image of the electroplated finished product, quality assessment indicators are extracted, including brightness uniformity index, sharpness index and reflection suppression index; The quality assessment indicators are weighted and integrated using preset weighting coefficients to calculate a comprehensive quality score. If the overall quality score is lower than the preset quality threshold, the image acquisition parameters will be adjusted and a new image of the electroplated finished product quality will be acquired.
5. The method for detecting the surface quality of electroplated finished products according to claim 1, characterized in that, The surface image of the electroplated finished product includes a first grayscale image acquired in orthogonal polarization mode and a second color image acquired in non-polarization mode. Extracting multidimensional optical features from the surface image of the electroplated finished product includes: Specular reflection suppression analysis was performed on the first grayscale image to extract diffuse reflection and specular reflection features; The second color image is converted to a uniform color space and compared with a preset standard color swatch to extract color features. A gray-level co-occurrence matrix is constructed for the first gray-level image, and surface texture features are extracted based on the gray-level co-occurrence matrix.
6. The method for detecting the surface quality of electroplated finished products according to claim 1, characterized in that, The process of fusing actual electroplating process parameters with multi-dimensional optical features to generate process-aware fusion features includes: Calculate the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters, and generate a process deviation vector, which includes the process deviation components corresponding to each process parameter. Based on the preset process-defect sensitivity mapping table, determine the defect type and defect-sensitive optical feature dimension corresponding to each process deviation component; Based on the absolute value of the process deviation component corresponding to each defect type, the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type is determined. The enhancement weights are applied to the corresponding feature dimensions in the multi-dimensional optical features to perform weighted enhancement of the optical features and generate process-aware fusion features.
7. The method for detecting the surface quality of electroplated finished products according to claim 6, characterized in that, The calculation of the weighted relative deviation between the actual electroplating process parameters and the preset standard process parameters, and the generation of a process deviation vector, includes: Based on the actual electroplating process parameters and the preset standard process parameters, the relative deviation between the actual value and the standard value of each process parameter is calculated. Based on the relative deviation of each process parameter, a weighted relative deviation is generated by pre-setting directional weighting coefficients; The weighted relative deviations of each process parameter are combined to generate a process deviation vector.
8. The method for detecting the surface quality of electroplated finished products according to claim 6, characterized in that, The step of determining the enhancement weight of the defect-sensitive optical feature dimension corresponding to each defect type based on the absolute value of the process deviation component corresponding to each defect type includes: For each defect type, the initial enhancement weight of the defect type is calculated based on the absolute value of the process deviation component corresponding to that defect type. When the same optical feature dimension corresponds to multiple defect types, determine whether the enhancement direction of each defect type is the same for that optical feature dimension; If the enhancement directions are the same, the weighted sum or maximum value of the initial enhancement weights for each defect type is taken as the final enhancement weight for that optical feature dimension. If the enhancement directions are different, the enhancement direction of the defect type with the largest initial enhancement weight shall be used to determine the final enhancement weight of that dimension.
9. A surface quality inspection system for electroplated finished products, characterized in that, include: The data acquisition module (101) is used to acquire information on the electroplated finished product to be tested, including finished product attribute information and actual electroplating process parameters; The image acquisition parameter determination module (102) is used to determine the image acquisition parameters based on the finished product attribute information and the actual electroplating process parameters through a preset parameter mapping table. The image acquisition parameters include lighting parameters and camera parameters. The surface image acquisition module (103) is used to acquire images of the electroplated finished product under test and obtain surface images of the electroplated finished product. The feature extraction and fusion module (104) is used to extract multi-dimensional optical features from the surface image of the electroplated finished product. The multi-dimensional optical features include specular reflection features, diffuse reflection features, color features and surface texture features. The actual electroplating process parameters are fused with the multi-dimensional optical features to generate process-aware fusion features. The detection result generation module (105) is used to obtain the surface quality detection results of electroplated finished products based on process perception fusion features and through a preset defect detection model.
10. A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing a method for inspecting the surface quality of electroplated finished products as described in any one of claims 1 to 8.