Material detection image defect enhancement method based on multi-light-source complementation

Abstract

The material detection image defect enhancement method based on multi-light source complementation of the application comprises the following steps: collecting and identifying parameters, processing the parameters by a ternary mapping model, and outputting noise feature prediction values and multi-light source initial parameters; adjusting multiple light sources after optimizing the multi-light source initial parameters to obtain optimized parameters, and collecting a noisy image of a material to be detected through a detection camera; inputting the noise feature prediction values, the optimized parameters, and the noisy image into a preset conditional GAN anti-noise model with light source constraints, and generating a defect enhancement image; when the parameter optimization criterion is not met, adjusting the optimized parameters, collecting a new noisy image, and repeating the steps until the parameter optimization criterion is met, and outputting the defect enhancement image, through the cooperation of multi-light source parameter adjustment and the introduction of the conditional GAN, a defect enhancement image with obvious and accurate features is generated, and through the introduction of an artificial intelligence function library and computer visual and auditory training and reinforcement of the model, the accuracy of subsequent defect recognition can be greatly improved.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a method for enhancing defects in material inspection images based on multi-source complementary technology. Background Technology

[0002] In the field of integrated circuit manufacturing, AVI inspection equipment is a commonly used automated visual defect inspection device. It can automatically load and unload materials through a robotic arm, carry out efficient appearance inspection by collecting image data, and form a multi-light source complementary lighting system through multiple sets of light sources. By optimizing the combination of light angle and brightness, it can improve the imaging contrast of material surface defects. In order to adapt to the inspection needs of materials of different sizes and avoid material displacement during the inspection process, the moving platform used in the inspection often adopts a blower adsorption method instead of traditional vacuum adsorption. The negative pressure generated by the blower firmly adsorbs the material on the inspection platform, ensuring the positioning accuracy requirements.

[0003] However, the airflow disturbance generated during the blower adsorption process, and the airflow jet and turbulence cause high-frequency micro-vibrations in the material, which in turn cause the light propagation path to deviate. This results in noise in the acquired image, such as pixel jitter, uneven brightness, and edge scattering, which can mask weak defects such as micron-level cracks. Existing multi-light source systems mostly use fixed parameters or simple time-series switching strategies, without adapting to the dynamic noise characteristics caused by airflow disturbance. Furthermore, conventional image noise reduction algorithms only focus on noise suppression at the pixel level and do not work in conjunction with multi-light source parameters. This can easily lead to over-smoothing, resulting in defect loss or noise residue affecting detection accuracy, making it difficult to meet the requirements for high-precision material defect detection. In addition, existing technologies do not utilize artificial intelligence function libraries to optimize model training efficiency and lack the ability of computer vision and audiovisual software to accurately identify dynamic noise and enhance the extraction of defect features, further limiting the realization of high-precision defect detection. Summary of the Invention

[0004] In view of this, the present invention proposes a material detection image defect enhancement method based on multi-source complementary technology, which can coordinate multiple light sources to optimize parameters in order to generate accurate defect enhancement images and improve defect detection accuracy.

[0005] The technical solution of this invention is implemented as follows:

[0006] A material inspection image defect enhancement method based on multi-light source complementarity is applied to an AVI inspection device including a feeding mechanism, a blower adsorption platform, an inspection camera, a backlight, a camera coaxial light source, a camera ring light source, and a unloading mechanism. The method includes the following steps:

[0007] Step S1: Collect the airflow parameters of the blower adsorption platform, the size parameters of the material to be tested, and the adsorption pressure parameters of the material to be tested as identification parameters.

[0008] Step S2: Input the identification parameters into the pre-trained ternary mapping model, and the ternary mapping model outputs the noise feature prediction value and the initial parameters of the multi-source light source, and determines the noise type;

[0009] Step S3: Optimize the initial parameters of the multi-light source according to the noise type and adsorption pressure parameters. After obtaining the optimized parameters, adjust the backlight, camera coaxial light source and camera ring light source, and acquire noisy images of the material to be detected by the detection camera.

[0010] Step S4: Input the noise feature prediction values, optimization parameters, and noisy image into the preset conditional GAN ​​noise resistance model with light source constraints, and generate a defect enhancement image;

[0011] Step S5: Determine whether the preset parameter optimization criteria are met based on the confidence levels of the noise discriminator and defect discriminator outputs of the conditional GAN ​​noise immunity model.

[0012] Step S6: If the parameter optimization criteria are not met, adjust the optimization parameters, acquire a new noisy image, and repeat steps S4-S5 until the parameter optimization criteria are met and the defect enhancement image is output.

[0013] Preferably, step S1 includes the following steps:

[0014] Step S11: Collect airflow velocity, pressure fluctuation amplitude, and turbulence frequency as airflow parameters using a miniature dynamic airflow sensor installed at the blower outlet.

[0015] Step S12: Collect the adsorption pressure parameters of the material to be tested through the pressure sensor at the bottom of the blower adsorption platform. The adsorption pressure parameters include local pressure distribution data and average pressure.

[0016] Step S13: Obtain the size parameters of the material to be tested based on the feeding mechanism, and normalize the airflow parameters, adsorption pressure parameters, and size parameters as identification parameters.

[0017] Preferably, the specific steps for training the ternary mapping model in step S2 include:

[0018] A BP neural network is constructed as the basic architecture of the ternary mapping model, and the basic architecture includes an input layer, a fully connected hidden layer, and an output layer.

[0019] Historical detection data from AVI inspection equipment is obtained as training samples. A ternary mapping model is trained by calling the gradient descent algorithm from an artificial intelligence function library. The loss function of the ternary mapping model is... for:

[0020]

[0021] Where M is the number of training samples. This represents the true output of the m-th sample group, including predicted noise features and initial parameters from multiple light sources. Let be the identification parameters for the m-th sample. For the BP neural network mapping function, For network parameters, It is an L2 norm.

[0022] Preferably, step S2 includes the following specific steps:

[0023] Step S21: The ternary mapping model receives the recognition parameters and uses the trained network parameters. Calculate the predicted noise features and initial parameters of multiple light sources. The predicted noise features include the amplitude and frequency of high-frequency vibration noise, the gray-level variance of brightness uneven noise, and the gradient amplitude distribution of edge scattering noise. The initial parameters of multiple light sources include the initial brightness, angle, and spectral parameters of the camera coaxial light source, the camera ring light source, and the back light source.

[0024] Step S22: Based on the predicted noise characteristic values, classify the noise type into high-frequency vibration noise, uneven brightness noise, edge scattering noise, or mixed noise.

[0025] Preferably, step S3 includes the following specific steps:

[0026] Step S31: For high-frequency vibration noise, set the focusing accuracy enhancement value of the camera coaxial light source; for uneven brightness noise, set the brightness uniformity enhancement value of the backlight; for edge scattering noise, set the illumination angle reduction value of the camera ring light source.

[0027] Step S32: Determine the pressure concentration area based on the adsorption pressure parameters, and set the local brightness enhancement value of the camera coaxial light source in the pressure concentration area;

[0028] Step S33: Output the focusing accuracy enhancement value, brightness uniformity enhancement value, illumination angle reduction value, and local brightness enhancement value as optimization parameters. Based on the optimization parameters, adjust the backlight, camera coaxial light source, and camera ring light source, and have the detection camera acquire a noisy image of the material to be detected.

[0029] Preferably, step S4 includes the following specific steps:

[0030] Step S41: Construct a conditional GAN ​​noise resistance model that includes a generator, a noise discriminator, and a defect discriminator. The conditional GAN ​​noise resistance model is trained using a multi-constraint loss function. The generator and the defect discriminator are integrated with computer audiovisual software.

[0031] Step S42: The generator is based on the U-Net architecture. It receives the predicted value of noise features, optimization parameters and noisy image. After normalizing the optimization parameters, it embeds them into the encoder and extracts multi-scale image features through the encoder. The decoder outputs the defect-enhanced image.

[0032] Step S43: The noise discriminator adopts the PatchGAN architecture, receives noisy images and defect enhancement images, and outputs noise suppression confidence.

[0033] Step S44: The defect discriminator adopts a CNN architecture, receives the defect enhancement image and the standard defect image, and outputs the defect retention confidence.

[0034] Preferably, the expression for the multi-constraint loss function is:

[0035]

[0036] in These are the weighting coefficients;

[0037] To mitigate losses, Wasserstein distance is used for calculation:

[0038]

[0039] in Represents the mathematical expectation. For standard defect images, The distribution of true standard images, The output is a combined output from two discriminators. Enhance images of defects. The distribution of the generated image;

[0040] For content loss, the following calculations are performed based on the features of the 5th convolutional layer of the VGG16 network:

[0041]

[0042] Where H, W, and C are the dimensions of the output feature map of the 5th convolutional layer of the VGG16 network, respectively. For the pixel coordinate index of the feature map, For VGG16 feature extraction function;

[0043] The light source adaptation loss is used to calculate the fit between brightness uniformity and optimization parameters:

[0044]

[0045] Where N is the total number of image patches into which the generated image is divided, and n is the index of the image patch. Let be the brightness variance of the nth image patch. The target brightness variance is calculated based on the optimized parameter L;

[0046] The defect feature loss is used to calculate the gradient difference in the defect region:

[0047]

[0048] in Standard defect image The total number of pixels in the defective area. For pixel coordinates only within the defect area Summation, This is the gradient operator.

[0049] Preferably, the specific steps of step S5 are as follows:

[0050] Step S51: Based on noise suppression confidence and defect retention confidence level Constructing a weighted comprehensive confidence level , , , These are the weighting coefficients;

[0051] Step S52, the parameter optimization criterion is: comprehensive confidence level. ,and ,in , The preset threshold;

[0052] Step S53: Determine whether the parameter optimization criteria are met based on the noise suppression confidence, defect retention confidence, and overall confidence.

[0053] Preferably, step S6 includes the following specific steps:

[0054] Step S61, if If the noise suppression is insufficient, the backlight brightness is increased and the camera ring light angle is reduced.

[0055] Step S62, if When the defect is deemed insufficient, the backlight brightness is reduced, the camera coaxial light source brightness is increased, and the camera ring light source angle is increased.

[0056] Step S63, if When the noise suppression is insufficient and the defect retention is insufficient, the optimization parameters corresponding to the noise type are optimized first.

[0057] Step S64: After adjusting the backlight, camera coaxial light source, and camera ring light source based on the new optimized parameters, new noisy images are acquired by the detection camera, and steps S4-S5 are repeated.

[0058] Step S65: When the parameter optimization criteria are met, output the defect enhancement image.

[0059] Preferably, the enhanced defect image output in step S6 is transmitted to the detection system of the AVI inspection equipment for defect type and location identification, and the defect false detection rate and false positive rate are calculated. If the false detection rate or false positive rate is greater than a preset value, the network parameters of the ternary mapping model are updated using the online gradient descent method. The updated formula is:

[0060]

[0061] in For the current network parameters, For the updated network parameters, The iteration step size, The gradient of the loss function under the current network parameters. Input parameters for the current operating condition. The output parameters are the actual parameters under the current operating conditions.

[0062] Compared with the prior art, the beneficial effects of the present invention are:

[0063] During the material loading process, the AVI detection equipment collects corresponding airflow parameters, size parameters, and adsorption pressure parameters as identification parameters. Then, a ternary mapping model generates initial parameters for multiple light sources, achieving initial preheating of the multiple light sources. The ternary mapping model also outputs predicted noise features and determines the noise type. Based on the noise type and adsorption pressure parameters, the initial parameters for the multiple light sources can be further optimized. After adjusting the multiple light sources, the AVI equipment can control the detection camera to acquire noisy images of the material. These noisy images, along with the predicted noise features and optimized parameters, can be input into a conditional GAN ​​noise resistance model. The dual discriminator of the conditional GAN ​​noise resistance model can output the corresponding confidence scores. The system determines whether the parameter optimization criteria are met. If not, the optimized parameters are further adjusted until the criteria are met. Then, the defect-enhanced image generated by the conditional GAN ​​noise-resistance model is output to the subsequent detection system to identify whether the material to be inspected has defects and the location of the defects. Through the coordinated adjustment and optimization of multi-source parameters, images with enhanced defect features can be obtained, improving the accuracy of subsequent defect detection. At the same time, the training efficiency and parameter convergence stability of the ternary mapping model are improved by using artificial intelligence function libraries, and the precision of defect feature extraction and noise suppression is enhanced by relying on computer vision and audiovisual software, further reducing the false negative rate and false positive rate of micron-level defects, and adapting to the high-precision detection needs under complex working conditions. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only preferred embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a flowchart of the material detection image defect enhancement method based on multi-light source complementarity of the present invention;

[0066] Figure 2 This is a structural diagram of the AVI inspection device used in the material inspection image defect enhancement method based on multi-light source complementarity of the present invention;

[0067] Figure 3 This is a structural diagram of the camera coaxial light source and camera ring light source of the AVI inspection device based on the material inspection image defect enhancement method of multi-light source complementarity of the present invention;

[0068] In the diagram, 1 is the feeding mechanism; 2 is the blower adsorption platform; 3 is the detection camera; 4 is the backlight; 5 is the camera coaxial light source; 6 is the camera ring light source; and 7 is the unloading mechanism. Detailed Implementation

[0069] To better understand the technical content of this invention, a specific embodiment is provided below, and the invention will be further described in conjunction with the accompanying drawings.

[0070] See Figures 1 to 3 The present invention provides a material inspection image defect enhancement method based on multi-light source complementarity, which is applied to an AVI inspection device including a feeding mechanism 1, a blower adsorption platform 2, an inspection camera 3, a backlight 4, a camera coaxial light source 5, a camera ring light source 6, and a unloading mechanism 7, and includes the following steps:

[0071] Step S1: Collect the airflow parameters of the blower adsorption platform 2, the size parameters of the material to be tested, and the adsorption pressure parameters of the material to be tested as identification parameters.

[0072] Step S2: Input the identification parameters into the pre-trained ternary mapping model, and the ternary mapping model outputs the noise feature prediction value and the initial parameters of the multi-source light source, and determines the noise type;

[0073] Step S3: Optimize the initial parameters of the multi-light source according to the noise type and adsorption pressure parameters. After obtaining the optimized parameters, adjust the backlight 4, the camera coaxial light source 5, and the camera ring light source 6, and acquire the noisy image of the material to be detected through the detection camera 3.

[0074] Step S4: Input the noise feature prediction values, optimization parameters, and noisy image into the preset conditional GAN ​​noise resistance model with light source constraints, and generate a defect enhancement image;

[0075] Step S5: Determine whether the preset parameter optimization criteria are met based on the confidence levels of the noise discriminator and defect discriminator outputs of the conditional GAN ​​noise immunity model.

[0076] Step S6: If the parameter optimization criteria are not met, adjust the optimization parameters, acquire a new noisy image, and repeat steps S4-S5 until the parameter optimization criteria are met and the defect enhancement image is output.

[0077] This invention relates to a material inspection image defect enhancement method based on multi-light source complementarity, applied in an AVI inspection device. The material to be inspected is automatically fed onto a feeding mechanism 1, and then onto a blower-adsorption platform 2. A blower ensures stable adsorption of the material. The blower-adsorption platform 2 can then move horizontally below an inspection camera 3 for image acquisition. To improve defect detection accuracy, a backlight 4 is installed on the blower-adsorption platform 2 to illuminate the bottom of the material, providing uniform transmitted light and enhancing edge contour integrity. A coaxial light source 5 and a ring light source 6 are provided. The coaxial light source 5 illuminates the material surface coaxially and perpendicularly, forming uniform diffuse / specular reflection control, focusing on small planar defects, and forming high reflectivity suppression. The ring light source 6 can be angled to focus on edge defects and identify concave and convex contours. Under the illumination of multiple light sources, after the detection camera 3 acquires the image of the material to be detected, the blower adsorption platform 2 can move and reset, so that the unloading mechanism 7 can automatically unload the material to be detected. The image of the material to be detected can be transmitted to the subsequent detection system for specific defect determination and location.

[0078] When the material to be tested is placed on the blower adsorption platform 2 and the blower is turned on for adsorption, the airflow parameters generated by the blower during operation can be collected. Simultaneously, the feeding mechanism 1 can collect the size parameters of the material to be tested. As the material is adsorbed onto the blower adsorption platform 2, it generates different pressures on the platform, thus allowing the collection of adsorption pressure parameters. These collected parameters are then combined to form identification parameters and input into a pre-trained airflow-noise-light source ternary mapping model. The ternary mapping model uses airflow parameters as the primary input, size parameters and adsorption pressure parameters as auxiliary inputs, noise feature prediction values ​​as intermediate variables, and initial parameters for multiple light sources as outputs, including brightness... Based on the multi-source parameters such as degree, angle, and focus, the backlight 4, camera coaxial light source 5, and camera ring light source 6 can be initialized, adjusted, and preheated to ensure that the parameters of the multi-source light sources are within a suitable range. At the same time, the noise type can be determined based on the noise characteristic prediction value. Different noise types have different requirements for different light sources. Therefore, based on the noise type and adsorption pressure parameters, the initial parameters of the multi-source light sources can be further optimized. After obtaining the optimized parameters, the multi-source light sources can be readjusted to illuminate different noise types. At this point, the multi-source parameter adjustment is completed, and the detection camera 3 can be started to acquire images of the material to be detected and obtain images containing noise.

[0079] To ensure accurate defect identification and localization by the subsequent detection system, the noisy image needs further processing to enhance it. To address this, this invention introduces a conditional GAN ​​noise-resistance model with light source constraints. Its inputs include predicted noise features, optimization parameters, and the noisy image. The built-in generator produces an enhanced defect image. To ensure accurate defect identification, it's necessary to verify that the enhanced image meets the requirements. Therefore, the conditional GAN ​​noise-resistance model incorporates dual discriminators: a noise discriminator and a defect discriminator. These discriminators output confidence scores to determine the noise removal effectiveness. The results and the integrity of the defect features are determined by the confidence levels of the noise discriminator and the defect discriminator outputs. It can be determined whether the parameter optimization criteria are met. If the parameter optimization criteria are not met, it indicates that the noise suppression may be insufficient or the defect retention may be insufficient. At this time, the optimization parameters can be further adjusted, and a new noisy image is acquired again after adjustment. Then, the processing process of the conditional GAN ​​noise resistance model and the judgment process of the parameter optimization criteria are repeated. When the parameter optimization criteria are met, the defect enhancement image generated by the conditional GAN ​​noise resistance model is output to the subsequent detection equipment. The detection equipment can then accurately identify the defect type and specific location on the material to be inspected.

[0080] Preferably, step S1 includes the following steps:

[0081] Step S11: Collect airflow velocity, pressure fluctuation amplitude, and turbulence frequency as airflow parameters using a miniature dynamic airflow sensor installed at the blower outlet.

[0082] Step S12: Collect the adsorption pressure parameters of the material to be tested through the pressure sensor at the bottom of the blower adsorption platform 2. The adsorption pressure parameters include local pressure distribution data and average pressure.

[0083] Step S13: Obtain the size parameters of the material to be tested based on the feeding mechanism 1, and normalize the airflow parameters, adsorption pressure parameters and size parameters as identification parameters.

[0084] After setting multiple sets of miniature dynamic airflow sensors at the blower outlet, airflow parameters can be collected. At the same time, pressure sensors can be evenly distributed on the blower adsorption platform 2 to collect the adsorption pressure parameters when the material to be tested is adsorbed. The data collected by the pressure sensors at different locations can determine the local pressure distribution data, and after averaging, the average pressure value can be obtained. In addition, the feeding mechanism 1 is equipped with sensors that can identify the size parameters of the material to be tested. Then, the collected three-dimensional parameters are normalized to unify the data dimensions and form an identification parameter vector, which serves as the input for the subsequent ternary mapping model.

[0085] Preferably, the specific steps for training the ternary mapping model in step S2 include:

[0086] A BP neural network is constructed as the basic architecture of the ternary mapping model, and the basic architecture includes an input layer, a fully connected hidden layer, and an output layer.

[0087] Historical detection data from AVI inspection equipment is obtained as training samples. A ternary mapping model is trained by calling the gradient descent algorithm from an artificial intelligence function library. The loss function of the ternary mapping model is... for:

[0088]

[0089] Where M is the number of training samples. This represents the true output of the m-th sample group, including predicted noise features and initial parameters from multiple light sources. Let be the identification parameters for the m-th sample. For the BP neural network mapping function, For network parameters, It is an L2 norm.

[0090] The input layer receives normalized recognition parameters, the hidden layer uses the ReLU activation function, and a Dropout layer is introduced to suppress overfitting. The output layer outputs predicted noise features and initial parameters from multiple light sources through a linear activation function. During model training, the airflow-pressure parameter interpolation-noise feature synthesis method is used to expand the training samples, and the gradient descent algorithm module and matrix operation module in the artificial intelligence function library are called. The optimization algorithm encapsulated in the function library improves training efficiency and ensures the network parameters are optimized. The convergence accuracy.

[0091] Preferably, step S2 includes the following specific steps:

[0092] Step S21: The ternary mapping model receives the recognition parameters and uses the trained network parameters. Calculate the predicted noise features and initial parameters of the multiple light sources. The predicted noise features include the amplitude and frequency of high-frequency vibration noise, the gray-level variance of brightness uneven noise, and the gradient amplitude distribution of edge scattering noise. The initial parameters of the multiple light sources include the initial brightness, angle, and spectral parameters of the camera coaxial light source 5, the camera ring light source 6, and the backlight 4.

[0093] Step S22: Based on the predicted noise characteristic values, classify the noise type into high-frequency vibration noise, uneven brightness noise, edge scattering noise, or mixed noise.

[0094] The ternary mapping model calculates noise feature predictions based on the input recognition parameters and the trained network parameters. Then, based on the different noise feature predictions, it can classify the noise type. For example, when the amplitude and frequency of high-frequency vibration noise are high, it is identified as high-frequency vibration noise, while when the gray-level variance is low, it is identified as uneven brightness noise. Simultaneously, the ternary mapping model can output initial parameters for multiple light sources, including the initial brightness, angle, and spectral parameters of the corresponding light sources. These initial parameters allow for rapid preheating of multiple light sources, ensuring that the parameters are within a reasonable illumination range and adapted to the corresponding noise type. Furthermore, during training, the model utilizes batch data processing interfaces and loss function calculation tools provided by the AI ​​function library to simplify the training process and reduce the computational complexity of network parameter iteration. Additionally, the numerical stability optimization of the function library avoids gradient vanishing or exploding problems during training.

[0095] Preferably, step S3 includes the following specific steps:

[0096] Step S31: For high-frequency vibration noise, set the focusing accuracy enhancement value of the camera coaxial light source 5; for uneven brightness noise, set the brightness uniformity enhancement value of the backlight 4; for edge scattering noise, set the illumination angle reduction value of the camera ring light source 6.

[0097] Step S32: Determine the pressure concentration area based on the adsorption pressure parameters, and set the local brightness enhancement value of the camera coaxial light source 5 in the pressure concentration area;

[0098] Step S33: Output the focusing accuracy enhancement value, brightness uniformity enhancement value, illumination angle reduction value, and local brightness enhancement value as optimization parameters. Based on the optimization parameters, adjust the backlight 4, camera coaxial light source 5, and camera ring light source 6, and have the detection camera 3 acquire a noisy image of the material to be detected.

[0099] The initial parameters of the multiple light sources are not the final adjustment parameters. Depending on the noise type and adsorption pressure parameters, the light source parameters need further optimization. Specifically, for different noise types, the parameters of the backlight 4, the camera coaxial light source 5, and the camera ring light source 6 need to be adjusted separately. For high-frequency vibration noise, the focusing accuracy of the camera coaxial light source 5 can be enhanced. For uneven brightness noise, the uniformity of brightness of the backlight 4 can be improved. For edge scattering noise, the illumination angle of the camera ring light source 6 can be reduced. In addition, the pressure concentration area can be determined by the adsorption pressure parameters. For the pressure concentration area, the corresponding light source parameters can be locally corrected, such as enhancing the local brightness of the camera coaxial light source 5, to ensure sufficient supplementary lighting in the material micro-deformation area caused by adsorption pressure differences.

[0100] After optimizing the multiple light sources based on the optimized parameters, the adjustment process of the multiple light sources is completed. At this time, the detection camera 3 can be started and acquire a noisy image containing the material to be detected.

[0101] Preferably, step S4 includes the following specific steps:

[0102] Step S41: Construct a conditional GAN ​​noise resistance model that includes a generator, a noise discriminator, and a defect discriminator. The conditional GAN ​​noise resistance model is trained using a multi-constraint loss function. The generator and the defect discriminator are integrated with computer audiovisual software.

[0103] Step S42: The generator is based on the U-Net architecture. It receives the predicted value of noise features, optimization parameters and noisy image. After normalizing the optimization parameters, it embeds them into the encoder and extracts multi-scale image features through the encoder. The decoder outputs the defect-enhanced image.

[0104] Step S43: The noise discriminator adopts the PatchGAN architecture, receives noisy images and defect enhancement images, and outputs noise suppression confidence.

[0105] Step S44: The defect discriminator adopts a CNN architecture, receives the defect enhancement image and the standard defect image, and outputs the defect retention confidence.

[0106] Defect features in noisy images may not be obvious enough, thus requiring feature enhancement to facilitate accurate defect identification by subsequent detection systems. To address this, this invention introduces a conditional GAN ​​noise-resistant model with a dual-discriminator structure. Its architecture consists of a lightweight generator and two discriminators. The generator, based on the U-Net architecture, receives the noisy image, predicted noise features, and optimized parameters. It extracts multi-scale image features through a four-layer convolutional encoder and embeds the normalized optimized parameters as a one-dimensional vector into the encoder feature map. This is then processed by a four-layer deconvolutional decoder and a skip connection structure to output an enhanced defect image. The noisy image is preprocessed using an adaptive noise reduction algorithm from computer vision software to enhance the grayscale differences of micron-level defects and preserve their characteristics. To ensure the integrity of defect features and the effectiveness of noise suppression, a dual discriminator is used to calculate confidence levels. The noise discriminator, based on the enhanced defect image and the noisy image, outputs a noise suppression confidence level that characterizes the noise removal effect. The defect discriminator, using contour matching tools from computer vision software, accurately identifies the edge gradient features of weak defects such as cracks and dents, preventing defect features from being masked by noise. It outputs a defect retention confidence level that characterizes the integrity of defect features based on the enhanced defect image and the standard defect image. The confidence levels output by the two discriminators are combined with parameter optimization criteria for judgment. If the parameter optimization criteria are met, the enhanced defect image can be output to the subsequent detection system for definitive detection and localization.

[0107] Preferably, the expression for the multi-constraint loss function is:

[0108]

[0109] in These are the weighting coefficients. ,Right now ;

[0110] To mitigate losses, Wasserstein distance is used for calculation:

[0111]

[0112] in Represents the mathematical expectation. For standard defect images, The distribution of true standard images, The output is a combined output from two discriminators. Enhance images of defects. The distribution of the generated image;

[0113] For content loss, the following calculations are performed based on the features of the 5th convolutional layer of the VGG16 network:

[0114]

[0115] Where H, W, and C are the dimensions of the output feature map of the 5th convolutional layer of the VGG16 network, respectively. For the pixel coordinate index of the feature map, For VGG16 feature extraction function;

[0116] The light source adaptation loss is used to calculate the fit between brightness uniformity and optimization parameters:

[0117]

[0118] Where N is the total number of image patches into which the generated image is divided, and n is the index of the image patch. Let be the brightness variance of the nth image patch. The target brightness variance is calculated based on the optimized parameter L;

[0119] The defect feature loss is used to calculate the gradient difference in the defect region:

[0120]

[0121] in Standard defect image The total number of pixels in the defective area. For pixel coordinates only within the defect area Summation, This is the gradient operator.

[0122] Adversarial loss can enhance the realism of generated defects, making it difficult for dual discriminators to distinguish between generated and standard defect images. Content loss is used to match the high-level semantic features of generated and standard images, preventing micron-level defect details from being over-smoothed. Light source adaptation loss allows the loudness and brightness distribution of generated images to be adapted to the actual parameters of the optimized light source, offsetting brightness fluctuations caused by airflow disturbances. Defect feature loss can focus on the gradient and edge features of defect areas, preventing the loss of weak defects such as microcracks.

[0123] Preferably, the specific steps of step S5 are as follows:

[0124] Step S51: Based on noise suppression confidence and defect retention confidence level Constructing a weighted comprehensive confidence level , , , These are the weighting coefficients;

[0125] Step S52, the parameter optimization criterion is: comprehensive confidence level. ,and ,in , The preset threshold;

[0126] Step S53: Determine whether the parameter optimization criteria are met based on the noise suppression confidence, defect retention confidence, and overall confidence.

[0127] The parameter optimization criteria need to be combined with the confidence scores of the dual discriminators, and three preset thresholds are set. Used for comparison with the overall confidence level, with a value set to 0.9. The confidence scores are used for comparison with the outputs of the dual discriminator, with values ​​of 0.85 and 0.9 respectively. The overall confidence score is calculated using a weighted summation method, where the weighting coefficients are... , The values ​​are set to 0.45 and 0.55 respectively to ensure that defect retention takes precedence over noise suppression.

[0128] When the threshold conditions of the parameter optimization criterion are met simultaneously, it is determined that the current optimized parameters and the output of the noise resistance model meet the optimization requirements. If either condition is not met, it is determined that the parameter optimization criterion is not met, and the optimized parameters need to be further adjusted.

[0129] Preferably, step S6 includes the following specific steps:

[0130] Step S61, if At that time, it was determined that the noise suppression was insufficient, so the brightness of the backlight 4 was increased and the angle of the camera ring light 6 was reduced;

[0131] Step S62, if When the defect is not retained, the brightness of the backlight 4 is reduced, the brightness of the camera coaxial light source 5 is increased, and the angle of the camera ring light source 6 is increased.

[0132] Step S63, if When the noise suppression is insufficient and the defect retention is insufficient, the optimization parameters corresponding to the noise type are optimized first.

[0133] Step S64: After adjusting the backlight 4, camera coaxial light source 5, and camera ring light source 6 based on the new optimized parameters, new noisy images are acquired by the detection camera 3, and steps S4-S5 are repeated.

[0134] Step S65: When the parameter optimization criteria are met, output the defect enhancement image.

[0135] when If the parameter optimization criteria are met, the corresponding defect enhancement image will be output to the subsequent detection system.

[0136] when When the noise suppression is insufficient, the adjustment strategy is to increase the brightness of the backlight 4, reduce the angle of the camera ring light source 6, and optimize the focusing parameters of the camera coaxial light source 5 to enhance the anti-vibration interference capability.

[0137] when When the defect was deemed to be insufficiently preserved, the brightness of the backlight 4 was reduced, the brightness of the camera coaxial light source 5 was increased, and the angle of the camera ring light source 6 was increased to highlight the edge of the defect.

[0138] when When the condition is determined to be double non-satisfied, the adjustment strategy is to prioritize optimizing the optimization parameters corresponding to the noise type based on the noise type output by the ternary mapping model. For example, when dealing with high-frequency vibration noise, the focus is on adjusting the camera coaxial light source 5, and then the parameters of other light sources are finely adjusted proportionally.

[0139] After fine-tuning the optimization parameters, new noisy images can be acquired again, and steps S4-S5 can be repeated until the parameter optimization criteria are met. Then, the defect enhancement image generated by the generator is output to the subsequent detection system.

[0140] Preferably, the enhanced defect image output in step S6 is transmitted to the detection system of the AVI inspection equipment for defect type and location identification, and the defect false detection rate and false positive rate are calculated. If the false detection rate or false positive rate is greater than a preset value, the network parameters of the ternary mapping model are updated using the online gradient descent method. The updated formula is:

[0141]

[0142] in For the current network parameters, For the updated network parameters, The iteration step size, The gradient of the loss function under the current network parameters. Input parameters for the current operating condition. The output parameters are the actual parameters under the current operating conditions.

[0143] After the enhanced defect image is output to the subsequent detection system, defects can be identified and located, and the defect missed detection rate and false judgment rate can be calculated. When the indicators exceed the standard, the recorded identification parameters, optimization parameters, noise feature prediction values, etc., can be used to update the network parameters of the ternary mapping model in real time with the help of online gradient descent. The optimized ternary mapping model can be applied to subsequent detection, forming a dynamic closed loop of detection-feedback-update-optimization. This effectively makes up for the insufficient adaptation of the pre-trained model to real-time operating condition fluctuations, continuously improves the accuracy of the initial parameter output of multi-source lights, reduces defect missed detection and false judgment caused by blower adsorption airflow disturbance, material characteristic differences, etc., and ensures the stability and high accuracy of the equipment in long-term detection.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for enhancing material inspection images based on multi-light source complementarity, applied to an AVI inspection device including a feeding mechanism, a blower adsorption platform, an inspection camera, a backlight, a camera coaxial light source, a camera ring light source, and a unloading mechanism, characterized in that, Includes the following steps: Step S1: Collect the airflow parameters of the blower adsorption platform, the size parameters of the material to be tested, and the adsorption pressure parameters of the material to be tested as identification parameters. Step S2: Input the identification parameters into the pre-trained ternary mapping model, and the ternary mapping model outputs the noise feature prediction value and the initial parameters of the multi-source light source, and determines the noise type; Step S3: Optimize the initial parameters of the multi-light source according to the noise type and adsorption pressure parameters. After obtaining the optimized parameters, adjust the backlight, camera coaxial light source and camera ring light source, and acquire noisy images of the material to be detected by the detection camera. Step S4: Input the noise feature prediction values, optimization parameters, and noisy image into the preset conditional GAN ​​noise resistance model with light source constraints, and generate a defect enhancement image; Step S5: Determine whether the preset parameter optimization criteria are met based on the confidence levels of the noise discriminator and defect discriminator outputs of the conditional GAN ​​noise immunity model. Step S6: If the parameter optimization criteria are not met, adjust the optimization parameters, acquire a new noisy image, and repeat steps S4-S5 until the parameter optimization criteria are met and the defect enhancement image is output. The specific steps for training the ternary mapping model in step S2 include: A BP neural network is constructed as the basic architecture of the ternary mapping model, and the basic architecture includes an input layer, a fully connected hidden layer, and an output layer. Historical detection data from AVI inspection equipment is obtained as training samples. A ternary mapping model is trained by calling the gradient descent algorithm from an artificial intelligence function library. The loss function of the ternary mapping model is... for: Where M is the number of training samples. This represents the true output of the m-th sample group, including predicted noise features and initial parameters from multiple light sources. Let be the identification parameters for the m-th sample. For the BP neural network mapping function, For network parameters, It is an L2 norm; The specific steps of step S2 include: Step S21: The ternary mapping model receives the recognition parameters and uses the trained network parameters. The noise feature prediction values ​​and initial parameters of the multiple light sources are calculated. The noise feature prediction values ​​include the amplitude and frequency of high-frequency vibration noise, the gray-level variance of brightness unevenness noise, and the gradient amplitude distribution of edge scattering noise. The initial parameters of the multiple light sources include the initial brightness, angle, and spectral parameters of the camera coaxial light source, the camera ring light source, and the back light source. Step S22: Based on the predicted noise characteristic values, classify the noise type into high-frequency vibration noise, uneven brightness noise, edge scattering noise, or mixed noise; The specific steps of step S3 include: Step S31: For high-frequency vibration noise, set the focusing accuracy enhancement value of the camera coaxial light source; for uneven brightness noise, set the brightness uniformity enhancement value of the backlight; for edge scattering noise, set the illumination angle reduction value of the camera ring light source. Step S32: Determine the pressure concentration area based on the adsorption pressure parameters, and set the local brightness enhancement value of the camera coaxial light source in the pressure concentration area; Step S33: Output the focus accuracy enhancement value, brightness uniformity enhancement value, illumination angle reduction value, and local brightness enhancement value as optimization parameters. Adjust the backlight, camera coaxial light source, and camera ring light source based on the optimization parameters, and have the detection camera acquire a noisy image of the material to be detected. The specific steps of step S4 include: Step S41: Construct a conditional GAN ​​noise resistance model that includes a generator, a noise discriminator, and a defect discriminator. The conditional GAN ​​noise resistance model is trained using a multi-constraint loss function. The generator and the defect discriminator are integrated with computer audiovisual software. Step S42: The generator is based on the U-Net architecture. It receives the predicted value of noise features, optimization parameters and noisy image. After normalizing the optimization parameters, it embeds them into the encoder and extracts multi-scale image features through the encoder. The decoder outputs the defect-enhanced image. Step S43: The noise discriminator adopts the PatchGAN architecture, receives noisy images and defect enhancement images, and outputs noise suppression confidence. Step S44: The defect discriminator adopts a CNN architecture, receives the defect enhancement image and the standard defect image, and outputs the defect retention confidence.

2. The method for enhancing material inspection images based on multi-source complementary technology according to claim 1, characterized in that, The specific steps of step S1 include: Step S11: Collect airflow velocity, pressure fluctuation amplitude, and turbulence frequency as airflow parameters using a miniature dynamic airflow sensor installed at the blower outlet. Step S12: Collect the adsorption pressure parameters of the material to be tested through the pressure sensor at the bottom of the blower adsorption platform. The adsorption pressure parameters include local pressure distribution data and average pressure. Step S13: Obtain the size parameters of the material to be tested based on the feeding mechanism, and normalize the airflow parameters, adsorption pressure parameters, and size parameters as identification parameters.

3. The method for enhancing material inspection images based on multi-source complementary technology according to claim 1, characterized in that, The expression for the multi-constraint loss function is: in These are the weighting coefficients; To mitigate losses, Wasserstein distance is used for calculation: in Represents the mathematical expectation. For standard defect images, The distribution of true standard images, The output is a combined output from two discriminators. Enhance images of defects. The distribution of the generated image; For content loss, the following calculations are performed based on the features of the 5th convolutional layer of the VGG16 network: Where H, W, and C are the dimensions of the output feature map of the 5th convolutional layer of the VGG16 network, respectively. For the pixel coordinate index of the feature map, For VGG16 feature extraction functions; The light source adaptation loss is used to calculate the fit between brightness uniformity and optimization parameters: Where N is the total number of image patches into which the generated image is divided, and n is the index of the image patch. Let be the brightness variance of the nth image patch. The target brightness variance is calculated based on the optimized parameter L; The defect feature loss is used to calculate the gradient difference in the defect region: in Standard defect image The total number of pixels in the defective area. For pixel coordinates only within the defect area Summation, This is the gradient operator.

4. The method for enhancing material inspection images based on multi-source complementary technology according to claim 1, characterized in that, The specific steps of step S5 are as follows: Step S51: Based on noise suppression confidence and defect retention confidence level Constructing a weighted comprehensive confidence level , , , These are the weighting coefficients; Step S52, the parameter optimization criterion is: comprehensive confidence level. ,and ,in , The preset threshold; Step S53: Determine whether the parameter optimization criteria are met based on the noise suppression confidence, defect retention confidence, and overall confidence.

5. The method for enhancing material inspection images based on multi-source complementary technology according to claim 4, characterized in that, The specific steps of step S6 include: Step S61, if If the noise suppression is insufficient, the backlight brightness is increased and the camera ring light angle is reduced. Step S62, if When the defect is deemed insufficient, the backlight brightness is reduced, the camera coaxial light source brightness is increased, and the camera ring light source angle is increased. Step S63, if When the noise suppression is insufficient and the defect retention is insufficient, the optimization parameters corresponding to the noise type are optimized first. Step S64: After adjusting the backlight, camera coaxial light source, and camera ring light source based on the new optimized parameters, new noisy images are acquired by the detection camera, and steps S4-S5 are repeated. Step S65: When the parameter optimization criteria are met, output the defect enhancement image.

6. The method for enhancing material inspection images based on multi-source complementary technology according to claim 1, characterized in that, The enhanced defect image output in step S6 is transmitted to the detection system of the AVI inspection equipment for defect type and location identification, and the defect false detection rate and false positive rate are calculated. If the false detection rate or false positive rate is greater than the preset value, the network parameters of the ternary mapping model are updated using the online gradient descent method. The updated formula is: in For the current network parameters, For the updated network parameters, The iteration step size, The gradient of the loss function under the current network parameters. Input parameters for the current operating condition. The output parameters are the actual parameters under the current operating conditions.

Images