Multi-scale aluminum plate surface defect detection system and method based on deep learning

Abstract

The present application relates to the technical field of machine vision detection and industrial surface defect detection, in particular to a multi-scale aluminum plate surface defect detection system and method based on deep learning, comprising: when the same aluminum plate passes through the detection station, a same imaging device collects first and second aluminum plate surface images in sequence according to first and second exposure parameters; the two images are registered along the conveying direction, and the first aluminum plate surface image after registration is subjected to intensity alignment based on an exposure proportionality coefficient; the residual error of the second aluminum plate surface image and the intensity alignment image is calculated, and a suppression mask is generated through morphological enhancement; shallow features are extracted from the second aluminum plate surface image, and the suppression mask is weighted bit by bit; the weighted feature map is input into a defect detection network containing a multi-scale feature fusion structure to generate an aluminum plate surface defect detection result, and optical ripple artifacts caused by mechanical micro-vibration can be separated and suppressed.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection and industrial surface defect detection technology, specifically to a multi-scale aluminum plate surface defect detection system and method based on deep learning. Background Technology

[0002] Currently, visual inspection of aluminum plate surface defects generally involves directly inputting acquired images into a neural network for discrimination, and improving stability through data enhancement or parameter tuning; however, existing technologies have the following drawbacks in industrial settings:

[0003] Due to the influence of mechanical micro-vibrations and lighting fluctuations in the production line, images are prone to producing stripe artifacts that closely resemble the morphology of real defects. Existing solutions mostly rely on single-frame or fixed-exposure imaging, lacking response difference information under multiple time-series and multiple-exposure conditions. This causes the network to confuse artifacts with real defects during shallow feature extraction, making it impossible to effectively distinguish and suppress artifacts. Under conditions of ambient light decay, equipment aging, or line speed fluctuations, fixed intensity alignment and feature suppression parameters cannot be dynamically corrected, easily causing local residual distortion, resulting in the suppression or missed detection of real defects. During continuous system operation, existing solutions lack a collaborative processing mechanism for sudden situations such as acquisition anomalies, registration errors, computation timeouts, and parameter drift, making it difficult to balance the real-time performance and accuracy of detection. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-scale aluminum plate surface defect detection system and method based on deep learning, which can effectively distinguish and suppress optical artifacts caused by mechanical micro-vibration to reduce the false defect rate, dynamically correct parameters such as feature suppression and exposure to avoid excessive suppression or missed detection of weak real defects, and enhance the collaborative processing capability for acquisition anomalies and computation timeouts to improve industrial site adaptability.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A deep learning-based multi-scale aluminum plate surface defect detection method includes: during the same process of the aluminum plate passing through the inspection station, the same imaging device sequentially acquires the first aluminum plate surface image and the second aluminum plate surface image according to the first exposure parameter and the second exposure parameter.

[0007] The first aluminum plate surface image and the second aluminum plate surface image are registered along the conveying direction of the aluminum plate at the inspection station. Based on the exposure ratio coefficient determined by the first exposure parameter and the second exposure parameter, the registered first aluminum plate surface image is intensity aligned to generate an intensity aligned image.

[0008] The residual between the surface image of the second aluminum plate and the intensity-aligned image is calculated to generate an initial residual map. The initial residual map is then subjected to morphological enhancement processing to generate an enhanced residual map. A suppression mask is generated based on the enhanced residual map.

[0009] Using the surface image of the second aluminum plate as the processing object, shallow features are extracted through a convolutional feature extraction network to generate a shallow feature map, and a weighted feature map is generated by performing bit-by-bit weighting on the shallow feature map using an inhibition mask.

[0010] The weighted feature map is input into a defect detection network containing a multi-scale feature fusion structure for feature analysis and defect discrimination, generating surface defect detection results for aluminum plates.

[0011] Furthermore, the first aluminum plate surface image and the second aluminum plate surface image are registered along the conveying direction of the aluminum plate at the detection station, and the registered first aluminum plate surface image is intensity aligned based on the exposure ratio coefficient determined by the first exposure parameter and the second exposure parameter to generate an intensity aligned image. Specifically, this includes: calculating the subpixel displacement between the first aluminum plate surface image and the second aluminum plate surface image in the conveying direction.

[0012] The first aluminum plate surface image is aligned based on the subpixel displacement to generate a registered image; the gain compensation parameters are determined based on the local grayscale relationship between the registered image and the second aluminum plate surface image.

[0013] Intensity compensation is performed on the registered image based on the exposure ratio coefficient and gain compensation parameters to generate an intensity-aligned image.

[0014] Furthermore, morphological enhancement processing is performed on the initial residual map to generate an enhanced residual map, and a suppression mask is generated based on the enhanced residual map. Specifically, this includes: smoothing the initial residual map using a two-dimensional Gaussian smoothing kernel to generate a smoothed residual map.

[0015] A linear morphological structural element with a preset length, set along the aluminum plate conveying direction, is used to perform morphological enhancement processing on the smooth residual map to generate an enhanced residual map.

[0016] The enhanced residual map is normalized to generate a normalized residual map; based on the preset minimum value of the suppression mask, the normalized residual map is subjected to exponential mapping to generate a suppression mask.

[0017] Furthermore, based on the preset minimum value of the suppression mask, an exponential mapping process is performed on the normalized residual map to generate a suppression mask, including: based on the preset residual suppression intensity coefficient, an exponential decay mapping is performed on the value of the normalized residual map at the corresponding position to generate the initial mapping value at the corresponding position.

[0018] The initial mapping value is compared with the minimum value of the suppression mask, and the larger value is taken as the suppression mask value at the corresponding position.

[0019] Furthermore, the shallow feature map is processed bitwise using the suppression mask to generate a weighted feature map. Specifically, this includes resampling the suppression mask to the size of the shallow feature map to generate a resampling suppression mask.

[0020] Based on the preset feature preservation coefficient, the shallow feature map is weighted and multiplied with the resampling suppression mask to generate a weighted feature map.

[0021] Furthermore, based on preset feature preservation coefficients, the shallow feature map is weighted and multiplied with the resampling suppression mask to generate a weighted feature map, including:

[0022] Element-wise weighted coefficients are generated based on the feature preservation coefficients and the resampling suppression mask; the shallow feature map is then weighted element-wise using these element-wise weighted coefficients to generate a weighted feature map.

[0023] Furthermore, the defect detection method is executed multiple times during the continuous passage of the aluminum plate through the inspection station, which also includes: based on the surface defect detection results of the aluminum plate and the results of manual review, the false defect rate and the missed detection rate within the preset time window are calculated.

[0024] Calculate the mean value of the single-frame residual of each enhanced residual map within the preset time window, and calculate the average value of the mean value of the single-frame residual as the mean value of the window residual.

[0025] According to the preset update rules, the exposure ratio coefficient is updated based on the mean of the window residual, and the residual suppression intensity coefficient and the minimum value of the suppression mask are updated based on the false defect rate and the missed detection rate.

[0026] A deep learning-based multi-scale aluminum plate surface defect detection system includes an imaging device, a processor, and a memory. The memory stores instructions, which are executed by the processor to implement the following modules:

[0027] The image acquisition module is used to acquire images of the surface of the first aluminum plate and the surface of the second aluminum plate sequentially by the imaging device according to the first exposure parameters and the second exposure parameters during the same process of the aluminum plate passing through the inspection station.

[0028] The image registration and intensity alignment module is used to register the first aluminum plate surface image and the second aluminum plate surface image along the conveying direction of the aluminum plate at the detection station, and to perform intensity alignment processing on the registered first aluminum plate surface image based on the exposure ratio coefficient determined by the first exposure parameter and the second exposure parameter to generate an intensity aligned image.

[0029] The residual mask generation module is used to calculate the residual between the second aluminum plate surface image and the intensity aligned image, generate an initial residual map, perform morphological enhancement processing on the initial residual map to generate an enhanced residual map, and generate a suppression mask based on the enhanced residual map.

[0030] The feature weighting module is used to extract shallow features from the surface image of the second aluminum plate through a convolutional feature extraction network to generate a shallow feature map, and then perform bit-by-bit weighting on the shallow feature map using an inhibition mask to generate a weighted feature map.

[0031] The defect detection module is used to input the weighted feature map into the defect detection network containing a multi-scale feature fusion structure for feature analysis and defect discrimination, and generate the surface defect detection results of the aluminum plate.

[0032] Furthermore, the instructions also implement a parameter adaptive update module during processor execution, which updates the exposure ratio coefficient, the residual suppression intensity coefficient used to generate the suppression mask based on the aluminum plate surface defect detection results, manual review results, and residual statistics information, and the minimum value of the suppression mask.

[0033] The residual statistics include at least the mean value of the single-frame residuals and the average value of each enhanced residual map within a preset time window.

[0034] The beneficial effects of this invention are:

[0035] 1. The deep learning-based multi-scale aluminum plate surface defect detection method of the present invention acquires two frames of images of the same area in sequence with different exposure parameters by the same imaging device during the same process of the aluminum plate passing through the inspection station. The two frames of images are registered along the conveying direction. Then, the intensity of the registered images is aligned by combining the exposure ratio coefficient. It can build a stable artifact discrimination basis by utilizing the response difference of the same area under different time sequence and different exposure conditions, effectively distinguishing real defects from optical ripple artifacts caused by mechanical micro-vibration. It solves the problem that it is difficult to stably suppress artifacts and the false defect rate is high when relying solely on a single frame image or fixed exposure imaging method.

[0036] 2. The deep learning-based multi-scale aluminum plate surface defect detection method in this invention generates an initial residual map by calculating the residual between the second aluminum plate surface image and the intensity-aligned image. It further generates a suppression mask by using Gaussian smoothing, linear morphological structural elements set along the conveying direction, and normalization and exponential mapping. This method can highlight artifact structures with directional continuity, filter out random noise and isolated thermal noise points, and make the suppression mask more accurately reflect the spatial distribution of artifacts. This avoids the fragmentation and destruction of the underlying features caused by directly using the initial residual map, and improves the directional suppression ability of vibration stripes and wavy artifacts.

[0037] 3. The deep learning-based multi-scale aluminum plate surface defect detection method of this invention takes the second aluminum plate surface image as the processing object, extracts shallow feature maps through a convolutional feature extraction network, and applies the suppression mask after resampling to the shallow feature maps for bit-weighted processing. Then, it is input into a defect detection network containing a multi-scale feature fusion structure for defect discrimination. It can actively reduce the channel response of the artifact region in the shallow visual feature stage, avoid the network from encoding artifacts and real defects together, and establish a feature baseline by using feature retention coefficients and the minimum value of the suppression mask to prevent real defects such as weak scratches from being over-removed, thus taking into account both the artifact suppression effect and the detection capability of weak real defects. Attached Figure Description

[0038] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0039] Figure 1 Flowchart of the multi-scale aluminum plate surface defect detection method based on deep learning of the present invention;

[0040] Figure 2 A block diagram of a deep learning-based multi-scale aluminum plate surface defect detection system according to an embodiment of the present disclosure is shown. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Please see Figure 1 A multi-scale aluminum plate surface defect detection method based on deep learning includes: during the same process of the aluminum plate passing through the inspection station, the same imaging device sequentially acquires the first aluminum plate surface image and the second aluminum plate surface image according to the first exposure parameter and the second exposure parameter.

[0043] The first aluminum plate surface image and the second aluminum plate surface image are registered along the conveying direction of the aluminum plate at the inspection station. Based on the exposure ratio coefficient determined by the first exposure parameter and the second exposure parameter, the registered first aluminum plate surface image is intensity aligned to generate an intensity aligned image.

[0044] The residual between the surface image of the second aluminum plate and the intensity-aligned image is calculated to generate an initial residual map. The initial residual map is then subjected to morphological enhancement processing to generate an enhanced residual map. A suppression mask is generated based on the enhanced residual map.

[0045] Using the surface image of the second aluminum plate as the processing object, shallow features are extracted through a convolutional feature extraction network to generate a shallow feature map, and a weighted feature map is generated by performing bit-by-bit weighting on the shallow feature map using an inhibition mask.

[0046] The weighted feature map is input into a defect detection network containing a multi-scale feature fusion structure for feature analysis and defect discrimination, generating surface defect detection results for aluminum plates.

[0047] In one embodiment, the method for separating and suppressing optical ripple artifacts caused by mechanical micro-vibrations at the underlying visual feature stage by utilizing temporal acquisition differences during continuous operation of the aluminum plate is as follows;

[0048] On a continuous annealing surface inspection production line for aluminum plates, a linear array camera is arranged above the inspection station. When the aluminum plate travels through the inspection station at high speed, the timing controller sends two trigger pulses in a single inspection cycle, driving the linear array camera to acquire the first and second aluminum plate surface images of the same physical area according to the first and second exposure parameters. The first exposure time corresponding to the first exposure parameter is less than the second exposure time corresponding to the second exposure parameter, and the image data is pushed into a dual-channel buffer queue.

[0049] The system synchronously extracts these two frames of images from the queue and performs spatial registration along the aluminum plate conveying direction to offset the physical displacement within the sampling time interval between the two frames;

[0050] After registration, the system calculates the exposure ratio coefficient by comparing the first exposure time corresponding to the first exposure parameter with the second exposure time corresponding to the second exposure parameter. The specific calculation formula is as follows:

[0051]

[0052] in, This is the exposure ratio factor. For the first exposure time, The second exposure time is used to enlarge the grayscale image of the first aluminum plate surface, generating an intensity-aligned image;

[0053] The system performs pixel-level subtraction between the surface image of the second aluminum plate and the intensity-aligned image to obtain an initial residual map. Since the reflectance of the real defect approximately follows a linear law with respect to exposure time, while the vibration artifact exhibits a nonlinear law due to the displacement integral during exposure, this residual map can highlight the artifact region.

[0054] The system further performs morphological enhancement on the residual map to filter out noise, and generates a suppression mask that reflects the spatial distribution of artifacts accordingly.

[0055] The system inputs the surface image of the second aluminum plate with a signal-to-noise ratio greater than a preset threshold into the convolutional feature extraction network to extract the shallow feature map, and applies the suppression mask to the shallow feature map for bit-by-bit weighting, thereby reducing the channel response of the artifact region in the feature space;

[0056] The weighted modulated feature map is pushed into a multi-scale feature fusion network for semantic aggregation and boundary regression, outputting the location and category of defects such as scratches and dents.

[0057] Preferably, the convolutional feature extraction network adopts a residual network architecture, which avoids feature degradation through cross-layer residual connections and preserves the visual features of the scratches in the lower layer;

[0058] The input feature vector of this extraction network is precisely set to a width. ,high A single-channel two-dimensional image tensor, i.e. The dimension of its output shallow feature map is... Multichannel feature tensors;

[0059] The multi-scale feature fusion network adopts a feature pyramid network structure, and its output vector dimension includes the confidence score tensor of each type of defect and the localization tensor composed of four bounding box coordinate offsets.

[0060] The training samples for the above detection network are obtained by manually selecting and labeling sample boxes containing real pressure marks and scratches from a historical production line image database, and simultaneously synthesizing negative samples with vibration ripples of different frequencies through pixel interpolation and displacement blurring algorithms to jointly construct the training set.

[0061] In backpropagation optimization, the loss function is a linearly weighted combination of the focus loss function used to address the imbalance between positive and negative samples and the intersection-union ratio loss function used for box regression constraints; the specific combination formula is as follows:

[0062]

[0063] in, For the total loss function, For focus loss function, The intersection-union loss function is... The classification loss weight is set to 1.0. The regression loss weight is set to 0.5;

[0064] If a trigger signal is lost or a single-frame image transmission times out during the data acquisition phase, the system will directly clear the incomplete data pairs in the current buffer queue, skip the residual calculation for the current period, and report the acquisition anomaly status to prevent mask misalignment caused by misaligned images.

[0065] In some possible implementations, the first aluminum plate surface image and the second aluminum plate surface image are registered along the conveying direction of the aluminum plate at the inspection station, and the registered first aluminum plate surface image is intensity aligned based on the exposure ratio coefficient determined by the first exposure parameters and the second exposure parameters to generate an intensity aligned image. Specifically, this includes: calculating the sub-pixel displacement between the first aluminum plate surface image and the second aluminum plate surface image in the conveying direction; and aligning the position of the first aluminum plate surface image based on the sub-pixel displacement to generate the registered image.

[0066] Gain compensation parameters are determined based on the local grayscale relationship between the registered image and the second aluminum plate surface image; intensity compensation is performed on the registered image based on the exposure ratio coefficient and the gain compensation parameters to generate an intensity-aligned image.

[0067] If intensity alignment is performed solely based on the global exposure ratio coefficient, fluctuations in local lighting on the production line or uneven reflection gain on the aluminum plate surface can lead to a substrate deviation in the aligned residual map that exceeds the area threshold.

[0068] After extracting the surface images of the first and second aluminum plates, the system calculates the subpixel displacement of the two frames in the transport direction using a phase correlation or cross-correlation peak localization algorithm, and strictly calls the bicubic interpolation algorithm to perform position interpolation alignment to generate the registered image.

[0069] The system uses a sliding window to calculate the average gray level of the registered image and the second aluminum plate surface image within the corresponding window, and uses the ratio of the two as the initial gain compensation parameter for the local area, thereby forming an initial gain matrix with reduced spatial resolution.

[0070] When the spatial dimension of the initial gain matrix is ​​smaller than the pixel dimension of the registered image, direct mapping will cause blocky edges in the aligned image. The system uses a bilinear interpolation algorithm to spatially upsample the initial gain matrix, expanding its dimension to be completely consistent with the width and height of the registered image, and generating a smooth local gain compensation parameter matrix.

[0071] The system multiplies the global exposure scaling factor, which is a scalar, by a smoothed local gain compensation parameter matrix to generate the final hybrid gain matrix, and applies it to the corresponding pixels of the registered image to generate an intensity-aligned image.

[0072] During this calculation process, if the average gray level within the current local window approaches zero due to black dirt or other reasons, the system will add a preset stability constant to the denominator of the division operation of the gain compensation parameter to prevent program interruption caused by division by zero overflow.

[0073] In some possible implementations, morphological enhancement processing is performed on the initial residual map to generate an enhanced residual map, and a suppression mask is generated based on the enhanced residual map. Specifically, this includes: smoothing the initial residual map using a two-dimensional Gaussian smoothing kernel to generate a smoothed residual map.

[0074] A linear morphological structural element with a preset length, set along the aluminum plate conveying direction, is used to perform morphological enhancement processing on the smooth residual map to generate an enhanced residual map; the enhanced residual map is then normalized to generate a normalized residual map.

[0075] Based on the preset minimum value of the suppression mask, an exponential mapping process is performed on the normalized residual map to generate the suppression mask.

[0076] The directly generated initial residual map usually contains random sensor noise or isolated thermal noise points. If it is used directly to generate a mask, it will cause fragmentation and destruction of the underlying features.

[0077] The system calls a two-dimensional Gaussian smoothing kernel to convolve and denoise the initial residual map, generating a smooth residual map. Since the artifacts formed on the aluminum plate surface by mechanical micro-vibration usually appear as stripes or ripples that extend continuously along the conveying direction, the system constructs a linear morphological structure element with a set length along the conveying direction, performs closing and opening operations on the smooth residual map, and subtracts the two results to extract the artifact structure with directional continuity, generating an enhanced residual map.

[0078] The preset length of the linear morphological structural element is dynamically calculated based on the conversion from physical dimension to pixel dimension. The conversion method includes: the system obtains the physical linear velocity of the production line in real time, and calculates the physical displacement of the aluminum plate during the two frames by combining the time interval corresponding to the first exposure parameter and the second exposure parameter.

[0079] Divide the physical displacement by the object-side pixel resolution of the imaging device to accurately map the physical displacement to the preset length of the structural element in the pixel domain.

[0080] The system normalizes the enhanced residual map according to the extreme values ​​in the current image, generating a normalized residual map with numerical distribution in a fixed interval;

[0081] Specifically, the system extracts the global maximum and global minimum values ​​of the current enhanced residual map, uses the maximum-minimum linear normalization formula to subtract the minimum value from the current pixel value and then divide by the difference between the maximum and minimum values, thereby accurately mapping all pixel values ​​to a fixed continuous interval of [0,1] and generating a normalized residual map.

[0082] The system uses a preset minimum value of the suppression mask as a lower limit constraint, performs exponential mapping on the normalized residual map, and generates the final suppression mask.

[0083] In this process, the preset minimum value of the suppression mask is represented as the bottom-line throughput that retains features without being cleared. Its calibration is determined by extracting features from normal historical aluminum plate images that do not have vibration artifacts on the production line and statistically analyzing the lowest response limit of real weak defects in the feature images. For example, it is a constant between 0.15 and 0.25.

[0084] If the system fails to obtain the current real-time line speed data of the production line during the morphological enhancement stage, resulting in the inability to dynamically calculate the length of the linear morphological structural element, the system will by default call the historical average length parameter stored in the local configuration file to maintain the calculation, ensuring that the morphological processing steps are not blocked.

[0085] In some possible implementations, an exponential mapping process is performed on the normalized residual map based on a preset minimum suppression mask value to generate a suppression mask, including: performing an exponential decay mapping on the value of the normalized residual map at the corresponding position based on a preset residual suppression intensity coefficient to generate an initial mapping value at the corresponding position.

[0086] The initial mapping value is compared with the minimum value of the suppression mask, and the larger value is taken as the suppression mask value at the corresponding position.

[0087] If the value of the mask mapping process is zero, the underlying features will be erased in the area where the residual exceeds the preset threshold, resulting in the missed detection of real scratches overlapping the artifact area.

[0088] Therefore, when performing exponential mapping, the system extracts the values ​​of the normalized residual map at each pixel position, and calculates an initial mapping value that decays exponentially as the residual increases, in combination with the preset residual suppression strength coefficient.

[0089] The system calls the natural exponential function, using the natural constant. As the base, the negative residual suppression strength coefficient will be used. The values ​​of the normalized residual plot at the corresponding positions The product of these factors is used as the exponent to generate the initial mapping value for that position. :

[0090]

[0091] in, This is the initial mapping value at the corresponding position. For the natural constant An exponential function with base 0. This is the residual suppression strength coefficient. Values ​​are assigned to the normalized residual plots at the corresponding locations; residual suppression strength coefficient. Used to control the steepness of the suppression mapping curve, its value is determined based on the baseline noise level of the on-site optical camera and the gray-scale contrast of the defect features. The lower the contrast, the smaller the value of this coefficient, in order to prevent accidental damage to weak defect features.

[0092] The system compares the initial mapping value with the preset minimum value of the suppression mask pixel by pixel, and forces the larger of the two values ​​to be written into the corresponding position of the suppression mask, so as to retain the feature flux in the residual peak region for multi-scale network analysis.

[0093] In some possible implementations, a weighted feature map is generated by performing bitwise weighted processing on the shallow feature map using a suppression mask. Specifically, this includes: resampling the suppression mask to the size of the shallow feature map to generate a resampling suppression mask; and performing a weighted multiplication of the shallow feature map and the resampling suppression mask based on a preset feature retention coefficient to generate a weighted feature map.

[0094] Since the size of shallow feature maps in convolutional networks is downsampled, it is usually smaller than the size of the original image and the corresponding generated suppression mask. Direct computation will lead to dimension mismatch.

[0095] The system reads the width and height parameters of the shallow feature map, and uses a bilinear interpolation algorithm to scale the suppression mask to the same size to generate a resampling suppression mask;

[0096] The system introduces feature preservation coefficients, combines them with resampling suppression masks in a preset ratio, and calculates element-wise weighted coefficients at each spatial location. The system broadcasts these element-wise weighted coefficients along the channel dimension of the feature map and performs element-wise multiplication with the shallow feature map to generate a weighted feature map.

[0097] When the resampling process fails to complete the interpolation operation due to the failure of the underlying memory allocation, the system will skip the mask weighting step, directly pass the unmodulated shallow feature map to the next network layer, and mark the current frame as unsuppressed in the log to ensure that the system continuously outputs detection results.

[0098] In some possible implementations, a weighted feature map is generated by performing a weighted multiplication process on the shallow feature map and the resampling suppression mask based on a preset feature preservation coefficient. This includes: generating element-wise weighting coefficients based on the feature preservation coefficients and the resampling suppression mask; and using the element-wise weighting coefficients to perform element-wise weighting on the shallow feature map to generate a weighted feature map.

[0099] When the resampling suppression mask is directly used to multiply modulate the shallow feature map, the scratch features accompanying the suppression mask will be cleared when the response of the suppression mask in the artifact region approaches 0.

[0100] To this end, the system establishes a baseline for feature strength by introducing a preset feature retention coefficient; when generating element-wise weighted coefficients, the system sets the preset feature retention coefficient to a basic constant, for example, 0.2; the specific formula for calculating the combined mapping is as follows:

[0101]

[0102] in, For the generated element-wise weighted coefficient matrix, The feature retention coefficient, This serves as a resampling suppression mask for the corresponding location.

[0103] This structured linear combination of weights ensures that the weighting coefficient at any pixel location will not be lower than the set base constant.

[0104] Since shallow feature maps are usually three-dimensional tensors containing width, height, and number of channels, while element-wise weighted coefficient matrices are two-dimensional containing width and height, the system performs a broadcast operation on the weighted coefficient matrix in the channel dimension of the feature map based on spatial alignment.

[0105] The system uses the broadcast weighted coefficient tensor and shallow feature map to perform element-wise multiplication to generate a weighted feature map. If a memory overflow or numerical anomaly occurs in the matrix operation to generate the weighted coefficients, the system will trigger an anomaly protection mechanism to forcibly reset the weighted coefficients to a matrix of all 1s, so that the low-level features are passed to the next network layer without modification, thus avoiding pipeline interruption.

[0106] In some possible implementations, the method further includes: based on the surface defect detection results of the aluminum plate and the results of manual review, calculating the false defect rate and the missed detection rate within a preset time window; calculating the mean value of the single-frame residual of each enhanced residual map within the preset time window, and calculating the average value of the mean value of the single-frame residual as the mean value of the window residual.

[0107] According to the preset update rules, the exposure ratio coefficient is updated based on the mean of the window residual, and the residual suppression intensity coefficient and the minimum value of the suppression mask are updated based on the false defect rate and the missed detection rate.

[0108] During long-term operation, the brightness decay of the production line light source or the aging of the mechanical roller assembly will cause the environmental parameters to drift. Fixed mask mapping parameters and exposure control parameters will lead to a gradual decline in detection performance.

[0109] To this end, the system maintains a sliding time window; within the preset time window, the system performs time-series alignment between the detection results of machine vision reasoning and the records of manual review based on the unique batch serial number;

[0110] The system continuously receives the detection results with timestamps and batch serial numbers output by the multi-scale defect detection network, and asynchronously retrieves the manual review records submitted by quality inspectors through the terminal; the system only extracts the closed-loop data that has completed serial number matching within the window, and calculates the false defect rate and missed detection rate within the current window accordingly;

[0111] Meanwhile, the system summarizes the average residual values ​​of all enhanced residual maps in the window in the background and calculates the average residual value of the window. The system updates the lookup table rules according to the preset parameters, checks the current average residual value of the window, outputs the new exposure ratio coefficient and sends it to the timing controller.

[0112] The specific threshold segmentation mapping and lookup table rules are as follows: the system presets a value such as the mean. satisfy In the low-noise reference range of the environment, when the mean value of the calculated window residual falls within this reference range, it is determined that there is no significant environmental shift, and the output exposure ratio coefficient maintains the current default reference value; when the mean value of the window residual is lower than the lower limit of this range, for example, when the mean value is less than 15, it indicates that the current reflected light flux is greater than the preset light flux threshold or the mechanical vibration is less than the preset vibration threshold. The system retrieves the dimension reduction gain parameter through a lookup table and reduces the exposure ratio coefficient from 1.0 to 0.9 to reduce the integral exposure time;

[0113] When the mean of the window residuals satisfy When the light intensity is in the moderate attenuation range, the system will increase the exposure ratio coefficient to 1.2 based on the lookup table to compensate for the illuminance; if the mean value of the window residual is greater than or equal to the abnormal upper limit of 40, the system will directly lock the exposure ratio coefficient without updating and trigger a light source aging alarm.

[0114] At the same time, the system assesses the degree to which the current false defect rate and missed detection rate deviate from the preset target, calculates the updated residual suppression strength coefficient and the minimum value of the suppression mask, and overwrites the old parameters in the local memory;

[0115] Specifically, the quantitative update rules based on false defect rate and missed detection rate are as follows: the system pre-sets a false defect rate tolerance threshold and a missed detection rate tolerance threshold;

[0116] When the statistical false defect rate is greater than the false defect rate tolerance threshold, it indicates that the current suppression strength is insufficient. The system increases the residual suppression strength coefficient with a preset first update step size and decreases the minimum value of the suppression mask with a preset second update step size.

[0117] Conversely, when the statistical false negative rate is greater than the false negative rate tolerance threshold, it indicates that the current suppression is excessive, resulting in the loss of real weak defects. The system reduces the residual suppression strength coefficient with the first update step and increases the minimum value of the suppression mask with the second update step.

[0118] If both the false defect rate and the missed detection rate are within the tolerance threshold, the original mapping parameters remain unchanged. Through this control command with quantitative closed-loop execution action, fine adjustment of system environment drift and complex working conditions is achieved.

[0119] If, at the end of the preset time window, the manual review results fail to be synchronized to the system in a timely manner due to high latency or network delay, resulting in insufficient closed-loop data, the system will only update the exposure ratio coefficient based on the mean residual value of the low-latency window, while keeping the residual suppression intensity coefficient and the minimum suppression mask value at the state of the previous cycle, thus avoiding drastic parameter fluctuations caused by missing feedback data.

[0120] Please see Figure 2 A deep learning-based multi-scale aluminum plate surface defect detection system includes an imaging device, a processor, and a memory. The memory stores instructions, which, when executed by the processor, implement the following modules: an image acquisition module, used to acquire first aluminum plate surface images and second aluminum plate surface images sequentially by the imaging device according to first exposure parameters and second exposure parameters during the same process of the aluminum plate passing through the inspection station.

[0121] The image registration and intensity alignment module is used to register the first aluminum plate surface image and the second aluminum plate surface image along the conveying direction of the aluminum plate at the detection station, and to perform intensity alignment processing on the registered first aluminum plate surface image based on the exposure ratio coefficient determined by the first exposure parameter and the second exposure parameter to generate an intensity aligned image.

[0122] The residual mask generation module is used to calculate the residual between the second aluminum plate surface image and the intensity aligned image, generate an initial residual map, perform morphological enhancement processing on the initial residual map to generate an enhanced residual map, and generate a suppression mask based on the enhanced residual map.

[0123] The feature weighting module is used to extract shallow features from the surface image of the second aluminum plate through a convolutional feature extraction network to generate a shallow feature map, and then perform bit-by-bit weighting on the shallow feature map using an inhibition mask to generate a weighted feature map.

[0124] The defect detection module is used to input the weighted feature map into the defect detection network containing a multi-scale feature fusion structure for feature analysis and defect discrimination, and generate the surface defect detection results of the aluminum plate.

[0125] The system is deployed in the industrial control computer at the aluminum plate inspection station; the industrial control computer is connected to the line scan camera above via a gigabit network cable; when the system is running, the image acquisition module controls the camera to complete the dual exposure timing acquisition through the hardware trigger interface, and writes the acquired first aluminum plate surface image and second aluminum plate surface image into the memory buffer.

[0126] The image registration and intensity alignment module reads buffer data, performs subpixel displacement calculation and local brightness gain calculation, and outputs the intensity-aligned image to the internal data bus.

[0127] The residual mask generation module subscribes to the above image data, extracts continuous stripe features through the morphological operation unit, and applies the lower limit truncated exponential mapping logic to store the generated suppression mask into the video memory.

[0128] The feature weighting module calls the convolution operator in the graphics processor environment to extract shallow feature maps from the surface image of the second aluminum plate, and reads the suppression mask in the video memory to perform matrix multiplication to complete the modulation of the low-level features;

[0129] The modulated feature map is transferred to the defect detection module. After inference by a deep network, the defect coordinates and category information are persistently stored in the database.

[0130] The system is equipped with a timeout monitoring mechanism between modules. If the single inference calculation time of a module such as the image registration and intensity alignment module or the feature weighting module exceeds the preset detection cycle threshold, the monitoring mechanism will directly stop the processing link of the current image frame, clear the corresponding memory and video memory buffers, and automatically release the blocked resources to ensure that subsequent incoming images can be responded to in real time and avoid data backlog that could cause abnormal system interruption.

[0131] In some possible implementations, the instructions also implement a parameter adaptive update module when executed by the processor, which updates the exposure ratio coefficient, the residual suppression intensity coefficient used to generate the suppression mask based on the enhanced residual map, and the minimum value of the suppression mask according to the aluminum plate surface defect detection results, manual review results, and residual statistics.

[0132] The residual statistics include at least the mean value of the single-frame residuals and the average value of each enhanced residual map within a preset time window.

[0133] In aluminum plate quality inspection production lines, due to the decay of ambient light, aging of camera sensors, and increased vibration of mechanical structures, fixed mask generation parameters often cannot maintain a stable defect detection rate for a long time and must be corrected based on closed-loop data.

[0134] Therefore, during system operation, the parameter adaptive update module is executed independently as a background daemon process; this module periodically scans the result data written by the defect detection module in the main database and the manual review records filled in by quality inspectors, and calculates the false defect rate and missed detection rate within a specific time window accordingly.

[0135] The specific calculation mapping relationship is as follows: The false defect rate is calculated by dividing the number of defects reported by the system within the preset time window that were manually reviewed and rejected and marked as normal board surfaces or mechanical artifacts by the total number of alarms in the system.

[0136] The missed detection rate is calculated by dividing the number of missed pieces that were manually discovered through random inspection or traced back to the next process, and which the system failed to capture and generate defect judgment in the corresponding batch by the sum of the total number of actual defects confirmed by manual review and the aforementioned number of missed pieces.

[0137] Meanwhile, the module summarizes the residual statistics during the period to assess the change in the current physical environment's floor noise. Once the false defect rate or missed detection rate is found to deviate from the preset threshold range, the module will recalculate and generate new exposure ratio coefficients, residual suppression strength coefficients, and minimum suppression mask values.

[0138] The image registration and intensity alignment module and the residual mask generation module read the latest parameter copy from shared memory and work non-blockingly before each new detection batch starts;

[0139] If the parameter adaptive update module times out during an update calculation due to database lockout or network latency, the module will directly terminate the current statistical task.

[0140] This ensures that the high-speed image detection pipeline at the front end continues to run based on existing stable parameters, completely unaffected by background update tasks blocking it, thereby guaranteeing the timing stability of the overall system detection process.

Claims

1. A multi-scale aluminum plate surface defect detection method based on deep learning, characterized in that, include: During the same inspection station process, the same imaging device sequentially acquires the surface images of the first aluminum plate and the second aluminum plate according to the first exposure parameters and the second exposure parameters. The first aluminum plate surface image and the second aluminum plate surface image are registered along the conveying direction of the aluminum plate at the detection station. Based on the exposure ratio coefficient determined by the first exposure parameter and the second exposure parameter, the registered first aluminum plate surface image is intensity aligned to generate an intensity aligned image. Calculate the residual between the second aluminum plate surface image and the intensity aligned image to generate an initial residual map, perform morphological enhancement processing on the initial residual map to generate an enhanced residual map, and generate a suppression mask based on the enhanced residual map; Using the second aluminum plate surface image as the processing object, shallow features are extracted through a convolutional feature extraction network to generate a shallow feature map, and the shallow feature map is then subjected to bit-weighted processing using the suppression mask to generate a weighted feature map. The weighted feature map is input into a defect detection network containing a multi-scale feature fusion structure for feature analysis and defect discrimination, generating surface defect detection results for aluminum plates.

2. The multi-scale aluminum plate surface defect detection method based on deep learning according to claim 1, characterized in that, The first aluminum plate surface image and the second aluminum plate surface image are registered along the conveying direction of the aluminum plate at the inspection station. Based on the exposure ratio coefficient determined by the first and second exposure parameters, the registered first aluminum plate surface image is then subjected to intensity alignment processing to generate an intensity-aligned image. Specifically, this includes: Calculate the subpixel displacement between the first aluminum plate surface image and the second aluminum plate surface image in the conveying direction; The image of the first aluminum plate surface is aligned based on the subpixel displacement to generate a registered image. Gain compensation parameters are determined based on the local grayscale relationship between the registered image and the image of the second aluminum plate surface. Intensity compensation is performed on the registered image based on the exposure ratio coefficient and the gain compensation parameter to generate the intensity aligned image.

3. The multi-scale aluminum plate surface defect detection method based on deep learning according to claim 1, characterized in that, The initial residual map is subjected to morphological enhancement processing to generate an enhanced residual map, and a suppression mask is generated based on the enhanced residual map, specifically including: The initial residual map is smoothed using a two-dimensional Gaussian smoothing kernel to generate a smoothed residual map; The smooth residual map is morphologically enhanced by using linear morphological structural elements of a preset length set along the aluminum plate conveying direction to generate the enhanced residual map. The enhanced residual map is normalized to generate a normalized residual map; The normalized residual map is subjected to exponential mapping processing based on the preset minimum value of the suppression mask to generate the suppression mask.

4. The multi-scale aluminum plate surface defect detection method based on deep learning according to claim 3, characterized in that, Based on a preset minimum suppression mask value, an exponential mapping process is performed on the normalized residual map to generate the suppression mask, including: Based on the preset residual suppression strength coefficient, the values ​​of the normalized residual map at the corresponding positions are subjected to exponential decay mapping to generate the initial mapping values ​​at the corresponding positions. The initial mapping value is compared with the minimum value of the suppression mask, and the larger value is taken as the suppression mask value at the corresponding position.

5. The multi-scale aluminum plate surface defect detection method based on deep learning according to claim 1, characterized in that, The shallow feature map is subjected to bit-weighted processing using the suppression mask to generate a weighted feature map, specifically including: The suppression mask is resampled to the size of the shallow feature map to generate a resampling suppression mask; Based on a preset feature retention coefficient, the shallow feature map is weighted and multiplied with the resampling suppression mask to generate the weighted feature map.

6. The multi-scale aluminum plate surface defect detection method based on deep learning according to claim 5, characterized in that, Based on a preset feature preservation coefficient, the shallow feature map is weighted and multiplied with the resampling suppression mask to generate the weighted feature map, including: Element-wise weighted coefficients are generated based on the feature retention coefficients and the resampling suppression mask; The shallow feature map is weighted element by element using the element-wise weighting coefficients to generate the weighted feature map.

7. The multi-scale aluminum plate surface defect detection method based on deep learning according to claim 4, characterized in that, The defect detection method is performed multiple times during the continuous passage of the aluminum plate through the inspection station, and also includes: Based on the surface defect detection results of the aluminum plate and the manual review results, the false defect rate and the missed detection rate within the preset time window are statistically analyzed. Calculate the mean value of the single-frame residual of each enhanced residual map within the preset time window, and calculate the average value of the mean values ​​of the single-frame residuals as the mean value of the window residuals. According to the preset update rules, the exposure ratio coefficient is updated based on the mean of the window residuals, and the residual suppression intensity coefficient and the minimum value of the suppression mask are updated based on the false defect rate and the missed detection rate.

8. A deep learning-based multi-scale aluminum plate surface defect detection system, applied to the deep learning-based multi-scale aluminum plate surface defect detection method as described in any one of claims 1 to 6, characterized in that, It includes an imaging device, a processor, and a memory, wherein the memory stores instructions that, when executed by the processor, implement the following modules: The image acquisition module is used to acquire images of the first aluminum plate surface and the second aluminum plate surface sequentially by the imaging device according to the first exposure parameter and the second exposure parameter during the same process of the aluminum plate passing through the inspection station. The image registration and intensity alignment module is used to register the first aluminum plate surface image and the second aluminum plate surface image along the conveying direction of the aluminum plate at the detection station, and to perform intensity alignment processing on the registered first aluminum plate surface image based on the exposure ratio coefficient determined by the first exposure parameters and the second exposure parameters to generate an intensity aligned image. The residual mask generation module is used to calculate the residual between the second aluminum plate surface image and the intensity alignment image, generate an initial residual map, perform morphological enhancement processing on the initial residual map to generate an enhanced residual map, and generate a suppression mask based on the enhanced residual map. The feature weighting module is used to take the second aluminum plate surface image as the processing object, extract shallow features through a convolutional feature extraction network to generate a shallow feature map, and perform bit-by-bit weighting processing on the shallow feature map using the suppression mask to generate a weighted feature map. The defect detection module is used to input the weighted feature map into a defect detection network containing a multi-scale feature fusion structure for feature analysis and defect discrimination, and generate the surface defect detection results of the aluminum plate.

9. The multi-scale aluminum plate surface defect detection system based on deep learning according to claim 8, characterized in that, When the instruction is executed by the processor, it also implements a parameter adaptive update module, which is used to update the exposure ratio coefficient, the residual suppression intensity coefficient used to generate the suppression mask based on the enhanced residual map, and the minimum value of the suppression mask according to the aluminum plate surface defect detection results, manual review results, and residual statistics. The residual statistics include at least the mean value of the single-frame residuals and the average value of each enhanced residual map within a preset time window.

Images