Image processing-based defect detection method and system in laminating process of filter assembly

Abstract

The invention belongs to the technical field of defect detection, and particularly relates to a defect detection method and system in a filter assembly laminating process based on image processing, so as to solve the technical problems that an existing detection method model is relatively weak in micro defect sensing capability and relatively low in detection accuracy. The detection method comprises the following steps: S1, carrying out block attention calculation on an image block sequence; s2, calculating an adjustment coefficient and adjusting the updating process of the row normalization factor by using the adjustment coefficient; s3, weighting corresponding row vectors in the initial context feature matrix by using the adjustment coefficient corresponding to each query block to obtain a weighted context feature matrix; and S4, outputting a mask pattern representing the position of the defect on the surface of the filter assembly. According to the detection method provided by the invention, secondary weighting and feature fusion are carried out on the initial context features, so that the accuracy of defect detection of the filter assembly is improved.

Description

Technical Field

[0001] This invention belongs to the technical field of defect detection, specifically relating to a defect detection method and system based on image processing during the bonding process of filter components. Background Technology

[0002] As a core component in precision optical systems and optoelectronic devices, the surface quality of optical filter components plays a decisive role in imaging quality, signal transmission stability, and service life. During production, transportation, and assembly, various types of minor defects such as scratches, pitting, dirt, and uneven coating may occur on the surface of optical filter components, which need to be detected and identified.

[0003] Traditional machine vision-based detection technologies identify defects by employing image processing algorithms such as threshold segmentation, edge detection, and Fourier transform. However, these technologies require the manual design of complex feature extractors and are sensitive to factors such as changes in illumination, background noise, and the diversity of defect morphology.

[0004] With the development of artificial intelligence technology, deep learning methods, represented by convolutional neural networks and Transformers, have been applied to the field of industrial defect detection. These deep learning methods improve the accuracy and automation level of detection by learning features from data. However, the computational complexity of the self-attention mechanism of the Transformer model is proportional to the square of the length of the input sequence, which leads to huge computational and memory overhead when processing high-resolution images. Although the standard FlashAttention alleviates the computational and memory overhead problem through a block-based strategy, it does not fully consider the relative spatial relationships between image blocks during block processing, and is prone to losing spatial information that is important for defect localization. In addition, the feature differences between defect areas and normal background areas in an image are huge, which leads to drastic fluctuations in the statistical variance of attention scores. The standard online Softmax normalization process becomes numerically unstable when faced with such drastic changes, thereby weakening the model's ability to perceive small defects.

[0005] Therefore, how to improve efficient attention models so that they can better preserve spatial information, adapt to drastic feature changes in defect detection scenarios, and generate more discriminative feature representations is a pressing problem that needs to be solved. Summary of the Invention

[0006] This invention provides a defect detection method and system based on image processing during the bonding process of filter components, in order to solve the technical problems of existing detection methods and models having weak perception ability for minute defects and low detection accuracy.

[0007] In a first aspect, the present invention provides a defect detection method for filter component bonding process based on image processing, comprising the following steps: S1. Obtain the original digital image of the filter component, and divide the original digital image into multiple non-uniformly sized image block sequences based on the local gradient magnitude and information entropy of the original digital image; perform block attention calculation on the image block sequence, and when calculating the dot product of the query matrix and the key matrix, superimpose the learnable position bias matrix to obtain the initial attention score. The learnable position bias matrix is ​​related to the relative spatial position of the image block in the original digital image. S2, when performing block-by-block online Softmax normalization on the initial attention score, calculate the adjustment coefficient and use the adjustment coefficient to adjust the update process of the row normalization factor to obtain the adjusted attention probability matrix. The adjustment coefficient is determined by the ratio of the statistical variance of the initial attention score in the current query block to the global cumulative variance. S3. The initial context feature matrix is ​​calculated using the adjusted attention probability matrix and value matrix. The corresponding row vectors in the initial context feature matrix are weighted using the adjustment coefficients corresponding to each query block to obtain the weighted context feature matrix. S4 concatenates the weighted context feature matrix and the adjusted attention probability matrix along the feature dimension to form a fused feature matrix, which is then input into the defect decoding network to output a mask map representing the location of defects on the surface of the filter component.

[0008] Further, in S1, when dividing the original digital image into multiple non-uniformly sized image block sequences, a 3×3 Sobel operator is used to calculate the local gradient magnitude of each pixel in the original digital image; with each pixel as the center, the information entropy of the pixel is calculated within a 9×9 neighborhood window; after normalizing the local gradient magnitude and information entropy of each pixel, they are weighted and summed with preset weights to obtain a complexity score map covering the entire original digital image; a quadtree partitioning method is used to recursively partition the original digital image according to the complexity score map: starting from the initial block containing the entire original digital image, for each image block, if the average complexity score of all pixels in the image block exceeds a preset threshold and the image block size is greater than the minimum size limit, then the image block is split into four equal-sized sub-blocks until all image blocks meet the termination condition, thereby forming a non-uniformly sized image block sequence.

[0009] Furthermore, in S1, when superimposing the learnable position bias matrix, the two-dimensional offset between the center pixel coordinates of the current query block and the queried key block in the original digital image is calculated. ; 2D offset As an index, the corresponding scalar bias value is queried in a pre-defined, parameter-learnable two-dimensional bias embedding table; The scalar bias values ​​obtained from the query are superimposed onto the corresponding elements of the dot product of the query matrix and the key matrix.

[0010] Furthermore, in S2, when calculating the adjustment coefficient, for the currently processed query block... Calculate the query block Statistical variance of all initial attention scores within the context ; To maintain the global cumulative variance, update it using the exponential moving average method. The update formula is as follows: ; in For smoothing coefficients, This is the updated global cumulative variance. This is the global cumulative variance obtained from the previous calculation; Adjustment coefficient The calculation formula is: ; in To prevent stable terms with a denominator of zero.

[0011] Furthermore, in S2, when adjusting the update process of the row normalization factor using the adjustment coefficient, in the online Softmax calculation, for those belonging to the query block... The Okay, normalization factor The update formula is: ; in, The updated normalization factor. This is the normalization factor obtained from the previous calculation. The maximum value in the row. This is the maximum row value obtained from the previous calculation. This is the updated maximum row value. For the dot product score, For query block The corresponding adjustment coefficient is adjusted by exponentially decaying the exponent of the new computation block.

[0012] Furthermore, in S3, when obtaining the context feature matrix, for the first element in the initial context feature matrix... row vector If the vector belongs to the query block Then the weighted context feature vector of this vector The calculation formula is: ; in, This is the adjustment coefficient corresponding to block b.

[0013] Furthermore, S4 includes the following steps: The weighted context feature matrix of dimension (N, d) and the adjusted attention probability matrix of dimension (N, N) are concatenated along the feature dimensions to obtain the fusion feature matrix of dimension (N, d+N), where N is the number of image patches and d is the feature dimension. The fused feature matrix is ​​input into a defect decoding network composed of multiple fully connected layers, and a defect probability score is output for each image patch, resulting in a probability vector of dimension (N, 1). Initialize an all-zero mask with the same size as the original digital image. Based on each probability score in the probability vector and the position and size of the image block corresponding to the probability score in the original digital image, fill the pixel values ​​of each image block region with the corresponding probability score to generate the mask.

[0014] Secondly, the present invention provides a defect detection system for the bonding process of filter components based on image processing, comprising the following modules: The overlay module acquires the original digital image of the filter component and divides the original digital image into multiple non-uniformly sized image block sequences based on the local gradient magnitude and information entropy of the original digital image; it performs block attention calculation on the image block sequences, and when calculating the dot product of the query matrix and the key matrix, it overlays a learnable position bias matrix to obtain an initial attention score, wherein the learnable position bias matrix is ​​related to the relative spatial position of the image block in the original digital image; The adjustment module calculates the adjustment coefficient when performing block-by-block online Softmax normalization on the initial attention score and uses the adjustment coefficient to adjust the update process of the row normalization factor to obtain the adjusted attention probability matrix. The adjustment coefficient is determined by the ratio of the statistical variance of the initial attention score in the current query block to the global cumulative variance. The weighting module calculates the initial context feature matrix using the adjusted attention probability matrix and value matrix, and then uses the adjustment coefficients corresponding to each query block to weight the corresponding row vectors in the initial context feature matrix to obtain the weighted context feature matrix. The output module concatenates the weighted context feature matrix and the adjusted attention probability matrix along the feature dimension to form a fused feature matrix, which is then input into the defect decoding network. The output is a mask map representing the location of defects on the surface of the filter component.

[0015] Furthermore, in the overlay module, when dividing the original digital image into multiple non-uniformly sized image block sequences, a 3×3 Sobel operator is used to calculate the local gradient magnitude of each pixel in the original digital image; with each pixel as the center, the information entropy of the pixel is calculated within a 9×9 neighborhood window; after normalizing the local gradient magnitude and information entropy of each pixel, they are weighted and summed with preset weights to obtain a complexity score map covering the entire original digital image; a quadtree partitioning method is used to recursively partition the original digital image according to the complexity score map: starting from the initial block containing the entire original digital image, for each image block, if the average complexity score of all pixels in the image block exceeds a preset threshold and the image block size is greater than the minimum size limit, then the image block is split into four equal-sized sub-blocks until all image blocks meet the termination condition, thereby forming a non-uniformly sized image block sequence.

[0016] Furthermore, in the overlay module, when overlaying the learnable position bias matrix, the two-dimensional offset between the center pixel coordinates of the current query block and the queried key block in the original digital image is calculated. ; 2D offset As an index, the corresponding scalar bias value is queried in a pre-defined, parameter-learnable two-dimensional bias embedding table; The scalar bias values ​​obtained from the query are superimposed onto the corresponding elements of the dot product of the query matrix and the key matrix.

[0017] The beneficial effects are as follows: Compared with existing technologies, this invention, through a non-uniform block-segmentation strategy based on local image gradients and information entropy, prioritizes the allocation of computational resources to information-rich potential defect regions. This reduces computational redundancy when processing high-resolution images without sacrificing key details, thus improving detection efficiency. In attention calculation, a learnable position bias matrix is ​​obtained by superimposing the dot product of the query matrix and the key matrix, compensating for the loss of relative spatial position information between image blocks in the block-segmentation mode and enhancing the model's ability to perceive the geometric location of defects. Furthermore, an adjustment coefficient based on the variance ratio of the attention score is introduced in the Softmax normalization stage, stabilizing the numerical calculation process and preventing large-area background features from overwhelming sparse but crucial defect features, thereby improving sensitivity to small or low-contrast defects. By performing secondary weighting and feature fusion on the initial context features, the accuracy of filter component defect detection is improved. Attached Figure Description

[0018] Figure 1 This is a flowchart of a defect detection method during the bonding process of image-processed filter components; Figure 2 A schematic diagram illustrating the introduction of positional bias in block attention; Figure 3This is a schematic diagram of the defect detection feature fusion and decoding process. Detailed Implementation

[0019] An embodiment of the defect detection method for filter assembly bonding process based on image processing provided by the present invention: like Figure 1 As shown, the defect detection method for filter assembly bonding process based on image processing includes the following steps: S1. Obtain the original digital image of the filter component, and divide the original digital image into multiple image block sequences of non-uniform size based on the local gradient magnitude and information entropy of the original digital image; perform block attention calculation on the image block sequence, and superimpose a learnable position bias matrix to obtain the initial attention score when calculating the dot product of the query matrix and the key matrix. The learnable position bias matrix is ​​related to the relative spatial position of the image block in the original digital image.

[0020] Specifically, a camera is used to acquire the raw digital image of the filter assembly. For example, the resolution of the raw digital image is 4096×4096 pixels. The raw digital image is initially divided into a uniform grid, for example, into 64×64 image blocks of size 64×64 pixels. For each image block, the Sobel operator is used to calculate the local gradient magnitude of all pixels within the image block and the average value is obtained. At the same time, the information entropy of the image block is calculated. The average local gradient magnitude and information entropy are normalized and then weighted and summed to obtain a comprehensive complexity score. A predetermined threshold is set. If the comprehensive complexity score of an image block is higher than the predetermined threshold, it indicates that the image block contains defects or complex textures. The image block is then further divided into four smaller image blocks, for example, 32×32 pixels. If the score is lower than the predetermined threshold, the original image block size remains unchanged. This splitting process is performed recursively until the comprehensive complexity score of all image blocks is lower than the predetermined threshold or reaches the preset minimum block size. All non-uniformly sized image blocks are flattened in order from left to right and from top to bottom to form a one-dimensional image block sequence.

[0021] The obtained image patch sequence is input into an embedding layer, where each image patch is converted into a fixed-dimensional feature vector, and then linearly transformed to obtain the query matrix Q, key matrix K, and value matrix V. Attention calculation is performed in a block-based manner, dividing the Q, K, and V corresponding to the image patch sequence into multiple blocks along the sequence dimension using the same block partitioning method. In the calculation of the... The query block and the first When performing the dot product of the two key blocks, the difference between their row indices and column indices is calculated based on their positions in the original digital image. These differences are then used as indices to search for the corresponding bias value in a predefined, learnable two-dimensional position bias matrix that is optimized during network training. The matrix formed by the found bias values ​​is then added element-wise to the dot product of the query block and the transpose of the key block, as shown below. Figure 2 This yields an initial attention score matrix containing explicit relative spatial location information.

[0022] In an optional embodiment, in S1, when the original digital image is divided into a sequence of multiple image blocks of non-uniform size, a 3×3 Sobel operator is used to calculate the local gradient magnitude of each pixel in the original digital image. The information entropy of each pixel is calculated within a 9×9 neighborhood window, centered on the pixel. After normalizing the local gradient magnitude and information entropy of each pixel, the values ​​are weighted and summed with preset weights to obtain a complexity score map covering the entire original digital image. The quadtree partitioning method is adopted to recursively partition the original digital image based on the complexity score map: starting from the initial block containing the entire original digital image, for each image block, if the average complexity score of all pixels in the image block exceeds the preset threshold and the image block size is greater than the minimum size limit, the image block is split into four equal-sized sub-blocks until all image blocks meet the termination condition, thus forming a sequence of image blocks with non-uniform size.

[0023] For example, for a pixel, the Sobel operator is used to calculate the horizontal and vertical gradients of that pixel, assuming they are 80 and 60 respectively. The local gradient magnitude of that pixel is then 100. Within a 9×9 neighborhood window surrounding the pixel, the frequency of each gray level is counted to calculate the information entropy: the information entropy within a textured neighborhood window might be 7.2, while the information entropy within a flat neighborhood window might be only 0.5.

[0024] The local gradient magnitude and information entropy are normalized, for example, linearly mapped to the [0, 1] interval. Assuming the normalized local gradient magnitude is 0.4 and the information entropy is 0.9, a weighted sum is then performed according to preset weights (e.g., the weight of the local gradient magnitude is 0.5, and the weight of the information entropy is 0.5), resulting in a complexity score of 0.65 for the pixel. Performing this operation on all pixels generates a complete complexity score map.

[0025] The quadtree partitioning is performed based on the complexity score map. This process starts with the entire original digital image, for example, using a 1024×1024 pixel original digital image as an initial block. The average complexity score of all pixels within this initial block is calculated. A preset threshold is set, and the minimum size limit for the image block is set to 16×16 pixels. Since the average complexity score of the initial block is higher than the preset threshold and the size is also larger than the minimum size limit, the initial block is split into four 512×512 pixel sub-blocks.

[0026] The above process is repeated for the four sub-blocks after splitting. If a sub-block is located in a smooth background area of ​​the original digital image, the average complexity score of that sub-block may be lower than a preset threshold, and the splitting of that sub-block is stopped. This recursive process continues until the average complexity score of all sub-blocks is lower than the preset threshold or the size reaches the minimum size limit, ultimately resulting in a series of image blocks of different sizes, which together constitute a non-uniform image block sequence.

[0027] In an optional embodiment, in S1, when superimposing the learnable position bias matrix, the two-dimensional offset between the center pixel coordinates of the current query block and the queried key block in the original digital image is calculated. ; 2D offset As an index, the corresponding scalar bias value is queried in a pre-defined, parameter-learnable two-dimensional bias embedding table; The scalar bias values ​​obtained from the query are superimposed onto the corresponding elements of the dot product of the query matrix and the key matrix.

[0028] Suppose that the center coordinates of the currently processed query block A in the original image are row 120, column 80, and the center coordinates of another query key block B are row 150, column 100. Then, the two-dimensional offsets of key block B relative to query block A are a horizontal offset of 20 and a vertical offset of 30. These horizontal and vertical offsets represent the spatial relationship between key block B and query block A. A pre-defined, learnable two-dimensional bias embedding table is queried using these two-dimensional offsets. The row and column indices of the two-dimensional bias embedding table correspond to the possible vertical and horizontal offsets. During training, each value in the table is continuously learned and optimized. Using the horizontal offset of 20 and the vertical offset of 30 as indices, the corresponding value is searched in the table. When calculating the dot product of the query vector of query block A and the key vector of key block B, the found bias value is added to the dot product to obtain a new attention score. This process is performed on all combinations of query blocks and key blocks, thereby incorporating spatial location information into the calculation of attention scores.

[0029] S2, when performing block-by-block online Softmax normalization on the initial attention score, calculate the adjustment coefficient and use the adjustment coefficient to adjust the update process of the row normalization factor to obtain the adjusted attention probability matrix. The adjustment coefficient is determined by the ratio of the statistical variance of the initial attention score in the current query block to the global cumulative variance.

[0030] Specifically, the online Softmax normalization process processes the key matrix block by block. For a given query block, when processing the initial attention score block calculated from the query block and the current key block, the statistical variance of all elements within the initial attention score block is calculated and denoted as the local variance. Simultaneously, a global cumulative variance variable for the query block is maintained, which sums the variances of the initial attention scores calculated from the query block and all processed key blocks.

[0031] Divide the current local variance by the updated global cumulative variance to obtain the adjustment coefficient for the current step. When updating the row normalization factor (i.e., the cumulative value of the Softmax denominator), multiply the exponential term calculated from the new score by this adjustment coefficient, and then scale and sum it with the row normalization factor.

[0032] This mechanism reduces the impact of attention scores generated by background regions with low information content on the row normalization factor, highlighting the contribution of sparse defective regions. After traversing all key blocks, normalization is performed to obtain the adjusted attention probability matrix.

[0033] In an optional embodiment, in S2, when calculating the adjustment coefficient, for the currently processed query block... Calculate the query block Statistical variance of all initial attention scores within the context ; To maintain the global cumulative variance, update it using the exponential moving average method. The update formula is as follows: ; in For smoothing coefficients, This is the updated global cumulative variance. This is the global cumulative variance obtained from the previous calculation; Adjustment coefficient The calculation formula is: ; in To prevent stable terms with a denominator of zero.

[0034] Specifically, the variance of attention scores within a specific query block is calculated. Assuming the currently processed query block b contains 4 query vectors, and each query vector, after being multiplied by all 100 key vectors, generates 100 initial attention scores, query block b generates a total of 400 initial attention scores. The statistical variance of these 400 initial attention scores is then calculated. This statistical variance reflects the degree of attention concentration in query block b. If the initial attention scores vary significantly, it indicates that attention is highly concentrated on a few key blocks, resulting in a large statistical variance. Conversely, if the initial attention scores are relatively uniform, the statistical variance is small.

[0035] Update the global cumulative variance and calculate the adjustment coefficient. Maintain a global cumulative variance to track the average variance level of all query blocks processed by the model; it is updated smoothly using an exponential moving average. Assume the previously calculated global cumulative variance was 8.0, and the smoothing coefficient... Let's set it to 0.1. Then the updated global cumulative variance is 8.15, and the adjustment coefficient calculated using the formula is approximately 1.1875. The adjustment coefficient reflects the dispersion of the current query block attention score relative to the global average.

[0036] In an optional embodiment, in S2, when adjusting the update process of the row normalization factor using the adjustment coefficient, in the online Softmax calculation, for those belonging to the query block... The Okay, normalization factor The update formula is: ; in, The updated normalization factor. This is the normalization factor obtained from the previous calculation. The maximum value in the row. This is the maximum row value obtained from the previous calculation. This is the updated maximum row value. For the dot product score, For query block The corresponding adjustment coefficient is adjusted by exponentially decaying the exponent of the new computation block.

[0037] Specifically, for the attention matrix of the th Before processing new key blocks, the previously calculated normalization factor (e.g., 120.5) and a previously calculated row maximum score (e.g., 5.2) are stored. A new set of dot product scores is calculated with the new key block; assuming the maximum value of this new set of dot product scores is 6.0, then the globally updated row maximum is 6.0. The first part of the update formula is the previously calculated normalization factor multiplied by a scaling factor, which is the exponent of the difference between the previously calculated row maximum and the updated row maximum, used to adjust the scale of the old factor.

[0038] The second part of the formula reflects the adjustment, calculating the exponential sum of the differences between the dot product scores of each new score block and the updated row maximum value; for example, this exponential sum is 30.8. This exponential sum is multiplied by an adjustment term, which is the exponent of a negative adjustment coefficient. Assuming the adjustment coefficient for the current query block b is calculated to be 1.1875, then the adjustment term is approximately 0.305. Therefore, the value of the second part is approximately 9.4. Adding the two results yields a new normalization factor. If the adjustment coefficient is large, it indicates high attention concentration, resulting in a small adjustment term, which suppresses the contribution of the new score block to the normalization denominator, making the attention probability distribution sharper.

[0039] S3. The initial context feature matrix is ​​calculated using the adjusted attention probability matrix and value matrix. The corresponding row vectors in the initial context feature matrix are weighted using the adjustment coefficients corresponding to each query block to obtain the weighted context feature matrix.

[0040] Specifically, the adjusted attention probability matrix is ​​multiplied by the corresponding value matrix blocks, and the results are accumulated to obtain the initial context feature vector for each query block. For each query block, the adjustment coefficients generated in all steps of the online Softmax calculation are averaged to obtain the adjustment coefficient for that query block. Each row vector in the initial context feature matrix is ​​multiplied element-wise by the adjustment coefficient of the corresponding query block, thereby weighting the context features. This enhances context features related to potential defect regions while suppressing context features related to flat background regions, thus generating a weighted context feature matrix.

[0041] In an optional embodiment, in S3, when obtaining the context feature matrix, for the first element in the initial context feature matrix... row vector If the vector belongs to the query block Then the weighted context feature vector of this vector The calculation formula is: ; in, This is the adjustment coefficient corresponding to block b.

[0042] Specifically, an initial context feature vector is generated for each query block using a standard computation process. For example, for the first... Each query block has an initial context feature vector, which is a vector containing information aggregated from all parts of the image. It is assumed to be a 64-dimensional vector with values ​​of 0.1, -2.3, etc. The initial context feature vector represents the preliminary feature representation of the query block in the global view.

[0043] The initial context feature vector is weighted using the adjustment coefficients previously calculated for each query block: assuming the first... The adjustment coefficient for the large query block b to which the current query block belongs is calculated to be 1.1875. First, a weighting factor is calculated, which is 2.1875. Then, the weighting factor is multiplied by the... The entire initial context feature vector of each query block. This weighting factor amplifies each element in the initial context feature vector. The significance of this mechanism is that query blocks with larger adjustment coefficients are more focused on the information they have detected, and are therefore considered to have detected more critical information. Thus, by amplifying the initial context feature vector, its influence in subsequent networks is enhanced.

[0044] S4 concatenates the weighted context feature matrix and the adjusted attention probability matrix along the feature dimension to form a fused feature matrix, which is then input into the defect decoding network to output a mask map representing the location of defects on the surface of the filter component.

[0045] Specifically, for the first image patch in the sequence Extract the th image patch from the weighted context feature vector. Row vectors, and the first row in the adjusted attention probability matrix Row vectors. Two vectors are concatenated along their feature dimensions to form a longer, more information-rich fused feature vector. The fused feature vectors of all image patches are reorganized into a two-dimensional feature map, the spatial layout of which corresponds to the non-uniform block division in the original digital image. This two-dimensional feature map is input into a decoder network consisting of multiple transposed convolutional layers and convolutional layers. The decoder network progressively upsamples the feature map, restoring its resolution to the size of the original input image. Finally, a 1×1 convolutional layer and a sigmoid activation function are applied to the last layer, outputting a single-channel probability map where each pixel value represents the probability of a defect at that location. A threshold (e.g., 0.5) is set to binarize the probability map, generating a black-and-white mask image of the defect, such as... Figure 3 The white pixel areas indicate the locations of defects in the filter components.

[0046] In an optional embodiment, S4 includes the following steps: The weighted context feature matrix of dimension (N, d) and the adjusted attention probability matrix of dimension (N, N) are concatenated along the feature dimensions to obtain the fusion feature matrix of dimension (N, d+N), where N is the number of image patches and d is the feature dimension. The fused feature matrix is ​​input into a defect decoding network composed of multiple fully connected layers, and a defect probability score is output for each image patch, resulting in a probability vector of dimension (N, 1). Initialize an all-zero mask with the same size as the original digital image. Based on each probability score in the probability vector and the position and size of the image block corresponding to the probability score in the original digital image, fill the pixel values ​​of each image block region with the corresponding probability score to generate the mask.

[0047] Assume the image is divided into 150 blocks (N=150), and each block's context feature vector has a dimension d=64. This results in a 150x64 context feature matrix. Simultaneously, there is also a 150x150 adjusted attention probability matrix. For the i-th block, extract its corresponding 64-dimensional feature vector and the 150 probability values ​​from the i-th row of the attention probability matrix, and concatenate them to form a 214-dimensional fused feature vector. Perform this operation on all 150 blocks to obtain a 150x214 fused feature matrix.

[0048] A 150×214 fused feature matrix is ​​input into a defect decoding network, which typically consists of several fully connected layers. The defect decoding network processes each row (i.e., the 214-dimensional fused feature of each block) and outputs a value between 0 and 1, representing the probability that the block contains a defect. For example, inputting the first row of features results in an output of 0.98, and inputting the second row results in an output of 0.05. After processing all 150 rows, a 150-row, 1-column defect probability vector is obtained.

[0049] Create a black image with the exact same dimensions as the original digital image, where all pixel values ​​are 0. Fill the black image with pixels based on the location, size, and defect probability of each block. For example, if the first block is a 256×256 pixel area in the top left corner of the image with a defect probability of 0.98, then all pixel values ​​within that area are set to 0.98. The second block might be a 64×64 pixel area with a probability of 0.05, so its pixel values ​​are set to 0.05. After iterating through all 150 blocks and filling them, a mask image is generated. The brightness of regions in the mask image corresponds to the probability of a defect in that region.

[0050] An embodiment of the defect detection system for the bonding process of filter components based on image processing provided by the present invention: A defect detection system for filter assembly bonding process based on image processing includes the following modules: The overlay module acquires the original digital image of the filter component and divides the original digital image into multiple non-uniformly sized image block sequences based on the local gradient magnitude and information entropy of the original digital image; it performs block attention calculation on the image block sequences, and when calculating the dot product of the query matrix and the key matrix, it overlays a learnable position bias matrix to obtain an initial attention score, wherein the learnable position bias matrix is ​​related to the relative spatial position of the image block in the original digital image; The adjustment module calculates the adjustment coefficient when performing block-by-block online Softmax normalization on the initial attention score and uses the adjustment coefficient to adjust the update process of the row normalization factor to obtain the adjusted attention probability matrix. The adjustment coefficient is determined by the ratio of the statistical variance of the initial attention score in the current query block to the global cumulative variance. The weighting module calculates the initial context feature matrix using the adjusted attention probability matrix and value matrix, and then uses the adjustment coefficients corresponding to each query block to weight the corresponding row vectors in the initial context feature matrix to obtain the weighted context feature matrix. The output module concatenates the weighted context feature matrix and the adjusted attention probability matrix along the feature dimension to form a fused feature matrix, which is then input into the defect decoding network. The output is a mask map representing the location of defects on the surface of the filter component.

[0051] In an optional embodiment, in the overlay module, when dividing the original digital image into multiple non-uniformly sized image block sequences, a 3×3 Sobel operator is used to calculate the local gradient magnitude of each pixel in the original digital image; with each pixel as the center, the information entropy of the pixel is calculated within a 9×9 neighborhood window; after normalizing the local gradient magnitude and information entropy of each pixel, they are weighted and summed with preset weights to obtain a complexity score map covering the entire original digital image; a quadtree partitioning method is used to recursively partition the original digital image according to the complexity score map: starting from the initial block containing the entire original digital image, for each image block, if the average complexity score of all pixels in the image block exceeds a preset threshold and the image block size is greater than the minimum size limit, then the image block is split into four equal-sized sub-blocks until all image blocks meet the termination condition, thereby forming a non-uniformly sized image block sequence.

[0052] In an optional embodiment, in the overlay module, when overlaying the learnable position bias matrix, the two-dimensional offset between the center pixel coordinates of the current query block and the queried key block in the original digital image is calculated. ; 2D offset As an index, the corresponding scalar bias value is queried in a pre-defined, parameter-learnable two-dimensional bias embedding table; The scalar bias values ​​obtained from the query are superimposed onto the corresponding elements of the dot product of the query matrix and the key matrix.

[0053] In addition, in the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

Claims

1. A method for defect detection in the process of aligning a filter assembly based on image processing, characterized in that, The method comprises the following steps: S1, obtaining an original digital image of the optical filter assembly, and dividing the original digital image into a plurality of image block sequences with non-uniform sizes based on local gradient amplitudes and information entropy of the original digital image; The image block sequence is subjected to block attention calculation, and a learnable position bias matrix is superimposed when calculating the dot product of the query matrix and the key matrix to obtain an initial attention score, the learnable position bias matrix being related to the relative spatial position of the image block in the original digital image; S2, when performing block-by-block online Softmax normalization on the initial attention score, an adjustment coefficient is calculated and the adjustment coefficient is used to adjust the update process of the row normalization factor to obtain an adjusted attention probability matrix, the adjustment coefficient being determined by the ratio of the statistical variance of the initial attention score in the current query block to the global cumulative variance; S3, using the adjusted attention probability matrix and the value matrix to calculate an initial context feature matrix, and weighting the corresponding row vectors in the initial context feature matrix using the adjustment coefficients corresponding to the query blocks to obtain a weighted context feature matrix; S4, splicing the weighted context feature matrix and the adjusted attention probability matrix along the feature dimension to form a fusion feature matrix and inputting the fusion feature matrix into a defect decoding network to output a mask image representing the position of the surface defects of the optical filter assembly.

2. The method of claim 1, wherein, In S1, when the original digital image is divided into a plurality of image block sequences with non-uniform sizes, a 3*3 Sobel operator is used to calculate the local gradient amplitude of each pixel in the original digital image; the information entropy of each pixel is calculated in a 9*9 neighborhood window centered on the pixel; after the local gradient amplitude and the information entropy of each pixel are normalized, the pre-set weight is used for weighted summation to obtain a complexity score map covering the entire original digital image; the original digital image is recursively divided according to the complexity score map using a quadtree division method: starting from an initial block containing the entire original digital image, for each image block, if the average value of the complexity scores of all pixels in the image block exceeds a pre-set threshold and the size of the image block is greater than a minimum size limit, the image block is divided into four equal sub-blocks, until all image blocks meet the termination condition, thereby forming a plurality of image block sequences with non-uniform sizes.

3. The method of claim 2, wherein the method further comprises: In S1, the two-dimensional offset of the center pixel coordinates of the current query block and the key block of the query in the original digital image is calculated when superimposing the learnable position bias matrix ; 2D offset As an index, the corresponding scalar bias value is queried in a pre-defined, parameter-learnable two-dimensional bias embedding table; The scalar bias value is superimposed on the corresponding elements of the dot product result of the query matrix and the key matrix.

4. The method of claim 1, wherein the method further comprises: In S2, when calculating the adjustment coefficient, for the current processing query block , calculate the statistical variance of all initial attention scores inside the query block ;​ The global accumulated variance is maintained and updated by an exponential moving average method, and the update formula is: ; wherein is a smoothing coefficient, is the updated global accumulated variance, is the global accumulated variance from the previous calculation; Adjustment coefficient The calculation formula is: ; wherein is a stabilizing term to prevent a denominator of zero.

5. The method of claim 4, wherein the method further comprises: In S2, when adjusting the update process of the row normalization factor using the adjustment coefficient, in the online Softmax calculation, for those belonging to the query block... The Okay, normalization factor The update formula is: ; wherein, is the updated normalization factor, is the last calculated normalization factor, is the row maximum, is the last calculated row maximum, is the updated row maximum, is the dot product score, is the query block is the corresponding adjustment factor, which adjusts the exponential decay of the index of the newly calculated block.

6. The method of claim 1, wherein, In S3, when obtaining the context feature matrix, for the first element in the initial context feature matrix... row vector If the vector belongs to the query block Then the weighted context feature vector of this vector The calculation formula is: ; wherein, is the adjustment factor corresponding to the block b of the query.

7. The method of claim 1-6, wherein, S4 comprises the following steps: The weighted context feature matrix with a dimension of (N, d) and the adjusted attention probability matrix with a dimension of (N, N) are spliced along the feature dimension to obtain a fusion feature matrix with a dimension of (N, d+N), wherein N is the number of image blocks and d is the feature dimension; The fusion feature matrix is input into a defect decoding network composed of a plurality of fully connected layers to output a defect probability score for each image block, thereby obtaining a probability vector with a dimension of (N, 1); An all-zero mask image with the same size as the original digital image is initialized, and according to each probability score in the probability vector and the position and size of the image block corresponding to the probability score in the original digital image, the pixel values of each image block region are filled with the corresponding probability score to generate the mask image.

8. A system for defect detection in the process of aligning a filter assembly based on image processing, characterized in that, Comprise the following modules: The superposition module obtains the original digital image of the optical filter assembly, and divides the original digital image into a plurality of image block sequences with non-uniform sizes based on the local gradient amplitude and information entropy of the original digital image; The image block sequence is subjected to block attention calculation, and in the calculation of the dot product of the query matrix and the key matrix, an initial attention score is obtained by superimposing a learnable position bias matrix, which is related to the relative spatial position of the image block in the original digital image; The adjustment module calculates an adjustment coefficient and adjusts the update process of the row normalization factor using the adjustment coefficient when performing block-by-block online Softmax normalization on the initial attention score to obtain an adjusted attention probability matrix, and the adjustment coefficient is determined by the ratio of the statistical variance of the initial attention score in the current query block to the global cumulative variance; The weighting module calculates an initial context feature matrix using the adjusted attention probability matrix and the value matrix, and weights the corresponding row vectors in the initial context feature matrix using the adjustment coefficients corresponding to each query block to obtain a weighted context feature matrix; The output module splices the weighted context feature matrix and the adjusted attention probability matrix along the feature dimension to form a fusion feature matrix and inputs it into the defect decoding network to output a mask image representing the position of the surface defects of the optical filter assembly.

9. The image processing based defect detection system in filter assembly bonding process according to claim 8, wherein, In the superposition module, when dividing the original digital image into a plurality of image block sequences with non-uniform sizes, a 3×3 Sobel operator is used to calculate the local gradient amplitude of each pixel in the original digital image; the information entropy of each pixel is calculated in a 9×9 neighborhood window centered on the pixel; after normalizing the local gradient amplitude and information entropy of each pixel, the sum of the weighted sums with preset weights is obtained to obtain a complexity score map covering the entire original digital image; using a quadtree division method, the original digital image is recursively divided according to the complexity score map: starting from an initial block containing the entire original digital image, for each image block, if the average value of the complexity scores of all pixels in the image block exceeds a preset threshold and the image block size is greater than the minimum size limit, the image block is divided into four equal sub-blocks, until all image blocks meet the termination condition, thereby forming a plurality of image block sequences with non-uniform sizes.

10. The image processing based defect detection system in filter assembly bonding process according to claim 9, wherein, In the superposition module, when superimposing the learnable position bias matrix, the two-dimensional offset of the center pixel coordinates of the current query block and the key block in the original digital image is calculated ; 2D offset As an index, the corresponding scalar bias value is queried in a pre-defined, parameter-learnable two-dimensional bias embedding table; The scalar bias value is superimposed on the corresponding elements of the dot product result of the query matrix and the key matrix.

Images