A fresnel lens butterfly pattern defect detection method based on an annular light source imaging scheme and improved RT-DETR

Abstract

The application discloses a Fresnel lens butterfly pattern defect detection method based on an annular light source imaging scheme and an improved RT-DETR, relates to the technical field of computer vision, and collects images, which are input into a pre-trained improved RT-DETR model after necessary input consistency processing. The model is optimized in the light of the difficulty in extracting the "strong periodic background and weak fine texture defect" of the Fresnel lens: in the C3 and C4 cross-scale fusion stage of the neck CCFM module, the interference of the normal ring pattern background on the fusion result is reduced, and the small-scale butterfly pattern defect fine texture features are protected and enhanced; in the neck AIFI stage, the attention mechanism is guided to focus on the suspected defect area, and the invalid occupation of the background area on the computing resources is reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a Fresnel lens butterfly defect detection method based on a ring light source imaging scheme and an improved RT-DETR. Background Technology

[0002] Fresnel lenses are widely used in various optical systems due to their thin and lightweight structure, low cost, and excellent light-gathering and diffusion properties. Their dense concentric circular serrations are a key structural feature for achieving optical functionality. During manufacturing processes such as injection molding and pressing, butterfly-shaped defects can easily appear on the lens surface due to variations in raw materials, fluctuations in process parameters, and molding stress. Butterfly patterns disrupt the consistency of the serration structure and the surface optical uniformity, leading to abnormal light refraction / reflection paths, thus affecting image quality and optical performance, and consequently impacting the stability and lifespan of the equipment. Therefore, precise and stable detection of butterfly-shaped defects is necessary during production to ensure product consistency and reliability.

[0003] Under coaxial uniform illumination conditions such as a ring light source, a relatively uniform illumination background can be provided, which helps to highlight local grayscale changes caused by defects. However, due to the strong periodic reflective properties of Fresnel teeth, if parameters such as illumination angle, luminous power, and illumination uniformity do not match the tooth structure characteristics, butterfly patterns can still be easily confused with normal tooth reflections, resulting in low defect contrast. In particular, tiny butterfly patterns are easily submerged by background reflections, leading to problems such as missed detections and false detections. Although a ring light source provides uniform illumination, its illumination parameters still need to be matched with the tooth structure of the lens to realize its advantages in detection.

[0004] Existing Fresnel lens butterfly defect detection technologies still suffer from the following shortcomings: manual visual inspection relies on human experience, is highly subjective and inefficient, and cannot meet the batch inspection needs of industrial production lines; under common lighting conditions such as natural light, surface light sources, or point light sources, the strong periodic background of Fresnel lenses makes it difficult to distinguish the butterfly pattern from the reflection of normal tooth patterns, resulting in low defect contrast, especially since tiny butterfly patterns are easily submerged by the background, leading to missed detections, false detections, and inconsistent detection results. Even with the use of general deep learning detection models (such as RT-DETR, YOLO, SSD, etc.), it is still difficult to effectively solve these problems: in the Neck cross-scale fusion stage, normal ring pattern background features are easily mixed into the fusion results and interfere with the extraction of defect features; small-scale butterfly pattern fine textures are easily covered by large-scale features during the fusion process, resulting in weakened details and reduced recognizability; general encoder attention mechanisms tend to allocate computational resources to background ring patterns or reflective areas, resulting in insufficient attention to defect areas and redundant computation, thus affecting detection accuracy and stability.

[0005] Furthermore, most existing ring light source defect detection solutions are designed for flat, transparent parts, failing to adequately consider the dense concentric serration structure of Fresnel lenses. They lack matching designs between key parameters such as illumination angle, luminous power, and illumination uniformity and the serration structure characteristics, making it difficult to effectively distinguish normal serrations from butterfly patterns under ring light projection. If a fixed projection angle is used without optimization based on the serration extension direction and light transmission characteristics, it can easily exacerbate the confusion of reflective features, making it difficult to stably highlight the brightness gradient of butterfly patterns, and may even introduce new periodic interference.

[0006] In summary, existing Fresnel lens butterfly defect detection technology, under directional illumination conditions such as ring light sources, still struggles to simultaneously achieve detection accuracy, stability, and real-time performance: Strong reflections from the concentric tooth pattern background make it difficult to distinguish the fine texture of the butterfly pattern from the background, resulting in low defect contrast; existing Neck fusion mechanisms cannot effectively suppress interference from normal ring pattern backgrounds, and small-scale defects are easily covered by large-scale features; encoder attention mechanisms focus excessively on reflective background areas, leading to insufficient defect feature extraction and redundant computation; furthermore, existing detection heads lack effective background prior and defect-priority recognition mechanisms, resulting in unstable detection results, inability to cope with operating condition fluctuations, and difficulty in meeting online real-time detection requirements (e.g., ≥30 frames / second). Therefore, there is an urgent need for a butterfly defect detection method that can be optimized under ring light illumination conditions through techniques such as background suppression during the fusion stage, small-scale feature enhancement, sparse attention focusing, and output stability coordination to improve detection accuracy, stability, and real-time performance, meeting the needs of industrial online and offline detection. Summary of the Invention

[0007] To overcome the shortcomings of the above technologies, this invention provides a method for achieving accurate and efficient detection of butterfly pattern defects, balancing detection accuracy and real-time performance, and meeting the online batch detection needs of industrial production lines.

[0008] The technical solution adopted by this invention to overcome its technical problems is: A method for detecting butterfly-shaped defects in Fresnel lenses based on a ring light source imaging scheme and an improved RT-DETR includes: S1. Obtain Zhang Fresnel lens image, to obtain the original Fresnel lens butterfly pattern image set. , , For the first Zhang Fresnel lens butterfly pattern image; S2. For the first Zhang Fresnel lens butterfly pattern image Preprocessing is performed to obtain the preprocessed Fresnel lens butterfly pattern image. All preprocessed Fresnel lens butterfly pattern images constitute a preprocessed Fresnel lens butterfly pattern image set. , ; S3. Assemble the preprocessed Fresnel lens butterfly pattern images. It is divided into training set, test set, and validation set; S4. Establish an improved RT-DETR model consisting of the Backbone network of the RT-DETR model, the Neck network, and the detection head of the RT-DETR model. The Neck network consists of the neck AIFI module and the neck CCFM module. S5. The training set of the first Zhang's processed Fresnel lens butterfly pattern image The input is fed into the backbone network of the improved RT-DETR model, and the output is the feature map. Feature map Feature map ; S6. Feature Map The input is fed into the neck AIFI module of the improved RT-DETR model's Neck network, and the output yields enhanced features. ; S7. Transfer the feature map Feature map Enhanced features The input is fed into the neck CCFM module of the improved RT-DETR model's Neck network, and the output is a fused feature map. ; S8. Merge feature maps The input is fed into the detection head of the improved RT-DETR model, and the output is a recognition image of the butterfly pattern defect detected in the Fresnel lens.

[0009] Furthermore, step S1 includes the following steps: S1-1. Selection A Fresnel lens with a butterfly-shaped defect; S1-2. Using an industrial camera positioned directly below the Fresnel lens and directly above it, each Fresnel lens with a butterfly-shaped defect is photographed to obtain... The image is formed by a Fresnel lens, wherein the axis of the annular light source is coaxial with the axis of the Fresnel lens, and the axis of the Fresnel lens is coaxial with the optical axis of the industrial camera lens.

[0010] Furthermore, step S2 includes the following steps: S2-1. The first Zhang Fresnel lens butterfly pattern image By using the `flip` function in the OpenCV library of Python and setting the coordinate axis parameter to 1 to perform a horizontal flip, we obtain the flipped Fresnel lens butterfly pattern image. ; S2-2. The flipped Fresnel lens butterfly pattern image The center of the Fresnel lens is located using the HoughCircles method from the OpenCV library in Python. and effective imaging radius Using the getRectSubPix function in the OpenCV library of Python, the center point is... The image is cropped to obtain the Fresnel lens butterfly pattern. ; S2-3. The cropped Fresnel lens butterfly pattern image The GaussianBlur function from the OpenCV library in Python is used to generate a low-frequency illumination background image, resulting in a Fresnel lens butterfly pattern image. The Fresnel lens butterfly pattern image After performing illumination correction using the cv2.divide function from the OpenCV library in Python, grayscale normalization was performed to obtain the corrected Fresnel lens butterfly pattern image. ; S2-4. The corrected Fresnel lens butterfly pattern image The contrast was adjusted using the convertScaleAbs function in the OpenCV library of Python to obtain the adjusted Fresnel lens butterfly pattern image. ; S2-5. Adjusted Fresnel lens butterfly pattern image The Fresnel lens butterfly pattern image was obtained by rotating the image within a range of ±3° using the warpAffine function in the OpenCV library of Python. ; S2-6. Rotate the Fresnel lens butterfly pattern image The image of the Fresnel lens butterfly pattern is obtained by converting it to grayscale using the cvtColor function in the OpenCV library of Python. ; S2-7. Process the Fresnel lens butterfly pattern image. The GaussianBlur function from the OpenCV library in Python is used to generate a low-frequency illumination background image, resulting in a Fresnel lens butterfly pattern image. ; S2-8. Fresnel lens butterfly pattern image The `createCLAHE` function from the OpenCV library in Python was used to enhance image contrast, resulting in a preprocessed collection of Fresnel lens butterfly patterns. .

[0011] Preferably, in step S2-3, the GaussianBlur function uses a 51×51 Gaussian kernel; in step S2-7, the GaussianBlur function uses a 3×3 Gaussian kernel with a standard deviation of 1.0; and in step S2-8, the createCLAHE function sets clipLimit to 1.81 and the grid size to 16×16.

[0012] Preferably, in step S3, the preprocessed Fresnel lens butterfly pattern image set is... The dataset is divided into training, testing, and validation sets in an 8:1:1 ratio.

[0013] Furthermore, step S6 includes the following steps: S6-1. The neck AIFI module of the improved RT-DETR model's Neck network consists of the input projection module of the RT-DETR model and the improved AIFI module. The improved AIFI module consists of the spatial dimension flattening layer of the AIFI module in the neck of the RT-DETR model, the 2D position encoding module in the neck of the RT-DETR model, the Transformer encoding layer, and the dimension reshaping layer of the RT-DETR model. S6-2. Feature Map The input is fed into the input projection module, and the output is the feature map. ; S6-3. Feature Map The input is fed into the spatial dimension flattening layer of the improved AIFI module to obtain a two-dimensional matrix. , two-dimensional matrix The input is fed into the 2D position encoding module of the improved AIFI module, and the output is a feature map. ; S6-4. The improved AIFI module's Transformer encoding layer consists of the first pre-LayerNorm layer of the RT-DETR model's Transformer encoding layer, a multi-head attention module, a first residual connection layer, the second pre-LayerNorm layer of the RT-DETR model's Transformer encoding layer, a feedforward network module of the RT-DETR model's Transformer encoding layer, and a second residual connection layer, which integrates the feature map. The input is fed into the first pre-LayerNorm layer of the Transformer encoding layer, and the output is the feature map. The multi-head attention module consists of the QKV linear projection layer of the multi-head self-attention module in the Transformer encoding layer of the RT-DETR model, the multi-head splitting and scaling dot product calculation unit of the RT-DETR model, the anchor-linked attention module, the Softmax function, the attention weight and V-value weighted summation unit of the RT-DETR model, and the multi-head merging and output linear projection layer of the RT-DETR model, which integrates the feature maps. The input is fed into the QKV linear projection layer to obtain the query vector. Key vector Value vector , query vector Key vector Value vector The input is fed into the multi-head splitting and scaling dot product computation unit, and the output is the attention score matrix. ; S6-5. Anchor Point Linkage in Multi-Head Attention Module: The multi-head attention module consists of an anchor point extraction unit, an attention allocation unit, and a residual fusion unit. The anchor point extraction unit comprises a 1×1 convolutional layer, a batch normalization layer, a SiLU activation function, a texture feature extraction branch, and a filtering output layer, which integrates the feature maps... The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the anchor point extraction unit, and the output is the feature map. , feature map The Sobel operator is used to process the feature map in the texture feature extraction branch of the anchor point extraction unit. Gradient calculations are performed in both the horizontal and vertical directions to obtain the feature map. The Middle Horizontal gradient of each pixel and vertical gradient Through formula The feature map is calculated. The Middle gradient magnitude of each pixel Through formula The feature map is calculated. The Middle gradient direction of each pixel If the gradient magnitude Greater than or equal to the threshold Then retain the first For each pixel, if the gradient magnitude Less than the threshold Then remove the first one. From each pixel, a binary mask image is obtained. Divide the gradient directions of all retained pixels into equal parts from 0° to 180°. By dividing the area into equal-width intervals, a direction histogram is obtained. The direction corresponding to the interval with the largest amplitude in the direction histogram is selected as the main direction of the butterfly pattern. Using connected component functions to analyze binary mask images Perform eight-neighbor connectivity detection to obtain The area value of the connected component, the th The area of ​​each connected region is Through formula Calculate the mean area of ​​the connected components , will the The area of ​​each connected region is and the average area of ​​connected regions The input is sent to the filtered output layer of the anchor point extraction unit. If the first... Area values ​​of connected components Greater than or equal to the threshold Then the connected component is taken as a candidate region for anchor point, if the first... Area values ​​of connected components Less than the threshold Then the connected component is removed, threshold The average area of ​​connected regions 1.3-1.7 times, through the formula Calculate the first The centroid x-coordinate of each anchor point candidate region Through formula Calculate the first The centroid ordinate of each anchor point candidate region ,in For the first The first anchor point candidate region The x-coordinate of each pixel For the first The first anchor point candidate region The ordinate of each pixel , For the first The number of pixels in the candidate region of the first anchor point, the number of pixels in the first anchor point. The centroid coordinates of the candidate anchor point regions are: The centroid coordinates of all anchor point candidate regions constitute the anchor point candidate set. ; S6-6. Generate a binary mask image. Blank images of the same size Blank image All pixels in the blank image are set to 0, and all anchor point candidate regions are filled into the blank image. In the corresponding region, the pixel values ​​of the filled pixels are modified to 255 to obtain the initial response map. The attention allocation unit of the anchor point linkage attention module consists of a linkage graph generation sublayer and an attention modulation sublayer. The linkage graph generation sublayer consists of a coordinate extraction layer, a direction vector encoding layer, a polar coordinate transformation layer, a dynamic gating fusion layer, and an associative attention fusion layer. The attention modulation sublayer consists of a Softmax function, a 1×1 convolutional layer, a SiLU activation function, and a dimension alignment and broadcast adaptation layer, which allocates the anchor point candidate set. Compared with the initial response diagram The coordinate extraction layer of the linkage graph generation sublayer, input to the attention allocation unit, is used in the initial response graph. Extract anchor point candidate set For all corresponding centroid coordinates, divide the x-coordinate of the centroid by . Divide the centroid ordinate by , Initial response diagram width, Initial response diagram The calculated centroid coordinates are mapped to the 0-1 interval, and all centroid coordinates mapped to the 0-1 interval constitute the set of anchor point positions. Set the anchor point locations The direction vector encoding layer, which is input into the linkage graph generation sublayer, calculates the difference in the x-coordinates of any two centroids. and the difference in the vertical axis Through formula The absolute orientation angles of the two centers of mass are calculated. To determine the absolute direction angle In the main direction of the butterfly pattern The absolute value of the difference is used as the direction deviation to construct the direction feature matrix. directional feature matrix The number of rows and columns are both related to the set of anchor points. With the same number of centroids, fill the directional deviation between any two centroids into the directional feature matrix. The corresponding position in the middle; S6-7. Set the anchor point positions The input is fed into the polar coordinate transformation layer of the linkage graph generation sub-layer, using the formula... The set of anchor point locations was calculated. The Middle The radial distance from the centroid to the origin In the formula, The x-coordinate of the Fresnel lens center as calibrated for the production line. The ordinate of the Fresnel lens center as calibrated for the production line. Anchor point location set The Middle The x-coordinate of the centroid, Anchor point location set The Middle The ordinate of each centroid is obtained through the formula. The set of anchor point locations was calculated. The Middle The centroid and the first Concentricity of the centroid Construct a concentricity matrix Concentricity matrix The number of rows and columns are both related to the set of anchor points. If the number of centroids is the same, fill the concentricity of any two centroids into the concentricity matrix. The corresponding position in the middle; S6-8. Directional Feature Matrix With concentricity matrix The directional feature matrix is ​​input into the dynamic gating fusion layer of the linkage graph generation sublayer. With concentricity matrix A concatenation operation is performed, where each element of the concatenated feature matrix is ​​sequentially input into a 1×1 convolutional layer and a sigmoid activation function to obtain the association weights of each element, thus constructing an oriented feature matrix. directional feature matrix The number of rows and columns are both related to the set of anchor points. If the number of centroids is the same, fill the correlation weights of any two centroids into the correlation gating matrix. The corresponding position in the matrix will be associated with the gating matrix. Compared with the initial response diagram The input is fed into the association attention fusion layer of the association graph generation sublayer, and the association gating matrix is ​​then applied. The dimensions were adjusted to match the initial response map. After the dimensions are the same, the associated gating matrix will be used. Compared with the initial response diagram Perform element-wise multiplication to obtain the initial anchor point linkage graph. Link the initial anchor point diagram The inputs are sequentially fed into the Softmax function of the attention modulation sublayer, the convolutional layer, and the SiLU activation function, and the output is the normalized enhanced weight map. Normalized weighted graph The input is dimensionally aligned to the attention modulation sublayer and its shape is adjusted to match the attention score matrix in the broadcast adaptation layer. Consistency is achieved, resulting in a weighted graph. , weight graph With attention score matrix Element-wise multiplication yields the modulated attention score matrix. ; S6-9. The residual fusion unit of the anchor-linked attention module consists of upsampling / downsampling units, a convolutional layer with a kernel size of 1×1, a batchNormalization layer, and a SiLU activation function, which modulates the attention score matrix. The modulated attention score matrix is ​​input into the upsampling / downsampling unit of the residual fusion unit. Adjust to the initial response diagram Consistency is achieved, resulting in the attention score matrix. The attention score matrix The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the residual fusion unit, and the output is a matrix. The attention score matrix With matrix Element-wise summation yields the final attention score matrix. ; S6-10. The final attention score matrix The input is fed into the Softmax function of the multi-head attention module in the Transformer encoding layer of the improved AIFI module, and the output is the attention weight matrix. ; S6-11. Adjust the attention weight matrix The attention weights and V-value weighted summation unit of the RT-DETR model in the multi-head attention module of the Transformer encoding layer of the improved AIFI module are input to obtain the single-head attention output features. ; S6-12. Output features of single-head attention The input to the multi-head attention module of the Transformer encoding layer of the improved AIFI module is combined with the multi-head merging and output linear projection layer of the RT-DETR model to obtain the feature output. ; S6-13. Features The features are input into the first residual connection layer of the Transformer encoding layer of the improved AIFI module. With features Element-wise addition is performed to obtain the features after residual connection. ; S6-14. Features after residual concatenation The input is fed into the second pre-LayerNorm layer of the Transformer encoding layer of the RT-DETR model in the improved AIFI module's Transformer encoding layer, and the output is the normalized feature. ; S6-15. Normalized features The input is fed into the feedforward network module of the Transformer encoding layer of the RT-DETR model in the improved AIFI module, and the output is obtained as the feedforward network output feature. ; S6-16. Output features of the feedforward network The second residual connection layer of the Transformer encoding layer in the improved AIFI module will feed forward the network output features. Features after connection with residuals Element-wise addition is performed to obtain the features after residual connection. ; S6-17. Features after residual concatenation The input is fed into the dimension reshaping layer of the RT-DETR model of the improved AIFI module, and the output is the enhanced feature. .

[0014] Preferably, the threshold The value is 0.4. The value is 16, the threshold. The average area of ​​connected regions 1.5 times.

[0015] Furthermore, step S7 includes the following steps: S7-1. The improved RT-DETR model's Neck CCFM module consists of a fine texture protection enhancement module, a feature dimension adaptation module, a ring-pattern periodic gated fusion unit, and a RepBlock-based multi-scale fusion path of the RT-DETR model's CCFM module. S7-2. The fine texture protection and enhancement module of the neck CCFM module consists of a Sobel convolutional layer, a convolutional layer with a kernel size of 1×1, and an SE module, which enhances the feature map. The input is fed into the Sobel convolutional layer of the fine texture preservation enhancement module, where the Sobel operator is used to process the feature map. Gradient calculations are performed in both the horizontal and vertical directions to obtain the feature map. The Middle Horizontal gradient of each pixel and vertical gradient Through formula The feature map is calculated. The Middle gradient magnitude of each pixel Construct a single-channel gradient magnitude map Single-channel gradient magnitude plot The number of rows and columns are both related to the feature map. With the same number of pixels, single-channel gradient magnitude map The Line 1 The value of the column is , In the formula, For feature map The maximum gradient magnitude of all pixels in the array. For feature map The minimum gradient magnitude of all pixels in the single-channel gradient magnitude map is obtained. The input is fed into the convolutional layer of the fine texture preservation and enhancement module, and the output is the gradient feature map. ; The S7-3. Fine Texture Protection Enhancement Module's SE module consists of a global average pooling layer, a first fully connected layer, a ReLU activation function, a second fully connected layer, and a Sigmoid activation function, sequentially processing the gradient feature map. The input is given to the SE module, and the output is the weight coefficient. Weighting coefficients Adjust to gradient feature map After obtaining the same dimension, multiply it element-wise to obtain the weighted enhanced feature. The weighted enhanced features With feature map Element-wise addition is performed to obtain the feature map after fine texture protection enhancement. ; S7-4. Enhanced features The feature dimension adaptation module input into the neck CCFM module is adjusted to match the feature map. With the same dimensions, feature maps are obtained. ; S7-5. The annular periodic gated fusion unit of the neck CCFM module consists of a grid sampling layer, a 3×3 convolutional layer, a batch normalization layer, a SiLU activation function, a first scale layer, a second scale layer, a sigmoid activation function, and a feature fusion layer, which integrates the feature maps. The feature map is input into the mesh sampling layer of the annular periodic gated fusion unit. The 1 pixel point generated The nth rotating sampling point, the th The pixel number The x-coordinate of each sampling point is , In the formula, For the first The x-coordinate of each pixel , No. The pixel number The ordinate of each sampling point is , , For the first The y-coordinate of the nth pixel, for the nth sampling points The feature vector is obtained by bilinear interpolation. For feature maps The first in For each pixel, the feature values ​​within its channel are extracted and concatenated in channel order to form a feature vector. Calculate the eigenvector With feature vectors The cosine similarity between them, taking the first... Pixel pairs The average cosine similarity of the nth rotated sampling point is used as the th The periodic intensity values ​​of each pixel, and the periodic intensity values ​​of all pixels according to the feature map. The coordinates in the graph are arranged in a one-to-one mapping to obtain a periodic intensity map. ; S7-6. Feature Map The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the ring-shaped periodic gated fusion unit, and the output is the global semantic feature map. The global semantic feature map Flattened to a length equal to one-dimensional array , Global semantic feature map The number of channels, Global semantic feature map of high, Global semantic feature map The width of the array is calculated. Overall variance , total variance The input is fed into the first scale layer of the annular periodic gated fusion unit, and the output is an adaptive global threshold. Adaptive global threshold Adjust the spatial dimensions to match the periodic intensity diagram When the sizes are the same, the global threshold will be adaptive. With periodic intensity map By subtracting the difference, we obtain the result. , the results The input is fed into the second scale layer of the annular periodic gated fusion unit, and the output is the scaled result. , scale the result The input to the sigmoid activation function of the ring-shaped periodic gated fusion unit outputs a pixel-level gated weight map. ; S7-7. Feature Map With feature map The feature map is input into the feature fusion layer of the annular periodic gated fusion unit. With feature map The addition operation yields the fused feature map. Pixel-level gated weight map The dimension is expanded to include the fused feature map. Consistent with each other, a pixel-level gated weight map is obtained. Pixel-level gated weight map With fusion feature map Perform element-wise multiplication to obtain the feature map. ; S7-8. Feature Map The input is fed into the RepBlock-based multi-scale fusion path of the CCFM module of the RT-DETR model, and the output is the fused feature map. .

[0016] Furthermore, this also includes through The total loss was calculated. In the formula, For DIou's loss, For Focal loss, use the Adam optimizer to utilize the total loss. Train the improved RT-DETR model.

[0017] Preferably, when training the improved RT-DETR model, the number of iterations is set to 300, the initial learning rate is set to 1×10^(-4), the learning rate is reduced to half of its original value every 20 iterations, and the batch size is set to 16.

[0018] The beneficial effects of this invention are: (1) Effectively alleviate strong periodic background interference. Through the imaging design of “matching the ring light source parameters with the tooth structure”, combined with the background suppression strategy such as the “ring periodic gating fusion mechanism” of the Neck layer, a synergistic suppression is formed in the imaging and feature fusion stage, reducing the false detection and false detection caused by the confusion between butterfly defects and normal tooth reflection.

[0019] (2) Improved detection accuracy and robustness: The improved RT-DETR uses mechanisms such as "fine texture protection enhancement, anchor point linkage attention focus, and defect priority prior collaborative optimization" and is combined with the DIoU+FocalLoss training strategy, which is more conducive to the stable learning of fine texture features and weak light and dark perturbations of small butterfly defects.

[0020] (3) It has strong industrial adaptability. By setting parameters such as the ring light source illumination angle of 45° and the working distance (0.8–1.8 times the outer diameter of the lens), it can adapt to the inspection needs of Fresnel lenses with different tooth spacing, outer diameter and materials, thereby reducing the production line transformation cost.

[0021] (4) It balances real-time performance and practicality, meets the requirements of online detection cycle time without introducing significant redundant calculations, and supports image annotation and text report output, detection data storage, and realizes quality traceability and process optimization.

[0022] (5) Enhanced efficiency through hardware and software collaboration: This invention achieves a more stable detection effect by adapting the link of "light source parameters - defect characterization - model mechanism - loss function" to the lighting advantages and the algorithm's targeting. Detailed Implementation

[0023] The present invention will be further described below.

[0024] A method for detecting butterfly-shaped defects in Fresnel lenses based on a ring light source imaging scheme and an improved RT-DETR includes: S1. Obtain Zhang Fresnel lens image, to obtain the original Fresnel lens butterfly pattern image set. , , For the first Zhang Fresnel lens butterfly pattern image.

[0025] S2. For the first Zhang Fresnel lens butterfly pattern image Preprocessing is performed to obtain the preprocessed Fresnel lens butterfly pattern image. All preprocessed Fresnel lens butterfly pattern images constitute a preprocessed Fresnel lens butterfly pattern image set. , .

[0026] S3. Assemble the preprocessed Fresnel lens butterfly pattern images. It is divided into training set, test set and validation set.

[0027] S4. Establish an improved RT-DETR model consisting of the Backbone network of the RT-DETR model, the Neck network, and the detection head of the RT-DETR model. The Neck network consists of the Neck AIFI module and the Neck CCFM module.

[0028] S5. The training set of the first Zhang's processed Fresnel lens butterfly pattern image The input is fed into the backbone network of the improved RT-DETR model, and the output is the feature map. Feature map Feature map .in It places greater emphasis on conveying the fine textures and local light and shadow disturbances related to butterfly-pattern defects. It focuses more on carrying the overall structure of the lens and the semantic information of the background. It focuses more on carrying global semantic information related to the global ring pattern distribution and geometric contour of the lens, and is only used for subsequent adaptive adjustment of the neck gating threshold. It does not participate in the attention modeling of the neck AIFI module or other feature calculations. Of the three , It serves as the primary input for subsequent attention modeling of the neck AIFI module and cross-scale fusion of the neck CCFM module. A threshold driving basis is provided separately for the neck gating unit.

[0029] S6. Feature Map The input is fed into the neck AIFI module of the improved RT-DETR model's Neck network, and the output yields enhanced features. .

[0030] S7. Transfer the feature map Feature map Enhanced features The input is fed into the neck CCFM module of the improved RT-DETR model's Neck network, and the output is a fused feature map. .

[0031] S8. Merge feature maps The input is fed into the detection head of the improved RT-DETR model, and the output is a recognition image of the butterfly pattern defect detected in the Fresnel lens.

[0032] The uniform illumination provided by the ring light source makes it easier to present the local changes caused by defects. On the other hand, targeted optimizations are made in the feature fusion, attention modeling and output stages of the detection network to reduce the impact of the normal tooth pattern period background on defect expression and to enhance the fine texture feature modeling and stable output of small butterfly pattern defects. Ultimately, accurate and efficient detection of butterfly pattern defects is achieved, taking into account both detection accuracy and real-time performance, to meet the online batch inspection needs of industrial production lines.

[0033] The improved RT-DETR model is the core component of this invention for achieving accurate identification of butterfly-pattern defects. After necessary input consistency processing, the acquired images are input into a pre-trained improved RT-DETR model. This model is specifically optimized to address the difficulty of extracting "strong periodic background and weak fine-texture defects" from Fresnel lenses: in the C3 and C4 cross-scale fusion stages of the neck CCFM module, the interference of normal ring-pattern background on the fusion results is reduced, and protective enhancement is implemented for the fine-texture features of small-scale butterfly-pattern defects; in the neck AIFI stage, the attention mechanism is guided to focus on suspected defect areas, reducing the ineffective use of computational resources by background areas; in the head output stage, background prior is combined to achieve defect priority discrimination and result stability control, ensuring that defect classification and localization results remain robust under fluctuating operating conditions, thereby significantly improving detection accuracy and meeting the requirements of industrial online real-time detection.

[0034] In one embodiment of the present invention, step S1 includes the following steps: S1-1. Selection A Fresnel lens with a butterfly-shaped defect.

[0035] S1-2. Using an industrial camera positioned directly below the Fresnel lens and directly above it, each Fresnel lens with a butterfly-shaped defect is photographed to obtain... The image is formed by a Fresnel lens, wherein the axis of the annular light source is coaxial with the axis of the Fresnel lens, and the axis of the Fresnel lens is coaxial with the optical axis of the industrial camera lens.

[0036] The illumination parameters of the ring light source here are designed with significant adaptability. They are not simply general parameters applied, but rather matched to the reflective patterns of the concentric serrations of the Fresnel lens. The core design includes three main indicators: First, the illumination angle and working distance are matched. The ring light incident angle is preferably 45°, and the working distance is preferably 0.8–1.8 times the lens outer diameter. This ensures a stable and consistent background response from normal serrations, while making the local brightness and darkness disturbances caused by butterfly-shaped defects more easily visible. Second, the light intensity and exposure are set in a coordinated manner to prevent saturation in the reflective areas of normal serrations, with the maximum grayscale not exceeding 85% of the saturation value, ensuring that the grayscale gradient in the defect area can still be completely recorded by the camera. Third, the illumination uniformity is controllable. Through ring light zoning adjustment or diffusion shaping, the field of view uniformity is ensured to reach at least 85% of the brightest area in the darkest part, reducing misjudgments caused by local bright spots. After the above matching, the lens surface presents a more stable "regular background," while butterfly-shaped defects disrupt this regularity, making the defects easier to distinguish in the image and providing clearer input for subsequent algorithm recognition. This step utilizes an industrial area array camera to acquire images of a Fresnel lens illuminated by a ring light. This industrial camera features high resolution and a high frame rate, enabling stable capture of subtle grayscale / texture changes caused by concentric tooth patterns and butterfly-shaped defects under online monitoring conditions, ensuring complete preservation of defect details. Acquired images are transmitted in real-time to the image processing unit via a high-speed data interface (such as USB 3.0 or GigE) and used as input to an improved RT-DETR detection network, providing high-quality data support for subsequent feature fusion, attention modeling, and defect output. A Hikvision MV-CS200-10GM industrial camera with an MVL-KF1224M-25MP lens can be selected. Camera parameters are set to BMP image output format, resolution adapted to inspection requirements, and frame rate adjusted to meet the speed requirements of industrial online inspection. The ring light source is a programmable ring light source (with a light-emitting area size adapted to the lens outer diameter). The controller supports programmable adjustment of light intensity (0–100%), zone brightness adjustment, and incident angle / working distance matching settings. The incident angle of the ring light is preferably 45°, and the working distance is preferably 0.8–1.8 times the lens outer diameter. The uniformity of the field of view is ensured to meet the requirement that the darkest part reaches at least 85% of the brightest part through zone adjustment or diffusion shaping.

[0037] In one embodiment of the present invention, step S2 includes the following steps: S2-1. The first Zhang Fresnel lens butterfly pattern image By using the `flip` function in the OpenCV library of Python and setting the coordinate axis parameter to 1 to perform a horizontal flip, we obtain the flipped Fresnel lens butterfly pattern image. It is used to simulate the left-right placement deviation of lenses from different batches at the inspection station, retain the brightness information under ring light illumination, reduce model input redundancy, improve the model's adaptability to changes in placement posture, and avoid deviations caused by learning fine textures of butterfly defects in only one direction.

[0038] S2-2. The flipped Fresnel lens butterfly pattern image The center of the Fresnel lens is located using the HoughCircles method from the OpenCV library in Python. and effective imaging radius Using the getRectSubPix function in the OpenCV library of Python, the center point is... The image is cropped to obtain the Fresnel lens butterfly pattern. Focus on the core detection area of ​​the lens and remove invalid background interference such as edge clamps and stray reflections from the outer ring.

[0039] S2-3. The cropped Fresnel lens butterfly pattern image The GaussianBlur function from the OpenCV library in Python is used to generate a low-frequency illumination background image, resulting in a Fresnel lens butterfly pattern image. The Fresnel lens butterfly pattern image After performing illumination correction using the cv2.divide function from the OpenCV library in Python, grayscale normalization was performed to obtain the corrected Fresnel lens butterfly pattern image. This reduces the interference of brightness fluctuations in non-defect areas on subsequent feature extraction, providing a more stable periodic background input for the periodic gating fusion of ring patterns.

[0040] S2-4. The corrected Fresnel lens butterfly pattern image The contrast was adjusted using the convertScaleAbs function in the OpenCV library of Python to obtain the adjusted Fresnel lens butterfly pattern image. Setting alpha to 1.2 and beta to 20 enhances the local light and dark perturbation gradient caused by the butterfly pattern defect, while avoiding overall overexposure or saturation of the regular tooth pattern area, thus improving the grayscale separability between the defect and the background.

[0041] S2-5. Adjusted Fresnel lens butterfly pattern image The Fresnel lens butterfly pattern image was obtained by rotating the image within a range of ±3° using the warpAffine function in the OpenCV library of Python. This complements the horizontal flipping mechanism, covering common pose perturbation scenarios in real-world online detection and improving the model's robustness to changes in texture direction of butterfly-pattern defects.

[0042] S2-6. Rotate the Fresnel lens butterfly pattern image The image of the Fresnel lens butterfly pattern is obtained by converting it to grayscale using the cvtColor function in the OpenCV library of Python. .

[0043] S2-7. Process the Fresnel lens butterfly pattern image. The GaussianBlur function from the OpenCV library in Python is used to generate a low-frequency illumination background image, resulting in a Fresnel lens butterfly pattern image. This processing can suppress random noise and tiny reflective burrs, while preserving as much of the continuous fine texture disturbance contour caused by butterfly defects as possible, providing a cleaner input for the subsequent fine texture protection and enhancement module to determine local texture mutations and continuity.

[0044] S2-8. Fresnel lens butterfly pattern image The `createCLAHE` function from the OpenCV library in Python was used to enhance image contrast, resulting in a preprocessed collection of Fresnel lens butterfly patterns. This avoids over-enhancing of regular tooth-pattern areas, which can lead to detail saturation. At the same time, it highlights the local gray-scale gradient differences and fine texture abrupt changes caused by minor butterfly-pattern defects, solving the problem that butterfly-pattern defect signals are weak and easily submerged by strong periodic backgrounds.

[0045] In this embodiment, preferably, the GaussianBlur function in step S2-3 uses a 51×51 Gaussian kernel; the GaussianBlur function in step S2-7 uses a 3×3 Gaussian kernel with a standard deviation of 1.0; and the createCLAHE function in step S2-8 sets clipLimit to 1.81 and the grid size to 16×16.

[0046] In step S3, the preprocessed Fresnel lens butterfly pattern images are collected. The dataset is divided into training, testing, and validation sets in an 8:1:1 ratio.

[0047] In one embodiment of the present invention, step S6 includes the following steps: S6-1. The neck AIFI module of the improved RT-DETR model's Neck network consists of the input projection module of the RT-DETR model and the improved AIFI module. The improved AIFI module consists of the spatial dimension flattening layer of the AIFI module in the neck of the RT-DETR model, the 2D position encoding module in the neck of the RT-DETR model, the Transformer encoding layer, and the dimension reshaping layer of the RT-DETR model.

[0048] S6-2. Feature Map The input is fed into the input projection module, and the output is the feature map. .

[0049] S6-3. Feature Map The input is fed into the spatial dimension flattening layer of the improved AIFI module to obtain a two-dimensional matrix. , two-dimensional matrix The input is fed into the 2D position encoding module of the improved AIFI module, and the output is a feature map. .

[0050] S6-4. The improved AIFI module's Transformer encoding layer consists of the first pre-LayerNorm layer of the RT-DETR model's Transformer encoding layer, a multi-head attention module, a first residual connection layer, the second pre-LayerNorm layer of the RT-DETR model's Transformer encoding layer, a feedforward network module of the RT-DETR model's Transformer encoding layer, and a second residual connection layer, which integrates the feature map. The input is fed into the first pre-LayerNorm layer of the Transformer encoding layer, and the output is the feature map. The multi-head attention module consists of the QKV linear projection layer of the multi-head self-attention module in the Transformer encoding layer of the RT-DETR model, the multi-head splitting and scaling dot product calculation unit of the RT-DETR model, the anchor-linked attention module, the Softmax function, the attention weight and V-value weighted summation unit of the RT-DETR model, and the multi-head merging and output linear projection layer of the RT-DETR model, which integrates the feature maps. The input is fed into the QKV linear projection layer to obtain the query vector. Key vector Value vector , query vector Key vector Value vector The input is fed into the multi-head splitting and scaling dot product computation unit, and the output is the attention score matrix. .

[0051] S6-5. Anchor Point Linkage in Multi-Head Attention Module: The multi-head attention module consists of an anchor point extraction unit, an attention allocation unit, and a residual fusion unit. The anchor point extraction unit comprises a 1×1 convolutional layer, a batch normalization layer, a SiLU activation function, a texture feature extraction branch, and a filtering output layer, which integrates the feature maps... The input is sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the anchor point extraction unit, and the output is the feature map. , feature map The Sobel operator is used to process the feature map in the texture feature extraction branch of the anchor point extraction unit. Gradient calculations are performed in both the horizontal and vertical directions to obtain the feature map. The Middle Horizontal gradient of each pixel and vertical gradient Through formula The feature map is calculated. The Middle gradient magnitude of each pixel Through formula The feature map is calculated. The Middle gradient direction of each pixel If the gradient magnitude Greater than or equal to the threshold Then retain the first For each pixel, if the gradient magnitude Less than the threshold Then remove the first one. From each pixel, a binary mask image is obtained. Divide the gradient directions of all retained pixels into equal parts from 0° to 180°. By dividing the area into equal-width intervals, a direction histogram is obtained. The direction corresponding to the interval with the largest amplitude in the direction histogram is selected as the main direction of the butterfly pattern. Using connected component functions to analyze binary mask images Perform eight-neighbor connectivity detection to obtain The area value of the connected component, the th The area of ​​each connected region is Through formula Calculate the mean area of ​​the connected components , will the The area of ​​each connected region is and the average area of ​​connected regions The input is sent to the filtered output layer of the anchor point extraction unit. If the first... Area values ​​of connected components Greater than or equal to the threshold Then the connected component is taken as a candidate region for anchor point, if the first... Area values ​​of connected components Less than the threshold Then the connected component is removed, threshold The average area of ​​connected regions 1.3-1.7 times, through the formula Calculate the first The centroid x-coordinate of each anchor point candidate region Through formula Calculate the first The centroid ordinate of each anchor point candidate region ,in For the first The first anchor point candidate region The x-coordinate of each pixel For the first The first anchor point candidate region The ordinate of each pixel , For the first The number of pixels in the candidate region of the first anchor point, the number of pixels in the first anchor point candidate region, The centroid coordinates of the candidate anchor point regions are: The centroid coordinates of all anchor point candidate regions constitute the anchor point candidate set. .

[0052] S6-6. Generate a binary mask image. Blank images of the same size Blank image All pixels in the blank image are set to 0, and all anchor point candidate regions are filled into the blank image. In the corresponding region, the pixel values ​​of the filled pixels are modified to 255 to obtain the initial response map. The attention allocation unit of the anchor point linkage attention module consists of a linkage graph generation sublayer and an attention modulation sublayer. The linkage graph generation sublayer consists of a coordinate extraction layer, a direction vector encoding layer, a polar coordinate transformation layer, a dynamic gating fusion layer, and an associative attention fusion layer. The attention modulation sublayer consists of a Softmax function, a 1×1 convolutional layer, a SiLU activation function, and a dimension alignment and broadcast adaptation layer, which allocates the anchor point candidate set. Compared with the initial response diagram The coordinate extraction layer of the linkage graph generation sublayer, input to the attention allocation unit, is used in the initial response graph. Extract anchor point candidate set For all corresponding centroid coordinates, divide the x-coordinate of the centroid by . Divide the centroid ordinate by , Initial response diagram width, Initial response diagram The calculated centroid coordinates are mapped to the 0-1 interval, and all centroid coordinates mapped to the 0-1 interval constitute the set of anchor point positions. Set the anchor point locations The direction vector encoding layer, which is input into the linkage graph generation sublayer, calculates the difference in the x-coordinates of any two centroids. and the difference in the vertical axis Through formula The absolute orientation angles of the two centers of mass are calculated. To determine the absolute direction angle In the main direction of the butterfly pattern The absolute value of the difference is used as the direction deviation to construct the direction feature matrix. directional feature matrix The number of rows and columns are both related to the set of anchor points. With the same number of centroids, fill the directional deviation between any two centroids into the directional feature matrix. The corresponding position in the middle.

[0053] S6-7. Set the anchor point positions The input is fed into the polar coordinate transformation layer of the linkage graph generation sub-layer, using the formula... The set of anchor point locations was calculated. The Middle The radial distance from the centroid to the origin In the formula, The x-coordinate of the Fresnel lens center as calibrated for the production line. The ordinate of the Fresnel lens center as calibrated for the production line. Anchor point location set The Middle The x-coordinate of the centroid, Anchor point location set The Middle The ordinate of each centroid is obtained through the formula. The set of anchor point locations was calculated. The Middle The centroid and the first Concentricity of the centroid Construct a concentricity matrix Concentricity matrix The number of rows and columns are both related to the set of anchor points. If the number of centroids is the same, fill the concentricity of any two centroids into the concentricity matrix. The corresponding position in the middle.

[0054] S6-8. Directional Feature Matrix With concentricity matrix The directional feature matrix is ​​input into the dynamic gating fusion layer of the linkage graph generation sublayer. With concentricity matrix A concatenation operation is performed, where each element of the concatenated feature matrix is ​​sequentially input into a 1×1 convolutional layer and a sigmoid activation function to obtain the association weights of each element, thus constructing an oriented feature matrix. directional feature matrix The number of rows and columns are both related to the set of anchor points. If the number of centroids is the same, fill the correlation weights of any two centroids into the correlation gating matrix. The corresponding position in the matrix will be associated with the gating matrix. Compared with the initial response diagram The input is fed into the association attention fusion layer of the association graph generation sublayer, and the association gating matrix is ​​then applied. The dimensions were adjusted to match the initial response map. After the dimensions are the same, the associated gating matrix will be used. Compared with the initial response diagram Perform element-wise multiplication to obtain the initial anchor point linkage graph. Link the initial anchor point diagram The inputs are sequentially fed into the Softmax function of the attention modulation sublayer, the convolutional layer, and the SiLU activation function, and the output is the normalized enhanced weight map. Normalized weighted graph The input is dimensionally aligned to the attention modulation sublayer and its shape is adjusted to match the attention score matrix in the broadcast adaptation layer. Consistency is achieved, resulting in a weighted graph. , weighted graph With attention score matrix Element-wise multiplication yields the modulated attention score matrix. .

[0055] S6-9. The residual fusion unit of the anchor-linked attention module consists of upsampling / downsampling units, a convolutional layer with a kernel size of 1×1, a batchNormalization layer, and a SiLU activation function, which modulates the attention score matrix. The input to the upsampling / downsampling unit of the residual fusion unit is used to modulate the attention score matrix. Adjust to the initial response diagram Consistency is achieved, resulting in the attention score matrix. The attention score matrix The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the residual fusion unit, and the output is a matrix. The attention score matrix With matrix Element-wise summation yields the final attention score matrix. .

[0056] S6-10. The final attention score matrix The input is fed into the Softmax function of the multi-head attention module in the Transformer encoding layer of the improved AIFI module, and the output is the attention weight matrix. .

[0057] S6-11. Adjust the attention weight matrix The attention weights and V-value weighted summation unit of the RT-DETR model in the multi-head attention module of the Transformer encoding layer of the improved AIFI module are input to obtain the single-head attention output features. .

[0058] S6-12. Output features of single-head attention The input to the multi-head attention module of the Transformer encoding layer of the improved AIFI module is combined with the multi-head merging and output linear projection layer of the RT-DETR model to obtain the feature output. .

[0059] S6-13. Features The features are input into the first residual connection layer of the Transformer encoding layer of the improved AIFI module. With features Element-wise addition is performed to obtain the features after residual connection. .

[0060] S6-14. Features after residual concatenation The input is fed into the second pre-LayerNorm layer of the Transformer encoding layer of the RT-DETR model in the improved AIFI module's Transformer encoding layer, and the output is the normalized feature. .

[0061] S6-15. Normalized features The input is fed into the feedforward network module of the Transformer encoding layer of the RT-DETR model in the improved AIFI module, and the output is obtained as the feedforward network output feature. .

[0062] S6-16. Output features of the feedforward network The second residual connection layer of the Transformer encoding layer in the improved AIFI module will feed forward the network output features. Features after connection with residuals Element-wise addition is performed to obtain the features after residual connection. .

[0063] S6-17. Features after residual concatenation The input is fed into the dimension reshaping layer of the RT-DETR model of the improved AIFI module, and the output is the enhanced feature. .

[0064] In this embodiment, the threshold is preferred. The value is 0.4. The value is 16, the threshold. The average area of ​​connected regions 1.5 times.

[0065] In one embodiment of the present invention, step S7 includes the following steps: S7-1. The improved RT-DETR model's neck CCFM module consists of a fine texture protection enhancement module, a feature dimension adaptation module, a ring-pattern periodic gated fusion unit, and a RepBlock-based multi-scale fusion path for the RT-DETR model's CCFM module.

[0066] S7-2. The fine texture protection and enhancement module of the neck CCFM module consists of a Sobel convolutional layer, a convolutional layer with a kernel size of 1×1, and an SE module, which enhances the feature map. The input is fed into the Sobel convolutional layer of the fine texture preservation enhancement module, where the Sobel operator is used to process the feature map. Gradient calculations are performed in both the horizontal and vertical directions to obtain the feature map. The Middle Horizontal gradient of each pixel and vertical gradient Through formula The feature map is calculated. The Middle gradient magnitude of each pixel Construct a single-channel gradient magnitude map Single-channel gradient magnitude plot The number of rows and columns are both related to the feature map. With the same number of pixels, single-channel gradient magnitude map The Line number The value of the column is , In the formula, For feature map The maximum gradient magnitude of all pixels in the array. For feature map The minimum gradient magnitude of all pixels in the single-channel gradient magnitude map is obtained. The input is fed into the convolutional layer of the fine texture preservation and enhancement module, and the output is the gradient feature map. .

[0067] The S7-3. Fine Texture Preservation Enhancement Module's SE module consists of a global average pooling layer, a first fully connected layer, a ReLU activation function, a second fully connected layer, and a Sigmoid activation function, sequentially processing the gradient feature map. The input is given to the SE module, and the output is the weight coefficient. Weighting coefficients Adjust to gradient feature map After obtaining the same dimension, multiply it element-wise to obtain the weighted enhanced feature. The weighted enhanced features With feature map Element-wise addition is performed to obtain the feature map after fine texture protection enhancement. .

[0068] S7-4. Enhanced features The feature dimension adaptation module of the neck CCFM module is adjusted to match the feature map. With the same dimensions, feature maps are obtained. .

[0069] S7-5. The annular periodic gated fusion unit of the neck CCFM module consists of a grid sampling layer, a 3×3 convolutional layer, a batch normalization layer, a SiLU activation function, a first scale layer, a second scale layer, a sigmoid activation function, and a feature fusion layer, which integrates the feature maps. The feature map is input into the mesh sampling layer of the annular periodic gated fusion unit. The 1 pixel point generated The nth rotating sampling point, the th The pixel number The x-coordinate of each sampling point is , In the formula, For the first The x-coordinate of each pixel , No. The pixel number The ordinate of each sampling point is , , For the first The ordinate of the nth pixel, for the nth sampling points The feature vector is obtained by bilinear interpolation. For feature maps The first in For each pixel, the feature values ​​within its channel are extracted and concatenated in channel order to form a feature vector. Calculate the eigenvector With feature vectors The cosine similarity between them, taking the first... Pixel pairs The average cosine similarity of the nth rotated sampling point is used as the th The periodic intensity values ​​of each pixel, and the periodic intensity values ​​of all pixels according to the feature map. The coordinates in the graph are arranged in a one-to-one mapping to obtain a periodic intensity map. .

[0070] S7-6. Feature Map The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the ring-shaped periodic gated fusion unit, and the output is the global semantic feature map. The global semantic feature map Flattened to a length equal to one-dimensional array , Global semantic feature map The number of channels, Global semantic feature map of high, Global semantic feature map Width of the array Overall variance , total variance The input is fed into the first scale layer of the annular periodic gated fusion unit, and the learnable parameter β (set to 1.0) in the first scale layer is used to calculate the overall variance. The adaptive global threshold is obtained by adding the scaling factor β to the preset initial threshold T0 (preferably set to 0.5). Adaptive global threshold Adjust the spatial dimensions to match the periodic intensity diagram When the sizes are the same, the global threshold will be adaptive. With periodic intensity map By subtracting the difference, we obtain the result. , the results The input is fed into the second scale layer of the toroidal periodic gated fusion unit, and the result is evaluated using the learnable parameter α (set to 1.0) in the second scale layer. Scale by a factor of α to obtain the scaled result. , scale the result The input is fed into the sigmoid activation function of the ring-shaped periodic gated fusion unit, and the output is a pixel-level gated weight map. .

[0071] S7-7. Feature Map With feature map The feature map is input into the feature fusion layer of the annular periodic gated fusion unit. With feature map The addition operation yields the fused feature map. Pixel-level gated weight map The dimension is expanded to include the fused feature map. Consistent with each other, a pixel-level gated weight map is obtained. Pixel-level gated weight map With fusion feature map Perform element-wise multiplication to obtain the feature map. .

[0072] S7-8. Feature Map The input is fed into the RepBlock-based multi-scale fusion path of the CCFM module of the RT-DETR model, and the output is the fused feature map. .

[0073] In one embodiment of the invention, it further includes... The total loss was calculated. In the formula, For DIou's loss, For Focal loss, use the Adam optimizer to utilize the total loss. Train the improved RT-DETR model.

[0074] In this embodiment, preferably, when training the improved RT-DETR model, the number of iterations is set to 300, the initial learning rate is set to 1×10^(-4), the learning rate is reduced to half of its original value every 20 iterations, and the batch size is set to 16.

[0075] By deeply adapting a ring light source to an improved RT-DETR model, this invention achieves efficient and accurate detection of butterfly-shaped defects in Fresnel lenses. Compared to traditional white light sources paired with conventional RT-DETR models, this invention effectively addresses the industry pain point of confusion between concentric tooth patterns and butterfly-shaped features in Fresnel lenses by adding an anchor-linked attention module, a fine texture protection enhancement module, and a periodic gated fusion unit for ring patterns. Simultaneously, the collaborative parameter design and cross-scale feature fusion optimization of the Backbone, Encoder, and Neck networks enable the model to possess excellent recognition capabilities for butterfly-shaped defects of different sizes and shapes, adapting to the actual needs of diverse lens specifications and complex defect morphologies in industrial scenarios. Both detection speed and accuracy meet industrial online inspection standards (≥30 frames / second).

[0076] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting butterfly-shaped defects in Fresnel lenses based on a ring light source imaging scheme and an improved RT-DETR, characterized in that, include: S1. Obtain Zhang Fresnel lens image, to obtain the original Fresnel lens butterfly pattern image set. , , For the first Zhang Fresnel lens butterfly pattern image; S2. For the first Zhang Fresnel lens butterfly pattern image Preprocessing is performed to obtain the preprocessed Fresnel lens butterfly pattern image. All preprocessed Fresnel lens butterfly pattern images constitute a preprocessed Fresnel lens butterfly pattern image set. , ; S3. Assemble the preprocessed Fresnel lens butterfly pattern images. It is divided into training set, test set, and validation set; S4. Establish an improved RT-DETR model consisting of the Backbone network, Neck network, and RT-DETR detection head of the RT-DETR model. The Neck network consists of an improved AIFI module and a neck CCFM module. The improved AIFI module consists of the spatial dimension flattening layer of the AIFI module in the neck of the RT-DETR model, the 2D position encoding module in the neck of the RT-DETR model, the Transformer encoding layer, and the dimension reshaping layer of the RT-DETR model. S5. The training set of the first Zhang's processed Fresnel lens butterfly pattern image The input is fed into the backbone network of the improved RT-DETR model, and the output is a feature map that carries the information of fine texture and local light and dark perturbations related to the butterfly pattern defect. The overall structure of the carrying lens and the semantic information of the background. Feature map Feature maps that carry global semantic information related to the global ring pattern distribution and geometric contours of the lens. ; S6. Feature Map The input is fed into the neck AIFI module of the improved RT-DETR model's Neck network, and the output yields enhanced features. ; S7. Transfer the feature map Feature map Enhanced features The input is fed into the neck CCFM module of the improved RT-DETR model's Neck network, and the output is a fused feature map. ; S8. Merge feature maps The input is fed into the detection head of the improved RT-DETR model, and the output is a recognition image of the butterfly pattern defect detected in the Fresnel lens.

2. The Fresnel lens butterfly defect detection method based on ring light source imaging scheme and improved RT-DETR according to claim 1, characterized in that, Step S1 includes the following steps: S1-1. Selection A Fresnel lens with a butterfly-shaped defect; S1-2. Using an industrial camera positioned directly below the Fresnel lens and directly above it, each Fresnel lens with a butterfly-shaped defect is photographed to obtain... The image is formed by a Fresnel lens, wherein the axis of the annular light source is coaxial with the axis of the Fresnel lens, and the axis of the Fresnel lens is coaxial with the optical axis of the industrial camera lens.

3. The Fresnel lens butterfly defect detection method based on a ring light source imaging scheme and improved RT-DETR according to claim 1, characterized in that, Step S2 includes the following steps: S2-1. The first Zhang Fresnel lens butterfly pattern image By using the `flip` function in the OpenCV library of Python and setting the coordinate axis parameter to 1 to perform a horizontal flip, we obtain the flipped Fresnel lens butterfly pattern image. ; S2-2. The flipped Fresnel lens butterfly pattern image The center of the Fresnel lens is located using the HoughCircles method from the OpenCV library in Python. and effective imaging radius Using the getRectSubPix function in the OpenCV library of Python, the center point is... The image is cropped to obtain the Fresnel lens butterfly pattern. ; S2-3. The cropped Fresnel lens butterfly pattern image The GaussianBlur function from the OpenCV library in Python is used to generate a low-frequency illumination background image, resulting in a Fresnel lens butterfly pattern image. The Fresnel lens butterfly pattern image After performing illumination correction using the cv2.divide function from the OpenCV library in Python, grayscale normalization was performed to obtain the corrected Fresnel lens butterfly pattern image. ; S2-4. The corrected Fresnel lens butterfly pattern image The contrast was adjusted using the convertScaleAbs function in the OpenCV library of Python to obtain the adjusted Fresnel lens butterfly pattern image. ; S2-5. Adjusted Fresnel lens butterfly pattern image The Fresnel lens butterfly pattern image was obtained by rotating the image within a range of ±3° using the warpAffine function in the OpenCV library of Python. ; S2-6. Rotate the Fresnel lens butterfly pattern image The image of the Fresnel lens butterfly pattern is obtained by converting it to grayscale using the cvtColor function in the OpenCV library of Python. ; S2-7. Process the Fresnel lens butterfly pattern image. The GaussianBlur function from the OpenCV library in Python is used to generate a low-frequency illumination background image, resulting in a Fresnel lens butterfly pattern image. ; S2-8. Fresnel lens butterfly pattern image The `createCLAHE` function from the OpenCV library in Python was used to enhance image contrast, resulting in a preprocessed collection of Fresnel lens butterfly patterns. .

4. The Fresnel lens butterfly defect detection method based on ring light source imaging scheme and improved RT-DETR according to claim 3, characterized in that: In step S2-3, the GaussianBlur function uses a 51×51 Gaussian kernel convolution; in step S2-7, the GaussianBlur function uses a 3×3 Gaussian kernel convolution and the standard deviation is set to 1.0; in step S2-8, the createCLAHE function sets clipLimit to 1.81 and the grid size to 16×16.

5. The Fresnel lens butterfly defect detection method based on ring light source imaging scheme and improved RT-DETR according to claim 1, characterized in that: In step S3, the preprocessed Fresnel lens butterfly pattern images are collected. The dataset is divided into training, testing, and validation sets in a ratio of 8:1:

1.

6. The Fresnel lens butterfly defect detection method based on the ring light source imaging scheme and improved RT-DETR according to claim 1, characterized in that, Step S6 includes the following steps: S6-1. The neck AIFI module of the improved RT-DETR model's Neck network consists of the input projection module of the RT-DETR model and the improved AIFI module. The improved AIFI module consists of the spatial dimension flattening layer of the AIFI module in the neck of the RT-DETR model, the 2D position encoding module in the neck of the RT-DETR model, the Transformer encoding layer, and the dimension reshaping layer of the RT-DETR model. S6-2. Feature Map The input is fed into the input projection module, and the output is the feature map. ; S6-3. Feature Map The input is fed into the spatial dimension flattening layer of the improved AIFI module to obtain a two-dimensional matrix. , two-dimensional matrix The input is fed into the 2D position encoding module of the improved AIFI module, and the output is a feature map. ; S6-4. The improved AIFI module's Transformer encoding layer consists of the first pre-LayerNorm layer of the RT-DETR model's Transformer encoding layer, a multi-head attention module, a first residual connection layer, the second pre-LayerNorm layer of the RT-DETR model's Transformer encoding layer, a feedforward network module of the RT-DETR model's Transformer encoding layer, and a second residual connection layer, which integrates the feature map. The input is fed into the first pre-LayerNorm layer of the Transformer encoding layer, and the output is the feature map. The multi-head attention module consists of the QKV linear projection layer of the multi-head self-attention module in the Transformer encoding layer of the RT-DETR model, the multi-head splitting and scaling dot product calculation unit of the RT-DETR model, the anchor-linked attention module, the Softmax function, the attention weight and V-value weighted summation unit of the RT-DETR model, and the multi-head merging and output linear projection layer of the RT-DETR model, which integrates the feature maps. The input is fed into the QKV linear projection layer to obtain the query vector. Key vector Value vector , query vector Key vector Value vector The input is fed into the multi-head splitting and scaling dot product computation unit, and the output is the attention score matrix. ; S6-5. Anchor Point Linkage in Multi-Head Attention Module: The multi-head attention module consists of an anchor point extraction unit, an attention allocation unit, and a residual fusion unit. The anchor point extraction unit comprises a 1×1 convolutional layer, a batch normalization layer, a SiLU activation function, a texture feature extraction branch, and a filtering output layer, which integrates the feature maps... The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the anchor point extraction unit, and the output is the feature map. , feature map The Sobel operator is used to process the feature map in the texture feature extraction branch of the anchor point extraction unit. Gradient calculations are performed in both the horizontal and vertical directions to obtain the feature map. The Middle Horizontal gradient of each pixel and vertical gradient Through formula The feature map is calculated. The Middle gradient magnitude of each pixel Through formula The feature map is calculated. The Middle gradient direction of each pixel If the gradient magnitude Greater than or equal to the threshold Then retain the first For each pixel, if the gradient magnitude Less than the threshold Then remove the first one. From each pixel, a binary mask image is obtained. Divide the gradient directions of all retained pixels into equal parts from 0° to 180°. By dividing the area into equal-width intervals, a direction histogram is obtained. The direction corresponding to the interval with the largest amplitude in the direction histogram is selected as the main direction of the butterfly pattern. Using connected component functions to analyze binary mask images Perform eight-neighbor connectivity detection to obtain The area value of the connected component, the th The area of ​​each connected region is Through formula Calculate the mean area of ​​the connected components , will the The area of ​​each connected region is and the average area of ​​connected regions The input is sent to the filtered output layer of the anchor point extraction unit. If the first... Area values ​​of connected components Greater than or equal to the threshold Then the connected component is taken as a candidate region for anchor point, if the first... Area values ​​of connected components Less than the threshold Then the connected component is removed, threshold The average area of ​​connected regions 1.3-1.7 times, through the formula Calculate the first The centroid x-coordinate of each anchor point candidate region Through formula Calculate the first The centroid ordinate of each anchor point candidate region ,in For the first The first anchor point candidate region The x-coordinate of each pixel For the first The first anchor point candidate region The ordinate of each pixel , For the first The number of pixels in the candidate region of the first anchor point, the number of pixels in the first anchor point candidate region, The centroid coordinates of the candidate anchor point regions are: The centroid coordinates of all anchor point candidate regions constitute the anchor point candidate set. ; S6-6. Generate a binary mask image. Blank images of the same size Blank image All pixels in the blank image are set to 0, and all anchor point candidate regions are filled into the blank image. In the corresponding region, the pixel values ​​of the filled pixels are modified to 255 to obtain the initial response map. The attention allocation unit of the anchor point linkage attention module consists of a linkage graph generation sublayer and an attention modulation sublayer. The linkage graph generation sublayer consists of a coordinate extraction layer, a direction vector encoding layer, a polar coordinate transformation layer, a dynamic gating fusion layer, and an associative attention fusion layer. The attention modulation sublayer consists of a Softmax function, a 1×1 convolutional layer, a SiLU activation function, and a dimension alignment and broadcast adaptation layer, which allocates the anchor point candidate set. Compared with the initial response diagram The coordinate extraction layer of the linkage graph generation sublayer, input to the attention allocation unit, is used in the initial response graph. Extract anchor point candidate set For all corresponding centroid coordinates, divide the x-coordinate of the centroid by . Divide the centroid ordinate by , Initial response diagram width, Initial response diagram The calculated centroid coordinates are mapped to the 0-1 interval, and all centroid coordinates mapped to the 0-1 interval constitute the set of anchor point positions. Set the anchor point locations The direction vector encoding layer, which is input into the linkage graph generation sublayer, calculates the difference in the x-coordinates of any two centroids. and the difference in the vertical axis Through formula The absolute orientation angles of the two centers of mass are calculated. To determine the absolute direction angle In the main direction of the butterfly pattern The absolute value of the difference is used as the direction deviation to construct the direction feature matrix. directional feature matrix The number of rows and columns are both related to the set of anchor points. With the same number of centroids, fill the directional deviation between any two centroids into the directional feature matrix. The corresponding position in the middle; S6-7. Set the anchor point positions The input is fed into the polar coordinate transformation layer of the linkage graph generation sub-layer, using the formula... The set of anchor point locations was calculated. The Middle The radial distance from the centroid to the origin In the formula, The x-coordinate of the Fresnel lens center as calibrated for the production line. The ordinate of the Fresnel lens center as calibrated for the production line. Anchor point location set The Middle The x-coordinate of the centroid, Anchor point location set The Middle The ordinate of each centroid is obtained through the formula. The set of anchor point locations was calculated. The Middle The centroid and the first Concentricity of the centroid Construct a concentricity matrix Concentricity matrix The number of rows and columns are both related to the set of anchor points. If the number of centroids is the same, fill the concentricity of any two centroids into the concentricity matrix. The corresponding position in the middle; S6-8. Directional Feature Matrix With concentricity matrix The directional feature matrix is ​​input into the dynamic gating fusion layer of the linkage graph generation sublayer. With concentricity matrix A concatenation operation is performed, where each element of the concatenated feature matrix is ​​sequentially input into a 1×1 convolutional layer and a sigmoid activation function to obtain the association weights of each element, thus constructing an oriented feature matrix. directional feature matrix The number of rows and columns are both related to the set of anchor points. If the number of centroids is the same, fill the correlation weights of any two centroids into the correlation gating matrix. The corresponding position in the matrix will be associated with the gating matrix. Compared with the initial response diagram The input is fed into the association attention fusion layer of the association graph generation sublayer, and the association gating matrix is ​​then applied. The dimensions were adjusted to match the initial response map. After the dimensions are the same, the associated gating matrix will be used. Compared with the initial response diagram Perform element-wise multiplication to obtain the initial anchor point linkage graph. Link the initial anchor point diagram The inputs are sequentially fed into the Softmax function of the attention modulation sublayer, the convolutional layer, and the SiLU activation function, and the output is the normalized enhanced weight map. Normalized weighted graph The input is dimensionally aligned to the attention modulation sublayer and its shape is adjusted to match the attention score matrix in the broadcast adaptation layer. Consistency is achieved, resulting in a weighted graph. , weighted graph With attention score matrix Element-wise multiplication yields the modulated attention score matrix. ; S6-9. The residual fusion unit of the anchor-linked attention module consists of an upsampling or downsampling unit, a convolutional layer with a kernel size of 1×1, a BatchNormalization layer, and a SiLU activation function, which modulates the attention score matrix. The modulated attention score matrix is ​​input into the upsampling or downsampling unit of the residual fusion unit. Adjust to the initial response diagram Consistency is achieved, resulting in the attention score matrix. The attention score matrix The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the residual fusion unit, and the output is a matrix. The attention score matrix With matrix Element-wise summation yields the final attention score matrix. ; S6-10. The final attention score matrix The input is fed into the Softmax function of the multi-head attention module in the Transformer encoding layer of the improved AIFI module, and the output is the attention weight matrix. ; S6-11. Adjust the attention weight matrix The attention weights and V-value weighted summation unit of the RT-DETR model in the multi-head attention module of the Transformer encoding layer of the improved AIFI module are input to obtain the single-head attention output features. ; S6-12. Output features of single-head attention The input to the multi-head attention module of the Transformer encoding layer of the improved AIFI module is combined with the multi-head merging and output linear projection layer of the RT-DETR model to obtain the feature output. ; S6-13. Features The features are input into the first residual connection layer of the Transformer encoding layer of the improved AIFI module. With features Element-wise addition is performed to obtain the features after residual connection. ; S6-14. Features after residual concatenation The input is fed into the second pre-LayerNorm layer of the Transformer encoding layer of the RT-DETR model in the improved AIFI module's Transformer encoding layer, and the output is the normalized feature. ; S6-15. Normalized features The input is fed into the feedforward network module of the Transformer encoding layer of the RT-DETR model in the improved AIFI module, and the output is the feedforward network output feature. ; S6-16. Output features of the feedforward network The second residual connection layer of the Transformer encoding layer in the improved AIFI module will feed forward the network output features. Features after connection with residuals Element-wise addition is performed to obtain the features after residual connection. ; S6-17. Features after residual concatenation The input is fed into the dimension reshaping layer of the RT-DETR model of the improved AIFI module, and the output is the enhanced feature. .

7. The Fresnel lens butterfly defect detection method based on ring light source imaging scheme and improved RT-DETR according to claim 6, characterized in that: threshold The value is 0.

4. The value is 16, the threshold. The average area of ​​connected regions 1.5 times.

8. The Fresnel lens butterfly defect detection method based on the ring light source imaging scheme and improved RT-DETR according to claim 6, characterized in that, Step S7 includes the following steps: S7-1. The improved RT-DETR model's Neck CCFM module consists of a fine texture protection enhancement module, a feature dimension adaptation module, a ring-pattern periodic gated fusion unit, and a RepBlock-based multi-scale fusion path of the RT-DETR model's CCFM module. S7-2. The fine texture protection and enhancement module of the neck CCFM module consists of a Sobel convolutional layer, a convolutional layer with a kernel size of 1×1, and an SE module, which enhances the feature map. The input is fed into the Sobel convolutional layer of the fine texture preservation enhancement module, where the Sobel operator is used to process the feature map. Gradient calculations are performed in both the horizontal and vertical directions to obtain the feature map. The Middle Horizontal gradient of each pixel and vertical gradient Through formula The feature map is calculated. The Middle gradient magnitude of each pixel Construct a single-channel gradient magnitude map Single-channel gradient magnitude plot The number of rows and columns are both related to the feature map. With the same number of pixels, single-channel gradient magnitude map The Line number The value of the column is , In the formula, For feature map The maximum gradient magnitude of all pixels in the array. For feature map The minimum gradient magnitude of all pixels in the single-channel gradient magnitude map is obtained. The input is fed into the convolutional layer of the fine texture preservation and enhancement module, and the output is the gradient feature map. ; The S7-3. Fine Texture Preservation Enhancement Module's SE module consists of a global average pooling layer, a first fully connected layer, a ReLU activation function, a second fully connected layer, and a Sigmoid activation function, which sequentially converts the gradient feature map... The input is given to the SE module, and the output is the weight coefficient. Weighting coefficients Adjust to gradient feature map After obtaining the same dimension, multiply it element-wise to obtain the weighted enhanced feature. The weighted enhanced features With feature map Element-wise addition is performed to obtain the feature map after fine texture protection enhancement. ; S7-4. Enhanced Features The feature dimension adaptation module of the neck CCFM module is adjusted to match the feature map. With the same dimensions, feature maps are obtained. ; S7-5. The annular periodic gated fusion unit of the neck CCFM module consists of a grid sampling layer, a 3×3 convolutional layer, a batch normalization layer, a SiLU activation function, a first scale layer, a second scale layer, a sigmoid activation function, and a feature fusion layer, which integrates the feature maps. The feature map is input into the mesh sampling layer of the annular periodic gated fusion unit. The 1 pixel point generated The nth rotating sampling point, the th The pixel number The x-coordinate of each sampling point is , In the formula, For the first The x-coordinate of each pixel , No. The pixel number The ordinate of each sampling point is , , For the first The ordinate of the nth pixel, for the nth sampling points The feature vector is obtained by bilinear interpolation. For feature maps The first in For each pixel, the feature values ​​within its channel are extracted and concatenated in channel order to form a feature vector. Calculate the eigenvector With feature vectors The cosine similarity between them, taking the first... Pixel pairs The average cosine similarity of the nth rotated sampling point is used as the th The periodic intensity values ​​of each pixel, and the periodic intensity values ​​of all pixels according to the feature map. The coordinates in the graph are arranged in a one-to-one mapping to obtain a periodic intensity map. ; S7-6. Feature Map The inputs are sequentially fed into the convolutional layer, batchNormalization layer, and SiLU activation function of the ring-shaped periodic gated fusion unit, and the output is the global semantic feature map. The global semantic feature map Flattened to a length equal to one-dimensional array , Global semantic feature map The number of channels, Global semantic feature map of high, Global semantic feature map The width of the array is calculated. Overall variance , total variance The input is fed into the first scale layer of the annular periodic gated fusion unit, and the output is an adaptive global threshold. Adaptive global threshold Adjust the spatial dimensions to match the periodic intensity diagram When the sizes are the same, the global threshold will be adaptive. With periodic intensity map By subtracting the difference, we obtain the result. , the results The input is fed into the second scale layer of the annular periodic gated fusion unit, and the output is the scaled result. , scale the result The input is fed into the sigmoid activation function of the ring-shaped periodic gated fusion unit, and the output is a pixel-level gated weight map. ; S7-7. Feature Map With feature map The feature map is input into the feature fusion layer of the annular periodic gated fusion unit. With feature map The addition operation yields the fused feature map. Pixel-level gated weight map The dimension is expanded to include the fused feature map. Consistent with each other, a pixel-level gated weight map is obtained. Pixel-level gated weight map With fusion feature map Perform element-wise multiplication to obtain the feature map. ; S7-8. Feature Map The input is fed into the RepBlock-based multi-scale fusion path of the CCFM module of the RT-DETR model, and the output is the fused feature map. .

9. The Fresnel lens butterfly defect detection method based on ring light source imaging scheme and improved RT-DETR according to claim 1, characterized in that: It also includes through The total loss was calculated. In the formula, For DIou's loss, For Focal loss, use the Adam optimizer to utilize the total loss. Train the improved RT-DETR model.

10. The Fresnel lens butterfly defect detection method based on the ring light source imaging scheme and improved RT-DETR according to claim 9, characterized in that: When training the improved RT-DETR model, the number of iterations was set to 300, the initial learning rate was set to 1×10^(-4), the learning rate was reduced to half of its original value every 20 iterations, and the batch size was set to 16.

Images