Ultrahigh-definition film defect detection method based on improved YOLOv8

Abstract

The invention provides an ultra-high-definition film defect detection method based on improved YOLOv8, and relates to the technical field of detection, and the method comprises the steps: obtaining an ultra-high-definition image of a to-be-detected film, adaptively adjusting the partitioning size and the overlapping rate based on a defect size distribution statistical result, and decomposing the ultra-high-definition image into a plurality of sub-image blocks with spatial redundancy; performing feature extraction on each sub-image block by using a feature extraction network; according to the boundary feature vectors of the adjacent sub-image blocks in the overlapping area, weighted interpolation is executed based on the spatial gradient and the semantic similarity, and feature maps of all the sub-image blocks subjected to feature compensation are generated; inputting the feature map into a defect detection network, wherein the network comprises an anchor frame configuration module designed for an extreme length-width ratio target, a shallow feature enhancement path and a cross-scale feature compensation fusion module; and determining the position and category of the defect in the ultra-high-definition image. The method can effectively improve the accuracy and efficiency of ultra-high-definition film defect detection, and is particularly suitable for detecting extreme length-width ratio and small-size defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of detection, and in particular to a super-high-definition thin film defect detection method based on improved YOLOv8. BACKGROUND

[0002] With the continuous progress of industrial manufacturing technology, thin film materials have been widely used in display panels, optical devices, packaging materials and other fields. The quality of thin film products directly affects the performance and reliability of the final product, so thin film defect detection has become a key link in the production process. Traditional manual visual inspection methods have low efficiency, strong subjectivity and are prone to fatigue, which cannot meet the high speed and high precision requirements of modern industrial production. With the development of computer vision and deep learning technology, automatic defect detection methods based on image processing have gradually become a research hotspot.

[0003] In recent years, target detection algorithms based on deep learning have made significant progress in defect detection. YOLO series algorithms are widely used in industrial defect detection scenarios due to their real-time performance and high detection accuracy. YOLOv8, as the latest version, has made great improvements in network structure and detection performance. However, in the actual application of super-high-definition thin film defect detection, existing technologies still have many limitations.

[0004] Super-high-definition images usually have a resolution of tens of millions or even hundreds of millions of pixels. Direct input into the detection network will cause memory overflow and insufficient computing resources. Existing methods usually use image blocking strategies to cut large images into small blocks for detection. However, this simple blocking method is prone to defect omission problems at block boundaries, especially when the defect target is located exactly at the blocking boundary, which is easily cut off, resulting in incomplete features and affecting detection accuracy. In addition, fixed blocking size and overlap rate are difficult to adapt to the detection needs of defects of different sizes. For small size defects, the blocking may be too large to lose detailed information, and for large size defects, the blocking may be too small to obtain complete target features. SUMMARY

[0005] The embodiment of the present application provides a super-high-definition thin film defect detection method based on improved YOLOv8, which can solve the problems in the prior art.

[0006] In a first aspect, the embodiment of the present application provides a super-high-definition thin film defect detection method based on improved YOLOv8, comprising: Obtaining a super-high-definition image of a thin film to be detected, adaptively adjusting the blocking size and overlap rate based on the defect size distribution statistical result, and decomposing the super-high-definition image into a plurality of sub-image blocks with spatial redundancy; Using a feature extraction network to extract features of each sub-image block to obtain a feature map corresponding to each sub-image block. For the boundary feature vectors of adjacent sub-image blocks in the overlapping area, weighted interpolation is performed based on spatial gradient and semantic similarity to generate feature-compensated feature maps of each sub-image block; The feature-compensated feature maps of each sub-image block are input into a defect detection network, which includes an anchor box configuration module designed for extreme aspect ratio targets, a shallow feature enhancement path, and a cross-scale feature compensation fusion module. The cross-scale feature compensation fusion module generates balanced semantic features by averaging the spatial sizes of different levels of features. The shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. Based on the output results of the defect detection network, the positions and categories of defects in the ultra-high definition image are determined.

[0007] Based on the defect size distribution statistical results, the block size and overlap rate are adaptively adjusted, and the ultra-high definition image is decomposed into multiple sub-image blocks with spatial redundancy, including: Statistical results of the average size and spatial distribution of defects in the historical or current batch are obtained. The block size is determined according to the defect size distribution statistical results. When the defect density exceeds the preset density threshold, the block size is reduced to retain the local context. When the average length of defects exceeds the preset length threshold, the block size is increased to reduce the number of blocks. The overlap rate is determined according to the average length of defects. The overlap rate increases with the increase of the average length of defects to ensure that the cross-border defects are fully covered in adjacent sub-image blocks. Based on the block size and overlap rate, the ultra-high definition image is blocked to generate multiple sub-image blocks with spatial redundancy.

[0008] For the boundary feature vectors of adjacent sub-image blocks in the overlapping area, weighted interpolation is performed based on spatial gradient and semantic similarity to generate feature-compensated feature maps of each sub-image block, including: The boundary feature vectors of adjacent sub-image blocks in the overlapping area are extracted. The spatial gradient value of the boundary feature vector is calculated, and the semantic similarity between the boundary feature vectors of adjacent sub-image blocks is calculated. The weighting coefficient is determined according to the spatial gradient value and semantic similarity. When the semantic similarity is higher than the preset similarity threshold, the weighting coefficient is increased. When the semantic similarity is lower than the preset similarity threshold, the weighting coefficient is reduced. Based on the weighting coefficient, the boundary feature vectors of adjacent sub-image blocks are weighted and interpolated to generate a continuous feature representation after healing. The continuous feature representation after healing is replaced by the original boundary feature vector to obtain the feature-compensated feature maps of each sub-image block.

[0009] The cross-scale feature compensation fusion module generates balanced semantic features by taking the average of different hierarchical features after spatial size unification, comprising: obtaining a plurality of feature maps of different hierarchical levels from the backbone network; performing down-sampling or up-sampling operations on the plurality of feature maps of different hierarchical levels respectively, so that the spatial sizes of the hierarchical feature maps are unified to a target size; performing element-wise averaging operation on the spatially uniform hierarchical feature maps at the corresponding spatial positions to generate balanced semantic features; The balanced semantic features are output as the output of the cross-scale feature compensation fusion module and passed to the subsequent hierarchical level of the defect detection network.

[0010] The shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer, and establishes a jump connection between the shallow features of the backbone network and the small target detection layer, so that the shallow features bypass the down-sampling operation of the intermediate level and are directly input to the small target detection layer.

[0011] The method further comprises: performing post-processing on the detection results, including dynamically adjusting the non-maximum suppression threshold according to the target size, and splicing judgment on the cross-block defects based on the edge continuity score, and merging adjacent detection boxes into a single defect when the edge continuity score exceeds a preset continuity threshold.

[0012] In a second aspect of the embodiments of the present application, an ultra-high-definition thin film defect detection system based on improved YOLOv8 is provided, comprising: A first unit is configured to obtain an ultra-high-definition image of a thin film to be detected, and based on a statistical result of defect size distribution, adaptively adjust the block size and overlap rate, and decompose the ultra-high-definition image into a plurality of sub-image blocks with spatial redundancy. A second unit is configured to extract features of each sub-image block using a feature extraction network to obtain a feature map corresponding to each sub-image block. A third unit is configured to perform weighted interpolation based on spatial gradient and semantic similarity on the boundary feature vectors of adjacent sub-image blocks in the overlapping area to generate feature-compensated sub-image block feature maps. A fourth unit is configured to input the feature-compensated sub-image block feature maps to a defect detection network, wherein the defect detection network comprises an anchor box configuration module designed for extreme aspect ratio targets, a shallow feature enhancement path, and a cross-scale feature compensation fusion module. A fifth unit is configured to determine the location and category of the defect in the ultra-high definition image based on the output result of the defect detection network.

[0013] A third aspect of the embodiments of the present application, An electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0014] A fourth aspect of the embodiments of the present application, A computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0015] The beneficial effects of the present application are as follows: The present application effectively solves the problems of large computational resource consumption and easy loss of small size defect features when directly processing ultra-high definition images by adaptively adjusting the block size and overlap rate based on the statistical results of defect size distribution, and by decomposing the ultra-high definition image into multiple sub-image blocks with spatial redundancy. At the same time, the adaptive block strategy can flexibly adjust the processing parameters according to the actual defect features, improving the adaptability and robustness of the detection method to different thin film defect scenarios.

[0016] The present application generates a feature map that has been compensated for features based on the spatial gradient and semantic similarity of the boundary feature vectors of adjacent sub-image blocks in the overlapping area, effectively eliminating the boundary effect and feature discontinuity caused by image block processing, avoiding missed detection and repeated detection of defects in the overlapping area, and significantly improving the detection accuracy and positioning precision of cross-block defects, ensuring the integrity and consistency of the detection results. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of the improved YOLOv8-based ultra-high definition thin film defect detection method of the embodiments of the present application. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0019] The technical solutions of the present application will be described in detail below with specific examples. The following specific examples can be combined with each other, and some examples may not be described in detail for the same or similar concepts or processes.

[0020] Figure 1 The flowchart of the method for detecting defects in ultra-high-definition thin films based on improved YOLOv8 is shown in FIG. 1. Figure 1 As shown in FIG. 1, the method comprises: Obtaining an ultra-high-definition image of a thin film to be detected, adaptively adjusting the block size and overlap rate based on the defect size distribution statistical result, and decomposing the ultra-high-definition image into a plurality of sub-image blocks with spatial redundancy. Using a feature extraction network to extract features from each sub-image block to obtain a corresponding feature map for each sub-image block. For the boundary feature vectors of adjacent sub-image blocks in the overlapping area, weighted interpolation is performed based on spatial gradient and semantic similarity to generate each sub-image block feature map after feature compensation. Inputting each sub-image block feature map after feature compensation into a defect detection network, the defect detection network comprising an anchor box configuration module designed for extreme aspect ratio targets, a shallow feature enhancement path, and a cross-scale feature compensation fusion module, wherein the cross-scale feature compensation fusion module generates balanced semantic features by averaging the spatial sizes of different levels of features, and the shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. Based on the output result of the defect detection network, the position and category of the defect in the ultra-high-definition image are determined.

[0021] In an optional implementation, based on the defect size distribution statistical result, the block size and overlap rate are adaptively adjusted, and the ultra-high-definition image is decomposed into a plurality of sub-image blocks with spatial redundancy, comprising: Statistically obtaining the average size and spatial distribution of defects in the historical or current batch to obtain the defect size distribution statistical result; Determining the block size according to the defect size distribution statistical result, reducing the block size to retain local context when the defect density exceeds the preset density threshold, and increasing the block size to reduce the number of blocks when the average length of defects exceeds the preset length threshold; Determining the overlap rate according to the average length of defects, the overlap rate increasing with the increase of the average length of defects to ensure that the cross-border defects are sufficiently covered in adjacent sub-image blocks; Based on the block size and overlap rate, the ultra-high-definition image is blocked to generate a plurality of sub-image blocks with spatial redundancy.

[0022] The implementation process of adaptive image segmentation processing includes statistical analysis of historical data and current batch data. Inspection records from the past thirty batches are read from the storage module; each record contains the pixel coordinates, length, width, and spatial location information of the defects within the image. For the current ultra-high-definition image to be processed, the identified defect information is also extracted. The statistical processor calculates the average length of all defects by summing the length values ​​of all defects and dividing by the total number of defects.

[0023] Defect density is measured by the number of defects per unit area. The average area occupied by each defect is obtained by dividing the entire image area by the total number of defects. When the number of defects within a grid region exceeds twice the overall average, that region is marked as a high-density region. A density threshold of 0.5 defects per square millimeter is set. When the detected actual density is 0.8 defects per square millimeter, it is considered to exceed the preset density threshold. In this case, the block size needs to be reduced to handle this high-density situation.

[0024] The block size is determined based on a comprehensive assessment of multiple factors. A preset baseline block size of 2000 x 2000 pixels is used, suitable for scenarios with medium defect density and length. When the detected defect density exceeds a threshold, the block size is adjusted using a density adjustment coefficient. This coefficient is calculated by multiplying the reciprocal of the actual density value by the threshold density value, and then multiplying by 0.8. This strategy of reducing the block size preserves more local contextual information, enabling the detection algorithm to more accurately identify densely distributed small defects.

[0025] To address situations where the average defect length exceeds a preset threshold, a strategy of increasing the block size is adopted. The length threshold is set to 800 pixels. When the statistically obtained average defect length reaches 1500 pixels, it is determined to exceed the threshold. In this case, the block size is increased using a length adjustment factor. The length adjustment factor is calculated by dividing the actual average length by the threshold length and then multiplying by 1.2. Increasing the block size allows the complete long defect to be accommodated within a single sub-image block, avoiding the segmentation of long defects into too many sub-blocks, thereby reducing the total number of blocks and improving processing efficiency.

[0026] When both high-density and long defects exist simultaneously, the two adjustment coefficients are weighted and fused. The density adjustment coefficient is weighted at 0.6, and the length adjustment coefficient is weighted at 0.4. This weighting mechanism ensures that a balance point is found in scenarios with complex defect distributions.

[0027] The overlap rate is directly related to the average defect length. The overlap rate is defined as the ratio of the width of the overlapping area between adjacent sub-image blocks to the block size. A baseline overlap rate of 15% is set, suitable for defects with an average length of 400 pixels. As the average defect length increases, the overlap rate needs to be increased accordingly to ensure that cross-boundary defects are adequately covered in adjacent sub-blocks.

[0028] The image segmentation module divides the ultra-high-definition image into segments based on the determined segment size and overlap ratio. The segmentation process starts from the top left corner of the image and slides sequentially to the right until it covers the entire image width, then moves downwards following the same overlap rule. The final generated sub-image blocks consist of 6 horizontal and 5 vertical blocks, for a total of 30 sub-image blocks. Each sub-image block, in addition to containing image data, also records its absolute coordinate position in the original image, providing spatial reference information for subsequent result fusion.

[0029] In one optional implementation, for the boundary feature vectors of adjacent sub-image patches in the overlapping region, weighted interpolation is performed based on spatial gradient and semantic similarity to generate feature maps of each sub-image patch after feature compensation, including: Extract the boundary feature vectors of adjacent sub-image patches at corresponding positions in the overlapping region; Calculate the spatial gradient value of the boundary feature vector, and calculate the semantic similarity between the boundary feature vectors of adjacent sub-image blocks; The weighting coefficient is determined based on the spatial gradient value and semantic similarity. When the semantic similarity is higher than the preset similarity threshold, the weighting coefficient is increased, and when the semantic similarity is lower than the preset similarity threshold, the weighting coefficient is decreased. Based on the weighting coefficients, the boundary feature vectors of adjacent sub-image blocks are weighted and interpolated to generate a continuous feature representation after bridging. The original boundary feature vector is replaced by the merged continuous feature representation to obtain feature maps of each sub-image block after feature compensation.

[0030] In image segmentation, after the input image is divided into multiple sub-image blocks with overlapping regions, the boundary feature vectors of these sub-image blocks in the overlapping regions need to be processed to eliminate stitching artifacts and generate smooth, continuous feature representations. Assume the input image size is 1024×1024 pixels, divided into four 512×512 pixel sub-image blocks, with a 64-pixel overlap between adjacent sub-image blocks. Each sub-image block is processed by a feature extraction network to generate a feature map with a spatial size of 64×64 and 256 channels.

[0031] For two adjacent sub-image blocks, denoted as sub-image block A and sub-image block B, boundary feature vectors are extracted within their overlapping region. Specifically, for sub-image block A, the boundary features closest to the overlapping region in its feature map are extracted. This boundary corresponds to the rightmost 8 columns of feature vectors in the feature map, forming a boundary feature tensor of size 64×8×256. For sub-image block B, the boundary features closest to the other side of the overlapping region in its feature map are extracted. This boundary corresponds to the leftmost 8 columns of feature vectors in the feature map, also forming a boundary feature tensor of size 64×8×256. Within the overlapping region, the feature vectors at corresponding positions should describe the same image content; however, due to incomplete contextual information caused by block processing, these feature vectors may differ.

[0032] The spatial gradient value of the boundary feature vector is calculated to measure the drastic change of the feature in spatial dimensions. For a feature vector at a certain position on the boundary of sub-image patch A, its spatial gradient value is obtained by calculating the difference between the feature vector and its adjacent feature vectors above and below it. Specifically, for the feature vector at position index i, the Euclidean distance between it and the feature vectors at positions i-1 and i-1 is calculated, and the two distance values ​​are added together and divided by 2 to obtain the spatial gradient value at that position. For example, the feature vector at position index 30 is a 256-dimensional vector. The Euclidean distances between it and the feature vectors at positions 29 and 31 are calculated respectively, yielding distance values ​​of 12.3 and 14.7. Therefore, the spatial gradient value at that position is 13.5. The same operation is performed on the feature vectors at all 64 positions on the boundary, resulting in 64 spatial gradient values.

[0033] The semantic similarity between feature vectors at the boundaries of adjacent sub-image blocks is calculated to measure the consistency of the content described by feature vectors at corresponding positions. For two feature vectors at corresponding positions within an overlapping region, one from the boundary of sub-image block A and the other from the boundary of sub-image block B, the semantic similarity value is obtained by calculating the cosine similarity of these two 256-dimensional feature vectors. Specifically, the two feature vectors are normalized to a magnitude of 1, and then the dot product of the two normalized vectors is calculated; the dot product value is the semantic similarity. For example, if the dot product of two feature vectors at a certain corresponding position, after normalization, is element-wise multiplied and summed, resulting in a value of 0.87, then the semantic similarity at that position is 0.87. The same operation is performed on all 64×8 pairs of feature vectors at corresponding positions within the overlapping region, resulting in 64×8 semantic similarity values.

[0034] The weighting coefficient is determined based on the spatial gradient value and semantic similarity. This weighting coefficient controls the mixing ratio of the two boundary feature vectors during interpolation. A preset similarity threshold of 0.75 is set. When the semantic similarity at a certain position is higher than this threshold, it indicates that the two boundary feature vectors describe highly consistent content, and the weighting coefficient should be increased to enhance the fusion degree. When the semantic similarity is lower than this threshold, it indicates that the two boundary feature vectors have significant differences, and the weighting coefficient should be decreased to retain their respective feature information. The calculation of the weighting coefficient comprehensively considers semantic similarity and spatial gradient value. For a position with a semantic similarity of 0.87, which is higher than the threshold of 0.75, the difference is 0.12. This difference is multiplied by a magnification factor of 2 to obtain 0.24. The spatial gradient value is also considered. If the spatial gradient value at this position is 13.5, it is normalized to the interval between 0 and 1. Assuming that the maximum value of the spatial gradient value on the boundary is 25, the normalized gradient value is 0.54. The initial value of the weighting coefficient is set to 0.5. Then, the semantic similarity contribution value of 0.24 is added, and the gradient contribution value of 0.27 is subtracted. The gradient contribution value is obtained by multiplying the normalized gradient value of 0.54 by the adjustment factor of 0.5. Finally, the weighting coefficient at this position is 0.47.

[0035] Based on the calculated weighting coefficients, weighted interpolation is performed on the boundary feature vectors of adjacent sub-image blocks to generate a bridged continuous feature representation. For a corresponding position within the overlapping region, let feature vector A be the feature vector of the boundary of sub-image block A and feature vector B be the feature vector of the boundary of sub-image block B. The weighting coefficient at this position is 0.47. The bridged feature vector is obtained by multiplying feature vector A by the weighting coefficient 0.47, multiplying feature vector B by 1 and subtracting the weighting coefficient, i.e., 0.53, and then adding the two weighted feature vectors element by element. For example, if the first element of feature vector A is 3.2 and the first element of feature vector B is 2.8, then the first element of the bridged feature vector is 3.2 multiplied by 0.47 plus 2.8 multiplied by 0.53, resulting in 2.988. The same operation is performed on all 256 elements of the feature vectors to obtain the complete bridged feature vector. Weighted interpolation is then performed on all 64×8 corresponding positions within the overlapping region to generate a complete bridged continuous feature representation, which also has a size of 64×8×256.

[0036] The generated, merged, continuous feature representation replaces the original boundary feature vectors, updating the feature maps of the sub-image blocks. For sub-image block A, the rightmost 8 columns of feature vectors in its feature map are replaced with the merged, continuous feature representation, while the updated feature map retains its original size of 64×64×256. For sub-image block B, the leftmost 8 columns of feature vectors in its feature map are similarly replaced with the merged, continuous feature representation. Through this replacement operation, the feature representations of adjacent sub-image blocks in the overlapping area become consistent, eliminating the feature discontinuity caused by block processing. The same boundary feature merging process is performed on all adjacent sub-image block pairs, ultimately yielding feature maps of each sub-image block after feature compensation. These feature maps can achieve a smooth transition when subsequently stitched into a complete image feature map, avoiding obvious stitching boundaries.

[0037] In one optional implementation, the cross-scale feature compensation fusion module generates balanced semantic features by averaging the spatial dimensions of features from different levels, including: Obtain feature maps from multiple different levels from the backbone network; Perform downsampling or upsampling operations on the feature maps of the multiple different levels respectively, so that the spatial size of the feature maps of each level is unified to the target size; After unifying the spatial dimensions, the feature maps of each level are averaged element by element at their corresponding spatial locations to generate balanced semantic features. The balanced semantic features are used as the output of the cross-scale feature compensation fusion module and passed to subsequent layers of the defect detection network.

[0038] The feature maps extracted from the backbone network have different spatial resolutions and semantic information intensities. In defect detection tasks, the backbone network typically includes multiple convolutional layers, with the spatial size of the feature maps extracted at each layer decreasing sequentially. Taking a five-layer backbone network as an example, with an input image size of 640 pixels by 640 pixels, the first-layer feature map has a spatial size of 640 pixels by 640 pixels and 64 channels; the second-layer feature map has a spatial size of 320 pixels by 320 pixels and 128 channels; the third-layer feature map has a spatial size of 160 pixels by 160 pixels and 256 channels; the fourth-layer feature map has a spatial size of 80 pixels by 80 pixels and 512 channels; and the fifth-layer feature map has a spatial size of 40 pixels by 40 pixels and 1024 channels. These feature maps from different layers are extracted and then fed into a cross-scale feature compensation fusion module for processing.

[0039] In the process of unifying spatial dimensions, a target size needs to be determined as a unified standard for all feature maps at all levels. The selection of the target size is based on the accuracy requirements of the detection task and computational resource constraints. In this embodiment, the target size is set to 160 pixels by 160 pixels, which is the same as the original size of the third-level feature map. For feature maps with spatial dimensions larger than the target size, a downsampling operation is performed to reduce the spatial dimension. The first-level feature map is downsampled from 640 pixels by 640 pixels to 160 pixels by 160 pixels using a max pooling operation with a stride of 4 and a pooling window size of 4 by 4. The second-level feature map is downsampled from 320 pixels by 320 pixels to 160 pixels by 160 pixels using a max pooling operation with a stride of 2 and a pooling window size of 2 by 2. The downsampling operation compresses spatial information while preserving key feature responses.

[0040] For feature maps with a spatial size smaller than the target size, an upsampling operation is performed to expand the spatial dimension. The fourth-level feature map is upsampled from 80 pixels by 80 pixels to 160 pixels by 160 pixels using bilinear interpolation, with a magnification factor of 2. The fifth-level feature map is upsampled from 40 pixels by 40 pixels to 160 pixels by 160 pixels, also using bilinear interpolation, with a magnification factor of 4. Bilinear interpolation generates a new pixel value by calculating the weighted average of the four nearest neighbor pixels around the target location, with the weights determined by the distance between the target location and its neighboring pixels. The upsampling operation expands the spatial dimension of the low-resolution feature map, facilitating subsequent fusion operations.

[0041] During the channel dimension unification process, the number of channels in feature maps at each level needs to be adjusted to the same value for element-wise operations. The number of channels in each level of feature map is adjusted using convolutional layers, with the kernel size being 1x1, without changing the spatial dimensions. The number of channels in the first-level feature map is adjusted from 64 to 256, using 64 1x1 convolutional kernels with 256 channels. The number of channels in the second-level feature map is adjusted from 128 to 256, using 128 1x1 convolutional kernels with 256 channels. The number of channels in the third-level feature map remains unchanged at 256. The number of channels in the fourth-level feature map is adjusted from 512 to 256, using 512 1x1 convolutional kernels with 256 channels. The number of channels in the fifth-level feature map is adjusted from 1024 to 256, using 1024 1x1 convolutional kernels with 256 channels. Each convolution operation is followed by a batch normalization layer and a ReLU activation function. The parameters of the batch normalization layer include mean, variance, scaling factor, and offset factor.

[0042] After unifying the spatial size and number of channels, the feature maps at all five levels have a dimensional shape of 160 pixels by 160 pixels by 256. The element-wise averaging operation is performed at the corresponding spatial locations of each feature map. For a location with row index i and column index j, where i ranges from 0 to 159 and j ranges from 0 to 159, there exists a 256-dimensional feature vector at this location on each feature map. The averaging operation is performed on the five feature vectors of the five feature maps at this location. Specifically, for each channel index k, where k ranges from 0 to 255, the feature values ​​of the five feature maps at locations i, j, and channel k are extracted respectively. These five values ​​are added together and divided by 5 to obtain the feature value of the balanced semantic feature at that location and channel.

[0043] Taking a specific numerical example, suppose that at spatial location row index 75, column index 100, and channel index 128, the eigenvalue of the first-level feature map is 0.82, the eigenvalue of the second-level feature map is 0.65, the eigenvalue of the third-level feature map is 0.91, the eigenvalue of the fourth-level feature map is 0.73, and the eigenvalue of the fifth-level feature map is 0.54. Adding these five values ​​together gives 3.65, and dividing by 5 gives 0.73. This value is the eigenvalue of the balanced semantic feature at that location and in that channel. This operation is repeated for all spatial locations and all channels, ultimately generating a complete balanced semantic feature map with dimensions of 160 pixels by 160 pixels by 256.

[0044] The balanced semantic feature incorporates both the detailed information from shallow features and the semantic information from deep features. Shallow feature maps retain rich spatial details and texture information, aiding in the accurate localization of minute defect boundaries. Deep feature maps contain high-level semantic information and global context, contributing to the understanding of defect categories and characteristics. By averaging, the advantages of features from different levels are integrated into the balanced semantic feature, avoiding the limitations of single-level features. This balanced semantic feature, as the output of the cross-scale feature compensation fusion module, is passed to subsequent layers of the defect detection network for feature pyramid construction or directly for the prediction task of the detection head.

[0045] In one optional implementation, the shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. By establishing a skip connection between the shallow features of the backbone network and the small target detection layer, the shallow features bypass the downsampling operation of the intermediate layers and are directly input to the small target detection layer.

[0046] The shallow feature enhancement path is implemented by extracting feature maps at specific layers of the backbone network. These feature maps contain high-resolution spatial detail information. The feature map output by the backbone network in the second convolutional stage is one-quarter the size of the original input image. This feature map contains information such as edges, texture, and basic shape, which is crucial for the recognition of small targets. Specifically, when the input image size is 640 pixels by 640 pixels, the output feature map at this stage is 160 pixels by 160 pixels with 128 channels. This feature map is recorded as a shallow feature tensor.

[0047] The backbone network continues to perform subsequent downsampling operations. After the third convolutional stage, the feature map size is reduced to 80 pixels by 80 pixels, and the number of channels increases to 256. The fourth convolutional stage further reduces the feature map size to 40 pixels by 40 pixels, and the number of channels increases to 512. The fifth convolutional stage finally reduces the feature map size to 20 pixels by 20 pixels, and the number of channels reaches 1024. Although these downsampling operations enhance the semantic expressiveness of the features, they lead to a continuous loss of spatial resolution. Small target features that were originally clearly visible in shallow features are gradually weakened or completely lost.

[0048] The skip connection is established through a feature transfer channel that extends directly from the shallow feature extraction location to the small object detection layer. This transfer channel uses a dedicated convolutional layer to process the shallow features. This convolutional layer has a kernel size of 1 pixel by 1 pixel and a stride of 1, used to adjust the number of feature channels to match the input requirements of the small object detection layer. When the shallow feature has 128 channels, this convolutional layer converts it into a 64-channel feature representation while maintaining the spatial size of the feature map at 160 pixels by 160 pixels.

[0049] No spatial downsampling is involved during feature transmission. Shallow feature tensors immediately enter the cross-layer transmission stage after channel adjustment. The transmission mechanism adopts a direct connection method, where shallow features bypass the third, fourth, and fifth convolutional stages of the backbone network along a preset transmission path, avoiding the degradation of feature resolution caused by the max pooling layers and convolutional operations with a stride of 2 in these stages. The data flow throughout the transmission path is unidirectional, directly pointing from the shallow feature extraction point to the input port of the small target detection layer.

[0050] The small object detection layer receives feature inputs from two sources: a shallow feature passed through skip connections, with a size of 160 pixels by 160 pixels and 64 channels; and a feature processed by a top-down path through a feature pyramid network, also with a spatial size of 160 pixels by 160 pixels but with 256 channels. Feature fusion is achieved by concatenating the two feature tensors along the channel dimensions, combining them into a single feature representation. The fused feature tensor has a size of 160 pixels by 160 pixels and 320 channels.

[0051] The fused features undergo further refinement, using a 3-pixel multiplication kernel for convolution with a stride of 1, padding of 1, and an output channel count adjusted to 128. This convolution operation enhances the consistency of feature representations while fusing feature information from different sources. The refined features then undergo batch normalization and non-linear activation function processing. The SiLU function is used, which calculates the product of the input value and its Sigmoid function value as the output, providing a smooth non-linear transformation.

[0052] The small object detection layer performs object detection based on enhanced feature representations. The detection head includes a classification branch and a regression branch. The classification branch processes features using two consecutive convolutional layers, each with a kernel size of 3 pixels by 3 pixels and 128 intermediate channels. The final output channel count equals the number of object categories. When the detection task includes 80 object categories, the final output of the classification branch is a tensor of 160 pixels by 160 pixels by 80. The regression branch also uses two convolutional layers, with 4 output channels corresponding to the four coordinate parameters of the object bounding box.

[0053] In practical applications, when detecting a pedestrian target measuring only 16 pixels by 16 pixels, the target occupies a pixel area of ​​256 pixels in the original input image. After the fifth stage of processing by the backbone network, the target corresponds to less than one feature point on a 20-pixel by 20-pixel feature map, leading to detection failure. Through a shallow feature enhancement path, the pedestrian target corresponds to a 4-pixel by 4-pixel area on a 160-pixel by 160-pixel shallow feature map, containing 16 feature points. These feature points preserve the pedestrian's contour and pose information, enabling the detection layer to accurately identify this small target.

[0054] For a vehicle target measuring 24 pixels by 24 pixels, its corresponding region on the shallow feature map is 6 pixels by 6 pixels, containing 36 feature points. These feature points capture the vehicle's shape features and local details. Combined with semantic information from the deep feature map, the detection layer can distinguish vehicle types and accurately locate the bounding box. Detection results show that the skip connection mechanism improves the recall rate of small targets by 12 percentage points and the localization accuracy by 8 percentage points.

[0055] The implementation of skip connections also includes the establishment of a gradient backpropagation path. During model training, the loss value calculated by the small object detection layer is directly passed to the shallow feature extraction location through skip connections. This backpropagation path allows the shallow parameters of the backbone network to receive optimization signals specifically for the small object detection task, promoting the shallow convolutional kernels to learn a weight distribution more suitable for small object feature extraction. During training, the weight update magnitude of shallow features increases by approximately 30% compared to when skip connections are not used, indicating that the shallow network has achieved more thorough optimization.

[0056] In one optional implementation, the method further includes: Post-processing is performed on the detection results, including dynamically adjusting the non-maximum suppression threshold according to the target size, and determining the splicing of cross-block defects based on the edge continuity score. When the edge continuity score exceeds the preset continuity threshold, adjacent detection boxes are merged into a single defect.

[0057] The post-processing flow of this invention begins with receiving the preliminary detection results. The detection model output contains a set of multiple detection boxes, each carrying location coordinates, a confidence score, and a category label. In industrial defect detection scenarios, the preliminary detection results often show that the same defect is detected repeatedly. Furthermore, due to the use of a block-based detection strategy, defects located at the edges of image blocks may be segmented into multiple discontinuous detection boxes.

[0058] The process of dynamically adjusting the non-maximum suppression threshold requires calculation based on the target size features. For each detection box, the pixel area it covers is calculated. For example, if the top-left corner of a detection box is at row 120, column 200, and the bottom-right corner is at row 180, column 280, then the width of this detection box is 80 pixels, the height is 60 pixels, and the area is 4800 square pixels. Multiple size segmentation intervals are preset, including small defect intervals (areas less than 2000 square pixels), medium defect intervals (areas between 2000 and 10000 square pixels), and large defect intervals (areas exceeding 10000 square pixels). For the aforementioned 4800 square pixel detection box, it is determined to be a medium-sized defect, and the corresponding non-maximum suppression threshold is set to 0.45. For small defects with an area of ​​1200 square pixels, the non-maximum suppression threshold is set to 0.30, because small defects are more prone to false detection and require a more stringent suppression strategy. For large defects with an area of ​​15,000 square pixels, the non-maximum suppression threshold is set to 0.60, allowing for higher overlap tolerance to avoid missegmenting a single large defect.

[0059] When performing non-maximum suppression, all bounding boxes of the same category are sorted in descending order of confidence score. Taking a certain defect category as an example, there are currently five bounding boxes with confidence scores of 0.92, 0.88, 0.76, 0.65, and 0.58. The bounding box with the highest confidence score is selected as the retained object; its coordinate range is from row 100 to row 150 and column 200 to column 260. The intersection ratio (Intersection over Union) of this bounding box with the other four bounding boxes is calculated. For the second bounding box with a confidence score of 0.88, its coordinate range is from row 110 to row 155 and column 210 to column 270. The intersection area of ​​the two bounding boxes is calculated as row 110 to row 150 and column 210 to column 260, with an intersection area of ​​2000 square pixels. The first detection box has an area of ​​3000 square pixels, the second detection box has an area of ​​2925 square pixels, and their union area is 3925 square pixels, with an intersection-union ratio (IU) of 0.51. Since this value exceeds the dynamically determined threshold of 0.45, the second detection box is marked as suppressed. For the third detection box with a confidence level of 0.76, its coordinates range from row 300 to row 340 and column 400 to column 450. It does not overlap with the retained detection boxes, and its IU value is 0; therefore, this detection box remains valid.

[0060] The cross-block defect stitching determination process handles detection results located near the boundaries of image blocks. In the block detection strategy, the original large-format image is divided into multiple 512-pixel by 512-pixel sub-blocks, with a 64-pixel overlap between adjacent sub-blocks to ensure complete capture of edge defects. All detection boxes located in edge regions are identified, specifically, if any boundary of the detection box is less than 32 pixels from the sub-block boundary. A detection box whose left boundary is located in the 8th column of a sub-block, only 8 pixels from the sub-block's left boundary, is marked as an edge detection object. Detection boxes also located in edge regions in adjacent image blocks are searched to find a pair of potential stitching candidates.

[0061] The edge continuity score is calculated by comprehensively considering three dimensions: spatial proximity, shape similarity, and category consistency. Spatial proximity is calculated by measuring the distance between two bounding boxes at their junction. The right boundary of the bounding box in the left sub-block is located at column 508 of the global coordinate system, and the left boundary of the bounding box in the right sub-block is located at column 516, with a gap of 8 pixels. This gap value is compared to a preset maximum allowable gap threshold of 20 pixels, resulting in a spatial proximity component value of 0.6. This value is obtained by dividing the difference between the gap value and the maximum threshold by the maximum threshold. Shape similarity is calculated by comparing the height and width ratios of the two bounding boxes. The left bounding box has a height of 45 pixels, and the right bounding box has a height of 48 pixels, a height difference of 3 pixels, with a relative difference rate of 0.065. In terms of width, the effective width of the left bounding box at its boundary is 30 pixels, and the effective width of the right bounding box at its boundary is 28 pixels, with a width difference rate of 0.069. The calculated shape similarity component value is 0.85, reflecting the degree of geometric consistency between the two bounding boxes. Category consistency assessment confirms whether two detection boxes belong to the same defect type. A value of 1.0 is assigned when the category labels match perfectly, and a value of 0 is assigned otherwise.

[0062] The edge continuity score is obtained by weighted combination of the three components mentioned above. The spatial proximity component is assigned a weight coefficient of 0.5, the shape similarity component a weight coefficient of 0.3, and the category consistency component a weight coefficient of 0.2. In the above example, the spatial proximity component (0.6) multiplied by a weight of 0.5 equals 0.3, the shape similarity component (0.85) multiplied by a weight of 0.3 equals 0.255, and the category consistency component (1.0) multiplied by a weight of 0.2 equals 0.2, with a sum of 0.755. A preset continuity threshold of 0.70 is set; if the currently calculated edge continuity score of 0.755 exceeds this threshold, a detection box merging operation is triggered.

[0063] The detection box merging process generates a new unified detection box to represent the complete cross-block defect. The boundary coordinates of the new detection box are determined: the left boundary is the minimum of the left boundaries of the two original detection boxes, the right boundary is the maximum of the right boundary, and the upper and lower boundaries follow the same logic. The coordinate range of the original left detection box is rows 200 to 245 and columns 478 to 508; the coordinate range of the original right detection box is rows 198 to 246 and columns 516 to 550; the coordinate range of the merged new detection box is rows 198 to 246 and columns 478 to 550. The confidence score of the new detection box is the maximum of the confidence scores of the two original detection boxes. If the confidence score of the left detection box is 0.82 and the confidence score of the right detection box is 0.79, then the confidence score of the new detection box is 0.82. The two original detection box records are deleted, and the newly generated merged detection box is added to the final output result set, completing the unified representation of the cross-block defect. The entire post-processing workflow outputs detection results optimized by nonmaximum suppression and cross-block stitching, with each detection box accurately corresponding to an independent defect instance.

[0064] This invention provides an ultra-high-definition thin film defect detection system based on an improved YOLOv8, comprising: The first unit is used to acquire ultra-high-definition images of the thin film to be inspected. Based on the statistical results of defect size distribution, it adaptively adjusts the block size and overlap rate to decompose the ultra-high-definition image into multiple sub-image blocks with spatial redundancy. The second unit is used to extract features from each of the sub-image blocks using a feature extraction network, thereby obtaining a feature map corresponding to each sub-image block. The third unit is used to perform weighted interpolation based on spatial gradient and semantic similarity on the boundary feature vectors of adjacent sub-image patches in the overlapping region, and generate feature maps of each sub-image patch after feature compensation; The fourth unit is used to input the feature maps of each sub-image block after feature compensation into the defect detection network. The defect detection network includes an anchor box configuration module designed for targets with extreme aspect ratios, a shallow feature enhancement path, and a cross-scale feature compensation fusion module. The cross-scale feature compensation fusion module generates balanced semantic features by averaging the spatial dimensions of features at different levels. The shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. The fifth unit is used to determine the location and category of defects in the ultra-high-definition image based on the output results of the defect detection network.

[0065] A third aspect of the present invention provides an electronic device, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0066] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0067] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting ultra-high-definition thin film defects based on improved YOLOv8, characterized in that, include: An ultra-high-resolution image of the thin film to be inspected is acquired. Based on the statistical results of defect size distribution, the block size and overlap rate are adaptively adjusted to decompose the ultra-high-resolution image into multiple sub-image blocks with spatial redundancy. A feature extraction network is used to extract features from each of the sub-image blocks to obtain a feature map corresponding to each sub-image block; For the boundary feature vectors of adjacent sub-image patches in the overlapping region, weighted interpolation is performed based on spatial gradient and semantic similarity to generate feature maps of each sub-image patch after feature compensation; The feature maps of each sub-image block after feature compensation are input into the defect detection network. The defect detection network includes an anchor box configuration module designed for targets with extreme aspect ratios, a shallow feature enhancement path, and a cross-scale feature compensation fusion module. The cross-scale feature compensation fusion module generates balanced semantic features by averaging the spatial dimensions of features at different levels. The shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. Based on the output of the defect detection network, the location and category of defects in the ultra-high-definition image are determined.

2. The method according to claim 1, characterized in that, Based on the statistical results of defect size distribution, the block size and overlap rate are adaptively adjusted to decompose the ultra-high-definition image into multiple sub-image blocks with spatial redundancy, including: By statistically analyzing the average size and spatial distribution of defects in historical or current batches, we can obtain statistical results on defect size distribution. The block size is determined based on the statistical results of the defect size distribution. When the defect density exceeds a preset density threshold, the block size is reduced to preserve the local context. When the average defect length exceeds a preset length threshold, the block size is increased to reduce the number of blocks. The overlap rate is determined based on the average length of the defect, and the overlap rate increases with the average length of the defect to ensure that cross-boundary defects have sufficient coverage in adjacent sub-image blocks; The ultra-high-definition image is divided into blocks based on the block size and overlap rate to generate multiple sub-image blocks with spatial redundancy.

3. The method according to claim 1, characterized in that, For the boundary feature vectors of adjacent sub-image patches in the overlapping region, weighted interpolation is performed based on spatial gradient and semantic similarity to generate feature maps of each sub-image patch after feature compensation, including: Extract the boundary feature vectors of adjacent sub-image patches at corresponding positions in the overlapping region; Calculate the spatial gradient value of the boundary feature vector, and calculate the semantic similarity between the boundary feature vectors of adjacent sub-image blocks; The weighting coefficient is determined based on the spatial gradient value and semantic similarity. When the semantic similarity is higher than the preset similarity threshold, the weighting coefficient is increased, and when the semantic similarity is lower than the preset similarity threshold, the weighting coefficient is decreased. Based on the weighting coefficients, the boundary feature vectors of adjacent sub-image blocks are weighted and interpolated to generate a continuous feature representation after bridging. The original boundary feature vector is replaced by the merged continuous feature representation to obtain feature maps of each sub-image block after feature compensation.

4. The method according to claim 1, characterized in that, The cross-scale feature compensation and fusion module generates balanced semantic features by averaging the spatial dimensions of features from different levels, including: Obtain feature maps from multiple different levels from the backbone network; Perform downsampling or upsampling operations on the feature maps of the multiple different levels respectively, so that the spatial size of the feature maps of each level is unified to the target size; After unifying the spatial dimensions, the feature maps of each level are averaged element by element at their corresponding spatial locations to generate balanced semantic features. The balanced semantic features are used as the output of the cross-scale feature compensation fusion module and passed to subsequent layers of the defect detection network.

5. The method according to claim 1, characterized in that, The shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. By establishing a skip connection between the shallow features of the backbone network and the small target detection layer, the shallow features bypass the downsampling operation of the intermediate layers and are directly input to the small target detection layer.

6. The method according to claim 1, characterized in that, The method further includes: Post-processing is performed on the detection results, including dynamically adjusting the non-maximum suppression threshold according to the target size, and determining the splicing of cross-block defects based on the edge continuity score. When the edge continuity score exceeds the preset continuity threshold, adjacent detection boxes are merged into a single defect.

7. An ultra-high-definition thin film defect detection system based on improved YOLOv8, used to implement the method as described in any one of claims 1-6, characterized in that, include: The first unit is used to acquire ultra-high-definition images of the thin film to be inspected. Based on the statistical results of defect size distribution, it adaptively adjusts the block size and overlap rate to decompose the ultra-high-definition image into multiple sub-image blocks with spatial redundancy. The second unit is used to extract features from each of the sub-image blocks using a feature extraction network, thereby obtaining a feature map corresponding to each sub-image block. The third unit is used to perform weighted interpolation based on spatial gradient and semantic similarity on the boundary feature vectors of adjacent sub-image patches in the overlapping region, and generate feature maps of each sub-image patch after feature compensation; The fourth unit is used to input the feature maps of each sub-image block after feature compensation into the defect detection network. The defect detection network includes an anchor box configuration module designed for targets with extreme aspect ratios, a shallow feature enhancement path, and a cross-scale feature compensation fusion module. The cross-scale feature compensation fusion module generates balanced semantic features by averaging the spatial dimensions of features at different levels. The shallow feature enhancement path directly transmits the shallow features of the backbone network to the small target detection layer. The fifth unit is used to determine the location and category of defects in the ultra-high-definition image based on the output results of the defect detection network.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images