Chip surface defect detection method and system based on improved YOLOv8

Abstract

The invention provides a chip surface defect detection method and system based on improved YOLOv8. The method comprises the following steps: (1) constructing a chip surface defect picture data set; (2) dividing the obtained data set into a training set, a verification set and a test set; (3) constructing an improved YOLOv8 network model; and (4) inputting the data set into the improved YOLOv8 network model for training and prediction to obtain a chip image containing surface defects. According to the invention, the improved YOLOv8 model can identify defects of different shapes and sizes through DCNv4 variable convolution, and the feature extraction capability of the network is enhanced; a Dysample up-sampling operator is used, so that the loss of defect edges and detail information is reduced, and more feature information is reserved; an original loss function is replaced with an MPDIOU boundary loss function, the convergence speed of network training is increased, and the defect positioning capacity of the model is improved. Compared with an original model, the method has the advantages that the detection effect is improved, and the method can be better applied to chip surface defect detection tasks.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to a chip defect detection method and system based on improved YOLOv8. Background Art

[0002] In the process of mass production of chips, due to the complexity of the chip manufacturing process, each link may cause defects on the chip surface, such as edge collapse, scratches and other small defects. These defects are often beyond the direct observation range of the human eye and are not easy to detect, so it is very difficult to detect them, which seriously affects the quality of the chip to a certain extent.

[0003] The traditional chip defect detection method is to use manual quality inspection, which is not only time-consuming and labor-intensive, but also has poor detection effects. With the increasing maturity of computer vision and neural network technology, target detection algorithms are widely used in defect detection systems. At present, target detection algorithms based on deep learning can be divided into two categories: two-stage algorithms and single-stage algorithms. Two-stage algorithms (such as R-CNN, Fast R-CNN, and Faster R-CNN) first generate candidate regions and then classify and regress objects. Single-stage algorithms, represented by SSD and YOLO series, only use convolutional neural networks to extract features and immediately classify and locate objects. Compared with the two-stage algorithm, the single-stage algorithm has a simpler structure, omits the step of generating candidate regions, and directly predicts the location and category of the target from the image, achieving faster detection speed while maintaining high accuracy, which is suitable for real-time detection applications.

[0004] As one of the most outstanding single-stage target detection algorithms, YOLOv8 performs well in detection speed and accuracy, and can effectively identify various defects on the chip surface. However, it still has some limitations in identifying complex and subtle features, such as poor recognition of occluded defects, poor target defect positioning, and prone to missed detection and false detection, making it difficult to meet the needs of high-precision detection. Summary of the invention

[0005] The purpose of the present invention is to provide a chip surface defect detection method and system based on improved YOLOv8, which can achieve higher detection accuracy and faster detection speed on industrial chip defect detection equipment, and can be more easily deployed in a defect diagnosis and analysis system. Inspectors can quickly identify chips containing defects based on the detection results.

[0006] On the one hand, the present invention provides a chip surface defect detection method based on improved YOLOv8, and the specific steps are as follows:

[0007] S1 builds a chip surface defect image dataset;

[0008] S2 divides the obtained data set into training set, validation set and test set;

[0009] S3 is improved based on the YOLOv8 model, DCN and C2f are fused to form a new C2f-DCN variable feature fusion module for feature extraction, a dynamic upsampling operator Dysample is introduced, and the MPDIOU boundary loss function is used to build an improved chip surface defect detection model, where the improved chip surface defect detection model is an improved YOLOv8 model;

[0010] S4 inputs the chip surface defect dataset into the improved YOLov8 model for training. By improving the ability to extract chip surface defect features and reducing the loss of detail feature information in the upsampling stage, it strengthens the positioning of target defects and optimizes the detection accuracy and generalization ability of the YOLov8 model.

[0011] S5 uses the test set to test the YOLOv8 and improved YOLOv8 models, compares the experimental results, and verifies the accuracy and efficiency of the two in chip surface defect detection, including defect location, defect category, and confidence.

[0012] In step S1, a chip surface defect image dataset is constructed, including:

[0013] (1.1) Obtaining chip surface defect images: Obtaining chip images with defects during chip production, which are divided into three types: missing pins, surface scratches, and bent pins;

[0014] (1.2) Image data enhancement: Use image enhancement methods to expand the original dataset and enhance its

[0015] Diversity;

[0016] (1.3) Image annotation: Use the LabelImg image annotation tool to mark the defect area with a box.

[0017] In step S3, an improved chip surface defect detection model is constructed, including:

[0018] (3.1) Configure the environment: Use Anaconda to create a virtual environment and debug the source code of the YOLov8 model;

[0019] (3.2) Data input: Input the constructed chip surface defect dataset into the YOLOv8 network structure model, including obtaining chip surface defect images, chip surface defect image labels, YOLOv8 network structure configuration files, and YOLOv8 model pre-training weights;

[0020] (3.3) Configure hyperparameters: set the size of the network input image, learning rate, batch size, and number of iterations;

[0021] (3.4) The original model used is YOLOv8, and the YOLOv8 network structure model is improved.

[0022] In step (3.4), the YOLOv8 network structure model is improved, including:

[0023] (3.4.1) Fusing DCN and C2f to form a new C2f-DCN variable feature fusion module

[0024] Row feature extraction;

[0025] (3.4.2) Modify the original upsampling module to Dysample;

[0026] (3.4.3) The MPDIoU boundary loss function is used to replace the original loss function.

[0027] In step (3.4.1), DCN and C2f are fused to form a new C2f-DCN variable feature fusion module for feature extraction. DCNv4_Net generates a mask through depth-wise calculation and calculates the offset using point-wise operation. Ultimately, the network is able to output the horizontal and vertical displacement and shape change parameters for each pixel. These parameter weights can be adaptively adjusted to better match the feature positions in the input data, thereby significantly improving the model's detection performance for irregular defects.

[0028] Furthermore, the steps for calculating the pixel offset and shape change parameters described in step (3.4.1) are:

[0029] (3.4.1.1) There are two convolution kernels in the C2f module Bottleneck of YOLOv8. The size of these two convolution kernels is 3*3. The size of the feature map output by the two convolution kernels remains unchanged. There is also a 3*3 convolution kernel in the DCNv4_Net structure, which can calculate the required offset and change parameters without changing the size of the feature map. Replace the two 3*3 convolutions in the C2f module with DCNv4_Net to obtain C2f-DCNv4;

[0030] (3.4.1.2) Input features are passed into C2f-DCNv4. The features first pass through a layer of 1*1 convolution kernel. A Split feature segmentation layer divides the features into two according to the number of channels. One part waits for feature concatenation Concat, and the other part is passed to the DCNv4 structure. In this structure, the features pass through two DCNv4 convolution layers. The DCNv4 convolution layer is a separable convolution composed of a 3*3 convolution and a linear projection. The original convolution weight wk is separated into a depth part and a point-by-point part. The depth part is responsible for the original position-aware modulation scalar mk, and the point-by-point part is the projection weight w shared between the sampling points. The feature passes through this separable convolution to obtain the horizontal and vertical coordinate offsets of the sampling points and the offset adjustment scale:

[0031]

[0032] Where G is the number of aggregation groups; for the g-th group, wg∈RC×C ′ ,C ′ = C / G is the projection weight at different positions: K is the number of sampling points; m gk ∈R is the modulation factor of the kth sampling point, and unlike DCNv3, DCNv4 discards the softmax normalization operation, which speeds up the operation of the model.

[0033] (3.4.1.3) Perform a BatchNorm2d normalization operation on the feature output including offset and scale obtained by DCNv4_Net:

[0034]

[0035] Where: x is the input eigenvalue, E[x] represents the mean, Var[x] represents the variance. These two parameters can be directly calculated in forward propagation. γ and β are the adjusted variance and adjusted mean, with initial values ​​of 1 and 0 respectively. These two parameters are continuously learned and updated by the model in back propagation. ε is to ensure the stability of the calculation and avoid the denominator being 0. The default value is 1e-5.

[0036] (3.4.1.4) Use the RELU function to perform an activation operation on the output result, outputting all input values ​​greater than 0 and setting the input values ​​less than 0 to 0.

[0037] In step (3.4.2), the original upsampling module is modified to Dysample, and Dysample upsampling uses linear projection to generate s 2 An offset set is used to construct new sampling points and the input feature map is resampled using bilinear interpolation.

[0038] Furthermore, the processing of the Dysample upsampling module in step (3.4.2) includes:

[0039] First, given the upsampling scale factor s and the feature map X of size C×H×W, the input feature map X is bilinearly interpolated, using the input channel C and the output channel 2s 2 The linear layer generates a size of 2s 2 ×H×W, control the offset O through the dynamic range factor, and then reshape the offset O to 2×sH×sW through pixel transformation, add the offset O to the original sampling grid G ​​to get the sampling set S, and resample the position of the sampling set S to the feature map X through the grid sampling function to generate an upsampled feature map X of size C×sH×sW.

[0040] In step (3.4.3), the MPDIoU boundary loss function is used to replace the original loss function. The MPDIoU loss function comprehensively considers the intersection ratio, center point distance, and width and height deviation between the predicted box and the true box; it takes the coordinates of the lower left and lower right corners of the predicted box as input, and defines a rectangle in a specific way so that the distance between the two points in the predicted box and the true box is minimized; it can accelerate the regression convergence of the network to the predicted box, thereby obtaining a more accurate prediction result. The calculation of the MPDIoU loss function is defined by the following formula:

[0041]

[0042] Among them: B pre represents the coordinates of the predicted bounding box, B gt Represents the coordinates of the actual annotation bounding box, x1, y1 represents the coordinates of the upper left corner of the box, x2, y2 represents the coordinates of the lower right corner of the box, is the distance between the upper left corner and the lower right corner of the predicted bounding box and the actual labeled bounding box, and w and h are the width and height of the chip image.

[0043] On the other hand, the present invention also provides a chip surface defect detection system based on improved YOLOv8, characterized in that it includes the following modules:

[0044] (1) Dataset building module: used to build a chip surface defect image dataset;

[0045] (2) Dataset partitioning module: used to partition the obtained dataset into training set, validation set and test set;

[0046] (3) Model building module: used to build an improved YOLOv8 network model;

[0047] (4) Training and prediction module: used to input the data set into the improved YOLO network model for training and prediction to obtain chip images containing surface defects.

[0048] On the other hand, the present invention also provides a computer device, including a camera, a memory and a processor, wherein the camera captures images of chips with surface defects; the internal memory provides an operating environment for an operating system and application programs; the external memory is used to store the operating system and computer programs; and the processor is configured to run the computer program of steps S1-S5.

[0049] Compared with the prior art, the present invention has the following significant advantages: the improved YOLOv8 model can identify defects of different shapes and sizes through DCNv4 variable convolution, which enhances the network's feature extraction capability; the use of the Dysample upsampling operator reduces the loss of defect edges and detail information, retaining more feature information; the use of the MPDIOU boundary loss function to replace the original loss function accelerates the convergence speed of network training and improves the model's ability to locate defects. Compared with the original model, the detection effect of the present invention has been improved, and it can be better applied to chip surface defect detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 A flow chart of a chip surface defect detection method based on improved YOLOv8 provided in an embodiment of the present invention;

[0052] Figure 2 A diagram of the YOLOv8 network structure used in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of the network structure of DCNv4 in an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the network structure of Dysample in an embodiment of the present invention;

[0055] Figure 5 FIG. 4 is a schematic diagram of the structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation of the present invention is described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many different ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0057] like Figure 1 As shown, the chip surface defect detection method based on improved YOLOv8 provided by the embodiment of the present invention includes:

[0058] Step 1: Build a chip surface defect image dataset. The specific steps include:

[0059] (1.1) Obtain chip surface defect images. The chip surface defect images used in the present invention are chip surface defect image data obtained by a chip company during the chip production process. The data set contains a total of 9880 images with a resolution of 1280*1080 and an RGB format. These images contain three types of defects: missing pins, surface scratches, and bent pins.

[0060] (1.2) Image data enhancement: In order to enhance the robustness of the model and improve the generalization ability of the model, image flipping, random scaling and Mosaic data enhancement techniques are applied to expand the original data set and enhance its diversity. Data enhancement techniques can help the model learn a wider range of feature representations, reduce overfitting, and improve the model's ability to detect chip surface defects in different scenarios, thereby improving industrial production efficiency and quality.

[0061] (1.3) Image annotation: Use the LabelImg image annotation tool to annotate the image and calibrate the actual defect area frame. For images with multiple defects, use multiple anchor frames to calibrate the defect area. The label of a missing pin is 0 (queshi), the label of a surface scratch is 1 (huaheng), and the label of a bent pin is 2 (wanqu). After the annotation is completed, the generated json format data is saved and converted into txt format data that can be processed by the YOLO network.

[0062] Step 2: Dataset division: Divide the sample-amplified and data-enhanced chip surface defect image data and the corresponding label files into training set, test set, and validation set in a ratio of 8:1:1.

[0063] Step 3: Build a chip surface defect detection model based on the improved YOLOv8 model.

[0064] (3.1) Environment configuration: Download the official source code of the YOLOv8 model from Ultralytics' official Github, use Anaconda to build a virtual environment, and install the following libraries according to the requirements file in the YOLOv8 source code: matplotlib>=3.3.0, numpy>=1.22.2, opencv-python>=4.6.0, pillow>=7.1.2, pyyaml>=5.3.1, requests>=2.23.0, scipy>=1.4.1, tqdm>=4.64.0, torch>=1.8.0, torchvision>=0.9.0.

[0065] (3.2) Data input: Input the divided chip surface defect training set and test set into the YOLOv8 network model, including chip surface defect images and corresponding labels, YOLOv8 network model pre-training weights, and YOLOv8 network structure configuration file. The network structure of YOLOv8 is as follows: Figure 2 shown.

[0066] (3.3) Configure hyperparameters: set learning rate, batch size, iteration number, image channels, image input size and optimizer. Specifically, the initial learning rate is 0.01, the minimum learning rate is set to 0.001, the weight decay is 0.0005, the batch size is 16, the iteration number is 300, the image channels are 3, the image input size is 640*640, and the optimizer is AdamW.

[0067] (3.4) Model improvement: The original model used in the present invention is YOLOv8, and the following three improvements are made to the YOLOv8 network structure model.

[0068] (3.4.1) Fuse DCN and C2f to form a new C2f-DCN variable feature fusion module for feature extraction:

[0069] (3.4.1.1) Introduce DCNv4 variable convolution in Bottleneck in C2f module to replace the original ordinary convolution to form C2f-DCNv4;

[0070] (3.4.1.2) Figure 3The figure shows a schematic diagram of the C2f-DCN structure formed by the fusion of DCNv4 and C2f. In the DCNv4 structure, the features will first pass through a layer of 1*1 convolution, and then pass through a layer of Split feature segmentation layer to divide the features into two according to the number of channels. One part is passed to the DCNv4 structure and waits for feature concatenation Concat, and the other part is passed to the DCNv4 structure. In this structure, the features pass through a layer of 3*3 layer and a DCNv4 convolution layer. The DCNv4 convolution layer is a separable convolution composed of a 3*3 convolution and a linear projection, which separates the original convolution weight wk into a depth part and a point-by-point part. The depth part is the original position-aware modulation scalar mk, and the point-by-point part is the projection weight w shared between sampling points. The feature pixel p0 can be output after this separable convolution:

[0071]

[0072] Where G is the number of aggregation groups; for the g-th group, wg∈RC×C ′ ,C ′ =C / G is the projection weight at different positions: K is the number of sampling points; mgk∈R is the modulation factor of the kth sampling point, and unlike DCNv3, DCNv4 cancels the final softmax normalization, reducing unnecessary calculations and speeding up the calculation of the model;

[0073] (3.4.1.3) Perform BatchNorm2d operation to standardize the feature output obtained by DCNv4:

[0074]

[0075] In the formula, x represents the input feature value, E[x] represents its mean, and Var[x] represents the variance. These two parameters can be directly calculated during the forward propagation process. γ and β are used to adjust the variance and mean, and their initial values ​​are 1 and 0 respectively. These two parameters are continuously learned and updated by the model during the training back propagation. ε is to ensure the stability of the calculation and avoid the denominator being 0. The default value is 1e-5.

[0076] (3.4.2) Modify the original upsampling module to Dysample

[0077] In order to improve the chip surface defect detection model's ability to capture target defect features and details, such as Figure 4 As shown, the present invention generates s by using efficient Dysample upsampling and linear projection. 2An offset set is constructed to construct new sampling points, and the input feature map is resampled using bilinear interpolation, thereby improving the chip surface defect detection model's ability to capture target defect features and details, reducing the inference cost and improving the reliability of the upsampling process.

[0078] The processing of the Dysample upsampling module includes:

[0079] In the grid sampling process, we first give a feature map X of size C×H1×W1 and a sampling set S of size 2×H2×W2, where the subscripts H and W represent the x and y coordinates. The grid sampling function uses the positions of the sampling set S to resample the assumed bilinear interpolation X to X ′ , the size is C×H2×W2, where the sampling function is defined as follows:

[0080] X ′ =grid_sample(X,S),

[0081] In the specific implementation, we first give the upsampling scale factor s and the feature map X of size C×H×W, and use the input and output channels C and 2gs respectively. 2 The linear layer generates a size of 2gs 2 ×H×W offset O, control the offset O through the dynamic range factor, and then reshape the offset O to 2×sH×sW through pixel transformation, add the offset O to the original sampling grid G ​​to get the sampling set S, and resample the feature map X of the sampling set S through the network sampling function to generate an upsampled feature map X of size C×sH×sW ′ .

[0082] Dysample upsampling generates a "dynamic range factor" point by point by linearly projecting the input features. By introducing a sigmoid function and a static factor of 0.5, the dynamic range is controlled within the range of [0,0,5], further increasing the flexibility of the offset. At this time, the offset is calculated by the following formula:

[0083] O=0.5sigmoid(linear1(X))*linear2(X)

[0084] In the present invention, compared with reconstructing the input features by generating content-aware upsampling kernels, Dysample upsampling is performed by generating sampling point positions, collecting a point at each position and dividing it into s 2 sampling points, as long as s 2If the sampling points can continue to be dynamically segmented, then the entire upsampling process only needs to collect one point. In addition, the kernel weight of Dysample upsampling is based on x and y as the position conditions, only 2 channels of feature maps are needed, and sampling is more efficient.

[0085] (3.4.3) Use MPDIoU boundary loss function to replace the original loss function

[0086] The MPDIoU loss function comprehensively considers the intersection ratio between the predicted box and the true box, the center point distance, and the width and height deviation; it takes the coordinates of the lower left and lower right corners of the predicted box as input, and defines a rectangle in a specific way so that the distance between the two points in the predicted box and the true box is minimized; it can accelerate the regression convergence of the network to the predicted box, thereby obtaining a more accurate prediction result. The calculation of the MPDIoU loss function is defined by the following formula:

[0087]

[0088] Among them: B pre represents the coordinates of the predicted bounding box, B gt Represents the coordinates of the actual annotation bounding box, x1, y1 represents the coordinates of the upper left corner of the box, x2, y2 represents the coordinates of the lower right corner of the box, is the distance between the upper left corner and the lower right corner of the predicted bounding box and the actual labeled bounding box, and w and h are the width and height of the chip image.

[0089] Step 4: Use the improved YOLOv8 model for training, including:

[0090] (5.1) The hardware environment used in the present invention is configured as follows: CPU: Intel Core i9-12900K, CPU main frequency: 3.2GHz, memory: 64G, GPU: NVIDIA GeForce RTX 3080, video memory: GDDR6X24G; the software environment is configured as follows: deep learning: Pytorch 1.12.1, Python 3.8, Cuda 11.3. The set hyperparameters are: initial learning rate 0.01, minimum learning rate is set to 0.001, weight decay is 0.0005, batch size is 16, number of iterations is 300, number of image channels is 3, image input size is 640*640, and optimizer is AdamW.

[0091] (5.2) During the model training process, in order to increase the network training speed, YOLOv8 pre-trained weights are used for training. The official provides several pre-trained weights, and different versions can be selected for training according to different needs. In the present invention, the selected pre-trained weight is YOLOv8s.pt. Then after every 50 iterations, the model is verified on the validation set. Secondly, the learning rate is adjusted, and the learning rate is dynamically adjusted according to the verification results to prevent overfitting and underfitting. The learning rate decay strategy (Learning Rate Decay) is used to gradually reduce the learning rate. Finally, the training process is completed until the preset number of iterations is reached or the performance of the validation set is no longer improved.

[0092] Step 5: Test and evaluate the improved YOLOv8 model.

[0093] Model testing and evaluation: The weight file generated after model training is loaded into the test file for model testing. The test results of the improved chip surface defect detection model on the test set are obtained, including defect location, defect category and confidence, and compared with the YOLOv8 model.

[0094] The improved chip surface defect detection model is improved based on the YOLOv8 model. DCN and C2f are fused to form a new C2f-DCN variable feature fusion module for feature extraction, and the original upsampling module is modified to Dysample. The MPDIOU boundary loss function is used to replace the original loss function to construct an improved chip surface defect detection model.

[0095] The improved algorithm not only performs better in overall detection accuracy, but also significantly improves the ability to identify subtle defects in complex backgrounds. The detection capability of irregular target defects is enhanced by introducing the DCN operator, and the positioning of target defects is enhanced by using MPDIoU.

[0096] The embodiment of the present invention also provides a chip surface defect detection system based on improved YOLOv8, including the following modules:

[0097] (1) Dataset building module: used to build a chip surface defect image dataset;

[0098] (2) Dataset partitioning module: used to partition the obtained dataset into training set, validation set and test set;

[0099] (3) Model building module: used to build an improved YOLOv8 network model;

[0100] (4) Training and prediction module: used to input the data set into the improved YOLOv8 network model for training and prediction to obtain chip images containing surface defects.

[0101] In this embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as follows: Figure 5 As shown. The computer device includes a processor, an internal memory, an external memory and a camera connected through a system bus. Among them, the processor is responsible for providing computing and control capabilities; the internal memory provides an operating environment for the operating system and application programs; and the external memory is a non-volatile storage medium for storing the operating system and computer programs. The camera of the computer device is responsible for collecting chip images with surface defects and displaying the processed results on the display screen. The communication interface of the computer device can communicate with the external terminal through WIFI technology, and the display screen can be a liquid crystal display screen.

[0102] It is worth noting that Figure 5 The structure depicted in the specification only represents some examples of hardware architectures related to the content of this patent application, and is not intended to limit the specific computer device type or configuration to which the present invention is applicable. In fact, in actual application, according to different needs, the above-mentioned components can be increased or decreased, or these components can be combined in different ways, or even different layout designs can be adopted. Therefore, professionals familiar with the technical background in this field should understand that there are many possible variations and are not limited to the specific examples described in the text.

[0103] Related vocabulary description:

[0104]

[0105] The above description only expresses the preferred implementation mode of the present invention, and is not intended to limit the present invention in other forms. Any technician familiar with the profession may use the above disclosure to modify or modify the equivalent embodiments with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiments in accordance with the technical essence of the present invention in order to deviate from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A chip surface defect detection method based on improved YOLOv8, characterized in that: The method includes: S1, build a chip surface defect image dataset; S2, divide the obtained data set into training set, validation set and test set; S3, based on the YOLOv8 network model, DCNv4 and C2f are fused to form a new C2f-DCN variable fusion module for feature extraction, a dynamic upsampling operator Dysample is introduced, and the MPDIoU boundary loss function is used to build an improved chip surface defect detection model, where the improved chip surface defect detection model is an improved YOLOv8 model; S4, input the chip surface defect dataset into the improved YOLov8 model for training; S5, use the test set to test the YOLOv8 and improved YOLOv8 models.

2. The chip surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The step S1 specifically includes the following steps: (1.1) Obtaining chip surface defect images: Obtaining chip images with defects during chip production and classifying them according to the types of chip defects; (1.2) Image data enhancement: Use data image enhancement methods to expand the original dataset and enhance its diversity; (1.3) Image annotation: Use the LabelImg image annotation tool to mark the defect area with a frame.

3. The chip surface defect detection method based on improved YOLOv8 according to claim 1 is characterized in that: In the step S2, the chip surface defect data set is divided into a training set, a verification set and a test set according to a certain ratio.

4. The chip surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: In step S3, an improved chip surface defect detection model is constructed, including: (3.1) Configure the environment: Use Anaconda to create a virtual environment and debug the source code of the YOLOv8 model; (3.2) Input data: Input the constructed chip surface defect dataset into the YOLOv8 network structure model, including obtaining chip surface defect images, surface defect image labels, YOLOv8 network structure configuration files, and YOLOv8 model pre-training weights; (3.3) Configure hyperparameters: set the size of the network input image, learning rate, batch size, and number of iterations; (3.4) The original model used is YOLOv8, and the YOLOv8 network structure model is improved.

5. The chip surface defect detection method based on improved YOLOv8 according to claim 4, characterized in that: In step (3.4), the YOLOv8 network structure model is improved, including: (3.4.1) Fusing DCN and C2f to form a new C2f-DCN variable feature fusion module for feature extraction; DCNv4_Net uses a method similar to separable convolution to generate a mask through depth-wise calculation and calculate the offset using point-wise operation. Finally, the network can output the horizontal and vertical displacement and shape change parameters for each pixel. (3.4.2) Modify the original upsampling module to Dysample to reduce the loss of detail feature information in the upsampling stage; Dysample upsampling uses linear projection to generate s 2 An offset set is used to construct new sampling points, and the input feature map is resampled using bilinear interpolation. (3.4.3) The MPDIoU boundary loss function is used to replace the original loss function to enhance the localization of target defects.

6. The chip surface defect detection method based on improved YOLOv8 according to claim 5, characterized in that: The steps for calculating the pixel offset and shape change parameters described in step (3.4.1) are: (3.4.1.1) There are two convolution kernels in the C2f module Bottleneck of YOLOv8. The size of these two convolution kernels is 3*3. The size of the feature map output by the two convolution kernels remains unchanged. There is also a 3*3 convolution kernel in the DCNv4_Net structure, which can calculate the required offset and change parameters without changing the size of the feature map. Replace the two 3*3 convolutions in the C2f module with DCNv4_Net to obtain C2f-DCNv4; (3.4.1.2) The input features are passed into C2f-DCNv4. The features first pass through a layer of 1*1 convolution kernel. A Split feature segmentation layer divides the features into two according to the number of channels. One part waits for feature concatenation Concat, and the other part is passed to the DCNv4 structure. In this structure, the features pass through two DCNv4 convolution layers. The DCNv4 convolution layer is a separable convolution composed of a 3*3 convolution and a linear projection. The original convolution weight wk is separated into a depth part and a point-by-point part. The depth part is responsible for the original position-aware modulation scalar mk, and the point-by-point part is the projection weight w shared between sampling points. The feature pixel p0 is output after this separable convolution as follows: Where G is the number of aggregation groups; for the g-th group, wg∈RC×C ′ ,C ′ =C / G is the projection weight at different positions: K is the number of sampling points; mgk∈R is the modulation factor of the kth sampling point, and unlike DCNv3, DCNv4 cancels the final softmax normalization, reducing unnecessary calculations and speeding up the calculation of the model; (3.4.1.3) Perform a BatchNorm2d operation on the feature output obtained by DCNv4_Net: Where: x is the input eigenvalue, E[x] represents the mean, Var[x] represents the variance. These two parameters can be directly calculated in forward propagation. γ and β are the adjusted variance and adjusted mean, with initial values ​​of 1 and 0 respectively. These two parameters are continuously learned and updated by the model in back propagation. ε is to ensure the stability of the calculation and avoid the denominator being 0. The default value is 1e-5. (3.4.1.4) Use the RELU function to perform an activation operation on the output result, outputting all input values ​​greater than 0 and setting the input values ​​less than 0 to 0.

7. The chip surface defect detection method based on improved YOLOv8 according to claim 5, characterized in that: In step (3.4.2), Dysample is introduced for upsampling, including: first, given the upsampling scale factor s and the feature map X of size C×H×W, the input feature map X is bilinearly interpolated, using the input channel C and the output channel 2s 2 The linear layer generates a size of 2s 2 ×H×W, control the offset O through the dynamic range factor, and then reshape the offset O to 2×sH×sW through pixel transformation, add the offset O to the original sampling grid G ​​to get the sampling set S, and resample the position of the sampling set S to the feature map X through the grid sampling function to generate an upsampled feature map X of size C×sH×sW.

8. The chip surface defect detection method based on improved YOLOv8 according to claim 5, characterized in that: In step (3.4.3), the MPDIoU boundary loss function is used to replace the original loss function, which is defined as: Among them: B pre represents the coordinates of the predicted bounding box, B gt Represents the coordinates of the actual annotation bounding box, x1, y1 represents the coordinates of the upper left corner of the box, x2, y2 represents the coordinates of the lower right corner of the box, is the distance between the upper left corner and the lower right corner of the predicted bounding box and the actual labeled bounding box, and w and h are the width and height of the chip image.

9. A chip surface defect detection system based on improved YOLOv8, characterized in that: Includes the following modules: (1) Dataset building module: used to build a chip surface defect image dataset; (2) Dataset partitioning module: used to partition the obtained dataset into training set, validation set and test set; (3) Model building module: used to build an improved YOLOv8 network model; (4) Training and prediction module: used to input the data set into the improved YOLOv8 network model for training and prediction to obtain chip images containing surface defects.

10. A computer device, comprising a camera, an internal memory, an external memory and a processor, wherein the camera is responsible for collecting images and obtaining images of chips with surface defects; the internal memory provides an operating environment for an operating system and application programs; and the external memory is a non-volatile storage medium, characterized in that: Used to store an operating system and a computer program, the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images