SOP chip packaging body defect detection method and system based on improved YOLO-v8

Abstract

The invention relates to an SOP chip packaging body defect detection method and system based on improved YOLO-v8, and the method comprises the following steps: S1, collecting and marking SOP chip packaging body defect pictures, and constructing an SOP chip packaging body defect data set; s2, performing data enhancement on the SOP chip package defect data set; s3, the YOLO-v8s model is optimized according to the defect characteristics of the SOP chip packaging body, and an SOP-YOLOv8s target detection model is obtained; s4, carrying out the training of an SOP-YOLOv8s target detection model through the SOP chip packaging body defect data set after the data enhancement; s5, constructing an SOP chip packaging body defect detection system, and deploying the trained SOP-YOLOv8s target detection model to an industrial personal computer system; and S6, acquiring an image through the industrial camera, and transmitting the image to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected. The method and the system are beneficial to improving the defect detection precision and speed of the SOP chip packaging body.

Description

Technical Field

[0001] The present invention relates to the technical field of chip defect detection, and in particular to a SOP chip package defect detection method and system based on improved YOLO-v8. Background Art

[0002] As a core component of integrated circuits, SOP chips have played a crucial role in various industries. However, due to the complexity of SOP chip production technology, slight changes in factors such as materials, environment, and process parameters often lead to defects in SOP chip packages, affecting product yield. Therefore, research on defect detection methods is particularly important.

[0003] In the early days, SOP chip package inspection largely relied on manual visual inspection. However, this method suffers from low efficiency, low precision, and inconsistent standards, and is gradually being replaced by machine vision inspection technology. Machine vision-based SOP chip package defect detection algorithms are simple and practical and have been deployed in actual production scenarios. However, in inspection scenarios with complex backgrounds and low defect contrast, traditional machine vision image processing algorithms perform poorly. Furthermore, machine vision algorithms rely on manually designed defect feature descriptions and are only applicable to simple, regular industrial scenarios. When faced with defects with complex morphologies, manual feature defect descriptions are difficult to accurately identify. In these cases, machine vision inspection methods are not only difficult to apply but may also require complex post-processing. Therefore, machine vision inspection methods have poor versatility for irregularly shaped package defects, and when multiple types of defects need to be detected simultaneously, their detection speed does not meet expectations. Therefore, new inspection methods are needed to address the problems existing in machine vision inspection methods. Summary of the Invention

[0004] The object of the present invention is to provide a SOP chip package defect detection method and system based on improved YOLO-v8, which is conducive to improving the accuracy and speed of SOP chip package defect detection.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: a SOP chip package defect detection method based on improved YOLO-v8, comprising the following steps:

[0006] S1. Collect and annotate SOP chip package defect images to build an SOP chip package defect dataset.

[0007] S2. Data enhancement of SOP chip package defect dataset;

[0008] S3. Optimize the YOLO-v8s model based on the defect characteristics of the SOP chip package to obtain the SOP-YOLOv8s target detection model;

[0009] S4. Train the SOP-YOLOv8s target detection model using the data-enhanced SOP chip package defect dataset.

[0010] S5. Build a SOP chip package defect detection system and deploy the trained SOP-YOLOv8s target detection model to the industrial computer system;

[0011] S6. Collect images through an industrial camera and transmit them to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected.

[0012] Furthermore, in step S1, the method for constructing the SOP chip package defect dataset is as follows:

[0013] Collect defect images of SOP chip packages; use Labelme software to annotate defects, including qualified chips, missing pins, broken pins, bent pins, and scratches on the package surface. After annotation, a label file is automatically generated and saved. The main contents of the label file are the defect type contained in the image and the diagonal pixel coordinates of the annotated defect box; the annotated data set is divided into training and test sets.

[0014] Furthermore, in step S2, the method for implementing data enhancement on the SOP chip package defect dataset is as follows:

[0015] Data enhancement is performed on the training set to expand the training set. Data enhancement operations include: image scaling, adjusting image brightness, adjusting image contrast, image flipping, adding noise, and image cropping. The training set is expanded through data enhancement processing and screening.

[0016] Furthermore, in step S3, the SOP-YOLOv8s target detection model includes four parts: input end, backbone network, neck network and head network, which is implemented by optimizing the YOLO-v8s model, including:

[0017] The C2f modules of the original backbone network and neck network are replaced with C2f-DOSA modules to improve the feature extraction capabilities of the original modules and reduce the number of module parameters;

[0018] Introducing the HWD downsampling module into the original backbone network and neck network to maximize the preservation of the details of the original feature map;

[0019] Introducing the bidirectional feature pyramid network BiPFPN to improve the accuracy of target detection;

[0020] The small target detection head P2 is introduced to optimize the feature fusion path to improve the model's detection performance for small defects.

[0021] The original IoU loss function is replaced by the FS-IoU loss function as the loss function of SOP-YOLOv8s. While introducing the shape and scale factors of the bounding box itself, the original IoU loss is reconstructed to improve the model training convergence speed and detection accuracy.

[0022] Furthermore, the C2f-DOSA module is implemented by reconstructing the C2f module through a self-attention module based on DOConv lightweight convolution and Dilated-G-CSA multi-head dilated convolution, specifically:

[0023] The input feature map is extracted by the DOConv convolution layer and then split into two branches in the channel dimension. The first branch then extracts global feature information through multiple series-connected Bottleneck-DOSA bottleneck modules, and the output feature map of each bottleneck module is spliced and output with the first and second branches, and then the feature information is fused through the DOConv convolution layer to obtain the output feature map of the C2f-DOSA module. The Bottleneck-DOSA bottleneck module extracts feature information from the input feature map through two DOConvs, and then the global feature information is constrained by the self-attention module of the Dilated-G-CSA multi-head dilated convolution to obtain the Bottleneck -The output feature map of the DOSA module; the self-attention module of the Dilated-G-CSA multi-head dilated convolution first generates query (Q), key (K) and value (V) feature maps for calculating the input feature map through 1×1 convolution and 3×3 hole convolution, and then divides the feature map into 3 groups along the channel dimension and reorganizes the dimensions according to the number of heads in the channel dimension to obtain Q, K, and V feature maps. Then, the Q feature map of each head is multiplied by the transposed matrix of the K feature map, and the attention score is obtained after scaling and normalization, and the attention score is weighted and summed with the V feature map of the corresponding head. Finally, the feature maps calculated by each head are spliced and reorganized and the number of channels is adjusted through 1×1 convolution to obtain the output of the module.

[0024] Furthermore, an HWD downsampling module is introduced into the original backbone network and the neck network. The implementation method is: replacing the downsampling Conv convolution module in the deep layer of the original backbone network and the downsampling Conv convolution module in the neck network with the HWD module.

[0025] Furthermore, a bidirectional feature pyramid network (BiPFPN) is introduced. Its implementation method is as follows: the path aggregation feature pyramid network (PAFPN) of the original YOLOv8 model is replaced by the bidirectional feature pyramid network (BiPFPN). That is, bidirectional cross-scale connections are added to the original network. At the same time, features at different levels are dynamically weighted through learnable weights to alleviate fusion conflicts caused by feature scale differences.

[0026] Furthermore, a small target detection head P2 is introduced. Its implementation method is as follows: the feature map obtained by upsampling and concat-ing the neck network twice is upsampled again to obtain a feature map of 160×160×256 size, and then spliced with the feature map output by the shallow C2f-DOSA module of the backbone network, and feature information is extracted again through the C2f-DOSA module; finally, a small target detection layer with a feature map size of 160×160×128 is obtained.

[0027] Furthermore, the FS-IoU loss function is implemented as follows:

[0028] Applying Focaler-IoU to the Shape-IoU loss function, we construct the FS-IoU loss function, which enables the model to dynamically focus on bounding box regression in a specific IoU area according to the difficulty of different targets in the detection task, while retaining the consideration of the bounding box shape similarity of Shape-IoU. Its expression is as follows:

[0029] L FS-IoU =L Shape-IoU +IoU-IoU Focaler

[0030] Among them, L FS-IoU Represents the loss function of SOP-YOLOv8s; L Shape-IoU Represents the Shape-IoU loss function; IoU represents the intersection-over-union ratio of the predicted box to the real box; IoU Focaler represents Focaler-IoU, which is expressed as follows:

[0031]

[0032] Among them, d and u are used to adjust the sensitivity of the target detection model to different IoU areas. The original IoU loss is reconstructed through linear interval mapping, and the original IoU interval [0,1] is mapped to the [d,u] interval. Therefore, according to the difficulty of detecting the target, the training strategy of the target detection model is dynamically adjusted, so that the model can focus more on specific IoU areas and improve the detection performance.

[0033] The present invention also provides a SOP chip package defect detection system for implementing the above method, comprising a base frame, an image acquisition mechanism, a feeding mechanism and an industrial computer system;

[0034] The image acquisition mechanism includes an adjustment frame, an industrial camera, a camera light source mounting plate, a ring light source mounting plate and a ring light source, the adjustment frame includes two longitudinal adjustment profiles and one transverse adjustment profile, each adjustment profile is fixedly connected by right-angled pieces and bolts, the shooting height of the industrial camera is determined by the installation height of the transverse adjustment profile on the longitudinal adjustment profile; the adjustment frame is installed on the mounting slot of the top plate of the base frame through right-angled pieces and bolts, and the position of the adjustment frame above the base frame is determined by its installation position with the mounting slot on the top plate of the base frame; the camera light source mounting plate is installed on the transverse adjustment profile through bolts and nuts, and the shooting position of the industrial camera is adjusted by adjusting its installation position on the transverse adjustment profile; the industrial camera and the ring light source mounting plate are fixed to the camera light source mounting plate through bolts and nuts, and the installation height of the ring light source mounting plate is determined by the installation position of the mounting slot on the camera light source mounting plate; the ring light source is fixed to the ring light source mounting plate by bolts, and its lighting height is determined by the installation height of the ring light source mounting plate;

[0035] The feeding mechanism includes a circular turntable, a connecting flange and a stepper motor; the circular turntable is connected to the connecting flange by bolts and nuts, and the groove above the circular turntable is used to place the SOP chip to be tested; the connecting flange is fixed to the stepper motor by set screws; the stepper motor is fixed to the mounting hole of the top plate of the base frame by bolts and nuts;

[0036] A trained SOP-YOLOv8s target detection model is deployed in the industrial computer system. The industrial camera and stepper motor are respectively connected to the industrial computer system. The industrial computer system controls the operation of the stepper motor, and collects images through the industrial camera and transmits them to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected.

[0037] Compared with the prior art, the present invention has the following beneficial effects: the present invention provides a SOP chip package defect detection method and system based on improved YOLO-v8, which solves the problem that the detection of SOP chip package defects by traditional machine vision algorithms relies on manually designed defect feature descriptions, resulting in poor detection accuracy, and improves the accuracy and speed of SOP chip package defect detection; the system can realize the detection of SOP chip package defects and the display of detection defect results by deploying the SOP-YOLOv8s target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 1 is a flow chart of an implementation method of a SOP chip package defect detection method according to an embodiment of the present invention;

[0039] Figure 2: This is an architecture diagram of the SOP-YOLOv8s target detection model in an embodiment of the present invention;

[0040] Figure 3 This is a graph of mAP@0.5:0.95 in an embodiment of the present invention;

[0041] Figure 4 is a loss value curve diagram in an embodiment of the present invention;

[0042] Figure 5 This is a SOP-YOLOv8s detection effect diagram in an embodiment of the present invention;

[0043] Figure 6 1 is a schematic structural diagram of a SOP chip package defect detection system according to an embodiment of the present invention.

[0044] In the figure: 1-display; 2-mouse; 3-keyboard; 4-industrial computer; 5-base frame; 6-adjustment frame; 7-camera light source mounting plate; 8-industrial camera; 9-ring light source mounting plate; 10-ring light source; 11-circular turntable; 12-connecting flange; 13-stepping motor. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] like Figure 1 As shown, this embodiment provides a SOP chip package defect detection method based on improved YOLO-v8, including the following steps:

[0049] S1. Collect and annotate SOP chip package defect images to build an SOP chip package defect dataset.

[0050] S2. Data enhancement of SOP chip package defect dataset;

[0051] S3. Optimize the YOLO-v8s model based on the defect characteristics of the SOP chip package to obtain the SOP-YOLOv8s target detection model;

[0052] S4. Train the SOP-YOLOv8s target detection model using the data-enhanced SOP chip package defect dataset.

[0053] S5. Build a SOP chip package defect detection system and deploy the trained SOP-YOLOv8s target detection model to the industrial computer system;

[0054] S6. Collect images through an industrial camera and transmit them to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected.

[0055] The following is a further detailed description of each part.

[0056] 1. Constructing a SOP chip package defect dataset

[0057] We collected 311 raw images of SOP chip package defects, all at a resolution of 1752×1750. Labelme software was used to annotate defects, including qualified chips, missing pins, broken pins, bent pins, and package surface scratches. A label file was automatically generated and saved after annotation. The label file contained the defect type within the image and the diagonal pixel coordinates of the annotated defect box. The labeled dataset was divided into a training set and a test set with a ratio of 4:1, consisting of 248 training images and 63 test images.

[0058] 2. Data enhancement processing

[0059] Data augmentation was performed on the training set to expand the training set. The data augmentation operations included image scaling, image brightness adjustment, image contrast adjustment, image flipping, noise addition, and image cropping. Through data augmentation processing and screening, the training set was expanded to 1,357 images.

[0060] 3. Optimize the YOLO-v8s model to obtain the SOP-YOLOv8s target detection model

[0061] The SOP-YOLOv8s target detection model structure in this embodiment is as follows Figure 2 The SOP-YOLOv8s target detection model consists of four parts: input end, backbone network, neck network and head network. It is implemented by optimizing the YOLO-v8s model, specifically:

[0062] 1) Replace the C2f modules of the original backbone network and neck network with C2f-DOSA modules to improve the feature extraction capability of the original modules and reduce the number of module parameters;

[0063] 2) Introducing the HWD downsampling module into the original backbone network and neck network to maximize the preservation of the details of the original feature map;

[0064] 3) Introducing the bidirectional feature pyramid network BiPFPN to optimize the feature fusion path to improve the accuracy of target detection;

[0065] 4) Introducing the small target detection head P2 to improve the model's detection performance for small defects;

[0066] 5) Replace the original IoU loss function with the FS-IoU loss function as the loss function of SOP-YOLOv8s. While introducing the shape and scale factors of the bounding box itself, the original IoU loss is reconstructed to improve the model training convergence speed and detection accuracy.

[0067] C2f-DOSA module:

[0068] The C2f-DOSA module is implemented by reconstructing the C2f module through a self-attention module based on DOConv lightweight convolution and Dilated-G-CSA multi-head dilated convolution.

[0069] After the input feature map is extracted by the DOConv convolution layer, it is split into two branches in the channel dimension. Then the first branch extracts global feature information through multiple series-connected Bottleneck-DOSA bottleneck modules, and the output feature map of each bottleneck module is spliced and output with the first and second branches, and then the feature information is fused through the DOConv convolution layer to obtain the output feature map of the C2f-DOSA module; the Bottleneck-DOSA bottleneck module extracts feature information from the input feature map through two DOConvs, and then the global feature information is constrained by the self-attention module of the Dilated-G-CSA multi-head dilated convolution to obtain the output feature map of the Bottleneck-DOSA module; the self-attention module of the Dilated-G-CSA multi-head dilated convolution first converts the input feature map X∈R B×C×H×W The feature maps for calculating query (Q), key (K) and value (V) are generated by 1×1 convolution and 3×3 dilated convolution (d=2) in sequence. The feature maps are then divided into 3 groups along the channel dimension and reorganized according to the number of heads in the channel dimension to obtain Q, K, and V feature maps. The Q feature map of each head is then multiplied by the transposed matrix of the K feature map, and then scaled and normalized to obtain the attention score, which is then weighted and summed with the V feature map of the corresponding head. Finally, the feature maps calculated for each head are concatenated and reorganized, and the number of channels is adjusted by 1×1 convolution to obtain the output of the module.

[0070]

[0071] Attention=Concat(Head1...Head i ),i∈(1,…,head)

[0072]

[0073] Among them, head represents the number of heads, s represents the scaling scale, Concat represents the concatenation operation, and Out represents the output feature map after concatenation and reorganization.

[0074] HWD downsampling module:

[0075] The downsampling Conv convolution module in the deep layer of the original backbone network and the downsampling Conv convolution module in the neck network are replaced with the HWD module.

[0076] Bidirectional Feature Pyramid Network BiPFPN:

[0077] The Bidirectional Feature Pyramid Network (BiPFPN) replaces the Path Aggregation Feature Pyramid Network (PAFPN) in the original YOLOv8 model. This adds bidirectional cross-scale connections to the original network. Furthermore, features at different levels are dynamically weighted using learnable weights to mitigate fusion conflicts caused by differences in feature scale.

[0078] Small target detection head P2:

[0079] The neck network is upsampled twice and the feature map obtained by concat is upsampled again to obtain a feature map of size 160×160×256, which is then spliced with the feature map output by the shallow C2f-DOSA module of the backbone network, and feature information is extracted again by the C2f-DOSA module; finally, a small target detection layer with a feature map size of 160×160×128 is obtained.

[0080] FS-IoU loss function:

[0081] Applying Focaler-IoU to the Shape-IoU loss function, we construct the FS-IoU loss function, which enables the model to dynamically focus on bounding box regression in a specific IoU area according to the difficulty of different targets in the detection task, while retaining the consideration of the bounding box shape similarity of Shape-IoU. Its expression is as follows:

[0082] L FS-IoU =L Shape-IoU +IoU-IoU Focaler

[0083] Among them, LFS-IoU Represents the loss function of SOP-YOLOv8s; L Shape-IoU Represents the Shape-IoU loss function; IoU represents the intersection-over-union ratio of the predicted box to the real box; IoU Focaler represents Focaler-IoU, which is expressed as follows:

[0084]

[0085] Among them, d and u are used to adjust the sensitivity of the target detection model to different IoU areas. The original IoU loss is reconstructed through linear interval mapping, and the original IoU interval [0,1] is mapped to the [d,u] interval. Therefore, according to the difficulty of detecting the target, the training strategy of the target detection model is dynamically adjusted, so that the model can focus more on specific IoU areas and improve the detection performance.

[0086] 4. SOP-YOLOv8s target detection model training

[0087] The training parameters are: set epoch to 300, set batch size to 32; training evaluation indicators are mAP@0.5:0.95 (calculating the average precision mAP of the model at different IoU thresholds (from 0.5 to 0.95) and training loss value. The training results of SOP-YOLOv8s are as follows Figures 3-4 As shown in Figure 2, it can be concluded that SOP-YOLOv8s has a significant improvement in detection performance compared to the baseline model YOLO-v8s. Figure 5 As shown. The calculation expression of mAP is:

[0088]

[0089] The calculation of AP is related to the precision rate P and recall rate R of each category detection, where Among them, TP represents the number of positive predictions that are actually positive, FP represents the number of positive predictions that are actually negative, and FN represents the number of negative predictions that are actually positive.

[0090] 5. Construction of SOP chip package defect detection system

[0091] Build a SOP chip package defect detection system, such as Figure 6 As shown in the figure, the trained SOP-YOLOv8s target detection model is deployed to the industrial computer system.

[0092] 6. SOP chip package defect detection

[0093] After receiving the start inspection command, the industrial camera is started to capture the SOP chip to be inspected in the circular turntable groove, and the captured image data is transmitted to the to-be-inspected folder of the industrial computer system; the SOP-YOLOv8s target detection model deployed in the system performs defect detection on the transmitted image data; the defect detection results are output to the display screen for display and the command is transmitted to the motor to detect the next SOP chip.

[0094] In order to implement the above method, Figure 6 As shown, this embodiment also provides a corresponding SOP chip package defect detection system, including a base frame 5, an image acquisition mechanism, a feeding mechanism and an industrial computer system.

[0095] The image acquisition mechanism includes an adjustment frame 6, an industrial camera 8, a camera light source mounting plate 7, an annular light source mounting plate 9 and an annular light source 10. The adjustment frame 6 includes two longitudinal adjustment profiles and one transverse adjustment profile. The adjustment profiles are fixedly connected by right-angled pieces and bolts. The shooting height of the industrial camera is determined by the installation height of the transverse adjustment profile on the longitudinal adjustment profile; the adjustment frame 6 is installed on the mounting groove of the top plate of the base frame 5 through right-angled pieces and bolts. The position of the adjustment frame 6 above the base frame 5 is fixed by its connection with the top plate of the base frame 5. The installation position of the upper mounting groove is determined by the camera light source mounting plate 7; the camera light source mounting plate 7 is installed on the horizontal adjustment profile by bolts and nuts, and the shooting position of the industrial camera is adjusted by adjusting its installation position on the horizontal adjustment profile; the industrial camera 8 and the ring light source mounting plate 9 are fixed to the camera light source mounting plate 7 by bolts and nuts, and the installation height of the ring light source mounting plate 9 is determined by the installation position of the mounting groove on the camera light source mounting plate 7; the ring light source 10 is fixed to the ring light source mounting plate 9 by bolts, and its lighting height is determined by the installation height of the ring light source mounting plate 9.

[0096] The feeding mechanism includes a circular turntable 11, a connecting flange 12 and a stepper motor 13; the circular turntable 11 is connected to the connecting flange 12 by bolts and nuts, and the groove above the circular turntable 11 is used to place the SOP chip to be tested; the connecting flange 12 is fixed to the stepper motor 13 by set screws; the stepper motor 13 is fixed to the mounting hole of the top plate of the base frame 5 by bolts and nuts.

[0097] The industrial computer system (including a display 1, a mouse 2, a keyboard 3 and an industrial computer cabinet 4) is deployed with a trained SOP-YOLOv8s target detection model. The industrial camera 8 and the stepper motor 13 are respectively connected to the industrial computer system. The industrial computer system controls the stepper motor 13 to work, and collects images through the industrial camera 8 and transmits them to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected.

[0098] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0099] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0100] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A SOP chip package defect detection method based on improved YOLO-v8, characterized in that: The following steps are involved: S1. Collect and annotate SOP chip package defect images to build an SOP chip package defect dataset. S2. Data enhancement of SOP chip package defect dataset; S3. Optimize the YOLO-v8s model based on the defect characteristics of the SOP chip package to obtain the SOP-YOLOv8s target detection model; S4. Train the SOP-YOLOv8s target detection model using the data-enhanced SOP chip package defect dataset. S5. Build a SOP chip package defect detection system and deploy the trained SOP-YOLOv8s target detection model to the industrial computer system; S6. Collect images through an industrial camera and transmit them to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected.

2. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 1, wherein In step S1, the implementation method of constructing the SOP chip package defect dataset is as follows: Collect defect images of SOP chip packages; use Labelme software to annotate defects, including qualified chips, missing pins, broken pins, bent pins, and scratches on the package surface. After annotation, a label file is automatically generated and saved. The main contents of the label file are the defect type contained in the image and the diagonal pixel coordinates of the annotated defect box; the annotated data set is divided into training and test sets.

3. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 1, is characterized in that, In step S2, the data enhancement method for the SOP chip package defect dataset is as follows: Data enhancement is performed on the training set to expand the training set. Data enhancement operations include: image scaling, adjusting image brightness, adjusting image contrast, image flipping, adding noise, and image cropping. The training set is expanded through data enhancement processing and screening.

4. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 1, is characterized in that, In step S3, the SOP-YOLOv8s target detection model includes four parts: input end, backbone network, neck network and head network, which is implemented by optimizing the YOLO-v8s model, including: The C2f modules of the original backbone network and neck network are replaced with C2f-DOSA modules to improve the feature extraction capabilities of the original modules and reduce the number of module parameters; Introducing the HWD downsampling module into the original backbone network and neck network to maximize the preservation of the details of the original feature map; Introducing the bidirectional feature pyramid network BiPFPN to optimize the feature fusion path to improve the accuracy of target detection; Introducing the small target detection head P2 to improve the model's detection performance for small defects; The original IoU loss function is replaced by the FS-IoU loss function as the loss function of SOP-YOLOv8s. While introducing the shape and scale factors of the bounding box itself, the original IoU loss is reconstructed to improve the model training convergence speed and detection accuracy.

5. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 4, is characterized in that, The C2f-DOSA module is implemented by reconstructing the C2f module based on the self-attention module of DOConv lightweight convolution and Dilated-G-CSA multi-head dilated convolution, specifically: The input feature map is extracted by the DOConv convolution layer and then split into two branches in the channel dimension. The first branch then extracts global feature information through multiple series-connected Bottleneck-DOSA bottleneck modules, and the output feature map of each bottleneck module is spliced and output with the first and second branches, and then the feature information is fused through the DOConv convolution layer to obtain the output feature map of the C2f-DOSA module. The Bottleneck-DOSA bottleneck module extracts feature information from the input feature map through two DOConvs, and then the global feature information is constrained by the self-attention module of the Dilated-G-CSA multi-head dilated convolution to obtain the Bottleneck -The output feature map of the DOSA module; the self-attention module of the Dilated-G-CSA multi-head dilated convolution first generates query (Q), key (K) and value (V) feature maps for calculating the input feature map through 1×1 convolution and 3×3 hole convolution, and then divides the feature map into 3 groups along the channel dimension and reorganizes the dimensions according to the number of heads in the channel dimension to obtain Q, K, and V feature maps. Then, the Q feature map of each head is multiplied by the transposed matrix of the K feature map, and the attention score is obtained after scaling and normalization, and the attention score is weighted and summed with the V feature map of the corresponding head. Finally, the feature maps calculated by each head are spliced and reorganized and the number of channels is adjusted through 1×1 convolution to obtain the output of the module.

6. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 4, is characterized in that, The HWD downsampling module is introduced into the original backbone network and the neck network. The implementation method is: replace the downsampling Conv convolution module in the deep layer of the original backbone network and the downsampling Conv convolution module in the neck network with the HWD module.

7. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 4, wherein A bidirectional feature pyramid network (BiPFPN) is introduced. Its implementation method is as follows: the Bidirectional Feature Pyramid Network (BiPFPN) replaces the Path Aggregation Feature Pyramid Network (PAFPN) of the original YOLOv8 model with the BiPFPN. That is, bidirectional cross-scale connections are added to the original network. At the same time, features at different levels are dynamically weighted through learnable weights to alleviate fusion conflicts caused by feature scale differences.

8. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 4, wherein The small target detection head P2 is introduced. Its implementation method is as follows: the feature map obtained by upsampling and concat-ing the neck network twice is upsampled again to obtain a feature map of size 160×160×256, and then spliced with the feature map output by the shallow C2f-DOSA module of the backbone network, and feature information is extracted again by the C2f-DOSA module; finally, a small target detection layer with a feature map size of 160×160×128 is obtained.

9. The SOP chip package defect detection method based on improved YOLO-v8 according to claim 4, wherein The implementation method of the FS-IoU loss function is: Applying Focaler-IoU to the Shape-IoU loss function, we construct the FS-IoU loss function, which enables the model to dynamically focus on bounding box regression in a specific IoU area according to the difficulty of different targets in the detection task, while retaining the consideration of the bounding box shape similarity of Shape-IoU. Its expression is as follows: L FS-IoU =L Shape-IoU +IoU-IoU Focaler Among them, L FS-IoU Represents the loss function of SOP-YOLOv8s; L Shape-IoU Represents the Shape-IoU loss function; IoU represents the intersection-over-union ratio of the predicted box to the real box; IoU Focaler represents Focaler-IoU, which is expressed as follows: Among them, d and u are used to adjust the sensitivity of the target detection model to different IoU areas. The original IoU loss is reconstructed through linear interval mapping, and the original IoU interval [0,1] is mapped to the [d,u] interval. Therefore, according to the difficulty of detecting the target, the training strategy of the target detection model is dynamically adjusted, so that the model can focus more on specific IoU areas and improve the detection performance.

10. A SOP chip package defect detection system for implementing the method according to any one of claims 1 to 9, characterized in that: It includes a base frame, an image acquisition mechanism, a feeding mechanism and an industrial computer system; The image acquisition mechanism includes an adjustment frame, an industrial camera, a camera light source mounting plate, a ring light source mounting plate and a ring light source, the adjustment frame includes two longitudinal adjustment profiles and one transverse adjustment profile, each adjustment profile is fixedly connected by right-angled pieces and bolts, the shooting height of the industrial camera is determined by the installation height of the transverse adjustment profile on the longitudinal adjustment profile; the adjustment frame is installed on the mounting slot of the top plate of the base frame through right-angled pieces and bolts, and the position of the adjustment frame above the base frame is determined by its installation position with the mounting slot on the top plate of the base frame; the camera light source mounting plate is installed on the transverse adjustment profile through bolts and nuts, and the shooting position of the industrial camera is adjusted by adjusting its installation position on the transverse adjustment profile; the industrial camera and the ring light source mounting plate are fixed to the camera light source mounting plate through bolts and nuts, and the installation height of the ring light source mounting plate is determined by the installation position of the mounting slot on the camera light source mounting plate; the ring light source is fixed to the ring light source mounting plate by bolts, and its lighting height is determined by the installation height of the ring light source mounting plate; The feeding mechanism includes a circular turntable, a connecting flange and a stepper motor; the circular turntable is connected to the connecting flange by bolts and nuts, and the groove above the circular turntable is used to place the SOP chip to be tested; the connecting flange is fixed to the stepper motor by set screws; the stepper motor is fixed to the mounting hole of the top plate of the base frame by bolts and nuts; A trained SOP-YOLOv8s target detection model is deployed in the industrial computer system. The industrial camera and stepper motor are respectively connected to the industrial computer system. The industrial computer system controls the operation of the stepper motor, and collects images through the industrial camera and transmits them to the SOP-YOLOv8s target detection model for detection to obtain the package defect type of the SOP chip to be detected.

Images