Chip blur defect detection model training method, defect detection method and device

Abstract

The present disclosure provides a chip blur defect detection model training method, a defect detection method and equipment, comprising: acquiring an image data set and an initial chip blur defect detection model; using a feature extractor to perform feature extraction on the image data to obtain image features; inputting the image features into an initial self-supervised classifier to determine a self-supervised loss function corresponding to the initial self-supervised classifier; inputting the image features into an initial detector, processing through the initial detector, and outputting predicted defect data; determining a detection loss function according to the real defect data and the predicted defect data; performing addition processing on the self-supervised loss function and the detection loss function to obtain a target loss function; training the initial chip blur defect detection model based on the target loss function until the number of training times of the initial chip blur defect detection model reaches a preset number of times to obtain a chip blur defect detection model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to a chip fuzzy defect detection model training method, a defect detection method and equipment. BACKGROUND

[0002] Due to the complexity of chip manufacturing process and the suboptimal production environment, some unintended structures are generated in the production process, such as scratches, cracks, bubbles, edge collapse, corrosion, residual glue and more than 20 kinds of defect problems. These defects not only affect the appearance of the components, but also affect the reliability of the components, and have a significant impact on the service life of the components. Therefore, the surface quality detection of the chip is a quality inspection process that deserves attention.

[0003] Due to the complex and changeable actual industrial production environment and other reasons, the image quality captured in the actual industrial production is uneven, and it is difficult to define the object features in the labeling process. Therefore, there is a problem of fuzzy labeling in the data labeling process. The defect detection method based on traditional image processing can only detect defects under specific conditions, resulting in a greater difficulty in detecting defects of chips. At the same time, the existing chip defect detection needs to rely on high-sensitivity detection devices to assist in completion, which has a high detection cost and requires a large amount of human resources. SUMMARY

[0004] Therefore, the purpose of the present disclosure is to provide a chip fuzzy defect detection model training method, a defect detection method and equipment to solve or partially solve the above problems.

[0005] To achieve the above purpose, the first aspect of the present disclosure provides a chip fuzzy defect detection model training method, which comprises:

[0006] obtaining an image data set and an initial chip fuzzy defect detection model, wherein the image data set comprises image data and real defect data corresponding to the image data, the image data is image data containing at least one real defect frame, and each real defect frame corresponds to at least one fuzzy defect category, the real defect frame and the fuzzy defect category corresponding to the real defect frame are the real defect data, and the at least one fuzzy defect category corresponding to each real defect frame includes the real defect category corresponding to the real defect frame;

[0007] using a feature extractor to extract features from the image data to obtain image features;

[0008] inputting the image features into an initial self-supervised classifier to determine a self-supervised loss function corresponding to the initial self-supervised classifier;

[0009] input the image features into an initial detector, process the image features through the initial detector, and output predicted defect data;

[0010] determine a detection loss function based on the real defect data and the predicted defect data;

[0011] add the self-supervised loss function and the detection loss function to obtain a target loss function;

[0012] train the initial chip blur defect detection model based on the target loss function until the number of training times of the initial chip blur defect detection model reaches a preset number of times, and obtain a chip blur defect detection model.

[0013] To achieve the above object, a second aspect of the present disclosure provides a chip blur defect detection method, which comprises:

[0014] obtain a chip image of a chip to be detected, and input the chip image into a chip blur defect detection model;

[0015] process the chip image through the chip blur defect detection model to obtain at least one defect detection frame corresponding to the chip image and a defect category corresponding to each defect detection frame.

[0016] Based on the same inventive concept, a third aspect of the present disclosure provides a training device of a chip blur defect detection model, which comprises:

[0017] a data acquisition module configured to acquire an image data set and an initial chip blur defect detection model, wherein the image data set comprises image data and real defect data corresponding to the image data, the image data is image data containing at least one real defect frame, and each real defect frame corresponds to at least one blur defect category, the real defect frame and the blur defect category corresponding to the real defect frame are the real defect data, and the at least one blur defect category corresponding to each real defect frame contains the real defect category corresponding to the real defect frame;

[0018] a feature extraction module configured to extract features of the image data by using a feature extractor to obtain image features;

[0019] a self-supervised loss function determination module configured to input the image features into an initial self-supervised classifier to determine a self-supervised loss function corresponding to the initial self-supervised classifier;

[0020] a predicted defect data determination module configured to input the image features into an initial detector, process the image features through the initial detector, and output predicted defect data;

[0021] The detection loss function determination module is configured to determine a detection loss function according to the real defect data and the predicted defect data.

[0022] The target loss function determination module is configured to add the self-supervised loss function and the detection loss function to obtain a target loss function.

[0023] The training module is configured to train the initial chip blur defect detection model based on the target loss function until the initial chip blur defect detection model reaches a preset number of training times, to obtain a chip blur defect detection model.

[0024] Based on the same inventive concept, a fourth aspect of the present disclosure provides a chip blur defect detection device, comprising:

[0025] The image acquisition module is configured to acquire a chip image of a chip to be detected and input the chip image into a chip blur defect detection model.

[0026] The defect detection module is configured to process via the chip blur defect detection model to obtain at least one defect detection frame corresponding to the chip image and a defect category corresponding to each defect detection frame.

[0027] Based on the same inventive concept, a fifth aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0028] Based on the same inventive concept, a sixth aspect of the present disclosure provides a non-transitory computer readable storage medium storing computer instructions for causing a computer to execute the method described above.

[0029] It can be seen from the above that the present disclosure proposes a chip fuzzy defect detection model training method, a defect detection method and equipment. Image data set and initial chip fuzzy defect detection model are obtained. The image data set includes image data and real defect data corresponding to the image data. The image data contains at least one real defect frame, and each real defect frame corresponds to at least one fuzzy defect category. The real defect frame and the fuzzy defect category corresponding to the real defect frame are the real defect data. The at least one fuzzy defect category corresponding to each real defect frame includes the real defect category corresponding to the real defect frame. The fuzzy defect category is the defect category for fuzzy labeling of image data, that is, multiple types of defects are labeled in the image data. Subsequently, the initial chip fuzzy defect detection model is trained using fuzzy image data, so that the chip fuzzy defect detection model can accurately identify and detect the defects of the image data with fuzzy labeling, and the defect detection capability of the chip fuzzy defect detection model is improved. The image data is feature extracted by using a feature extractor to obtain image features for subsequent training of an initial self-supervised classifier and an initial detector using the image features. The image features are input into the initial self-supervised classifier to determine the self-supervised loss function corresponding to the initial self-supervised classifier. The image features are input into the initial detector, and the initial detector is processed to output predicted defect data. Based on the real defect data and the predicted defect data, a detection loss function is determined. The self-supervised loss function and the detection loss function are added to obtain a target loss function. The initial chip fuzzy defect detection model is trained based on the target loss function until the initial chip fuzzy defect detection model training times reach a preset number, and a chip fuzzy defect detection model is obtained. The self-supervised classifier and the detector are used to optimize the feature extractor to improve the representation ability of the feature extractor, and thus the extracted image features are more accurate. At the same time, the trained chip fuzzy defect detection model is used for subsequent chip fuzzy defect detection without relying on high-sensitivity detector devices for assistance and without the need for personnel involvement, thereby reducing the cost of defect detection. At the same time, the model is used for defect detection without the need to find a specific detection environment, thereby improving the efficiency of chip detection. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present disclosure or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art descriptions. Obviously, the drawings in the following description are only embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0031] Figure 1A flowchart of a training method of a chip blur defect detection model of an embodiment of the present disclosure;

[0032] Figure 2 A schematic diagram of an initial chip blur defect detection model corresponding to the present embodiment;

[0033] Figure 3 A flowchart of a chip blur defect detection method of an embodiment of the present disclosure;

[0034] Figure 4 A structural block diagram of a training device of a chip blur defect detection model of an embodiment of the present disclosure;

[0035] Figure 5 A structural block diagram of a chip blur defect detection device of an embodiment of the present disclosure;

[0036] Figure 6 A structural schematic diagram of an electronic device of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to specific embodiments and drawings.

[0038] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood as the common meanings understood by those skilled in the art to which the present disclosure belongs. The terms “first”, “second” and similar terms used in the embodiments of the present disclosure do not represent any order, number or importance, but are only used to distinguish different components. The terms “include” or “contain” and similar terms mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, without excluding other elements or objects. The terms “connect” or “connected” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “up”, “down”, “left”, “right” and the like only represent relative positional relationships, which can change accordingly when the absolute positions of the described objects change.

[0039] The terms involved in the present disclosure are explained as follows:

[0040] Mobilenet model: The Mobilenet model is a commonly used lightweight network model, the core of which is to introduce a depth separable convolution to divide a standard convolution into a depth convolution and a pointwise convolution. In the Mobilenet-SSD model, the bottom network of the Mobilenet basic classification network model is first used to extract position edges and other information, and the upper network is used to extract more concrete features.

[0041] Due to the complexity of chip manufacturing processes and the less-than-ideal production environment, some undesirable structures may occur during production, resulting in more than twenty defects such as scratches, cracks, bubbles, edge chipping, corrosion, and residual adhesive. These defects not only affect the appearance of components but also their reliability and lifespan. Therefore, surface quality inspection of chips is a crucial quality control process.

[0042] Traditional image processing-based defect detection methods can only detect defects under specific conditions, making defect detection on chips quite challenging. Furthermore, current chip defect detection requires the assistance of high-sensitivity detection devices, resulting in high costs and a significant expenditure of human resources.

[0043] Faced with diverse defect forms, accurately extracting defect attribute features in complex scenarios and improving the accuracy of feature extraction and detector performance is a challenging problem in chip defect detection.

[0044] Based on the above description, this embodiment proposes a training method for a chip fuzzy defect detection model, such as... Figure 1 As shown, the method includes:

[0045] Step 101: Obtain an image dataset and an initial chip fuzzy defect detection model. The image dataset includes image data and real defect data corresponding to the image data. The image data is image data containing at least one real defect box and each real defect box corresponds to at least one fuzzy defect category. The real defect box and the fuzzy defect category corresponding to the real defect box are the real defect data. At least one fuzzy defect category corresponding to each real defect box contains the real defect category corresponding to the real defect box.

[0046] In specific implementation, an image dataset and an initial chip fuzzy defect detection model are acquired. The initial defect model is a neural network model, taking into account the requirements for chip defect detection accuracy and real-time performance.

[0047] The image dataset includes image data and corresponding real defect data. The image data is chip surface defect images acquired by an industrial camera. The image data contains at least one real defect box and each real defect box corresponds to at least one fuzzy defect category. The real defect box and the fuzzy defect category corresponding to the real defect box are the real defect data. The at least one fuzzy defect category corresponding to each real defect box contains the real defect category corresponding to the real defect box.

[0048] For example, the image dataset contains image data A, which contains a true defect bounding box and two fuzzy defect categories, namely scratches and cracks.

[0049] Step 102: Use a feature extractor to extract features from the image data to obtain image features.

[0050] In specific implementation, a feature extractor is used to extract features from the image data to obtain image features. The feature extractor uses the Mobilenet network. In this embodiment, the Mobilenet network uses the Mobilenet_v1 version.

[0051] Step 103: Input the image features into the initial self-supervised classifier and determine the self-supervised loss function corresponding to the initial self-supervised classifier.

[0052] In specific implementation, image features are input into an initial self-supervised classifier, and the initial self-supervised classifier is trained using the image features to determine the self-supervised loss function corresponding to the initial self-supervised classifier. For the initial chip fuzzy defect detection model, the process of the self-supervised classifier optimizing the feature extractor is an auxiliary task for the initial chip fuzzy defect detection model. By locating salient targets in the image, identifying their direction and object type, and associating the object direction with the original image, the network's representation extraction capability is enhanced.

[0053] Step 104: Input the image features into the initial detector, process them through the initial detector, and output the predicted defect data.

[0054] In practice, image features are input into an initial detector, which processes the image features and outputs predicted defect data, which is then used to determine the target loss function.

[0055] Step 105: Determine the detection loss function based on the actual defect data and the predicted defect data.

[0056] In practice, the real defect data consists of the actual defect categories and locations corresponding to the image data, while the predicted defect data consists of the defect categories and locations identified using the model. By comparing the real defect data and the predicted defect data, the target loss function corresponding to the initial chip fuzzy defect detection model is determined.

[0057] Step 106: Sum the self-supervised loss function and the detection loss function to obtain the target loss function.

[0058] In practice, after determining the self-supervised loss function and the detection loss function respectively, the self-supervised loss function and the detection loss function are added together to obtain the target loss function corresponding to the initial chip fuzzy defect detection model.

[0059] Step 106: Train the initial chip fuzzy defect detection model based on the target loss function until the initial chip fuzzy defect detection model has been trained a preset number of times, thereby obtaining the chip fuzzy defect detection model.

[0060] In practice, the initial chip fuzzy defect detection model is trained based on the objective loss function. When the initial chip fuzzy defect detection model has been trained a preset number of times, the preset training termination condition is met, the initial chip fuzzy defect detection model is trained, and the chip fuzzy defect detection model is obtained.

[0061] The target loss function includes at least one of the following types: mean squared error loss function, cross-entropy loss function, logarithmic loss function, exponential loss function, squared loss function, or absolute value loss function.

[0062] The above scheme acquires an image dataset and an initial chip fuzzy defect detection model. The image dataset includes image data and corresponding real defect data. The image data contains at least one real defect box, and each real defect box corresponds to at least one fuzzy defect category. The real defect boxes and their corresponding fuzzy defect categories constitute the real defect data. Each real defect box contains at least one fuzzy defect category, which is the fuzzy defect category annotated with the image data. In other words, the image data contains multiple types of defects. The initial chip fuzzy defect detection model is then trained using the fuzzy image data, enabling it to accurately identify and detect defects in the fuzzy-annotated image data, thus improving its defect detection capability. A feature extractor is used to extract features from the image data, obtaining image features that are then used to train the initial self-supervised classifier and the initial detector. The image features are input into an initial self-supervised classifier to determine the corresponding self-supervised loss function. The image features are then input into an initial detector, which processes the data and outputs predicted defect data. Based on the actual defect data and the predicted defect data, a detection loss function is determined. The self-supervised loss function and the detection loss function are summed to obtain the target loss function. The initial chip fuzzy defect detection model is trained based on the target loss function until the initial chip fuzzy defect detection model has been trained a preset number of times, resulting in a chip fuzzy defect detection model. The feature extractor is optimized using the self-supervised classifier and detector to improve its representational ability, thereby making the extracted image features more accurate. Furthermore, subsequent chip fuzzy defect detection using the trained chip fuzzy defect detection model eliminates the need for high-sensitivity detection devices and human intervention, reducing defect detection costs. Additionally, relying on the model for defect detection eliminates the need for specific detection environments, improving chip detection efficiency.

[0063] In some embodiments, step 101 specifically includes:

[0064] Step 1011: Obtain the original image data, perform annotation processing on the original image data, and determine the actual defect data corresponding to the original image data.

[0065] Step 1012: Rotate the original image data according to multiple preset angles to obtain image data, wherein the image data includes the image corresponding to each preset angle among the multiple preset angles.

[0066] Step 1013: Construct an image dataset based on the image data and the actual defect data.

[0067] In practice, raw image data is acquired, and the raw image data is labeled to determine the actual defect data corresponding to the raw image data. The actual defect data includes at least one of the following: actual defect bounding boxes, the location of the actual defect bounding boxes, and at least one fuzzy defect category, etc.

[0068] The original image data is rotated according to multiple preset angles to obtain image data, wherein the image data contains an image corresponding to each preset angle. An image dataset is constructed based on the image data and the real defect data.

[0069] For example, multiple preset angles are {0°, 90°, 180°, 270°}. The original image data is rotated clockwise according to the multiple preset angles to obtain the original image data, the image data after rotating the original image data 90° clockwise, the image data after rotating the original image data 180° clockwise, and the image data after rotating the original image data 270° clockwise.

[0070] The above scheme involves rotating the original image data to construct an image dataset. This dataset is then used to perform auxiliary tasks based on image rotation, thereby improving the accuracy of subsequent image feature extraction.

[0071] In some embodiments, step 103 specifically includes:

[0072] Step 1031: Input the image features into the initial self-supervised classifier, and use the initial self-supervised classifier to map the image features to obtain the classification probability vector of the image corresponding to each preset angle.

[0073] Step 1032: Obtain the one-hot vector of the image corresponding to each preset angle, and determine the self-supervised loss function corresponding to the initial self-supervised classifier based on the total classification probability vector and the total one-hot vector.

[0074] In practice, the image features are input into an initial self-supervised classifier, where each image feature corresponds to image data at a preset angle. The initial self-supervised classifier is then used to map these image features, resulting in a classification probability vector for each preset angle's image data.

[0075] Obtain the one-hot vector of the image corresponding to each preset angle. The one-hot vector refers to the encoding of N states using N bits of 0 or 1. Each state has its own independent representation, and only one bit is 1 while the other bits are 0.

[0076] In this embodiment, the rotation angle corresponding to the image data can be obtained based on the one-hot vector of the image corresponding to each preset angle. For example, the one-hot vector is z. r ={z 0° ,z 90° ,z 180° ,z 270° If the one-hot vector of the first image is z r ={0,0,1,0}, which means that the rotation angle of the first image is 180 degrees, that is, the first image is obtained by rotating the original image data 180 degrees clockwise.

[0077] The self-supervised loss function corresponding to the initial self-supervised classifier is determined based on all classification probability vectors and all one-hot vectors. The self-supervised loss function is expressed by the formula:

[0078]

[0079] Where L is the self-supervised loss function, p r For classification probability vectors, It is the classification probability vector p r The j-th element in z r It is the one-hot vector corresponding to the rotated image, where r∈R, R={0 ° 90 ° 180 ° 270 °}, z r ={z 0° ,z 90° ,z 180° ,z 270°}

[0080] The above scheme uses a self-supervised auxiliary task to locate salient targets in an image, identify their orientation and object type, and associate the object orientation with the original image to enhance the network's representation extraction capabilities.

[0081] In some embodiments, the initial chip fuzzy defect detection model includes multiple convolutional layers, and step 104 specifically includes:

[0082] Step 1041: Input the image features into each of the plurality of convolutional layers respectively. For each convolutional layer:

[0083] The convolutional layer is used to extract features from the original image to obtain multiple prior boxes and initial image sub-features corresponding to the convolutional layer.

[0084] The multiple prior boxes and the initial image sub-features are input into the initial detector. After processing by the initial detector, the predicted defect box position and the predicted defect category corresponding to each of the multiple prior boxes corresponding to the convolutional layer are obtained.

[0085] Step 1042: Determine the predicted defect data based on the positions of all predicted defect boxes and the predicted defect categories corresponding to all predicted defect boxes, and output the predicted defect data.

[0086] In practice, the initial chip blur defect detection model contains multiple convolutional layers. The image features are input into each of these convolutional layers, and the convolutional layers process the image features. The scales of the multiple convolutional layers are different.

[0087] For each convolutional layer, the image features are extracted and processed to obtain multiple prior boxes and initial image sub-features corresponding to that convolutional layer. The number of prior boxes output by each convolutional layer is preset. Then, after the image features are input into the convolutional layer, the preset number of prior boxes are output. The multiple prior boxes and initial image sub-features are input into an initial detector. After processing by the initial detector, the predicted defect box position and the predicted defect category corresponding to each of the multiple prior boxes corresponding to that convolutional layer are obtained.

[0088] That is, the initial detector outputs multiple predicted defect box locations and the predicted defect categories corresponding to the multiple predicted defect boxes. The number of predicted defect box locations and the number of predicted defect categories are the same as the number of predicted defect boxes.

[0089] The predicted defect data is determined based on the positions of all predicted defect boxes and the predicted defect categories corresponding to all predicted defect boxes, and the predicted defect data is output.

[0090] In some embodiments, step 1042 specifically includes:

[0091] Step A: Determine the confidence level of the predicted defect category corresponding to each predicted defect box;

[0092] Step B: The predicted defect category with the highest confidence level is taken as the target predicted defect category, and the predicted defect box corresponding to the predicted defect category with the highest confidence level is taken as the target predicted defect box.

[0093] Step C: Based on the target predicted defect box position corresponding to the target predicted defect box and the other predicted defect box positions among all predicted defect box positions excluding the target predicted defect box position, calculate the first intersection-union ratio between the target predicted defect box and other predicted defect boxes.

[0094] Step D: Select other predicted defect boxes with a first intersection-union ratio greater than a first preset intersection-union ratio threshold as predicted defect boxes to be processed, and delete the predicted defect boxes to be processed.

[0095] Step E: Determine whether there are any remaining predicted defect boxes among all predicted defect boxes;

[0096] Step F: In response to the existence of remaining predicted defect boxes, determine the remaining predicted defect box corresponding to the predicted defect category with the highest confidence from all remaining predicted defect boxes as the new target predicted defect box. Identify and delete new predicted defect boxes to be processed until no new remaining predicted defect boxes exist. Use the target predicted defect category, target predicted defect box position, new target predicted defect category, and new target predicted defect box position as the predicted defect data; or...

[0097] Step G: In response to the absence of any remaining predicted defect boxes, the target predicted defect category and the position of the target predicted defect box are used as the predicted defect data.

[0098] In practice, the initial detector outputs multiple predicted defect box locations and the predicted defect categories corresponding to the multiple predicted defect boxes. The initial detector includes a progressive classifier and a locator. The confidence level corresponding to each predicted defect category is determined based on the classification probability vector output by the progressive classifier.

[0099] The predicted defect category with the highest confidence level is taken as the target predicted defect category, and the predicted defect box corresponding to the predicted defect category with the highest confidence level is taken as the target predicted defect box. The position of the target predicted defect box is calculated with the positions of all other predicted defect boxes except the position of the target predicted defect box, to obtain the first intersection-union ratio between the target predicted defect box and each other predicted defect box.

[0100] Each first cross-union ratio (CUNR) is compared with a first preset CUNR threshold. Other predicted defect boxes with a first CUNR greater than the first preset CUNR threshold are selected as predicted defect boxes to be processed and then deleted.

[0101] Determine if any remaining predicted defect boxes exist besides the target predicted defect box and the deleted pending predicted defect boxes from all predicted defect boxes. If remaining predicted defect boxes exist, identify the remaining predicted defect box corresponding to the predicted defect category with the highest confidence from all remaining predicted defect boxes as the new target predicted defect box. Identify and delete new pending predicted defect boxes until no new remaining predicted defect boxes exist or the number of new remaining predicted defect boxes is 1. Collect the target predicted defect category, target predicted defect box position, new target predicted defect category, and new target predicted defect box position as the predicted defect data.

[0102] If there are no remaining predicted defect boxes, the target predicted defect category and the position of the target predicted defect box are used as the predicted defect data.

[0103] For example, the number of predicted defect boxes is 10. The predicted defect box A with the highest confidence in predicting the defect category among the 10 predicted defect boxes is identified and used as the target predicted defect box. A first intersection-union ratio (IUR) is calculated between predicted defect box A and each of the other 9 predicted defect boxes. If there are 5 IUR values ​​that are all greater than a first preset IUR threshold, the predicted defect boxes corresponding to those 5 IUR values ​​are deleted.

[0104] At this point, four predicted defect boxes remain. The predicted defect box B with the highest confidence level for its predicted defect category among these four boxes is identified and designated as the new target predicted defect box. The first intersection-union ratio (IUR) is calculated between predicted defect box B and each of the other three predicted defect boxes. If two IURs are both greater than a first preset IUR threshold, the predicted defect boxes corresponding to these two IURs are deleted.

[0105] At this point, there is still one predicted defect box C remaining among all the predicted defect boxes. The predicted defect category and position of predicted defect box A, the predicted defect category and position of predicted defect box B, and the predicted defect category and position of predicted defect box C are then used as the predicted defect data.

[0106] The above scheme uses non-maximum suppression to eliminate redundant predicted defect boxes and suppress and filter redundant real defect boxes, thus ensuring the accuracy of classification by the classification network at the center of the convolution kernel.

[0107] In some embodiments, the real defect data includes the position of the real defect box corresponding to the real defect box and the fuzzy defect category corresponding to the real defect box. Step 105 specifically includes:

[0108] Step 1051: For each predicted defect box: Calculate the second intersection-union ratio corresponding to the predicted defect box based on the position of the predicted defect box and the position of the actual defect box;

[0109] Step 1052: The predicted defect boxes with a second cross-union ratio greater than the second preset cross-union ratio threshold are taken as the target predicted defect boxes;

[0110] Step 1053: Determine the detection loss function based on the target predicted defect box location, target predicted defect category, actual defect box location, and fuzzy defect category.

[0111] In practice, for each predicted defect box: based on the position of the predicted defect box and the position of the actual defect box, the second cross-union ratio (CUI) corresponding to the predicted defect box is calculated. Each second CUI is compared with a second preset CUI threshold, and predicted defect boxes with a second CUI greater than the second preset CUI threshold are selected as target predicted defect boxes.

[0112] The detection loss function is determined based on the location of the predicted defect box corresponding to the predicted defect box, the predicted defect category, the location of the actual defect box, and the fuzzy defect category.

[0113] Specifically, the target loss function is determined as follows:

[0114] A location loss function is determined based on the actual defect box location and the target predicted defect box location. The location loss function is expressed by the following formula:

[0115]

[0116] Among them, L loc (x,d,g) represents the location loss function. Pos and Neg are classified based on the intersection-union ratio (IU) of the prior bounding box and the ground truth defect bounding box. If the IU is greater than a threshold, it is classified as Pos (positive class, containing a certain object category); if the IU is less than a threshold, it is classified as Neg (negative class). m∈{c x ,c y ,w,h} are the position parameters of the anchor frame, representing the center coordinates and dimensions of the anchor frame; Predict the location of the defect box for the target; The true defect box location; the smooth L1 loss function expression is: if if When the absolute error is less than 1, the smooth L1 loss function is a quadratic function; when the absolute error is greater than 1, the smooth L1 loss function is a linear function, where k is the k-th class. Let be the similarity coefficient between the predicted defect box of the i-th target and the j-th real defect box of category k. If they are a perfect match, then... If it does not match, then S j It is the set of candidate categories for the j-th true defect box. It is the classification probability vector q of the predicted defect box of the i-th target. i The c-th element in These are the weighting coefficients.

[0117] A category loss function is determined based on the fuzzy defect category and the target predicted defect category. The category loss function is expressed by the following formula:

[0118]

[0119] Among them, L cls (x,q) is the class loss function, and c is the c-th class. Let be the similarity coefficient between the predicted defect box of the i-th target and the j-th true defect box of category c. If they are a perfect match, then... If it does not match, then S j It is the fuzzy defect category of the j-th real defect box. It is the classification probability vector q of the predicted defect box of the i-th target. i For the c-th element in the dataset, considering the ambiguity of the defect labels on the actual defect boxes, it is necessary to gradually identify the actual labels from the candidate label set. These are the weighting coefficients.

[0120] The number of predicted defect boxes is obtained, and a detection loss function is determined based on the number of predicted defect boxes, the location loss function, and the category loss function, wherein the detection loss function is expressed by the formula:

[0121]

[0122] Where L is the detection loss function and I is the number of predicted defect boxes for the target.

[0123] like Figure 2 As shown, Figure 2 A schematic diagram of the initial chip fuzzy defect detection model corresponding to this embodiment is shown. Figure 2 The SSD-PL detector is the initial detector described in this embodiment. The initial detector includes a progressive classifier and a localizer. The category loss function is the loss function corresponding to the training of the progressive classifier, and the location loss function is the loss function corresponding to the training of the localizer.

[0124] The training method for the chip blur defect detection model proposed in this disclosure employs a lightweight network model with low latency and high accuracy, resulting in a small number of network parameters and lower equipment deployment requirements. A self-supervised auxiliary task is used to locate salient targets in images, identify their orientation and object type, and associate the object orientation with the original image, enhancing the network's representation extraction capability. In real-world industrial applications, images may suffer from label blurring during data annotation. Designing a target detector based on progressive disambiguation can reduce data annotation costs and train a more discriminative target detector.

[0125] Another embodiment of this disclosure provides a method for detecting chip fuzzy defects, such as... Figure 3 As shown, the method includes:

[0126] Step 201: Obtain a chip image of the chip to be inspected, and input the chip image into the chip fuzzy defect detection model;

[0127] Step 202: Through the chip fuzzy defect detection model, at least one defect detection box corresponding to the chip image and the defect category corresponding to each defect detection box are obtained.

[0128] In practice, an industrial camera captures an image of the chip to be inspected, and this image is then input into a trained chip fuzzy defect detection model. The model processes the image to obtain at least one defect detection box corresponding to the chip image and the defect category corresponding to each box. In other words, the chip fuzzy defect detection model described in this embodiment can detect all defects contained in the chip image, and the number of defects can be one or more.

[0129] The above approach utilizes a trained chip fuzzy defect detection model for chip defect detection, eliminating the need for high-sensitivity detection devices and human intervention, thus reducing defect detection costs. Furthermore, by relying on a model for defect detection, specific detection environments are eliminated, improving chip inspection efficiency.

[0130] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0131] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a training device for a chip fuzzy defect detection model.

[0133] refer to Figure 4 , Figure 4 The training apparatus for the chip fuzzy defect detection model in this embodiment includes:

[0134] The data acquisition module 301 is configured to acquire an image dataset and an initial chip fuzzy defect detection model. The image dataset includes image data and real defect data corresponding to the image data. The image data is image data containing at least one real defect box and each real defect box corresponds to at least one fuzzy defect category. The real defect box and the fuzzy defect category corresponding to the real defect box are the real defect data. At least one fuzzy defect category corresponding to each real defect box contains the real defect category corresponding to the real defect box.

[0135] Feature extraction module 302 is configured to extract features from the image data using a feature extractor to obtain image features;

[0136] The self-supervised loss function determination module 303 is configured to input the image features into an initial self-supervised classifier and determine the self-supervised loss function corresponding to the initial self-supervised classifier;

[0137] The defect prediction data determination module 304 is configured to input the image features into the initial detector, process them through the initial detector, and output the defect prediction data.

[0138] The detection loss function determination module 305 is configured to determine the detection loss function based on the actual defect data and the predicted defect data.

[0139] The target loss function determination module 306 is configured to sum the self-supervised loss function and the detection loss function to obtain the target loss function;

[0140] The training module 307 is configured to train the initial chip fuzzy defect detection model based on the target loss function until the initial chip fuzzy defect detection model has been trained a preset number of times, thereby obtaining the chip fuzzy defect detection model.

[0141] In some embodiments, the data acquisition module 301 is specifically configured as follows:

[0142] Obtain raw image data, perform annotation processing on the raw image data, and determine the actual defect data corresponding to the raw image data;

[0143] The original image data is rotated according to multiple preset angles to obtain image data, wherein the image data includes an image corresponding to each preset angle among the multiple preset angles;

[0144] An image dataset is constructed based on the image data and the actual defect data.

[0145] In some embodiments, the self-supervised loss function determination module 303 specifically includes:

[0146] The mapping processing unit is configured to input the image features into an initial self-supervised classifier, and use the initial self-supervised classifier to perform mapping processing on the image features to obtain the classification probability vector of the image corresponding to each preset angle.

[0147] The loss function determination unit is configured to obtain the one-hot vector of the image corresponding to each preset angle, and determine the self-supervised loss function corresponding to the initial self-supervised classifier based on the total classification probability vector and the total one-hot vector.

[0148] In some embodiments, the loss function determination unit is specifically configured as follows:

[0149] The self-supervised loss function corresponding to the initial self-supervised classifier is determined based on all classification probability vectors and all one-hot vectors. The self-supervised loss function is expressed by the formula:

[0150]

[0151] Where L is the self-supervised loss function, p r For classification probability vectors, It is the classification probability vector p r The j-th element in z r Let z be the one-hot vector corresponding to the rotated image, where r∈R, R={0°,90°,180°,270°}, and z is the one-hot vector. r ={z 0° ,z 90° ,z 180° ,z270°}

[0152] In some embodiments, the initial chip fuzzy defect detection model includes multiple convolutional layers, and the predicted defect data determination module 304 specifically includes:

[0153] The input unit is configured to input the image features into each of the plurality of convolutional layers, for each convolutional layer:

[0154] The convolutional layer is used to extract features from the original image to obtain multiple prior boxes and initial image sub-features corresponding to the convolutional layer.

[0155] The multiple prior boxes and the initial image sub-features are input into the initial detector. After processing by the initial detector, the predicted defect box position and the predicted defect category corresponding to each of the multiple prior boxes corresponding to the convolutional layer are obtained.

[0156] The predicted defect data determination unit is configured to determine the predicted defect data based on the positions of all predicted defect boxes and the predicted defect categories corresponding to all predicted defect boxes, and output the predicted defect data.

[0157] In some embodiments, the predicted defect data determination unit is specifically configured to:

[0158] Determine the confidence level of the predicted defect category corresponding to each predicted defect box;

[0159] The predicted defect category with the highest confidence level is taken as the target predicted defect category, and the predicted defect box corresponding to the predicted defect category with the highest confidence level is taken as the target predicted defect box.

[0160] Based on the target predicted defect box position corresponding to the target predicted defect box, and the other predicted defect box positions among all predicted defect box positions excluding the target predicted defect box position, the first intersection-union ratio between the target predicted defect box and other predicted defect boxes is calculated.

[0161] Other predicted defect boxes with a first intersection-union ratio greater than a first preset intersection-union ratio threshold are selected as predicted defect boxes to be processed and deleted.

[0162] Determine whether there are any remaining predicted defect boxes among all predicted defect boxes;

[0163] In response to the existence of remaining predicted defect boxes, the remaining predicted defect box corresponding to the predicted defect category with the highest confidence among all remaining predicted defect boxes is identified as the new target predicted defect box. New predicted defect boxes to be processed are identified and deleted until no new remaining predicted defect boxes exist. The target predicted defect category, target predicted defect box position, new target predicted defect category, and new predicted defect box position are then used as the predicted defect data; or...

[0164] In response to the absence of any remaining predicted defect boxes, the target predicted defect category and the position of the target predicted defect box are used as the predicted defect data.

[0165] In some embodiments, the real defect data includes the position of the real defect box corresponding to the real defect box and the fuzzy defect category corresponding to the real defect box, and the detection loss function determination module 305 specifically includes:

[0166] The intersection-union ratio (CUI) calculation unit is configured to calculate, for each predicted defect box, a second CUI corresponding to the predicted defect box based on the position of the predicted defect box and the position of the actual defect box.

[0167] The target prediction defect box determination unit is configured to use the prediction defect boxes whose second intersection-union ratio is greater than the second preset intersection-union ratio threshold as target prediction defect boxes.

[0168] The loss function determination unit is configured to determine the detection loss function based on the target predicted defect box location, the target predicted defect category, the real defect box location, and the fuzzy defect category.

[0169] In some embodiments, the loss function determination unit is specifically configured as follows:

[0170] A location loss function is determined based on the actual defect box location and the target predicted defect box location. The location loss function is expressed by the following formula:

[0171]

[0172] Among them, L loc (x,d,g) represents the location loss function. Pos and Neg are classified based on the intersection-union ratio (IU) of the prior bounding box and the ground truth defect bounding box. If the IU is greater than a threshold, it is classified as Pos (positive class, containing a certain object category); if the IU is less than a threshold, it is classified as Neg (negative class). m∈{c x ,c y ,w,h} are the position parameters of the anchor frame, representing the center coordinates and dimensions of the anchor frame; Predict the location of the defect box for the target; The true defect box location; the smooth L1 loss function expression is: if if When the absolute error is less than 1, the smooth L1 loss function is a quadratic function; when the absolute error is greater than 1, the smooth L1 loss function is a linear function, where k is the k-th class. Let be the similarity coefficient between the predicted defect box of the i-th target and the j-th real defect box of category k. If they are a perfect match, then... If it does not match, then S j It is the set of candidate categories for the j-th true defect box. It is the classification probability vector q of the predicted defect box of the i-th target. i The c-th element in These are the weighting coefficients.

[0173] A category loss function is determined based on the fuzzy defect category and the target predicted defect category. The category loss function is expressed by the following formula:

[0174]

[0175] Among them, L cls (x,q) is the class loss function, and c is the c-th class. Let be the similarity coefficient between the predicted defect box of the i-th target and the j-th true defect box of category c. If they are a perfect match, then... If it does not match, then S j It is the fuzzy defect category of the j-th real defect box. It is the classification probability vector q of the predicted defect box of the i-th target. i The c-th element in These are the weighting coefficients.

[0176] The number of predicted defect boxes is obtained, and a detection loss function is determined based on the number of predicted defect boxes, the location loss function, and the category loss function, wherein the detection loss function is expressed by the formula:

[0177]

[0178] Where L is the detection loss function and I is the number of predicted defect boxes for the target.

[0179] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides a chip fuzzy defect detection device.

[0180] refer to Figure 5 , Figure 5 The chip blur defect detection device of the embodiment includes:

[0181] The image acquisition module 401 is configured to acquire a chip image of the chip to be inspected and input the chip image into the chip blur defect detection model.

[0182] The defect detection module 402 is configured to process the chip fuzzy defect detection model to obtain at least one defect detection box corresponding to the chip image and a defect category corresponding to each defect detection box.

[0183] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0184] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0185] Based on the same inventive concept, corresponding to any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the above embodiments.

[0186] Figure 6 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0187] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0188] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0189] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0190] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0191] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0192] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0193] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0194] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0195] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0196] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0197] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0198] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0199] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0200] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0201] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0202] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0203] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0204] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A training method for a chip fuzzy defect detection model, characterized in that, include: Acquire an image dataset and an initial chip fuzzy defect detection model. The image dataset includes image data and real defect data corresponding to the image data. The image data is image data containing at least one real defect box and each real defect box corresponds to at least one fuzzy defect category. The real defect box and the fuzzy defect category corresponding to the real defect box are the real defect data. At least one fuzzy defect category corresponding to each real defect box contains the real defect category corresponding to the real defect box. The image data is used to extract features using a feature extractor to obtain image features; The image features are input into an initial self-supervised classifier, and the self-supervised loss function corresponding to the initial self-supervised classifier is determined. The image features are input into the initial detector, processed by the initial detector, and the predicted defect data is output. Based on the actual defect data and the predicted defect data, determine the detection loss function; The self-supervised loss function and the detection loss function are summed to obtain the target loss function; The initial chip fuzzy defect detection model is trained based on the objective loss function until the initial chip fuzzy defect detection model is trained a preset number of times to obtain the chip fuzzy defect detection model. The actual defect data includes the actual defect box location and the fuzzy defect category corresponding to the actual defect box. The step of determining the detection loss function based on the actual defect data and the predicted defect data includes: For each predicted defect box: calculate the second intersection-union ratio corresponding to the predicted defect box based on the position of the predicted defect box and the position of the actual defect box; The predicted defect boxes whose second cross-union ratio is greater than the second preset cross-union ratio threshold are used as the target predicted defect boxes; The detection loss function is determined based on the location of the predicted defect box, the category of the predicted defect, the location of the actual defect box, and the category of the fuzzy defect. The step of determining the detection loss function based on the target predicted defect box location, the target predicted defect category, the actual defect box location, and the fuzzy defect category includes: A location loss function is determined based on the actual defect box location and the target predicted defect box location. The location loss function is expressed by the following formula: in, The location loss function, Pos and Neg, is used to classify objects based on the intersection-union ratio (IU) of the prior bounding box and the ground truth defect bounding box. If the IU is greater than a threshold, it is classified as Pos (positive class, containing a certain object category); if the IU is less than a threshold, it is classified as Neg (negative class). These are the position parameters of the anchor frame, representing the center coordinates and dimensions of the anchor frame; Predict the location of the defect box for the target; The actual defect bounding box location; smooth The loss function expression is: ,if ; if When the absolute value error is less than 1, smoothness The loss function is a quadratic function; when the absolute value error is greater than 1, smoothness is achieved. The loss function is a linear function, and k is the k-th class. For the first The target prediction defect box and the category k are the first The similarity coefficient that matches each real defect box; if there is a perfect match, then... If it does not match, then , It is the first A set of candidate categories for each real defect bounding box. It is the first The classification probability vector of each target predicted defect box The first in One element, These are the weighting coefficients. ; A category loss function is determined based on the fuzzy defect category and the target predicted defect category. The category loss function is expressed by the following formula: in, For the category loss function, It is the first kind, For the first Predicting defect boxes and categories for each target The The similarity coefficient that matches each real defect box; if there is a perfect match, then... If it does not match, then , It is the first Fuzzy defect categories for each real defect bounding box. It is the first The classification probability vector of each target predicted defect box The first in One element, These are the weighting coefficients. ; The number of predicted defect boxes is obtained, and a detection loss function is determined based on the number of predicted defect boxes, the location loss function, and the category loss function, wherein the detection loss function is expressed by the formula: in, To detect the loss function, Predict the number of defect boxes for the target.

2. The method according to claim 1, characterized in that, The acquisition of the image dataset includes: Obtain raw image data, perform annotation processing on the raw image data, and determine the actual defect data corresponding to the raw image data; The original image data is rotated according to multiple preset angles to obtain rotated image data, wherein the rotated image data contains an image corresponding to each preset angle among the multiple preset angles; An image dataset is constructed based on the original image data, the rotated image data, and the actual defect data. The step of using a feature extractor to extract features from the image data to obtain image features includes: The original image data is used to extract features using a feature extractor to obtain the original image features; The rotating image data is used to extract features using a feature extractor to obtain rotating image features; The original image features and the rotated image features are used as image features.

3. The method according to claim 2, characterized in that, The step of inputting the image features into an initial self-supervised classifier and determining the self-supervised loss function corresponding to the initial self-supervised classifier includes: The rotated image features are input into an initial self-supervised classifier, and the initial self-supervised classifier is used to map the rotated image features to obtain the classification probability vector of the image corresponding to each preset angle. Obtain the one-hot vector of the image corresponding to each preset angle, and determine the self-supervised loss function corresponding to the initial self-supervised classifier based on the total classification probability vector and the total one-hot vector.

4. The method according to claim 3, characterized in that, The step of determining the self-supervised loss function corresponding to the initial self-supervised classifier based on all classification probability vectors and all one-hot vectors includes: The self-supervised loss function corresponding to the initial self-supervised classifier is determined based on all classification probability vectors and all one-hot vectors. The self-supervised loss function is expressed by the formula: Where L is the self-supervised loss function, For classification probability vectors, It is a classification probability vector The first in One element, It is the one-hot vector corresponding to the rotated image, where, , , .

5. The method according to claim 2, characterized in that, The initial chip fuzzy defect detection model contains multiple convolutional layers. The process of inputting the image features into an initial detector, processing them, and outputting predicted defect data includes: The original image features are input into each of the multiple convolutional layers, and for each convolutional layer: The convolutional layer is used to extract features from the original image to obtain multiple prior boxes and initial image sub-features corresponding to the convolutional layer. The multiple prior boxes and the initial image sub-features are input into the initial detector. After processing by the initial detector, the predicted defect box position and the predicted defect category corresponding to each of the multiple prior boxes corresponding to the convolutional layer are obtained. The predicted defect data is determined based on the positions of all predicted defect boxes and the predicted defect categories corresponding to all predicted defect boxes, and the predicted defect data is output.

6. The method according to claim 5, characterized in that, The step of determining the predicted defect data based on the locations of all predicted defect boxes and all predicted defect categories includes: Determine the confidence level of the predicted defect category corresponding to each predicted defect box; The predicted defect category with the highest confidence level is taken as the target predicted defect category, and the predicted defect box corresponding to the predicted defect category with the highest confidence level is taken as the target predicted defect box. Based on the target predicted defect box position corresponding to the target predicted defect box, and the other predicted defect box positions among all predicted defect box positions excluding the target predicted defect box position, the first intersection-union ratio between the target predicted defect box and other predicted defect boxes is calculated. Other predicted defect boxes with a first intersection-union ratio greater than a first preset intersection-union ratio threshold are selected as predicted defect boxes to be processed and deleted. Determine whether there are any remaining predicted defect boxes among all predicted defect boxes; In response to the existence of remaining predicted defect boxes, the remaining predicted defect box corresponding to the predicted defect category with the highest confidence among all remaining predicted defect boxes is identified as the new target predicted defect box. New predicted defect boxes to be processed are identified and deleted until no new remaining predicted defect boxes exist. The target predicted defect category, target predicted defect box position, new target predicted defect category, and new predicted defect box position are then used as the predicted defect data; or... In response to the absence of any remaining predicted defect boxes, the target predicted defect category and the position of the target predicted defect box are used as the predicted defect data.

7. A method for detecting fuzzy defects in chips, characterized in that, The chip fuzzy defect detection model obtained by applying the training method of the chip fuzzy defect detection model according to any one of claims 1 to 6 includes: Obtain a chip image of the chip to be inspected, and input the chip image into the chip fuzzy defect detection model; The chip fuzzy defect detection model is used to process the chip image to obtain at least one defect detection box and the defect category corresponding to each defect detection box.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 7.

Images