Ingot defect detection method, device and system
By assigning the ingot defect detection task to three different models, each focusing on a specific task, the problems of missed detection and false detection in ingot defect detection in the existing technology are solved, and more efficient and accurate defect identification and positioning are achieved.
Patent Information
- Application Number
- CN202510591790.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2025-09-26
AI Technical Summary
Existing ingot defect detection methods have problems of missed detection and false detection, making it difficult to accurately identify and locate defects.
Three different models are used to respectively handle the detection tasks of whether the ingot has defects, defect location and defect type, namely binary classification model, defect location model and defect classification model. By separating the tasks, the detection efficiency and accuracy of each model are improved.
The accuracy of ingot defect detection is improved, the probability of missed detection and false detection is reduced, and the detection efficiency is improved.
Smart Images
Figure CN120703090A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of defect detection technology, and in particular relates to a method, device and system for defect detection of an ingot. Background Art
[0002] Due to the difficulty of manufacturing ingots, defects are easily formed during the production process. The accuracy of defect detection has a vital impact on the subsequent processing of the ingots and the optimization of the production process. Currently, the defect detection methods for ingots have many missed detections and false detections. Summary of the Invention
[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a method, device and system for detecting defects in ingots, wherein each model focuses on a specific detection task, thereby reducing the probability of missed defect detection and false detection.
[0004] In a first aspect, the present application provides a method for detecting defects in an ingot, the method comprising:
[0005] Acquiring an image of an ingot to be inspected, wherein the ingot image includes a target ingot;
[0006] Inputting the ingot image into a binary classification model to obtain a classification result output by the binary classification model, the classification result including whether the target ingot has a defect or whether the target ingot does not have a defect;
[0007] When the classification result indicates that the target ingot has a defect, inputting the ingot image into a defect localization model to obtain a defect detection area output by the defect localization model;
[0008] The defect image corresponding to the defect detection area is input into the defect classification model, and the type of defects in the defect image is analyzed to obtain a defect type label output by the defect classification model. The defect image is obtained by cropping the ingot image, and the defect detection result of the target ingot includes the defect detection area and the defect type label.
[0009] According to the defect detection method of the ingot of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of the defects, and detecting the type of defects to three different models for execution, each model focuses on a specific detection task, and can more deeply learn and optimize the characteristics and patterns of the corresponding tasks, thereby improving the execution efficiency and detection accuracy of each task, and reducing the probability of missed defect detection and false detection.
[0010] According to one embodiment of the present application, after inputting the ingot image into the defect localization model and before inputting the defect image corresponding to the defect detection area into the defect classification model, the method further includes:
[0011] In a case where the defect localization model does not output a defect detection area, it is determined that the target ingot has no defects.
[0012] According to one embodiment of the present application, after inputting the ingot image into the binary classification model and before inputting the ingot image into the defect localization model, the method further includes:
[0013] If the classification result is that the target ingot does not have defects, it is determined that the target ingot does not have defects.
[0014] According to one embodiment of the present application, acquiring an image of the ingot to be inspected includes:
[0015] Acquiring a plurality of partial images of an ingot, wherein the partial images of the ingot include partial information of the target ingot;
[0016] The ingot image is obtained by splicing a plurality of partial images of the ingot.
[0017] According to one embodiment of the present application, the binary classification model is trained based on images of sample ingots with defects and images of sample ingots without defects.
[0018] According to one embodiment of the present application, the defect localization model is trained unsupervised using sample ingot images without defects.
[0019] In a second aspect, the present application provides a device for detecting defects in an ingot, the device comprising:
[0020] an acquisition module, configured to acquire an image of an ingot to be detected, wherein the ingot image includes a target ingot;
[0021] a first processing module, configured to input the ingot image into a binary classification model to obtain a classification result output by the binary classification model, wherein the classification result includes whether the target ingot has a defect or whether the target ingot does not have a defect;
[0022] a second processing module, configured to, when the classification result indicates that the target ingot has a defect, input the ingot image into a defect localization model to obtain a defect detection area output by the defect localization model;
[0023] a third processing module, configured to input the defect image corresponding to the defect detection area into a defect classification model, analyze the type of defects in the defect image, and obtain a defect type label output by the defect classification model, wherein the defect image is obtained by cropping the ingot image, and the defect detection result of the target ingot includes the defect detection area and the defect type label.
[0024] According to the defect detection device for ingots of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of defects, and detecting the type of defects to three different models for execution, each model focuses on a specific detection task, and can more deeply learn and optimize the characteristics and patterns of the corresponding tasks, thereby improving the execution efficiency and detection accuracy of each task, and reducing the probability of missed defect detection and false detection.
[0025] In a third aspect, the present application provides a defect detection system for an ingot, comprising:
[0026] An image acquisition device, used for acquiring images of the ingot;
[0027] As in the ingot defect detection device described in the second aspect above, the ingot defect detection device is connected to the image acquisition device.
[0028] According to the defect detection system for ingots of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of defects, and detecting the type of defects to three different models for execution, each model focuses on a specific detection task, and can more deeply learn and optimize the characteristics and patterns of the corresponding tasks, thereby improving the execution efficiency and detection accuracy of each task, and reducing the probability of missed defect detection and false detection.
[0029] According to one embodiment of the present application, the image acquisition device is provided on a moving mechanism, and the moving mechanism is used to drive the image acquisition device to move to acquire multiple local images of the ingot, wherein the local images of the ingot include local information of the target ingot, and the ingot image is obtained by splicing multiple local images of the ingot.
[0030] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting defects in ingots as described in the first aspect above is implemented.
[0031] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for detecting defects in an ingot as described in the first aspect above is implemented.
[0032] In a sixth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the ingot defect detection method as described in the first aspect above.
[0033] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0035] Figure 1 This is one of the flow charts of the ingot defect detection method provided in the embodiment of the present application;
[0036] Figure 2 This is the second flow chart of the ingot defect detection method provided in the embodiment of the present application;
[0037] Figure 3 This is the third flow chart of the ingot defect detection method provided in the embodiment of the present application;
[0038] Figure 4 This is the fourth flow chart of the ingot defect detection method provided in the embodiment of the present application;
[0039] Figure 5 1 is a schematic structural diagram of an ingot defect detection device provided in an embodiment of the present application;
[0040] Figure 6 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.
[0042] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims refers to at least one of the connected objects, and the character " / " generally indicates that the objects connected are in an "or" relationship.
[0043] The following describes in detail the ingot defect detection method, ingot defect detection device, ingot defect detection system, electronic device and readable storage medium provided by the embodiments of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0044] The defect detection method for ingots provided in the embodiments of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the defect detection method for ingots. The electronic devices mentioned in the embodiments of the present application include but are not limited to computers, etc. The defect detection method for ingots provided in the embodiments of the present application is described below using an electronic device as an example of the execution subject.
[0045] like Figure 1 As shown, the ingot defect detection method includes: step 110, step 120 and step 130.
[0046] Step 110 : Acquire an image of the ingot to be inspected, where the ingot image includes the target ingot.
[0047] The ingot image is an image obtained by photographing a target ingot, and the ingot image includes the target ingot, which is the ingot to be inspected for defects.
[0048] It is understandable that the target ingot may include multiple surfaces, and the ingot image may be an image obtained by photographing the bottom of the target ingot. The ingot defect detection method provided in the embodiment of the present application can perform defect detection on the bottom of the ingot.
[0049] In this step, an image of the ingot may be captured by an image capture device.
[0050] Step 120: Input the ingot image into the binary classification model to obtain a classification result output by the binary classification model.
[0051] The classification results include whether the target ingot has defects or not, and the binary classification model is a model for detecting whether the target ingot has defects.
[0052] In this step, the ingot image is input into a binary classification model, the features of the ingot image are extracted by the binary classification model, and the extracted features are analyzed. The binary classification model outputs a probability value that characterizes the presence of defects in the target ingot. When the probability value is greater than the set probability threshold, the target ingot is considered to have defects, otherwise it is considered that the target ingot does not have defects.
[0053] Step 130 : When the classification result indicates that the target ingot has defects, the ingot image is input into a defect location model to obtain a defect detection area output by the defect location model.
[0054] The defect location model is a model that can detect whether a target ingot has defects and can locate the defect position of the target ingot, and the defect detection area is an area in the target ingot where defects exist.
[0055] In this embodiment, when the binary classification model detects that the target ingot has defects, the ingot image is input into the defect localization model, the defects of the target ingot are detected by the defect localization model, and the area where the defects exist is located and divided to obtain a defect detection area.
[0056] It should be noted that the binary classification model detects that there are defects in the target ingot. The defects in the target ingot may be small and can be ignored when the defect localization model performs defect area detection, so that the defect detection area cannot be detected in the ingot image. When the defect detection area is detected in the ingot image, it indicates that the defects in the target ingot are large and cannot be ignored.
[0057] Step 140: Input the defect image corresponding to the defect detection area into the defect classification model, analyze the type of defects in the defect image, and obtain the defect type label output by the defect classification model. The defect detection result of the target ingot includes the defect detection area and the defect type label.
[0058] The defect image is an image including a defect detection area, the defect classification model is a model that can detect defect categories, and the defect type label is a label indicating the defect type of the defect detection area.
[0059] It can be understood that the defect image is obtained by cropping the ingot image.
[0060] In this step, when a defect detection area is detected, the defect image corresponding to the defect detection area is input into the defect classification model, features are extracted from the defect image by the defect classification model, and the defects are classified into predefined cracks, dents or scratches, etc. according to the extracted features, and a defect type label corresponding to the defect image is assigned.
[0061] In this embodiment, the defect detection result includes a defect detection area and a defect type label. The defect detection area and the defect type label correspond one to one. The location of the defect in the target ingot and the type of the defect can be determined based on the defect detection area and the corresponding defect type label.
[0062] In related technologies, a target detection model is typically used to detect defects in ingots and obtain defect-labeled images. This model needs to handle multiple tasks simultaneously, which increases the complexity and burden of the model. Each task has its own focus, and a single model may not be able to achieve optimal performance on all tasks. Furthermore, different tasks may require the extraction of different types of features. When balancing these different requirements, a single model may encounter feature conflicts, resulting in performance degradation for certain tasks. In addition, in order to train a model that can handle multiple tasks simultaneously, more labeled data is required, which not only increases the difficulty of data collection but also may introduce more labeling errors, affecting the training effect of the model. Ingots may have multiple defects during the production process. For complex defects, a single target detection model handles multiple tasks simultaneously and analyzes different types of features at the same time. In addition, the training effect is poor and there may be more missed detections and false detections.
[0063] In an embodiment of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of defects, and detecting the type of defects to three different models for execution, each model can focus more on its own task, reduce the complexity and burden caused by multi-tasking, and improve the performance of model task processing; by separating tasks, each model can more effectively extract and utilize features suitable for its task, reduce feature conflicts, and improve the performance of each task; even if each model still requires corresponding labeled data, the collection and labeling of data can be more targeted, reducing the complexity and risk of labeling errors caused by multi-task data requirements. Each model only needs to focus on the data related to its own task, making the data management and labeling process more efficient and accurate, thereby improving the model training effect. Each model focuses on a specific detection task and can learn and optimize the features and patterns of the corresponding task more deeply, thereby improving the execution efficiency and detection accuracy of each task, and reducing the probability of missed defect detection and false detection.
[0064] According to the defect detection method for ingots provided in an embodiment of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of defects, and detecting the type of defects to three different models for execution, each model focuses on a specific detection task, and can more deeply learn and optimize the characteristics and patterns of the corresponding tasks, thereby improving the execution efficiency and detection accuracy of each task, and reducing the probability of missed defect detection and false detection.
[0065] In some embodiments, after inputting the ingot image into the defect localization model and before inputting the defect image corresponding to the defect detection area into the defect classification model, the method further includes:
[0066] In the case that the defect localization model does not output the defect detection area, it is determined that the target ingot does not have defects.
[0067] In this embodiment, after the binary classification model detects that there is a defect in the target ingot, the ingot image is input into the defect localization model. The defect in the target ingot may be small and can be ignored when the defect localization model performs defect area detection, so that the defect detection area cannot be detected in the ingot image. The defect localization model does not output the defect detection area, and the defect detection result can be determined as no defect in the target ingot, that is, the smaller defect detected by the binary classification model is identified as a non-defect.
[0068] In this embodiment, after the binary classification model detects that the target ingot has defects, the ingot image is input into the defect localization model. The defect localization model focuses on the ingots in which defects have been detected, and can analyze the size and area of the defects more carefully, which can effectively reduce the false alarms that may be generated by the binary classification model regarding whether the target ingot has defects.
[0069] In some embodiments, after inputting the ingot image into the binary classification model and before inputting the ingot image into the defect localization model, the method further comprises:
[0070] In a case where the classification result is that the target ingot does not have a defect, it is determined that the target ingot does not have a defect.
[0071] In this embodiment, the binary classification model detects whether the target ingot has defects. When it is detected that the target ingot does not have defects, the defect detection result is directly confirmed as the target ingot does not have defects, and the ingot image is not input into the defect location model. This can reduce unnecessary further analysis of the non-defective ingot, thereby improving detection efficiency.
[0072] In some embodiments, acquiring an image of an ingot to be inspected includes:
[0073] Acquire multiple partial images of the ingot;
[0074] Multiple ingot partial images are spliced together to obtain an ingot image.
[0075] The partial image of the ingot includes partial information of the target ingot, that is, the partial image of the ingot includes a partial area of the target ingot.
[0076] For example, the bottom of the target ingot is photographed and divided into 4 areas in the horizontal direction and 4 areas in the vertical direction, so that the bottom of the target ingot is divided into 16 areas of 4×4 in total. A local image of the ingot is taken for each divided area, and adjacent parts of the local images of the ingots in adjacent areas may have overlapping information.
[0077] In this embodiment, multiple partial images of the ingot can be stitched together in sequence according to corresponding regional positions to obtain an ingot image, that is, based on the local information captured by each partial image of the ingot, a complete view of the entire target ingot can be restored by combining and stitching to obtain an ingot image.
[0078] In actual implementation, the local images of the ingots corresponding to the local areas of each row can be horizontally spliced, two local images of the ingots are spliced each time, and the splicing is performed from left to right. The scale-invariant feature transformation method is used to detect feature points, and then the random sampling consistency algorithm is used to achieve feature description matching of the horizontal images, and finally the horizontal image splicing is achieved.
[0079] After the partial images of the ingot in each row are horizontally stitched, each row corresponds to a stitched image, and the multiple stitched images are stitched from top to bottom using the same method as the horizontal stitching, and finally a complete ingot image is output.
[0080] In this embodiment, a local image of the ingot is obtained, which can capture details more finely. Multiple local images of the ingot are spliced together. Compared with directly obtaining a complete image of the ingot, a higher resolution and clearer ingot image can be obtained after splicing, thereby improving the accuracy of defect detection.
[0081] In some embodiments, the binary classification model is trained based on images of sample ingots with defects and images of sample ingots without defects.
[0082] The sample ingot image includes a sample ingot, which may be an ingot of the same model and material as the target ingot. The sample ingot in the defective sample ingot image has defects, and the sample ingot in the non-defective sample ingot image does not have defects.
[0083] In this embodiment, the binary classification model may be supervisedly trained based on images of sample ingots with defects and images of sample ingots without defects.
[0084] The defective sample ingot image may include a sample ingot image obtained by photographing a defective ingot, a sample ingot image obtained by simulating possible defects in the ingot, and a sample ingot image having an unusual residual compared to a sample ingot image without defects in the feature space.
[0085] In this embodiment, the binary classification model is trained based on images of sample ingots with defects and images of sample ingots without defects. The binary classification model can learn to identify whether an ingot has defects.
[0086] The following describes a specific implementation example for training a binary classification model.
[0087] The binary classification model may be a distribution-relevant attack model (Dra).
[0088] like Figure 2 As shown, in step S13, the abnormal image, ie, the image of the sample ingot with the defect label and the image of the sample ingot without the defect are input.
[0089] Step S14: Generate a known abnormal image, i.e., a known defect image. The known defect is a defect directly given in the training data, i.e., a defect in a sample ingot image with a defect label. The known defect image corresponds to a sample ingot image obtained by photographing an ingot with a defect.
[0090] Step S15: Generate a pseudo-abnormal image, i.e., a pseudo-defect image. The pseudo-defect is a simulated defect generated by an external data source, not an actual defect sample, but is equivalent to a defect image after data enhancement. The pseudo-defect image corresponds to a sample ingot image obtained by simulating possible defects in the ingot.
[0091] Step S16: Generate a potential residual abnormality image, i.e., a potential residual defect image. A potential residual defect refers to a sample having unusual residuals compared with a normal sample in the feature space. The potential residual defect image corresponds to a sample ingot image having unusual residuals compared with a sample ingot image without defects in the feature space.
[0092] Step S17: Input the images generated in steps S14, S15, and S16 and the image of the sample ingot without defects into a feature extraction network, and a residual neural network (ResNet) 18 model can be used as the feature extraction network.
[0093] Step S18: After training with a known anomaly learning head, the known anomaly learning head, namely the known defect learning head, is specifically used to learn representations of known defects and learn how to distinguish normal samples from known defect samples.
[0094] Step S19: After training with a pseudo-anomaly learning head, the pseudo-anomaly learning head, i.e., the pseudo-defect learning head, is specifically designed to learn representations of pseudo-defects, which can help the model learn a wider range of defect features.
[0095] Step S20: After training with a latent residual abnormality learning head, the latent residual abnormality learning head, namely the latent residual defect learning head, is specifically used to learn residuals based on latent features, which helps to capture subtle defects that are not easily noticeable in the original feature space.
[0096] Step S21: Output the label of the binary classification prediction result, where 0 represents normal and 1 represents defective.
[0097] In some embodiments, the defect localization model is trained unsupervised using images of sample ingots without defects.
[0098] In this embodiment, the defect localization model is trained to identify image features in which no defects exist. When the defect localization model performs defect localization, the image features of the ingot image are compared with the learned image features in which no defects exist, and the area different from the learned features is marked as a defect detection area. The defect detection area can be composed of multiple pixels with abnormal features.
[0099] A specific embodiment of training a defect localization model is introduced below.
[0100] The defect location model may be a simple network (SimpleNet) model.
[0101] like Figure 3 As shown, in step S22, only normal images, that is, images of sample ingots without defects, are used as input for model training.
[0102] Step S23: Input the image of the sample ingot without defects into a feature extractor. The feature extractor may use a pre-trained WideResNet-50 or WideResNet-101 or other wider residual network.
[0103] Step S24 : extracting features of the sample ingot image using the network of step S23 to obtain a local feature map of the sample ingot image.
[0104] Step S25: The feature adapter can transfer the trained features to the features of the target domain, and can use a fully connected layer or a multi-layer perceptron.
[0105] Step S26: After the feature adapter is used to adapt the feature in step S25, an output image after the feature adaptation is output.
[0106] Step S27: Add Gaussian noise to the output image after feature adaptation to generate a pseudo-defect feature adaptation image for use in model training.
[0107] Step S28: simultaneously inputting the pseudo-defect feature adaptation map with Gaussian noise added and the normal feature adaptation map into the discriminator to train the discriminator.
[0108] Step S29: judging whether the input ingot image has any abnormality, that is, whether it has any defects, based on the output of the discriminator.
[0109] Step S30: If there is no abnormality, a label indicating that the ingot image has no abnormality is output.
[0110] Step S31: If there is an abnormality, a mask image corresponding to the abnormality of the ingot image is output.
[0111] Step S32: locate the abnormal area, that is, the defect detection area, according to the output mask image, and obtain the position information of the abnormal area in the image.
[0112] In related technologies, defect detection of ingots mostly uses a single target detection model, such as yolov5, yolov8 and Region with Convolutional Neural Networks (RCNN), etc., which may result in many missed detections and false detections in actual complex images.
[0113] The defect detection method for ingots provided in the embodiment of the present application first uses a binary classification model to detect whether the target ingot in the ingot image has defects, then uses a defect localization model to locate the specific defect position, and finally uses a defect classification model to implement defect classification. This can ensure to the greatest extent that the missed detection and false detection rates are reduced in complex ingot images, while improving the flexibility of the detection method.
[0114] A specific embodiment of a method for detecting defects in an ingot is described below.
[0115] like Figure 4 As shown, in step S1, the bottom of the target ingot is divided into 4 areas in the horizontal direction and 4 areas in the vertical direction, so that the bottom of the target ingot is divided into 16 areas of 4×4 in total, and a local image of the ingot is captured for each divided area, thereby obtaining 16 local images of the ingot.
[0116] Step S2: stitching the 16 partial images of the ingots. The partial images of the ingots corresponding to the local areas of each row can be stitched horizontally, two partial images of the ingots are stitched each time, and the stitching is performed from left to right. The scale-invariant feature transformation method is used to detect feature points, and then the random sampling consistency algorithm is used to achieve feature description matching of the horizontal images, and finally the horizontal images are stitched.
[0117] After the partial images of the ingot in each row are horizontally stitched, each row corresponds to a stitched image, and the multiple stitched images are stitched from top to bottom using the same method as the horizontal stitching, and finally a complete ingot image is output.
[0118] Step S3: Processing the ingot image outputted in step S2, using an image filter such as a Gaussian filter to smooth the image and remove some uneven points.
[0119] Step S4: input the image of the ingot after image processing into a trained binary classification anomaly detection model, i.e., a binary classification model.
[0120] Step S5: Determine whether an abnormality is detected based on the output of the binary classification model in step S4, that is, whether the bottom of the target ingot has a defect.
[0121] Step S6: If it is determined as no defect in step S5, the result display image has no abnormality, that is, the detection result that the bottom of the target ingot has no defect is output.
[0122] Step S7: If the ingot is judged to be defective in step S5, the ingot image is further input into a trained anomaly detection model with an abnormal position output, namely a defect localization model. The output of the defect localization model can not only distinguish whether there is a defect, but also determine the size and location of the defect.
[0123] Step S8: Determine whether an abnormality is detected in step S7. If no abnormality is detected, go to step S6; if an abnormality is detected, go to step S9.
[0124] Step S9: According to step S7, the defect detection area is acquired, the specific defect position is located, and it is cut out to obtain a defect image.
[0125] Step S10: Input the defect image intercepted in step S9 into the trained defect classification model. The defect classification model is trained using the intercepted image containing the ingot defect. The defect classification model has the characteristics of having pre-trained parameters and a moderate number of model layers. The defect classification model can be a classification model such as a visual geometry group (vgg16) network and a residual network (ResNet18) for classification.
[0126] Step S11 : Match the classification result obtained in step S10 with the captured image, that is, match the defect type label with the defect detection area, and record them.
[0127] Step S12: Output an image with defect annotations based on the defect type label and the corresponding defect detection area in step S11.
[0128] Based on the defect detection method for the ingot provided in the embodiment of the present application, the ingot image was collected by an industrial camera, and two rounds of tests were performed on various defects on the back of the ingot. The test results are shown in Table 1. It can be seen from Table 1 that the overall accuracy of the defect detection method for the ingot provided in the embodiment of the present application is high.
[0129] Table 1
[0130]
[0131] The ingot defect detection method provided in the embodiment of the present application can be executed by an ingot defect detection device. In the embodiment of the present application, the ingot defect detection method performed by the ingot defect detection device is taken as an example to illustrate the ingot defect detection device provided in the embodiment of the present application.
[0132] An embodiment of the present application also provides a device for detecting defects in an ingot.
[0133] like Figure 5 As shown, the defect detection device for the ingot includes:
[0134] An acquisition module 510 is configured to acquire an image of an ingot to be detected, wherein the ingot image includes a target ingot;
[0135] A first processing module 520 is configured to input the ingot image into a binary classification model to obtain a classification result output by the binary classification model, wherein the classification result includes whether the target ingot has defects or not;
[0136] The second processing module 530 is configured to input the ingot image into a defect localization model to obtain a defect detection area output by the defect localization model when the classification result indicates that the target ingot has a defect;
[0137] The third processing module 540 is used to input the defect image corresponding to the defect detection area into the defect classification model, analyze the type of defects in the defect image, and obtain the defect type label output by the defect classification model. The defect image is obtained by cropping the ingot image. The defect detection result of the target ingot includes the defect detection area and the defect type label.
[0138] According to the defect detection device for ingots provided in an embodiment of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of defects, and detecting the type of defects to three different models for execution, each model focuses on a specific detection task, and can more deeply learn and optimize the characteristics and patterns of the corresponding tasks, thereby improving the execution efficiency and detection accuracy of each task and reducing the probability of missed defect detection and false detection.
[0139] In some embodiments, the second processing module 530 is further configured to determine that the target ingot does not have defects when the defect localization model does not output a defect detection region.
[0140] In some embodiments, the first processing module 520 is further configured to determine that the target ingot does not have defects when the classification result indicates that the target ingot does not have defects.
[0141] In some embodiments, the acquisition module 510 is configured to acquire a plurality of ingot partial images, where the ingot partial images include partial information of the target ingot;
[0142] Multiple ingot partial images are spliced together to obtain an ingot image.
[0143] In some embodiments, the binary classification model is trained based on images of sample ingots with defects and images of sample ingots without defects.
[0144] In some embodiments, the defect localization model is trained unsupervised using images of sample ingots without defects.
[0145] The ingot defect detection device in the embodiment of the present application may be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip.
[0146] The ingot defect detection device in the embodiment of the present application can be a device having an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0147] The defect detection device for ingots provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented in the method embodiment are not described here.
[0148] The embodiment of the present application also provides a defect detection system for an ingot.
[0149] The defect detection system comprises an image acquisition device and the above-mentioned ingot defect detection device, and the ingot defect detection device is connected to the image acquisition device.
[0150] The image acquisition device is used to acquire images of the ingot.
[0151] According to the defect detection system for ingots provided in an embodiment of the present application, by assigning the processes of detecting whether the target ingot has defects, detecting the location of defects, and detecting the type of defects to three different models for execution, each model focuses on a specific detection task, and can more deeply learn and optimize the characteristics and patterns of the corresponding tasks, thereby improving the execution efficiency and detection accuracy of each task and reducing the probability of missed defect detection and false detection.
[0152] In some embodiments, the image acquisition device is provided on a moving mechanism, which is used to drive the image acquisition device to move to acquire multiple local images of the ingot, the local images of the ingot including local information of the target ingot, and the ingot image is obtained by splicing multiple local images of the ingot.
[0153] Among them, the moving mechanism is a mechanical device that can drive an image acquisition device such as a camera or a scanner to move in space according to a predetermined path or pattern. The movement can be linear, rotational, or a combination of the two, so that the image acquisition device can capture the target ingot at different positions to obtain a local image of the ingot.
[0154] In some embodiments, as Figure 6 As shown, an embodiment of the present application further provides an electronic device 600, including a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the program is executed by the processor 601, each process of the above-mentioned ingot defect detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0155] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.
[0156] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned ingot defect detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0157] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0158] An embodiment of the present application further provides a computer program product, including a computer program, which implements the above-mentioned ingot defect detection method when executed by a processor.
[0159] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0160] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned ingot defect detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0161] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0162] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0164] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.
[0165] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
[0166] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A method for detecting defects in an ingot, characterized in that: include: Acquiring an image of an ingot to be inspected, wherein the ingot image includes a target ingot; Inputting the ingot image into a binary classification model to obtain a classification result output by the binary classification model, the classification result including whether the target ingot has a defect or whether the target ingot does not have a defect; When the classification result indicates that the target ingot has a defect, inputting the ingot image into a defect localization model to obtain a defect detection area output by the defect localization model; The defect image corresponding to the defect detection area is input into the defect classification model, and the type of defects in the defect image is analyzed to obtain a defect type label output by the defect classification model. The defect image is obtained by cropping the ingot image, and the defect detection result of the target ingot includes the defect detection area and the defect type label.
2. The method for detecting defects in an ingot according to claim 1, wherein: After inputting the ingot image into the defect localization model and before inputting the defect image corresponding to the defect detection area into the defect classification model, the method further includes: In a case where the defect localization model does not output a defect detection area, it is determined that the target ingot has no defects.
3. The method for detecting defects in an ingot according to claim 1, wherein: After inputting the ingot image into the binary classification model and before inputting the ingot image into the defect localization model, the method further includes: If the classification result is that the target ingot does not have defects, it is determined that the target ingot does not have defects.
4. The method for detecting defects in an ingot according to any one of claims 1 to 3, characterized in that: The step of obtaining an image of an ingot to be detected includes: Acquiring a plurality of partial images of an ingot, wherein the partial images of the ingot include partial information of the target ingot; The ingot image is obtained by splicing a plurality of partial images of the ingot.
5. The method for detecting defects in an ingot according to any one of claims 1 to 3, characterized in that: The binary classification model is trained based on images of sample ingots with defects and images of sample ingots without defects.
6. The method for detecting defects in an ingot according to any one of claims 1 to 3, characterized in that: The defect localization model is trained unsupervised using images of sample ingots without defects.
7. A defect detection device for an ingot, characterized in that: include: an acquisition module, configured to acquire an image of an ingot to be detected, wherein the ingot image includes a target ingot; a first processing module, configured to input the ingot image into a binary classification model to obtain a classification result output by the binary classification model, wherein the classification result includes whether the target ingot has a defect or whether the target ingot does not have a defect; a second processing module, configured to, when the classification result indicates that the target ingot has a defect, input the ingot image into a defect localization model to obtain a defect detection area output by the defect localization model; a third processing module, configured to input the defect image corresponding to the defect detection area into a defect classification model, analyze the type of defects in the defect image, and obtain a defect type label output by the defect classification model, wherein the defect image is obtained by cropping the ingot image, and the defect detection result of the target ingot includes the defect detection area and the defect type label.
8. A defect detection system for an ingot, characterized in that: include: An image acquisition device, used for acquiring images of the ingot; The ingot defect detection device according to claim 7, wherein the ingot defect detection device is connected to the image acquisition device.
9. The ingot defect detection system according to claim 8, characterized in that: The image acquisition device is provided on a moving mechanism, and the moving mechanism is used to drive the image acquisition device to move to acquire multiple partial images of the ingot, wherein the partial images of the ingot include local information of the target ingot, and the ingot image is obtained by splicing the multiple partial images of the ingot.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the defect detection method for the ingot according to any one of claims 1 to 6 is implemented.