Defect detection methods, electronic devices and storage media

Abstract

This application relates to the field of defect detection technology, and provides a defect detection method, electronic device, and storage medium. The defect detection method includes: inputting a target image into a pre-trained first classification model to obtain a first classification result; if the first classification result indicates that there are no defects in the target image, inputting the target image into a defect segmentation model to obtain a segmentation result; the first classification model is used to identify the defect type in the input image. The above defect detection method can solve the problem of inaccurate defect detection results. Since the first classification model only needs to detect the defect type in the target image, there is no need to label defect regions during the training process of the first classification model. Furthermore, if the first classification model does not detect defects, the defect segmentation model is used to supplement the detection of defects in the target image. This avoids the problem of inaccurate defect detection results caused by missed detections by the first classification model, thereby helping to improve the accuracy of defect detection results.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, specifically to defect detection methods, electronic devices, and storage media. Background Technology

[0002] Currently, image recognition is increasingly being applied in industrial production processes. For example, in the process of bar inspection, defects in the bars can be detected by recognizing their images.

[0003] In related technologies, an initial segmentation model is trained using training data to obtain a defect detection model. During the detection process, the image is input into the defect detection model to obtain the defect detection result.

[0004] However, since the defect regions corresponding to some defect types are difficult to label, the initial segmentation model is difficult to train for that type of defect, which leads to inaccurate defect detection results. Summary of the Invention

[0005] To help solve the problem of inaccurate defect detection results, this application provides a defect detection method, electronic device, and storage medium.

[0006] Firstly, a defect detection method is provided, employing the following technical solution:

[0007] A defect detection method, the method comprising:

[0008] The target image is input into the pre-trained first classification model to obtain the first classification result;

[0009] If the first classification result indicates that there are no defects in the target image, the target image is input into the defect segmentation model to obtain the segmentation result;

[0010] The first classification model is used to identify the type of defects in the input image, and the defect segmentation model is used to detect defect regions in the input image.

[0011] By adopting the above technical solution, the problem of inaccurate defect detection results can be solved. Since the target image is first detected based on the first classification model, and the first classification model only needs to detect the defect type in the target image without determining the defect region, it is not necessary to label the defect region corresponding to the defect type during the training of the first classification model. This makes it easier to train the first classification model. Furthermore, if the first classification model does not detect defects, the defect segmentation model is used to supplement the detection of the target image. This can help avoid the problem of inaccurate defect detection results caused by the first classification model missing detections, thereby helping to improve the accuracy of defect detection results.

[0012] Optionally, the segmentation result includes detected and undetected segments. After inputting the target image into the defect segmentation model to obtain the segmentation result, the method further includes:

[0013] If the segmentation result is detected, the defective image is determined based on the segmentation result and the target image;

[0014] The defective image is input into a pre-trained second classification model to obtain the second classification result;

[0015] The second classification model is used to identify the types of defects in the input image.

[0016] In the above technical solution, since the defect segmentation model is only used to detect whether there is a defect region in the input image, and when a defect region is detected, the second classification model is used to determine the defect type corresponding to the defect region, which can help improve the accuracy of defect detection results.

[0017] Optionally, the first classification model is trained using a first dataset and a second dataset, and the second classification model is trained using the second dataset, wherein the defect types of the training data in the first dataset are different from the defect types of the training data in the second dataset.

[0018] In the above technical solution, the first classification model is trained using the first dataset and the second dataset, while the second classification model is trained using only the second dataset. This can improve the sensitivity of the second classification model to the defect types in the training data of the second dataset, thereby improving the accuracy of the classification results of the second classification model and thus improving the accuracy of the defect detection results.

[0019] Optionally, after inputting the target image into the defect segmentation model to obtain the segmentation result, the method further includes:

[0020] If the segmentation result is not detected, the target image is input into the third classification model to obtain the third classification result;

[0021] The third classification model is used to identify the type of defects in the input image, and the third classification model is trained using the first dataset.

[0022] In the above technical solution, the first classification model is trained using the first dataset and the second dataset, while the third classification model is trained using only the first dataset. This can improve the sensitivity of the third classification model to the defect types in the training data of the second dataset, thereby improving the accuracy of the detection results of the third classification model, and further improving the accuracy of the defect detection results.

[0023] Optionally, the defect segmentation model is trained using a third dataset, wherein the defect types in the training data of the third dataset are the same as the defect types in the training data of the second dataset.

[0024] In the above technical solution, since the defect types in the training data of the third dataset used to train the defect segmentation model are the same as the defect types in the training data of the second dataset used to train the second classification model, the second classification model and the defect segmentation model can be sensitive to the same defect types. This makes it easier for the second classification model to classify the defects detected by the defect segmentation model and improve the accuracy of the defect detection results.

[0025] Optionally, the target image is obtained by image acquisition of the bar material to be inspected;

[0026] The types of defects in the training data in the first dataset include: dirt, rust, and / or slippage and scratches;

[0027] The types of defects in the training data in the second dataset include: rough marks, scratches, holes, cracks, gaps, and / or lack of light exposure.

[0028] In the above technical solution, since the training data is classified based on the type of defects in the bar stock, the training data corresponding to defect types that are difficult to mark, such as dirt, rust spots, and / or scratches, are assigned to the first dataset, while the training data corresponding to defect types that are easy to mark, such as rough marks, scratches, holes, cracks, gaps, and / or lack of light exposure, are assigned to the second dataset. This makes it convenient to use the appropriate dataset to train the classification model according to actual needs, thereby obtaining classification models with different classification functions.

[0029] Optionally, determining the defective image based on the segmentation result and the target image includes:

[0030] The target image is flipped to obtain the flipped image;

[0031] Based on the segmentation results, the target defect region is determined in the flipped image;

[0032] The flipped image is cropped with the target defect area as the center to obtain the defective image.

[0033] In the above technical solution, the target image is first flipped during the defect image determination process to obtain a flipped image. Then, the target defect region is determined in the flipped image based on the segmentation results. Finally, the flipped image is cropped with the target defect region as the center to obtain the defect image. This avoids the problem of inaccurate defect detection results of the second classification model when the defect region is located at the edge of the target image. Since the defect image is cropped with the target defect region as the center to obtain the defect image, the defect region can be located in the center of the defect image, which can help improve the accuracy of the defect detection results of the second classification model.

[0034] Optionally, before inputting the target image into the pre-trained first classification model, the method further includes:

[0035] Acquire the original image, which is obtained by scanning the side of the bar to be inspected using a line scan camera;

[0036] The image of the region corresponding to the side of the bar to be detected in the original image is determined as the target image.

[0037] In the above technical solution, since the original image is obtained by scanning the side of the bar to be inspected using a line scan camera, and the image of the area corresponding to the side of the bar can be extracted from the original image as the target image, it is convenient to realize the automatic detection of bar defects, reduce manual intervention in the bar inspection process, and improve the efficiency of defect detection.

[0038] Secondly, an electronic device is provided, employing the following technical solution:

[0039] An electronic device, the electronic device comprising:

[0040] At least one processor;

[0041] Memory;

[0042] At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: execute any of the defect detection methods provided in the first aspect.

[0043] Thirdly, a computer-readable storage medium is provided, employing the following technical solution:

[0044] A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform any of the defect detection methods provided in the first aspect.

[0045] In summary, this application includes at least one of the following beneficial technical effects:

[0046] 1. It can solve the problem of inaccurate defect detection results. Since the target image is first detected based on the first classification model, and the first classification model only needs to detect the defect type in the target image without determining the defect region, there is no need to label the defect region corresponding to the defect type during the training of the first classification model. Furthermore, if the first classification model does not detect defects, the defect segmentation model is used to supplement the detection of defects in the target image. This can help avoid the problem of inaccurate defect detection results caused by the first classification model missing detections, thereby helping to improve the accuracy of defect detection results.

[0047] 2. It can avoid the problem of inaccurate defect detection results of the second classification model when the defect area is located at the edge of the target image. Since the defect image is obtained by cropping the flipped image with the target defect area as the center, the defect area can be located in the center of the defect image, which can help improve the accuracy of the defect detection results of the second classification model. Attached Figure Description

[0048] Figure 1 This is a flowchart of a defect detection method provided in an embodiment of this application;

[0049] Figure 2 This is a flowchart of another defect detection method provided in the embodiments of this application;

[0050] Figure 3 This is a flowchart of yet another defect detection method provided in the embodiments of this application;

[0051] Figure 4 This is a flowchart of the defect image determination process provided in the embodiments of this application;

[0052] Figure 5 This is a flowchart of the target image acquisition process provided in the embodiments of this application;

[0053] Figure 6 This is a schematic diagram of the target image provided in the embodiments of this application;

[0054] Figure 7 This is a flowchart of a defect detection example provided in an embodiment of this application;

[0055] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0056] To make the purpose, technical solution, and advantages of this application clearer, the following description is provided in conjunction with the appendix. Figure 1-8The present application will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the application.

[0057] This application uses the example of a defect detection method running in an electronic device as an illustration. The electronic device is a terminal or a server. The terminal can be a computer, mobile phone, tablet computer, etc. This application does not limit the type of electronic device.

[0058] This application provides a defect detection method, referring to... Figure 1 The defect detection method includes the following steps:

[0059] Step 101: Input the target image into the pre-trained first classification model to obtain the first classification result.

[0060] The first classification model is used to identify the types of defects in the input image.

[0061] Optionally, the first classification model is obtained by training the initial classification model using training data. Each set of training data includes a sample image and the corresponding image label. The image label is used to indicate the defect type of the sample image or whether the sample image is a good product.

[0062] To illustrate, the training process of the first classification model includes: creating an initial classification model; inputting sample images into the initial classification model to obtain model results; iteratively updating the model parameters of the initial classification model based on the model results and the image labels corresponding to the sample images; and obtaining the first classification model when the number of iterations reaches a preset number or the updated model converges.

[0063] In one example, the initial classification model is the ResNet50 model. In actual implementation, other models can also be used to construct the initial classification model; this embodiment does not limit the type of the initial classification model.

[0064] Optionally, the loss function used during the initial classification model training is the cross-entropy loss function.

[0065] In the above technical solution, since the first classification model only needs to identify the type of defect in the input image, and does not need to specifically identify the location of the defect area in the image, there is no need to label the location of the defect area in the training image during the training process of the first classification model. This makes it easier to label the training image, thereby reducing the training difficulty of the first classification model.

[0066] Optionally, the training data for the first classification model includes all types of defects that need to be detected. This allows for comprehensive defect detection in the target image based on the first classification model.

[0067] In this embodiment, the defect detection method is applied to the defect detection of bar stock as an example. In actual implementation, the defect detection method proposed in this embodiment can also be used to detect defects in other objects according to actual needs. This embodiment does not limit the application scenarios of the defect detection method.

[0068] In one instance, the types of defects to be detected in the bar stock include: dirt, rust spots, slippage scratches, rough marks, scratches, holes, cracks, gaps, and lack of light exposure. Correspondingly, the types of defects in the training data of the first classification model include: dirt, rust spots, slippage scratches, rough marks, scratches, holes, cracks, gaps, and lack of light exposure.

[0069] Among them, slippage and abrasion refers to abrasion caused by slippage during the production or movement of the bar stock.

[0070] The absence of light is usually caused by the roughness of the product surface. Because the amount of light reflected from a rough surface is less than that from a smooth surface during image acquisition, this can lead to the defect of not seeing light.

[0071] In one instance, if the outer diameter of a part of the product is too small, it will result in that part not being polished smoothly during the grinding process, resulting in an original rough surface. Based on this, during the inspection process, this part will be detected as not being exposed to light.

[0072] In actual implementation, the defect type can be set according to the actual detection needs. This embodiment does not limit the defect type of the training data of the first classification model.

[0073] Optionally, in order to improve the training effect of the first classification model, the training data of the first classification model may also include good data, that is, flawless data.

[0074] In one example, the initial classification model includes an attention module. This improves the initial classification model's sensitivity to defects in the image. By weighting the channels, it highlights defect information in the sample image, thereby increasing the model's sensitivity to defects. This helps improve the accuracy of the trained defect recognition model in identifying defects.

[0075] In one instance, the initial classification model was obtained by adding the SENet attention module to the base block of the ResNet50 model.

[0076] Step 102: If the first classification result indicates that there are no defects in the target image, input the target image into the defect segmentation model to obtain the segmentation result.

[0077] If the first classification result indicates that there are defects in the target image, the detection ends.

[0078] Among them, the defect segmentation model is used to detect defective regions in the input image.

[0079] In one example, the defect segmentation model is trained on the initial segmentation model using a second dataset. Each training dataset includes a sample image and its corresponding image label, which indicates the location of the defective region in the sample image.

[0080] The training process of the defect segmentation model, as illustrated, includes: creating an initial segmentation model; inputting sample images into the initial segmentation model to obtain model results; iteratively updating the model parameters of the initial segmentation model based on the model results and the image labels corresponding to the sample images; and obtaining the defect segmentation model when the number of iterations reaches a preset number or the updated model converges.

[0081] In this example, the initial segmentation model is the UNet model. In actual implementation, other models can also be used to construct the initial segmentation model; this embodiment does not limit the type of the initial segmentation model.

[0082] Optionally, the loss function used in the initial segmentation model training process is the cross-entropy loss function plus the Dice loss function.

[0083] Optionally, the training data for the defect segmentation model may include defect types that are easy to label as defect regions. This allows the defect segmentation model to supplement the identification of defect types in easily labeled regions, helping to avoid missed detections caused by the first classification model failing to identify that type of defect, thus improving the accuracy of defect detection results.

[0084] In one example, the training data for the defect segmentation model includes defects such as rough marks, scratches, holes, cracks, gaps, and / or defects that have not been exposed to light. Since the defect regions corresponding to these types of defects are easy to label, it is convenient to train the initial segmentation model to obtain the defect segmentation model.

[0085] Optionally, in order to improve the training effect of the defect segmentation model, the training data of the defect segmentation model also includes good product data, that is, data without defects.

[0086] In the above technical solution, since the defect segmentation model only needs to detect defective regions in the input image without distinguishing specific defect types, the training difficulty of the defect segmentation model can be reduced, and the segmentation efficiency of the defect segmentation model can also be improved.

[0087] In actual implementation, the training data corresponding to the defect segmentation model can also include the defect type corresponding to the defect region. Accordingly, the trained defect segmentation model can not only identify the defect region in the input image, but also identify the defect type corresponding to the defect region. Thus, defects can be processed based on the defect detection results. This embodiment does not limit the type of output content of the defect segmentation model.

[0088] The implementation principle of the defect detection method provided in this embodiment is as follows: The target image is input into a pre-trained first classification model to obtain a first classification result; if the first classification result indicates that there are no defects in the target image, the target image is input into a defect segmentation model to obtain a segmentation result; the first classification model is used to identify the defect type in the input image, and the defect segmentation model is used to detect defect regions in the input image. This defect detection method can solve the problem of inaccurate defect detection results. Since defect detection is first performed on the target image based on the first classification model, and the first classification model only needs to detect the defect type in the target image without determining the defect region, there is no need to label the defect region corresponding to the defect type during the training process of the first classification model. This facilitates the training of the first classification model. Furthermore, if the first classification model does not detect defects, the defect segmentation model is used to supplement the detection of the target image. This helps avoid the problem of inaccurate defect detection results caused by missed detections in the first classification model, thereby improving the accuracy of the defect detection results.

[0089] Based on the above technical solution, the segmentation results include detected and undetected data. At this point, refer to... Figure 2 Step 102, after inputting the target image into the defect segmentation model and obtaining the segmentation result, also includes the following steps:

[0090] Step 201: If the segmentation result is detected, determine the defective image based on the segmentation result and the target image.

[0091] Among them, detection refers to the detection of defective areas in the target image.

[0092] "Not detected" means that no defective areas were detected in the target image.

[0093] In one example, the segmentation result includes a mask corresponding to the defective region. Determining the defective image based on the segmentation result and the target image includes: determining the defective region from the target image based on the mask corresponding to the defective region; and determining the defective image based on the defective region.

[0094] Optionally, the method of determining the defective image based on the defective region can be to directly determine the image corresponding to the defective region as the defective image, or to crop the target image based on the defective region to obtain the defective image. This embodiment does not limit the method of determining the defective image based on the defective region.

[0095] Step 202: Input the defective image into the pre-trained second classification model to obtain the second classification result.

[0096] The second classification model is used to identify the types of defects in the input image.

[0097] Optionally, the second classification model is obtained by training the initial classification model using training data. Each set of training data includes a sample image and the corresponding image label for the sample image. The image label is used to indicate the type of defect corresponding to the sample image.

[0098] The training process of the second classification model is the same as that of the first classification model described above, and will not be repeated here in this embodiment.

[0099] In the above technical solution, since the segmentation result is either detected or not detected, without needing to detect specific defect types, this helps reduce the training difficulty of the segmentation model and improve the segmentation speed. Furthermore, when the segmentation result is detected, the defect image is determined based on the segmentation result and the target image, and the defect image is further identified using a second classification model to obtain the defect type. This helps solve the problem of low accuracy caused by the low sensitivity of multiple segmentation models leading to direct defect type detection based on the defect segmentation model. Since the defect segmentation model is only used to detect whether there are defective regions in the input image, and when defective regions are detected, the second classification model is used to determine the defect type corresponding to the defective region, thus helping to improve the accuracy of defect detection results.

[0100] Optionally, the first classification model is trained using the first dataset and the second dataset, and the second classification model is trained using the second dataset.

[0101] The defect types in the training data of the first dataset are different from those in the training data of the second dataset.

[0102] In one example, the defect type of the training data in the first dataset is the defect type of the defect area that is difficult to label, while the defect type of the training data in the second dataset is the defect type of the defect area that is easy to label.

[0103] For example, if a certain type of defect is usually distributed in patches on the object to be inspected, then it is difficult to mark the defect area for that type of defect.

[0104] For example, if the shape and / or size of a certain type of defect is relatively fixed, then the defect area is easy to mark.

[0105] In one instance, the target image is obtained by image acquisition of the bar material to be inspected; the types of defects to be detected include: dirt, rust spots, scratches, rough marks, scratches, holes, cracks, gaps and / or lack of light exposure; the types of defects in the training data of the first dataset include: defects that are difficult to label (such as: dirt, rust spots and / or scratches); the types of defects in the training data of the second dataset include: defects that are easy to label (rough marks, scratches, holes, cracks, gaps and / or lack of light exposure).

[0106] Optionally, to improve the training effect of the classification model, the first dataset also includes good data, that is, data without defects.

[0107] In the above technical solution, since the training data is classified based on the type of defects in the bar stock, the training data corresponding to defect types that are difficult to mark, such as dirt, rust spots, and / or scratches, are assigned to the first dataset, while the training data corresponding to defect types that are easy to mark, such as rough marks, scratches, and / or unexposed areas, are assigned to the second dataset. This makes it convenient to use the appropriate dataset to train the classification model according to actual needs, thereby obtaining classification models with different classification functions.

[0108] In actual implementation, the first dataset and the second dataset can also be divided in other ways. For example, the defect type corresponding to the sample data in the second dataset can be determined based on the defect type that the defect detection model can detect. This embodiment does not limit the way the first dataset and the second dataset are divided.

[0109] In the above technical solution, the corresponding defect image is generated and input into the second classification model only when the first classification model fails to detect a defect but the defect segmentation model does, i.e., when the defect segmentation model determines that the first classification model has missed a detection. Since the defect segmentation model is only sensitive to some defect types, the second classification model does not need to detect all types of defects. Therefore, in the above technical solution, the first classification model is trained using the first dataset and the second dataset, while the second classification model is trained using only the second dataset. This can improve the sensitivity of the second classification model to the defect types in the training data of the second dataset, thereby improving the accuracy of the classification results of the second classification model and thus improving the accuracy of the defect detection results.

[0110] In other examples, the first classification model can be trained using only the first dataset. This allows for a reasonable division of the first and second datasets based on the sensitivity of the classification and segmentation models to different types of defects, which can help improve the training speed of the model.

[0111] Furthermore, the segmentation results include detected and undetected, as referenced. Figure 3 Step 102, after inputting the target image into the defect segmentation model and obtaining the segmentation result, also includes the following steps:

[0112] Step 301: If the segmentation result is not detected, input the target image into the third classification model to obtain the third classification result.

[0113] The third classification model is used to identify the types of defects in the input image. The third classification model is trained using the first dataset.

[0114] Optionally, the third classification model is obtained by training the initial classification model using training data. Each set of training data includes a sample image and the corresponding image label for the sample image. The image label is used to indicate the type of defect corresponding to the sample image.

[0115] The training process of the third classification model is the same as that of the first classification model described above, and will not be repeated here in this embodiment.

[0116] In the above technical solution, when neither the first classification model nor the defect segmentation model detects defects, there are two possibilities: first, the target image indeed has no defects; second, both the first classification model and the defect segmentation model miss the defects. In the above technical solution, when the segmentation result is that no defects are detected, a third classification model is further used to detect defects in the target image. This can help avoid the problem of inaccurate defect detection results caused by both the first classification model and the defect detection model missing the defects, thus further improving the accuracy of defect detection results.

[0117] Furthermore, since the defect segmentation model is only sensitive to some defect types, the third classification model does not need to detect all types of defects. In the above technical solution, the first classification model is trained using the first dataset and the second dataset, while the third classification model is trained using only the first dataset. This can improve the sensitivity of the third classification model to the defect types in the training data of the second dataset, thereby improving the accuracy of the detection results of the third classification model, and further improving the accuracy of the defect detection results.

[0118] Furthermore, the defect segmentation model was trained using a third dataset, in which the defect types in the training data were the same as those in the training data in the second dataset.

[0119] In one example, the defect types in the training data of the third dataset and the defect types in the training data of the second dataset are both defect types that the defect segmentation model is highly sensitive to. For example, it is easy to determine the defect type of the defect region. This makes it easier to train the defect segmentation model and also helps to improve the accuracy of the defect segmentation model's segmentation results, thereby improving the accuracy of the defect detection results.

[0120] Optionally, to improve the training effect of the defect segmentation model, the third dataset also includes good product data, i.e., data without defects.

[0121] In the above technical solution, since the defect types in the training data of the third dataset used to train the defect segmentation model are the same as the defect types in the training data of the second dataset used to train the second classification model, the second classification model and the defect segmentation model can be sensitive to the same defect types. This makes it easier for the second classification model to classify the defects detected by the defect segmentation model and improve the accuracy of the defect detection results.

[0122] Furthermore, since the defect types in the training data of the first dataset are different from those in the training data of the second dataset, and the defect types in the training data of the first dataset are different from those in the training data of the third dataset, the third classification model and the defect segmentation model are sensitive to different defect types. This makes it easier for the third segmentation model to supplement the detection of target images that have not been detected by the defect segmentation model, thereby further improving the accuracy of defect detection results.

[0123] Optional, see reference Figure 4 In step 202 above, the defective image is determined based on the segmentation result and the target image, specifically including the following steps:

[0124] Step 401: Flip the target image to obtain the flipped image.

[0125] Optionally, the target image can be flipped in a pre-set manner, or it can be flipped based on the location of the defective area. This embodiment does not limit the way the target image is flipped.

[0126] In one example, flipping the target image to obtain a flipped image includes: flipping the target image horizontally to obtain an initial flipped image; and flipping the initial flipped image vertically to obtain the flipped image. At this point, the target image is located in the center of the flipped image.

[0127] Step 402: Determine the target defect region in the flipped image based on the segmentation results.

[0128] In one example, determining the target defect region in the flipped image based on the segmentation result includes: determining the target region corresponding to the target image in the flipped image; determining the target defect region in the target region based on the segmentation result; and determining the cropping region centered on the defect region.

[0129] In this case, the target region corresponding to the target image in the flipped image is determined based on the flipping method of the target image. For example, if the flipped image is obtained by flipping the target image horizontally and then flipping the horizontally flipped image vertically, there are 9 image regions in the flipped image that have the same content as the target image, and the target region is located in the middle position of the flipped image.

[0130] In actual implementation, the target defect area can also be determined based on other methods, such as: determining all defect areas in the flipped image based on the segmentation results, and determining the defect area closest to the center of the flipped image as the target defect area. This embodiment does not limit the method of determining the target defect area.

[0131] Step 403: Crop the flipped image with the target defect area as the center to obtain the defect image.

[0132] Optionally, the cutting area can be determined centered on the defective area, including: determining the cutting area centered on a preset position (e.g., the center position) of the defective area.

[0133] In one example, cropping the flipped image with the target defect area as the center to obtain a defective image includes: determining a cropping area of ​​a preset cropping size with the target defect area as the center; and cropping the flipped image based on the cropping area to obtain the defective image.

[0134] In one instance, the preset crop size is pre-set based on the input requirements of the second classification model.

[0135] In the above technical solution, the target image can be flipped first during the defect image determination process to obtain a flipped image. Then, the target defect region is determined in the flipped image based on the segmentation results. Finally, the flipped image is cropped with the target defect region as the center to obtain the defect image. This avoids the problem of inaccurate defect detection results of the second classification model when the defect region is located at the edge of the target image. Since the defect image is cropped with the target defect region as the center to obtain the defect image, the defect region can be located in the center of the defect image, which can help improve the accuracy of the defect detection results of the second classification model.

[0136] Furthermore, before training the initial classification model using the training data corresponding to the second classification model to obtain the second classification model, the process includes: flipping the sample image to obtain a flipped sample image; cropping the flipped sample image with the defective region in the region corresponding to the sample image in the flipped sample image as the center to obtain a preprocessed sample image; and training the initial classification model based on the preprocessed sample image to obtain the second classification model.

[0137] The method of flipping the sample image to obtain the flipped sample image is the same as the method of flipping the target image to obtain the flipped image in step 401 above; the method of cropping the flipped sample image with the defect area in the region corresponding to the sample image in the flipped sample image as the center to obtain the preprocessed sample image is the same as the method of cropping the flipped image with the target defect area as the center to obtain the defective image in step 403 above. This embodiment will not be described again here.

[0138] In the above technical solution, by flipping the sample images in the training data of the second classification model and cropping the sample images with the defect area as the center, a preprocessed sample image is obtained. Then, the model is trained based on the preprocessed sample image, which can make the defect area located in the center of the preprocessed sample image. Thus, training the initial classification model based on the preprocessed sample image can improve the learning effect of the initial classification model on defects, thereby improving the accuracy of the defect detection results of the trained second classification model.

[0139] Optional, see reference Figure 5 Step 101, before inputting the target image into the pre-trained first classification model, also includes:

[0140] Step 501: Obtain the original image.

[0141] The original image was obtained by scanning the side of the bar to be inspected using a line scan camera.

[0142] In one instance, the original image includes a complete image of the outer periphery of the bar.

[0143] In one example, the line scanner is fixedly mounted, and its image acquisition range includes the side area of ​​the bar. During the initial image acquisition process, the bar is rotated so that the line scanner can acquire a complete image of the bar's outer perimeter. This facilitates comprehensive defect detection on the side of the bar.

[0144] Step 502: Determine the image of the region corresponding to the side of the bar to be detected in the original image as the target image.

[0145] In one example, the target image is shown below. Figure 6 .

[0146] Optionally, the method for determining the image of the region corresponding to the side of the bar to be detected from the original image can be to determine the image of the region corresponding to the side of the bar to be detected within a preset region in the original image, or it can be to determine the image of the continuous region in the original image that meets the preset conditions as the image of the region corresponding to the bar to be detected. This embodiment does not limit the method of determining the image of the region corresponding to the side of the bar to be detected.

[0147] In the above technical solution, since the original image is obtained by scanning the side of the bar to be inspected using a line scan camera, and the image of the area corresponding to the side of the bar can be extracted from the original image as the target image, it is convenient to realize the automatic detection of bar defects, reduce manual intervention in the bar inspection process, and improve the efficiency of defect detection.

[0148] To better understand the defect detection method provided in this application's embodiments, this embodiment also provides an example to illustrate the method, such as... Figure 7 As shown, the method includes the following steps:

[0149] Step 601: Input the target image into the pre-trained first classification model to obtain the first classification result; if the first classification result indicates that the target image has no defects, proceed to step 602.

[0150] Step 602: Input the target image into the defect segmentation model to obtain the segmentation result; if the defect segmentation result is detected, proceed to step 603; if the defect detection result is not detected, proceed to step 605.

[0151] Step 603: Determine the defective image based on the segmentation results and the target image.

[0152] Step 604: Input the defective image into the pre-trained second classification model to obtain the second classification result.

[0153] Step 605: Input the target image into the third classification model to obtain the third classification result.

[0154] For relevant details, please refer to the image cropping method described above; this embodiment will not repeat them here.

[0155] This application also provides an electronic device, such as... Figure 8 As shown, Figure 8The illustrated electronic device 700 includes a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, for example, via a bus 702. Optionally, the electronic device 700 may also include a transceiver 704. It should be noted that in practical applications, the transceiver 704 is not limited to one type, and the structure of this electronic device 700 does not constitute a limitation on the embodiments of this application.

[0156] Processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 701 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0157] Bus 702 may include a pathway for transmitting information between the aforementioned components. Bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 702 can be divided into address bus, data bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0158] The memory 703 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0159] The memory 703 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 701. The processor 701 is used to execute the application code stored in the memory 703 to implement the content shown in the foregoing method embodiments.

[0160] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, PDAs (personal digital assistants), and PADs (tablet computers), as well as fixed terminals such as digital TVs and desktop computers. They can also serve as server-side components. Figure 8 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0161] This application also provides a computer-readable storage medium storing a computer program, which, when executed in a computer, causes the computer to perform the defect detection method provided in the above embodiments.

[0162] It should be understood that although the steps in the flowcharts in the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise expressly stated herein, there is no strict order in which these steps are performed, and they may be performed in other orders.

[0163] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A defect detection method, characterized in that, The method includes: The target image is input into the pre-trained first classification model to obtain the first classification result; If the first classification result indicates that there are no defects in the target image, the target image is input into the defect segmentation model to obtain the segmentation result; The first classification model is used to identify the type of defects in the input image, and the defect segmentation model is used to detect defect regions in the input image. The segmentation result includes detected and undetected segments. After inputting the target image into the defect segmentation model to obtain the segmentation result, the process further includes: If the segmentation result is detected, the defective image is determined based on the segmentation result and the target image; The defective image is input into a pre-trained second classification model to obtain the second classification result; The second classification model is used to identify the types of defects in the input image; The first classification model is trained using a first dataset and a second dataset, and the second classification model is trained using the second dataset. The defect types in the training data of the first dataset are different from the defect types in the training data of the second dataset. After inputting the target image into the defect segmentation model to obtain the segmentation result, the process further includes: If the segmentation result is not detected, the target image is input into the third classification model to obtain the third classification result; The third classification model is used to identify the type of defects in the input image, and the third classification model is trained using the first dataset. The defect segmentation model is trained using a third dataset, in which the defect types of the training data are the same as those of the training data in the second dataset.

2. The method according to claim 1, characterized in that, The target image is obtained by image acquisition of the bar material to be tested; The types of defects in the training data in the first dataset include: dirt, rust, and / or slippage and scratches; The types of defects in the training data in the second dataset include: rough marks, scratches, holes, cracks, gaps, and / or lack of light exposure.

3. The method according to claim 1, characterized in that, The step of determining the defective image based on the segmentation result and the target image includes: The target image is flipped to obtain the flipped image; Based on the segmentation results, the target defect region is determined in the flipped image; The flipped image is cropped with the target defect area as the center to obtain the defective image.

4. The method according to claim 1, characterized in that, Before inputting the target image into the pre-trained first classification model, the process also includes: Acquire the original image, which is obtained by scanning the side of the bar to be inspected using a line scan camera; The image of the region corresponding to the side of the bar to be detected in the original image is determined as the target image.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, said at least one application being configured to: perform the defect detection method according to any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed in a computer, the computer is instructed to perform the defect detection method according to any one of claims 1 to 4.

Images