A steel surface defect detection method, device, detection platform and storage medium

Abstract

The application provides a steel surface defect detection method, device, detection platform and storage medium. The method comprises the following steps: obtaining a steel surface sample image of a known defect type of a steel production line; building an algorithm model based on a YOLO algorithm, and training the algorithm model based on the steel surface sample image of the known defect type to obtain a target detection model; performing quantitative error analysis on the calculation speed and calculation accuracy of the target detection model based on a quantitative method to obtain a defect detection model; and determining the defect type of a to-be-detected steel surface sample image based on the defect detection model and the to-be-detected steel surface sample image. The application can improve the calculation speed and calculation accuracy of the defect detection model, thereby improving the detection accuracy and detection speed, and more accurately and efficiently meeting the actual detection requirements of the work line.

Description

Technical Field

[0001] This application relates to the field of material quality prediction and health management technology, and in particular to a method, apparatus, detection platform and storage medium for detecting defects on the surface of steel. Background Technology

[0002] In the field of steel flaw detection, existing technologies for detecting surface defects include both contact and non-contact methods. Contact inspection methods may damage the steel surface, creating new defects. Therefore, non-contact methods for non-destructive testing of steel surfaces to obtain material quality information have received increasing attention and application. The accuracy and speed of the inspection method are crucial to the quality inspection process. Among non-destructive testing methods, computer vision-based inspection is a common approach. This method uses a camera to image the sample surface and then identifies, locates, and analyzes the image data for defects.

[0003] Because steel has a large surface area and the online inspection platform requires the equipment to achieve a certain detection frame rate, the amount of data processed by the equipment is substantial. Using high computing power will increase the overall cost of the inspection equipment. Using low computing power may lead to decreased accuracy or excessively slow inspection speed. Summary of the Invention

[0004] This application provides a method, apparatus, detection platform, and storage medium for detecting surface defects in steel, in order to solve the problem of the imbalance between accuracy and speed in the prior art for detecting surface defects in steel.

[0005] Firstly, this application provides a method for detecting surface defects in steel, including:

[0006] Obtain surface sample images of steel from steel production lines with known defect types;

[0007] An algorithm model was built based on the YOLO algorithm, and the algorithm model was trained based on sample images of steel surfaces with known defect types to obtain a target detection model.

[0008] Based on the quantization method, the computational speed and accuracy of the target detection model are analyzed by quantization error analysis to obtain the defect detection model;

[0009] Based on the defect detection model and the surface sample image of the steel to be detected, the defect type of the surface sample image of the steel to be detected is determined.

[0010] Secondly, this application provides a steel surface defect detection device, comprising:

[0011] The image acquisition module is used to acquire surface sample images of steel from steel production lines for known defect types.

[0012] The model training module is used to build an algorithm model based on the YOLO algorithm, and to train the algorithm model based on sample images of steel surfaces with known defect types to obtain a target detection model.

[0013] The model quantization module is used to perform quantization error analysis on the calculation speed and accuracy of the target detection model based on quantization methods, so as to obtain the defect detection model.

[0014] The type detection module is used to determine the defect type of the surface sample image of the steel to be detected based on the defect detection model and the surface sample image of the steel to be detected.

[0015] Thirdly, this application provides a detection platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect or any possible implementation of the first aspect.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any possible implementation of the first aspect.

[0017] This application provides a method, apparatus, detection platform, and storage medium for detecting surface defects in steel. The method involves acquiring surface sample images of steel from a steel production line, identifying known defect types; building an algorithm model based on the YOLO algorithm; training the algorithm model using the surface sample images of steel with known defect types to obtain a target detection model; performing quantization error analysis on the computational speed and accuracy of the target detection model using quantization methods to obtain a defect detection model; and determining the defect type of the surface sample image of the steel to be detected based on the defect detection model and the surface sample image. This application improves the computational speed and accuracy of the defect detection model by quantifying the computational speed and accuracy of the trained target detection model, thereby improving detection accuracy and speed, and more accurately and efficiently meeting the actual detection needs of the production line. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the implementation of the steel surface defect detection method provided in the embodiments of this application;

[0020] Figure 2 This is a schematic diagram of the process of calling CSI camera real-time detection provided in an embodiment of this application;

[0021] Figure 3 This is a flowchart illustrating the quantization process of the target detection model provided in this application embodiment;

[0022] Figure 4 This is a schematic diagram of the steel surface defect detection device provided in the embodiments of this application;

[0023] Figure 5 This is a schematic diagram of the detection platform provided in the embodiments of this application. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0026] Figure 1 The implementation flowchart of the steel surface defect detection method provided in the embodiments of this application is described in detail below:

[0027] In step 101, images of steel surface samples with known defect types from the steel production line are obtained.

[0028] In this application embodiment, the defect types include indentations, scratches, oil stains, and dirt, and a single steel surface sample image may include multiple defect types.

[0029] In this embodiment of the application, a video stream of the steel production line is acquired in real time by a camera device on an online steel inspection platform, and images of steel surface samples with several known defect types are obtained from the acquired video stream. The camera device may include a CSI camera.

[0030] In one possible implementation, acquiring a sample image of a steel surface with a known defect type includes:

[0031] Select real steel surface sample images from the steel production line, and label the steel surface sample images with defect type labels based on expert prior knowledge to obtain steel surface sample images with known defect types.

[0032] Because the steel surface defect dataset has an uneven number of defect types, when labeling samples with mixed defect types, we should try to keep the number of labels for each type of defect equal, that is, we do not need to label all defects.

[0033] In one possible implementation, when building a model using sample images of steel surfaces with known defect types, data augmentation is also required to augment these images. For example, if the number of steel surface sample images with different defect types is 500, to achieve a sufficient number of samples for more accurate model training, data augmentation is used to augment these 500 samples to 2000.

[0034] Data augmentation methods can include Mosaic enhancement, Copy Paste enhancement, Random affire enhancement, and Mixup enhancement.

[0035] In this embodiment of the application, the acquired sample images have known defect types, which provide samples for subsequent model training, and the number of samples is expanded through data augmentation to improve the accuracy of model training.

[0036] In step 102, an algorithm model is built based on the YOLO algorithm, and the algorithm model is trained based on sample images of steel surfaces with known defect types to obtain a target detection model.

[0037] The YOLO algorithm can include YOLOv5, YOLOv6, YOLOv7, and YOLOv8. An algorithm model is built based on these algorithms, and the algorithm with the highest computational accuracy is selected as the basis for the defect detection model.

[0038] In this embodiment of the application, an algorithm model is built based on the above-mentioned algorithms, and the above-mentioned algorithm models are trained according to the steel surface sample images of known defect types obtained in step 101 to obtain a target detection model.

[0039] In one possible implementation, step 102 may include:

[0040] The first algorithm model was built using YOLOv5, the second algorithm model was built using YOLOv6, the third algorithm model was built using YOLOv7, and the fourth algorithm model was built using YOLOv8, respectively.

[0041] Using steel surface sample images as input and the corresponding defect types as output, the first, second, third, and fourth algorithm models are trained respectively; the defect types include indentations, scratches, oil stains, and dirt.

[0042] The areas enclosed by the ROC curves and coordinate axes of the first, second, third, and fourth algorithm models after training are obtained respectively, to obtain the first AUC of the first algorithm model, the second AUC of the second algorithm model, the third AUC of the third algorithm model, and the fourth AUC of the fourth algorithm model.

[0043] The algorithm model corresponding to the maximum value among the first, second, third, and fourth AUCs is selected as the object detection model.

[0044] Optionally, YOLOv5, YOLOv6, YOLOv7, and YOLOv8 algorithms are used to build corresponding first, second, third, and fourth algorithm models, respectively. Then, the augmented steel surface sample images of known defect types are divided into training and validation sets according to a preset ratio. The preset ratio can be 7:3 or 8:2, depending on the actual needs.

[0045] Since the steel surface defect type dataset has the characteristics of fewer defect pixels compared to normal areas and an imbalance between positive and negative samples, accuracy should not be used as the evaluation metric for the model during training. Instead, the area under the ROC curve can be used.

[0046] The Area Under the Curve (AUC) is defined as the area enclosed by the ROC curve and the coordinate axes; obviously, this area will not exceed 1. Since the ROC curve is generally above the line y = x, the AUC value ranges between 0.5 and 1. The closer the AUC is to 1.0, the higher the realism of the detection method; when it equals 0.5, the realism is the lowest, and it has no application value.

[0047] After dividing the training and validation sets, the steel surface sample images in the training set are used as input, and the corresponding defect type labels are used as output to train the first, second, third, and fourth algorithm models, respectively. After training, the steel surface sample images in the validation set are used as input, and the corresponding defect type labels are used as output to validate the trained first, second, third, and fourth algorithm models, respectively.

[0048] After training and validation, the area enclosed by the ROC curve and the coordinate axis of the first algorithm model is obtained, which is the first AUC; the area enclosed by the ROC curve and the coordinate axis of the second algorithm model is obtained, which is the second AUC; the area enclosed by the ROC curve and the coordinate axis of the third algorithm model is obtained, which is the third AUC; and the area enclosed by the ROC curve and the coordinate axis of the fourth algorithm model is obtained, which is the fourth AUC.

[0049] Since the AUC value is required to be above 0.75 when judging whether the computational accuracy of the algorithm model meets the requirements, it is necessary to first determine whether the first AUC, second AUC, third AUC and fourth AUC are all greater than 0.75. If there are AUC values ​​that are not greater than 0.75, the corresponding algorithm is not considered directly; if they are all greater than 0.75, the algorithm model with the largest value is selected as the target detection model according to the numerical size.

[0050] In one possible implementation, after training the algorithm model, the trained algorithm model can be subjected to necessary format conversion and acceleration processing, and then deployed on Jetson Orin NX for accuracy and speed testing.

[0051] Optionally, the algorithm model that meets the requirements of the test set can be converted into formats such as OpenNeural Network Exchange (Onnx) or Engine format. During invocation, the open-source computer vision processing software library (opencv-python) is recompiled to enable the open-source multimedia framework (Gstreamer).

[0052] The compilation process is as follows: Uninstall the existing version of opencv-python; install cmake and gcc compiler tools; install dependencies libgtk and libpng; check the version of opencv-python and determine the maximum frame rate limit for the video stream; configure opencv-python options: check the parameters of the image acquisition device and change the frame rate limit of opencv-python to be greater than or equal to the frame rate acquired by the image device; change the resolution format of the input image to adaptive to the sample; compile the C++ project and install it.

[0053] To enable Gstreamer, follow these steps: Change cv2.VideoCapture(0) to: cap = cv2.VideoCapture(“nvarguscamerasrc”, cv2.CAP_GSTREAMER).

[0054] Modify the data loading file to convert the inference data format into a video stream, and then use the CSI camera for real-time detection. After real-time detection is implemented, use TensorrRt to accelerate the video stream acquired by the CSI camera. Modify the files for TensorrRt to call YOLO series models to input the video stream acquired by the CSI camera, and the deployment is complete.

[0055] The specific process of accelerating the video stream is as follows: Modify tensorrtx / yolov5 / yolov5_det_trt.py (this script performs inference on images) to obtain yolov5_det_trt_1.py, and then modify the camera cap = cv2.videocapture() in it, referring to the above method of enabling Gstreamer.

[0056] The process of calling the CSI camera for real-time detection is as follows: Figure 2 .

[0057] For example, speed and accuracy tests were performed on the pt, fp16, and int8 models of the YOLOv5, YOLOv6, YOLOv7, and YOLOv8 algorithms:

[0058] The algorithm model was trained using 5000 training samples of steel surface sample images with known defect types. The script export.py was used to input the pt model and output the onnx and engine models. A dataset of 2400 steel surface defect types was selected, and the three formats of the model were tested using test.py.

[0059] Based on the above tests, it can be concluded that, on existing production line data, the detection accuracy of the algorithm models built by YOLOv5 and YOLOv6 is low both before and after quantization, failing to meet the detection requirements. Furthermore, the detection speed cannot reach the 400fps required for real-time detection even after quantization.

[0060] The test results are shown in Table 1. The detection accuracy of the algorithm models built with YOLOv7 and YOLOv8 is low after quantization and does not meet the detection requirements. The detection speed before quantization does not meet the real-time detection requirements. Therefore, the models can be quantized.

[0061] Table 1. YOLO series algorithm model detection results test

[0062]

[0063]

[0064] In this embodiment of the application, by comparing the AUC of each algorithm model, a target detection model with higher computational accuracy can be obtained, thereby enabling the detection accuracy to meet the actual detection requirements.

[0065] In step 103, based on the quantization method, the calculation speed and accuracy of the target detection model are analyzed by quantization error analysis to obtain the defect detection model.

[0066] In this embodiment of the application, a quantification method is used to perform quantification error analysis on the calculation speed and calculation accuracy of the target detection model obtained in step 102, and finally a defect detection model that can meet the actual detection requirements is obtained.

[0067] For example, a target detection model whose computational accuracy reaches a preset accuracy is quantized:

[0068] The quantization rule is as follows: First, the entire network is quantized. If the speed requirement is met but the accuracy requirement is not, quantization error statistics are performed, and the layers are sorted by the size of the quantization error. The layer with the largest error is then dequantized.

[0069] The steps for quantization error statistics are as follows: Select 2000 images from the COCO validation set released in 2017. For each network layer, set breakpoint variables, calculate the absolute error of the quantized output, and then normalize the output before quantization. After traversing all network layers, obtain the error statistics.

[0070] If the accuracy still does not meet the requirements after dequantization, continue quantizing the next layer in sequence until the accuracy meets the requirements. If the speed does not meet the requirements at this point, sort the quantized layers according to the number of parameters from most to least, and requantize them layer by layer in sequence until both accuracy and speed meet the requirements.

[0071] In one possible implementation, step 103 may include:

[0072] Determine whether the computation speed of the target detection model has reached the preset speed;

[0073] If the computation speed of the target detection model reaches the preset speed, then the target detection model will be used as the first detection model.

[0074] Determine whether the calculation accuracy of the first detection model reaches the preset accuracy;

[0075] If the calculation accuracy of the first detection model reaches the preset accuracy, then the first detection model will be used as the defect detection model.

[0076] Among them, the object detection model has the highest computational accuracy but its computational speed is insufficient. Therefore, the initial object detection model has the highest computational accuracy. The model's computational accuracy includes three levels: fp32, fp16, and int8, with fp32 having the highest accuracy and int8 the lowest. The only difference between these accuracies is the number of decimal places.

[0077] Optionally, refer to Figure 3 First, determine if the computation speed of the target detection model reaches the preset speed. If the computation speed of the target detection model reaches the preset speed, then the target detection model with the qualified computation speed is used as the first detection model, and determine if the computation accuracy of the first detection model reaches the preset accuracy. If the computation accuracy reaches the preset accuracy, then the first detection model is used as the defect detection model.

[0078] In one possible implementation, determining whether the computational speed of the object detection model has reached a preset speed may further include:

[0079] If the computation speed of the target detection model does not reach the preset speed, the computation precision of each layer in the target detection model will be reduced, and the target detection model with reduced precision will be returned to the step of judging whether the computation speed of the target detection model has reached the preset speed to continue execution until the computation speed of the target detection model reaches the preset speed.

[0080] Reference Figure 3 If the computation speed of the target detection model does not reach the preset speed, the computation accuracy of each layer in the initial target detection model will be reduced, and the target detection model with reduced computation accuracy will be returned to the step of judging whether the computation speed of the target detection model has reached the preset speed to continue execution until the computation speed of the target detection model reaches the preset speed.

[0081] For example, if the computation speed of the target detection model does not reach the preset speed, the computation precision fp32 of each layer in the target detection model is reduced to fp16, and the target detection model with reduced precision is returned to the step of judging whether the computation speed of the target detection model has reached the preset speed to continue execution until the computation speed of the target detection model reaches the preset speed.

[0082] For example, if the computational precision of each layer in the target detection model has been reduced to int8, but the computational speed of the corresponding target detection model has not yet reached the preset speed, then the judgment is stopped, and it is determined whether to use this target detection model according to the user's needs. If not, the model can be retrained; if it is used, the initial target detection model is directly used as the defect detection model, which can make the computational precision of the defect detection model meet the detection requirements.

[0083] In one possible implementation, determining whether the computational accuracy of the first detection model reaches the preset accuracy may further include:

[0084] If the computational accuracy of the first detection model does not reach the preset accuracy, then the quantization error corresponding to each layer in the first detection model is calculated based on the computational accuracy of the initial target detection model and the computational accuracy of the first detection model.

[0085] Determine whether the quantization error of each layer is 0;

[0086] If the quantization error of each layer is 0, then the model selected by the user from the first detection model and the target detection model is obtained, and the model selected by the user is used as the defect detection model.

[0087] If the quantization error of each layer of the first detection model is not all zero, then the quantization error of each layer in the first detection model is sorted in descending order according to the error value to obtain the error value sequence.

[0088] Select the layer corresponding to the first N quantization errors in the error value sequence as the first target layer, and update the calculation accuracy of the corresponding layer in the first detection model using the calculation accuracy of the first target layer of the initial target detection model. Return to the step of judging whether the calculation accuracy of the first detection model has reached the preset accuracy and continue to execute until the calculation accuracy of the first detection model reaches the preset accuracy. Then, use the first detection model with the calculation accuracy reaching the preset accuracy as the defect detection model.

[0089] Reference Figure 3 If the computation speed of the first detection model reaches the preset speed, but the computation accuracy does not reach the preset accuracy, then the computation accuracy of each layer in the initial target detection model and the computation accuracy of the first detection model are input into the first formula to calculate the quantization error corresponding to each layer in the first detection model. The first formula is:

[0090]

[0091] Where, ε i The quantization error corresponding to the i-th layer is... Let be the computational precision corresponding to the i-th layer in the initial target detection model. This represents the computational precision corresponding to the i-th layer in the first detection model.

[0092] After obtaining the quantization error corresponding to each layer in the first detection model, it is determined whether the quantization temperature difference of each layer is 0. If it is 0, it is determined that the calculation speed of the initial target detection model directly reaches the preset speed without performing full model calculation accuracy adjustment. At this point, it is necessary to select the required model from the first detection model and the target detection model as the defect detection model according to the user's detection requirements.

[0093] If not all values ​​are 0, the quantization errors of each layer in the first detection model are sorted in descending order of error value to obtain an error value sequence. The layers corresponding to the top N quantization errors in the error value sequence are selected as the first target layer, and the calculation precision of the first target layer in the first detection model is updated using the calculation precision of the first target layer of the initial target detection model. The step of judging whether the calculation precision of the first detection model has reached the preset precision is returned and continued until the calculation precision of the first detection model reaches the preset precision. The first detection model with the preset precision is then used as the defect detection model.

[0094] For example, when the quantization errors of each layer in the first detection model are sorted in descending order of quantization error as {ε1, ε2, ..., ε...}, the results are: n When N is set to 3, the calculation precision of the layers corresponding to ε1, ε2, and ε3 is updated to the corresponding calculation precision in the initial target detection model, and the process is repeated until the calculation precision of the first detection model reaches the preset precision.

[0095] In one possible implementation, using a first detection model with a computational accuracy reaching a preset precision as the defect detection model may include:

[0096] The first detection model, whose calculation accuracy reaches the preset accuracy, is used as the second detection model. The calculation speed of the second detection model is then judged to see if it reaches the preset speed.

[0097] If the calculation speed of the second detection model reaches the preset speed, then the second detection model will be used as the defect detection model.

[0098] If the computation speed of the second detection model does not reach the preset speed, the parameter quantity of each first target layer in the second detection model is obtained, and the parameter quantity of each first target layer in the second detection model is sorted in descending order to obtain the parameter quantity sequence.

[0099] Select the layer corresponding to the first M parameters in the parameter sequence as the second target layer, update the calculation precision of the second target layer in the second detection model to the calculation precision of the layer corresponding to the initial first detection model, return to the step of judging whether the calculation speed of the second detection model has reached the preset speed, continue to execute until the calculation speed of the second detection model reaches the preset speed, and use the second detection model with the calculation speed reaching the preset speed as the defect detection model.

[0100] Reference Figure 3After adjusting the computational accuracy of the first detection model, the first detection model with the preset accuracy is used as the second detection model, and it is determined whether the computational speed of the second detection model reaches the preset speed. If the computational speed of the second detection model reaches the preset speed, then the second detection model is used as the defect detection model. If the computational speed of the second detection model does not reach the preset speed, then the parameter values ​​of each first target layer in the second detection model are obtained, and the parameter values ​​of each first target layer are sorted in descending order to obtain a parameter value sequence.

[0101] Select the layer corresponding to the top M parameters from the parameter ranking as the second target layer. Update the calculation accuracy of the second target layer in the second detection model to the calculation accuracy of the corresponding layer in the initial first detection model. Return to the step of judging whether the calculation speed of the second detection model has reached the preset speed and continue to execute until the calculation speed of the second detection model reaches the preset speed. Then, use the second detection model with the preset speed as the defect detection model.

[0102] For example, if the computation speed of the second detection model does not reach the preset speed, the parameter count of each first target layer in the second detection model is obtained and sorted in descending order to obtain the parameter count sequence, i.e., {n1, n2, ..., nn}. n If M is set to 2, then the layers corresponding to n1 and n2 are taken as the second target layers, and the calculation precision of the second target layers is updated to the calculation precision of the corresponding layers of the initial first detection model. This process is repeated until the calculation speed of the second detection model reaches the preset speed.

[0103] In this embodiment, the defect detection model that meets the detection requirements is determined by sequentially judging the calculation speed and calculation depth of the target detection model, thereby improving the detection speed and accuracy of the defect detection model.

[0104] In step 104, based on the defect detection model and the surface sample image of the steel to be inspected, the defect type of the surface sample image of the steel to be inspected is determined.

[0105] In this embodiment of the application, based on the defect detection model obtained in step 103, a sample image of the steel surface to be detected is obtained, and the sample image of the steel surface to be detected is input into the defect detection model to determine the defect type of the sample image of the steel surface to be detected.

[0106] This application provides a method for detecting surface defects in steel. The method involves acquiring surface sample images of steel from a steel production line, identifying known defect types; building an algorithm model based on the YOLO algorithm; training the algorithm model using the surface sample images of steel with known defect types to obtain a target detection model; performing quantization error analysis on the computational speed and accuracy of the target detection model using quantization methods to obtain a defect detection model; and determining the defect type of the surface sample image of the steel to be detected based on the defect detection model and the surface sample image. This application improves the computational speed and accuracy of the defect detection model by quantifying the computational speed and accuracy of the trained target detection model, thereby improving both detection accuracy and speed, and enabling more accurate and efficient fulfillment of the actual detection needs of the production line.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0109] Figure 4 A schematic diagram of the steel surface defect detection device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown, and are described in detail below:

[0110] like Figure 4 As shown, the steel surface defect detection device 4 includes:

[0111] Image acquisition module 41 is used to acquire images of steel surface samples with known defect types from the steel production line;

[0112] The model training module 42 is used to build an algorithm model based on the YOLO algorithm and train the algorithm model based on sample images of steel surfaces with known defect types to obtain a target detection model.

[0113] The model quantization module 43 is used to perform quantization error analysis on the calculation speed and accuracy of the target detection model based on the quantization method, so as to obtain the defect detection model.

[0114] The type detection module 44 is used to determine the defect type of the surface sample image of the steel to be inspected based on the defect detection model and the surface sample image of the steel to be inspected.

[0115] This application provides a steel surface defect detection device. It acquires surface sample images of steel from a steel production line, showing known defect types. An algorithm model is built based on the YOLO algorithm and trained using these sample images to obtain a target detection model. A quantization method is used to analyze the computational speed and accuracy of the target detection model, resulting in a defect detection model. Based on the defect detection model and the surface sample images of the steel to be inspected, the defect type of the sample images is determined. This application improves the computational speed and accuracy of the defect detection model by quantifying the computational speed and accuracy of the trained target detection model, thereby improving both detection accuracy and speed, and enabling more accurate and efficient fulfillment of the actual inspection needs of the production line.

[0116] In one possible implementation, the model quantization module can be used for:

[0117] Determine whether the computation speed of the target detection model has reached the preset speed;

[0118] If the computation speed of the target detection model reaches the preset speed, then the target detection model will be used as the first detection model.

[0119] Determine whether the calculation accuracy of the first detection model reaches the preset accuracy;

[0120] If the calculation accuracy of the first detection model reaches the preset accuracy, then the first detection model will be used as the defect detection model.

[0121] In one possible implementation, the model quantization module can also be used for:

[0122] If the computation speed of the target detection model does not reach the preset speed, the computation precision of each layer in the target detection model will be reduced, and the target detection model with reduced precision will be returned to the step of judging whether the computation speed of the target detection model has reached the preset speed to continue execution until the computation speed of the target detection model reaches the preset speed.

[0123] In one possible implementation, the model quantization module can also be used for:

[0124] If the computational accuracy of the first detection model does not reach the preset accuracy, then the quantization error corresponding to each layer in the first detection model is calculated based on the computational accuracy of the initial target detection model and the computational accuracy of the first detection model.

[0125] Determine whether the quantization error of each layer is 0;

[0126] If the quantization error of each layer is 0, then the model selected by the user from the first detection model and the target detection model is obtained, and the model selected by the user is used as the defect detection model.

[0127] If the quantization errors of each layer of the first detection model are not all zero, then the quantization errors of each layer in the first detection model are sorted in descending order according to the error values ​​to obtain the error value sequence.

[0128] Select the layer corresponding to the first N quantization errors in the error value sequence as the first target layer, and update the calculation accuracy of the corresponding layer in the first detection model using the calculation accuracy of the first target layer of the initial target detection model. Return to the step of judging whether the calculation accuracy of the first detection model has reached the preset accuracy and continue to execute until the calculation accuracy of the first detection model reaches the preset accuracy. Then, use the first detection model with the calculation accuracy reaching the preset accuracy as the defect detection model.

[0129] In one possible implementation, the model quantization module can also be used for:

[0130] The quantization error for each layer in the first detection model is calculated using the first formula, which is:

[0131]

[0132] Where, ε i Let be the quantization error corresponding to the i-th layer. Let be the computational precision corresponding to the i-th layer in the initial target detection model. This represents the computational precision corresponding to the i-th layer in the first detection model.

[0133] In one possible implementation, the model quantization module can also be used for:

[0134] The first detection model, whose calculation accuracy reaches the preset accuracy, is used as the second detection model. The calculation speed of the second detection model is then judged to see if it reaches the preset speed.

[0135] If the calculation speed of the second detection model reaches the preset speed, then the second detection model will be used as the defect detection model.

[0136] If the computation speed of the second detection model does not reach the preset speed, the parameter quantity of each first target layer in the second detection model is obtained, and the parameter quantity of each first target layer in the second detection model is sorted in descending order to obtain the parameter quantity sequence.

[0137] Select the layer corresponding to the first M parameters in the parameter sequence as the second target layer, update the calculation precision of the second target layer in the second detection model to the calculation precision of the layer corresponding to the initial first detection model, return to the step of judging whether the calculation speed of the second detection model has reached the preset speed, continue to execute until the calculation speed of the second detection model reaches the preset speed, and use the second detection model with the calculation speed reaching the preset speed as the defect detection model.

[0138] In one possible implementation, the YOLO algorithm may include the YOLOv5 algorithm, the YOLOv6 algorithm, the YOLOv7 algorithm, and the YOLOv8 algorithm;

[0139] The model training module can be used for:

[0140] The first algorithm model was built using YOLOv5, the second algorithm model was built using YOLOv6, the third algorithm model was built using YOLOv7, and the fourth algorithm model was built using YOLOv8, respectively.

[0141] Using steel surface sample images as input and the corresponding defect types as output, the first, second, third, and fourth algorithm models are trained respectively; the defect types include indentations, scratches, oil stains, and dirt.

[0142] The areas enclosed by the ROC curves and coordinate axes of the first, second, third, and fourth algorithm models after training are obtained respectively, to obtain the first AUC of the first algorithm model, the second AUC of the second algorithm model, the third AUC of the third algorithm model, and the fourth AUC of the fourth algorithm model.

[0143] The algorithm model corresponding to the maximum value among the first, second, third, and fourth AUCs is selected as the object detection model.

[0144] Figure 5 This is a schematic diagram of the detection platform provided in an embodiment of this application. Figure 5 As shown, the detection platform 5 in this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, it implements the steps in the various steel surface defect detection method embodiments described above, for example... Figure 1 Steps 101 to 104 are shown. Alternatively, when the processor 50 executes the computer program 52, it implements the functions of each module in the above-described device embodiments, for example... Figure 4 The functions of each module are shown.

[0145] For example, the computer program 52 can be divided into one or more modules, which are stored in the memory 51 and executed by the processor 50 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 52 in the detection platform 5. For example, the computer program 52 can be divided into... Figure 4 The modules shown.

[0146] The detection platform 5 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The detection platform 5 may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that... Figure 5 This is merely an example of the detection platform 5 and does not constitute a limitation on the detection platform 5. It may include more or fewer components than shown, or combine certain components, or different components. For example, the detection platform may also include input / output devices, network access devices, buses, etc.

[0147] The processor 50 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0148] The memory 51 can be an internal storage unit of the detection platform 5, such as a hard drive or memory of the detection platform 5. The memory 51 can also be an external storage device of the detection platform 5, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD card, Flash Card), etc., equipped on the detection platform 5. Furthermore, the memory 51 can include both internal storage units and external storage devices of the detection platform 5. The memory 51 is used to store the computer program and other programs and data required by the detection platform. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0150] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] In the embodiments provided in this application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0155] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various steel surface defect detection method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0156] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for detecting surface defects in steel, characterized in that, include: Obtain surface sample images of steel from steel production lines with known defect types; An algorithm model was built based on the YOLO algorithm, and the algorithm model was trained based on sample images of steel surfaces with known defect types to obtain a target detection model. Based on the quantization method, the computational speed and accuracy of the target detection model are analyzed by quantization error analysis to obtain the defect detection model; Based on the defect detection model and the surface sample image of the steel to be detected, the defect type of the surface sample image of the steel to be detected is determined; The step of performing quantization error analysis on the computational speed and accuracy of the target detection model based on quantization methods to obtain the defect detection model includes: Determine whether the computation speed of the target detection model has reached the preset speed; If the calculation speed of the target detection model reaches the preset speed, then the target detection model is used as the first detection model; Determine whether the calculation accuracy of the first detection model reaches the preset accuracy; If the calculation accuracy of the first detection model reaches the preset accuracy, then the first detection model is used as the defect detection model; The step of determining whether the calculation accuracy of the first detection model reaches the preset accuracy further includes: If the calculation accuracy of the first detection model does not reach the preset accuracy, then based on the calculation accuracy of the initial target detection model and the calculation accuracy of the first detection model, the quantization error corresponding to each layer in the first detection model is calculated. Determine whether the quantization error of each layer is 0; If the quantization error of each layer is 0, then the model selected by the user from the first detection model and the target detection model is obtained, and the model selected by the user is used as the defect detection model. If the quantization error of each layer of the first detection model is not all zero, then the quantization error of each layer in the first detection model is sorted in descending order according to the error value to obtain the error value sequence. Select the layer corresponding to the first N quantization errors in the error value sequence as the first target layer, and update the calculation accuracy of the corresponding layer in the first detection model using the calculation accuracy of the first target layer of the initial target detection model. Return to the step of judging whether the calculation accuracy of the first detection model has reached the preset accuracy and continue to execute until the calculation accuracy of the first detection model reaches the preset accuracy. Then, use the first detection model with the calculation accuracy reaching the preset accuracy as the defect detection model. The step of using a first detection model with a calculation accuracy reaching the preset accuracy as the defect detection model includes: The first detection model whose calculation accuracy reaches the preset accuracy is used as the second detection model, and it is determined whether the calculation speed of the second detection model reaches the preset speed. If the calculation speed of the second detection model reaches the preset speed, then the second detection model will be used as the defect detection model. If the calculation speed of the second detection model does not reach the preset speed, the parameter quantity of each first target layer in the second detection model is obtained, and the parameter quantity of each first target layer in the second detection model is sorted in descending order to obtain a parameter quantity sequence. Select the layer corresponding to the first M parameters in the parameter sequence as the second target layer, update the calculation accuracy of the second target layer in the second detection model to the calculation accuracy of the layer corresponding to the initial first detection model, return to the step of judging whether the calculation speed of the second detection model has reached the preset speed and continue to execute until the calculation speed of the second detection model reaches the preset speed, and use the second detection model with the calculation speed reaching the preset speed as the defect detection model.

2. The method for detecting surface defects in steel according to claim 1, characterized in that, The step of determining whether the computation speed of the target detection model has reached a preset speed further includes: If the computation speed of the target detection model does not reach the preset speed, the computation precision of each layer in the target detection model is reduced, and the target detection model with reduced precision is returned to the step of judging whether the computation speed of the target detection model has reached the preset speed to continue execution until the computation speed of the target detection model reaches the preset speed.

3. The method for detecting surface defects in steel according to claim 1, characterized in that, The calculation of the quantization error corresponding to each layer in the first detection model, based on the computational accuracy of the initial target detection model and the computational accuracy of the first detection model, includes: The quantization error of each layer in the first detection model is calculated using a first formula, which is: in, For the first The quantization error corresponding to the layer, For the initial object detection model, the first The calculation precision corresponding to the layer, For the first detection model, the first The calculation precision corresponding to each layer.

4. The method for detecting surface defects in steel according to claim 1, characterized in that, The YOLO algorithms include YOLOv5, YOLOv6, YOLOv7, and YOLOv8. The algorithm model is built based on the YOLO algorithm, and trained on steel surface sample images with known defect types to obtain a target detection model, including: The first algorithm model was built using YOLOv5, the second algorithm model was built using YOLOv6, the third algorithm model was built using YOLOv7, and the fourth algorithm model was built using YOLOv8. Using a steel surface sample image as input and the corresponding defect type as output, the first algorithm model, the second algorithm model, the third algorithm model, and the fourth algorithm model are trained respectively; the defect types include indentations, scratches, oil stains, and dirt. The areas enclosed by the ROC curves and coordinate axes of the first, second, third, and fourth algorithm models after training are obtained respectively, to obtain the first AUC of the first algorithm model, the second AUC of the second algorithm model, the third AUC of the third algorithm model, and the fourth AUC of the fourth algorithm model. The algorithm model corresponding to the maximum value among the first AUC, the second AUC, the third AUC, and the fourth AUC is selected as the target detection model.

5. A steel surface defect detection device, characterized in that, include: The image acquisition module is used to acquire surface sample images of steel from steel production lines for known defect types. The model training module is used to build an algorithm model based on the YOLO algorithm, and to train the algorithm model based on sample images of steel surfaces with known defect types to obtain a target detection model. The model quantization module is used to perform quantization error analysis on the calculation speed and accuracy of the target detection model based on quantization methods, so as to obtain the defect detection model. The type detection module is used to determine the defect type of the surface sample image of the steel to be inspected based on the defect detection model and the surface sample image of the steel to be inspected. The model quantization module is used for: Determine whether the computation speed of the target detection model has reached the preset speed; If the calculation speed of the target detection model reaches the preset speed, then the target detection model is used as the first detection model; Determine whether the calculation accuracy of the first detection model reaches the preset accuracy; If the calculation accuracy of the first detection model reaches the preset accuracy, then the first detection model is used as the defect detection model; The model quantization module is also used for: If the calculation accuracy of the first detection model does not reach the preset accuracy, then based on the calculation accuracy of the initial target detection model and the calculation accuracy of the first detection model, the quantization error corresponding to each layer in the first detection model is calculated. Determine whether the quantization error of each layer is 0; If the quantization error of each layer is 0, then the model selected by the user from the first detection model and the target detection model is obtained, and the model selected by the user is used as the defect detection model. If the quantization error of each layer of the first detection model is not all zero, then the quantization error of each layer in the first detection model is sorted in descending order according to the error value to obtain the error value sequence. Select the layer corresponding to the first N quantization errors in the error value sequence as the first target layer, and update the calculation accuracy of the corresponding layer in the first detection model using the calculation accuracy of the first target layer of the initial target detection model. Return to the step of judging whether the calculation accuracy of the first detection model has reached the preset accuracy and continue to execute until the calculation accuracy of the first detection model reaches the preset accuracy. Then, use the first detection model with the calculation accuracy reaching the preset accuracy as the defect detection model. The model quantization module is also used for: The first detection model whose calculation accuracy reaches the preset accuracy is used as the second detection model, and it is determined whether the calculation speed of the second detection model reaches the preset speed. If the calculation speed of the second detection model reaches the preset speed, then the second detection model will be used as the defect detection model. If the calculation speed of the second detection model does not reach the preset speed, the parameter quantity of each first target layer in the second detection model is obtained, and the parameter quantity of each first target layer in the second detection model is sorted in descending order to obtain a parameter quantity sequence. Select the layer corresponding to the first M parameters in the parameter sequence as the second target layer, update the calculation accuracy of the second target layer in the second detection model to the calculation accuracy of the layer corresponding to the initial first detection model, return to the step of judging whether the calculation speed of the second detection model has reached the preset speed and continue to execute until the calculation speed of the second detection model reaches the preset speed, and use the second detection model with the calculation speed reaching the preset speed as the defect detection model.

6. A detection platform, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the steel surface defect detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the steel surface defect detection method as described in any one of claims 1 to 4.

Images