Strip steel surface defect detection method and system based on depth map

Abstract

The invention provides a strip steel surface defect detection method and system based on a depth map, and the method comprises the steps: S1, carrying out data collection, obtaining steel plate surface point cloud data, and then generating a point cloud data set; s2, after a point cloud data set is obtained, preprocessing is carried out, and defect detection is carried out; s3, judging the category of the defect image; and S4, respectively displaying the defects of the steel plate in combination with detection results. Through the man-machine interaction module provided by the invention, detection results including depth, size, proportion and category conclusions of defects can be visually displayed to a user, so that the user can conveniently know the defect condition of the surface of the steel plate.

Description

Technical Field

[0001] The present invention relates to the technical field of steel plate surface defect detection, and in particular to a strip steel surface defect detection method and system based on a depth map. Background Art

[0002] Traditional defect detection based on 2D images relies on the contrast of the object being tested. The detection effect is easily disturbed by the background texture, and it is difficult to accurately detect surface defects with depth. In recent years, 3D machine vision technology has achieved rapid development. Compared with 2D technology, 3D technology can provide the depth Z value of the object in addition to displaying the brightness value of the object.

[0003] Patent document CN116539628A discloses a steel plate surface defect detection system based on 3D vision technology, including: a data acquisition terminal, which is arranged on the upper and lower sides of the surface of the steel plate to be detected, and is used to collect the surface texture image of the steel plate at a specific angle to generate point cloud data; a central server, which is connected to the data acquisition terminal by communication; a monitoring device, including a human-computer interaction unit and a control unit, the control unit is used to control the data acquisition work of the data acquisition terminal, the human-computer interaction unit is used to control the control unit, and the quality defect detection results obtained by the central server after processing the point cloud data. However, the invention does not involve a strip surface defect detection algorithm based on a depth map.

[0004] Patent document CN115564720A discloses a strip defect detection method and system based on 3D scanning and machine vision, which includes: using a camera and a 3D scanning device to obtain 2D image information and 3D contour information of the strip at the same position; obtaining the boundary of the strip according to the 2D image information and 3D contour information; using a preset machine learning model according to preset features to identify and classify the defects in the image, and locate the acquired defects based on the boundary; outputting the identified defect image and issuing an alarm. However, the invention does not involve a strip surface defect detection algorithm based on a depth map. Summary of the invention

[0005] In view of the defects in the prior art, the object of the present invention is to provide a method and system for detecting surface defects of strip steel based on a depth map.

[0006] A strip steel surface defect detection method based on a depth map provided by the present invention comprises:

[0007] Step S1: perform data collection, obtain point cloud data of the steel plate surface and generate a point cloud data set;

[0008] Step S2: After obtaining the point cloud data set, pre-process it and perform defect detection;

[0009] Step S3: Determine the category of the defect image;

[0010] Step S4: display the steel plate defects respectively in combination with the detection results.

[0011] Preferably, in step S1:

[0012] The data acquisition end is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to different angles to face the steel plate surface. The camera obtains two-dimensional image information and three-dimensional point cloud information of the steel plate surface in real time. The 3D laser scanning camera obtains the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set.

[0013] Preferably, in step S2:

[0014] After obtaining the point cloud data set, process it, including:

[0015] Step S2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters;

[0016] Step S2.2: preprocessing the point cloud data set: performing multiple filtering operations such as straight-through filtering, line-by-line outlier filtering and smoothing filtering, and full-image Z-mean normalization on the point cloud data to filter out the noise points of the point cloud data and restore the point cloud data of the real shape of the steel plate;

[0017] Step S2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, extract the suspicious area through binarization and mark the defect points according to the preset threshold.

[0018] Preferably, the process of calculating the relative depth includes:

[0019] Taking the current pixel as the center, calculate the average height of the surrounding pixels, and obtain the relative depth of the pixel relative to the surrounding area by calculating the difference between the height of the current pixel and the average height of the surrounding pixels;

[0020] The process of extracting suspicious areas includes:

[0021] Based on the gray value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to black 0, and pixels with values ​​greater than or equal to the threshold are set to white 255, converting the image into a binary image with only black and white colors;

[0022] Step S2.4: Execute a clustering algorithm on the defect points to form a complete defect, determine the defect location, and calculate the distance between the defect and the strip head. The location of the strip head in the image and the location of the defect in the image are known to obtain the distance between the defect and the strip head, which is used for the next unit to locate the longitudinal position of the defect;

[0023] Step S2.5: Filter the detected pseudo defects using defect filtering rules;

[0024] Step S2.6: Convert the point cloud data height value into RGB color value according to the color table, and transmit the obtained color strip image to the defect classification module.

[0025] Preferably, in step S3:

[0026] Provide a deep learning classification model, through which the category of defect images is determined. The deep learning classification model is trained based on a neural network algorithm and a large-scale training data set, and can automatically learn and extract features from defect images and classify them into different categories;

[0027] The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps:

[0028] Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels; resize the images in the dataset to 100x100 and perform preprocessing operations;

[0029] Model construction: Use the PyTorch deep learning framework, import the ResNet18 model, and modify the last fully connected layer to adapt to the number of categories of the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers;

[0030] Loss function definition: Use the cross entropy loss function to measure the gap between the model prediction results and the true label;

[0031] Data loading and training: Divide the prepared data set into a training set and a validation set, and use the data loader to input the data into the model. Use the training set to train the model, and update the model parameters by calculating the loss function and using the optimization algorithm. Use the learning rate scheduler to dynamically adjust the learning rate during the training process.

[0032] Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, stop training; otherwise, continue to adjust the model's hyperparameters or increase the number of training rounds;

[0033] Model saving and deployment: After training is completed, the model is saved to disk and used and deployed in actual applications.

[0034] A strip steel surface defect detection system based on a depth map provided by the present invention comprises:

[0035] Module M1: perform data acquisition, obtain the point cloud data of the steel plate surface and generate a point cloud data set;

[0036] Module M2: After obtaining the point cloud data set, pre-process it and perform defect detection;

[0037] Module M3: Determine the category of defective images;

[0038] Module M4: Display steel plate defects based on the test results.

[0039] Preferably, in the module M1:

[0040] The data acquisition end is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to different angles to face the steel plate surface. The camera obtains two-dimensional image information and three-dimensional point cloud information of the steel plate surface in real time. The 3D laser scanning camera obtains the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set.

[0041] Preferably, in the module M2:

[0042] After obtaining the point cloud data set, process it, including:

[0043] Module M2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters;

[0044] Module M2.2: Preprocessing of point cloud data set: performing straight-through filtering, line-by-line outlier filtering and smoothing filtering, full-image Z-mean normalization and other filtering operations on the point cloud data to filter out the noise points of the point cloud data and restore the point cloud data of the real shape of the steel plate;

[0045] Module M2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, extract the suspicious area through binarization and mark the defect points according to the preset threshold.

[0046] Preferably, the process of calculating the relative depth includes:

[0047] Taking the current pixel as the center, calculate the average height of the surrounding pixels, and obtain the relative depth of the pixel relative to the surrounding area by calculating the difference between the height of the current pixel and the average height of the surrounding pixels;

[0048] The process of extracting suspicious areas includes:

[0049] Based on the gray value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to black 0, and pixels with values ​​greater than or equal to the threshold are set to white 255, converting the image into a binary image with only black and white colors;

[0050] Module M2.4: Execute clustering algorithm on defect points to form a complete defect, determine the defect position, and calculate the distance between the defect and the strip head. The distance between the defect and the strip head is obtained by knowing the location of the strip head in the image and the location of the defect in the image, which is used for the next unit to locate the longitudinal position of the defect;

[0051] Module M2.5: Use defect filtering rules to filter detected pseudo defects;

[0052] Module M2.6: Convert the point cloud data height value into RGB color value according to the color table, obtain the color strip image and transmit it to the defect classification module.

[0053] Preferably, in the module M3:

[0054] Provide a deep learning classification model, through which the category of defect images is determined. The deep learning classification model is trained based on a neural network algorithm and a large-scale training data set, and can automatically learn and extract features from defect images and classify them into different categories;

[0055] The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps:

[0056] Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels; resize the images in the dataset to 100x100 and perform preprocessing operations;

[0057] Model construction: Use the PyTorch deep learning framework, import the ResNet18 model, and modify the last fully connected layer to adapt to the number of categories of the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers;

[0058] Loss function definition: Use the cross entropy loss function to measure the gap between the model prediction results and the true label;

[0059] Data loading and training: Divide the prepared data set into a training set and a validation set, and use the data loader to input the data into the model. Use the training set to train the model, and update the model parameters by calculating the loss function and using the optimization algorithm. Use the learning rate scheduler to dynamically adjust the learning rate during the training process.

[0060] Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, stop training; otherwise, continue to adjust the model's hyperparameters or increase the number of training rounds;

[0061] Model saving and deployment: After training is completed, the model is saved to disk and used and deployed in actual applications.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. The data acquisition module provided by the present invention uses a 3D laser scanning camera for data acquisition, which can obtain point cloud data of the steel plate surface in real time. By processing the data set, it can accurately detect defects on the steel plate surface. At the same time, it provides brightness and height maps of defects, and users can easily identify the type and severity of defects by observing 2D and 3D images.

[0064] 2. The present invention provides a set of deep learning classification models through the defect classification module, which can classify and determine the detected defects. The deep learning model has certain versatility and flexibility, and can be trained and adjusted according to actual needs to adapt to different types of defect detection tasks. This can better understand the nature and severity of defects, help users make more accurate judgments and decisions, realize automated defect detection, reduce the need for manual operation, and improve the efficiency and accuracy of detection. At the same time, the accuracy and robustness of the model have also been verified and optimized, and can be effectively applied in actual industrial environments.

[0065] 3. The present invention provides a human-computer interaction module, which can display the detection results to the user in an intuitive manner, including the depth, size, proportion and category conclusion of the defects, so that the user can easily understand the defects on the surface of the steel plate. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0067] Figure 1 A schematic diagram of the defect two-dimensional image information and three-dimensional contour information provided by the present invention;

[0068] Figure 2 A flow chart of a strip steel surface defect detection algorithm based on a depth map provided by the present invention;

[0069] Figure 3 A schematic diagram of the steps of the defect detection algorithm provided by the present invention;

[0070] Figure 4 This is an example diagram of defect classification results provided by the present invention, from left to right, they are pits, cracks, peeling, and embossing;

[0071] Figure 5 A schematic diagram of the module structure of a strip steel surface defect detection algorithm based on a depth map provided by the present invention.

[0072] Figure 6 The human-computer interaction module display interface provided by the present invention. DETAILED DESCRIPTION

[0073] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0074] Embodiment 1:

[0075] The development and application of the present invention will fill the shortcomings of traditional 2D imaging technology and bring unique progress in industrial object detection. Especially for the surface state of steel plates, it can overcome the complexity of the surface texture, focus on the detection of deep defects to achieve quantitative measurement, and thus improve the quality and safety of steel production. By using a 3D vision system to directly measure the height data of the object surface, the sensitivity to the surface texture of the object and changes in ambient light is reduced, and the stability of the detection results is improved. At the same time, based on the 3D features of the defects, this method can significantly improve the accuracy of defect classification. In addition, the method can also provide brightness maps and height maps of the defects, allowing users to easily judge the type and severity of the defects by observing 2D and 3D images. Through these advantages, the 3D steel plate surface defect detection method can more accurately detect and evaluate the defects on the surface of the steel plate, and improve the efficiency and accuracy of quality inspection.

[0076] According to a strip steel surface defect detection method based on a depth map provided by the present invention, Figure 1-Figure 6 As shown, including:

[0077] Step S1: perform data collection, obtain point cloud data of the steel plate surface and generate a point cloud data set;

[0078] Specifically, in step S1:

[0079] The data acquisition end is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to different angles to face the steel plate surface. The camera obtains two-dimensional image information and three-dimensional point cloud information of the steel plate surface in real time. The 3D laser scanning camera obtains the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set.

[0080] Step S2: After obtaining the point cloud data set, pre-process it and perform defect detection;

[0081] Specifically, in step S2:

[0082] After obtaining the point cloud data set, process it, including:

[0083] Step S2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters;

[0084] Step S2.2: preprocessing the point cloud data set: performing multiple filtering operations such as straight-through filtering, line-by-line outlier filtering and smoothing filtering, and full-image Z-mean normalization on the point cloud data to filter out the noise points of the point cloud data and restore the point cloud data of the real shape of the steel plate;

[0085] Step S2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, extract the suspicious area through binarization and mark the defect points according to the preset threshold.

[0086] Specifically, the relative depth calculation process includes:

[0087] Taking the current pixel as the center, calculate the average height of the surrounding pixels, and obtain the relative depth of the pixel relative to the surrounding area by calculating the difference between the height of the current pixel and the average height of the surrounding pixels;

[0088] The process of extracting suspicious areas includes:

[0089] Based on the gray value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to black 0, and pixels with values ​​greater than or equal to the threshold are set to white 255, converting the image into a binary image with only black and white colors;

[0090] Step S2.4: Execute a clustering algorithm on the defect points to form a complete defect, determine the defect location, and calculate the distance between the defect and the strip head. The location of the strip head in the image and the location of the defect in the image are known to obtain the distance between the defect and the strip head, which is used for the next unit to locate the longitudinal position of the defect;

[0091] Step S2.5: Filter the detected pseudo defects using defect filtering rules;

[0092] Step S2.6: Convert the point cloud data height value into RGB color value according to the color table, and transmit the obtained color strip image to the defect classification module.

[0093] Step S3: Determine the category of the defect image;

[0094] Specifically, in step S3:

[0095] Provide a deep learning classification model, through which the category of defect images is determined. The deep learning classification model is trained based on a neural network algorithm and a large-scale training data set, and can automatically learn and extract features from defect images and classify them into different categories;

[0096] The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps:

[0097] Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels; resize the images in the dataset to 100x100 and perform preprocessing operations;

[0098] Model construction: Use the PyTorch deep learning framework, import the ResNet18 model, and modify the last fully connected layer to adapt to the number of categories of the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers;

[0099] Loss function definition: Use the cross entropy loss function to measure the gap between the model prediction results and the true label;

[0100] Data loading and training: Divide the prepared data set into a training set and a validation set, and use the data loader to input the data into the model. Use the training set to train the model, and update the model parameters by calculating the loss function and using the optimization algorithm. Use the learning rate scheduler to dynamically adjust the learning rate during the training process.

[0101] Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, stop training; otherwise, continue to adjust the model's hyperparameters or increase the number of training rounds;

[0102] Model saving and deployment: After training is completed, the model is saved to disk and used and deployed in actual applications.

[0103] Step S4: display the steel plate defects respectively in combination with the detection results.

[0104] Embodiment 2:

[0105] Embodiment 2 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0106] The present invention also provides a strip surface defect detection system based on a depth map, and the strip surface defect detection system based on a depth map can be realized by executing the process steps of the strip surface defect detection method based on a depth map, that is, technical personnel in this field can understand the strip surface defect detection method based on a depth map as a preferred implementation of the strip surface defect detection system based on a depth map.

[0107] A strip steel surface defect detection system based on a depth map provided by the present invention comprises:

[0108] Module M1: perform data acquisition, obtain the point cloud data of the steel plate surface and generate a point cloud data set;

[0109] Specifically, in the module M1:

[0110] The data acquisition end is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to different angles to face the steel plate surface. The camera obtains two-dimensional image information and three-dimensional point cloud information of the steel plate surface in real time. The 3D laser scanning camera obtains the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set.

[0111] Module M2: After obtaining the point cloud data set, pre-process it and perform defect detection;

[0112] Specifically, in the module M2:

[0113] After obtaining the point cloud data set, process it, including:

[0114] Module M2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters;

[0115] Module M2.2: Preprocessing of point cloud data set: performing straight-through filtering, line-by-line outlier filtering and smoothing filtering, full-image Z-mean normalization and other filtering operations on the point cloud data to filter out the noise points of the point cloud data and restore the point cloud data of the real shape of the steel plate;

[0116] Module M2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, extract the suspicious area through binarization and mark the defect points according to the preset threshold.

[0117] Specifically, the relative depth calculation process includes:

[0118] Taking the current pixel as the center, calculate the average height of the surrounding pixels, and obtain the relative depth of the pixel relative to the surrounding area by calculating the difference between the height of the current pixel and the average height of the surrounding pixels;

[0119] The process of extracting suspicious areas includes:

[0120] Based on the gray value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to black 0, and pixels with values ​​greater than or equal to the threshold are set to white 255, converting the image into a binary image with only black and white colors;

[0121] Module M2.4: Execute clustering algorithm on defect points to form a complete defect, determine the defect position, and calculate the distance between the defect and the strip head. The distance between the defect and the strip head is obtained by knowing the location of the strip head in the image and the location of the defect in the image, which is used for the next unit to locate the longitudinal position of the defect;

[0122] Module M2.5: Use defect filtering rules to filter detected pseudo defects;

[0123] Module M2.6: Convert the point cloud data height value into RGB color value according to the color table, obtain the color strip image and transmit it to the defect classification module.

[0124] Module M3: Determine the category of defective images;

[0125] Specifically, in the module M3:

[0126] Provide a deep learning classification model, through which the category of defect images is determined. The deep learning classification model is trained based on a neural network algorithm and a large-scale training data set, and can automatically learn and extract features from defect images and classify them into different categories;

[0127] The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps:

[0128] Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels; resize the images in the dataset to 100x100 and perform preprocessing operations;

[0129] Model construction: Use the PyTorch deep learning framework, import the ResNet18 model, and modify the last fully connected layer to adapt to the number of categories of the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers;

[0130] Loss function definition: Use the cross entropy loss function to measure the gap between the model prediction results and the true label;

[0131] Data loading and training: Divide the prepared data set into a training set and a validation set, and use the data loader to input the data into the model. Use the training set to train the model, and update the model parameters by calculating the loss function and using the optimization algorithm. Use the learning rate scheduler to dynamically adjust the learning rate during the training process.

[0132] Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, stop training; otherwise, continue to adjust the model's hyperparameters or increase the number of training rounds;

[0133] Model saving and deployment: After training is completed, the model is saved to disk and used and deployed in actual applications.

[0134] Module M4: Display steel plate defects based on the test results.

[0135] Embodiment 3:

[0136] Embodiment 3 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0137] The present invention provides a strip steel surface defect detection algorithm based on a depth map, comprising the following steps:

[0138] S1: Data acquisition module: The data acquisition terminal is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to face the steel plate surface at different angles. The camera acquires the 2D image information and 3D point cloud information of the steel plate surface in real time. The 3D laser scanning camera acquires the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set;

[0139] S2: Defect detection module: The data acquisition module acquires the point cloud data set and processes it, including:

[0140] S2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters.

[0141] S2.2: Preprocessing the point cloud data set: performing multiple filtering operations such as straight-through filtering, row-by-row outlier filtering and smoothing filtering, and full-image Z-mean normalization on the point cloud data to filter out the noise points in the point cloud data and restore the point cloud data of the real shape of the steel plate.

[0142] S2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, and extract the suspicious area through binarization. Mark the defect points according to the preset threshold.

[0143] The relative depth calculation process includes:

[0144] Taking the current pixel as the center, calculate the average height of the surrounding pixels. By calculating the difference between the height of the current pixel and the average height of the surrounding pixels, the relative depth of the pixel relative to the surrounding area can be obtained.

[0145] The process of extracting suspicious areas includes:

[0146] Based on the grayscale value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to 0 (black), and pixels with values ​​greater than or equal to the threshold are set to 255 (white), thereby converting the image into a binary image with only black and white colors.

[0147] The binarization formula can be expressed as:

[0148] if pixel_value <threshold:

[0149] binary_value=0

[0150] else:

[0151] binary_value=255

[0152] Among them, pixel_value represents the pixel value in the original image, and threshold represents the set threshold.

[0153] S2.4: Perform clustering algorithm on defect points to form a complete defect. Determine the defect location and calculate the distance between the defect and the head. The distance between the defect and the head can be obtained by knowing the location of the head in the image and the location of the defect in the image.

[0154] S2.5: Use defect filtering rules to filter the detected pseudo defects.

[0155] S2.6: Convert the height value of the point cloud data into RGB color value according to the color table, and transmit the obtained color strip image to the defect classification module.

[0156] S3: 2D image shows defect grayscale information, 3D image shows defect depth information. 2D image and 3D image are displayed synchronously at pixel level. Figure 1 shown.

[0157] The defect category is determined by the defect classification module, and the human-machine module displays the depth, size, proportion and category conclusion of the defect for users to use. The display interface is as follows: Figure 6 shown.

[0158] Defect classification module: provides a deep learning classification model that determines the category of defect images.

[0159] This deep learning model is trained based on advanced neural network algorithms and large-scale training data sets. It can automatically learn and extract features from defect images and classify them into different categories.

[0160] The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps:

[0161] Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels. Resize the images in the dataset to 100x100 and perform preprocessing operations such as normalization.

[0162] Model construction: Use deep learning frameworks such as PyTorch to import the ResNet18 model and modify the final fully connected layer to adapt to the number of categories in the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers.

[0163] Loss function definition: The cross entropy loss function is used to measure the gap between the model prediction results and the true labels.

[0164] Data loading and training: The prepared dataset is divided into training set and validation set, and the data is input into the model using the data loader. Then, the model is trained using the training set, and the parameters of the model are updated by calculating the loss function and using the optimization algorithm. The learning rate scheduler is used to dynamically adjust the learning rate during the training process.

[0165] Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, you can stop training; otherwise, you can continue to adjust the model's hyperparameters or increase the number of training rounds.

[0166] Model saving and deployment: After training is complete, save the model to disk so that it can be used and deployed in actual applications.

[0167] S4: Human-computer interaction module: Combined with the test results, it displays the depth of steel plate defects, length and width of defects, defect proportion and defect category conclusion.

[0168] The present invention can be promoted and applied in 3D detection of surface quality of cold-rolled strip steel, as well as in 3D detection of surface quality of medium and thick plates, continuous casting billets and steel pipes.

[0169] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0170] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for detecting surface defects of strip steel based on a depth map, characterized in that: include: Step S1: perform data collection, obtain point cloud data of the steel plate surface and generate a point cloud data set; Step S2: After obtaining the point cloud data set, pre-process it and perform defect detection; Step S3: Determine the category of the defect image; Step S4: display the steel plate defects respectively in combination with the detection results.

2. The method for detecting surface defects of strip steel based on depth map according to claim 1, characterized in that: In step S1: The data acquisition end is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to different angles to face the steel plate surface. The camera obtains two-dimensional image information and three-dimensional point cloud information of the steel plate surface in real time. The 3D laser scanning camera obtains the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set.

3. The method for detecting surface defects of strip steel based on depth map according to claim 1, characterized in that: In step S2: After obtaining the point cloud data set, process it, including: Step S2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters; Step S2.2: preprocessing the point cloud data set: performing multiple filtering operations such as straight-through filtering, line-by-line outlier filtering and smoothing filtering, and full-image Z-mean normalization on the point cloud data to filter out the noise points of the point cloud data and restore the point cloud data of the real shape of the steel plate; Step S2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, extract the suspicious area through binarization and mark the defect points according to the preset threshold.

4. The method for detecting surface defects of strip steel based on depth map according to claim 3, characterized in that: The relative depth calculation process includes: Taking the current pixel as the center, calculate the average height of the surrounding pixels, and obtain the relative depth of the pixel relative to the surrounding area by calculating the difference between the height of the current pixel and the average height of the surrounding pixels; The process of extracting suspicious areas includes: Based on the gray value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to black 0, and pixels with values ​​greater than or equal to the threshold are set to white 255, converting the image into a binary image with only black and white colors; Step S2.4: Execute a clustering algorithm on the defect points to form a complete defect, determine the defect location, and calculate the distance between the defect and the strip head. The location of the strip head in the image and the location of the defect in the image are known to obtain the distance between the defect and the strip head, which is used for the next unit to locate the longitudinal position of the defect; Step S2.5: Filter the detected pseudo defects using defect filtering rules; Step S2.6: Convert the point cloud data height value into RGB color value according to the color table, and transmit the obtained color strip image to the defect classification module.

5. The method for detecting surface defects of strip steel based on depth map according to claim 1, characterized in that: In step S3: Provide a deep learning classification model, through which the category of defect images is determined. The deep learning classification model is trained based on a neural network algorithm and a large-scale training data set, and can automatically learn and extract features from defect images and classify them into different categories; The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps: Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels; resize the images in the dataset to 100x100 and perform preprocessing operations; Model construction: Use the PyTorch deep learning framework, import the ResNet18 model, and modify the last fully connected layer to adapt to the number of categories of the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers; Loss function definition: Use the cross entropy loss function to measure the gap between the model prediction results and the true label; Data loading and training: Divide the prepared data set into a training set and a validation set, and use the data loader to input the data into the model. Use the training set to train the model, and update the model parameters by calculating the loss function and using the optimization algorithm. Use the learning rate scheduler to dynamically adjust the learning rate during the training process. Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, stop training; otherwise, continue to adjust the model's hyperparameters or increase the number of training rounds; Model saving and deployment: After training is completed, the model is saved to disk and used and deployed in actual applications.

6. A strip steel surface defect detection system based on depth map, characterized in that: include: Module M1: perform data acquisition, obtain the point cloud data of the steel plate surface and generate a point cloud data set; Module M2: After acquiring the point cloud data set, preprocess it and perform defect detection; Module M3: Determine the category of defective images; Module M4: Display steel plate defects based on the test results.

7. The strip surface defect detection system based on depth map according to claim 6 is characterized in that: In the module M1: The data acquisition end is set on the upper and lower sides of the steel plate surface, and the 3D laser scanning camera is adjusted to different angles to face the steel plate surface. The camera obtains two-dimensional image information and three-dimensional point cloud information of the steel plate surface in real time. The 3D laser scanning camera obtains the point cloud data of the steel plate surface in real time and transmits it to the defect detection module to generate a point cloud data set.

8. The strip steel surface defect detection system based on depth map according to claim 6, characterized in that: In the module M2: After obtaining the point cloud data set, process it, including: Module M2.1: The software reads the images output by each camera and stitches them into a complete steel plate point cloud image according to the calibration parameters; Module M2.2: Preprocessing of point cloud data set: performing straight-through filtering, line-by-line outlier filtering and smoothing filtering, full-image Z-mean normalization and other filtering operations on the point cloud data to filter out the noise points of the point cloud data and restore the point cloud data of the real shape of the steel plate; Module M2.3: Calculate the gradient of the point cloud data in the height direction, calculate the relative depth of each pixel relative to the surrounding area in real time, extract the suspicious area through binarization and mark the defect points according to the preset threshold.

9. The strip surface defect detection system based on depth map according to claim 8, characterized in that: The relative depth calculation process includes: Taking the current pixel as the center, calculate the average height of the surrounding pixels, and obtain the relative depth of the pixel relative to the surrounding area by calculating the difference between the height of the current pixel and the average height of the surrounding pixels; The process of extracting suspicious areas includes: Based on the gray value of the pixel, a threshold is set, and pixels with values ​​less than the threshold are set to black 0, and pixels with values ​​greater than or equal to the threshold are set to white 255, converting the image into a binary image with only black and white colors; Module M2.4: Execute clustering algorithm on defect points to form a complete defect, determine the defect position, and calculate the distance between the defect and the strip head. The distance between the defect and the strip head is obtained by knowing the location of the strip head in the image and the location of the defect in the image, which is used for the next unit to locate the longitudinal position of the defect; Module M2.5: Use defect filtering rules to filter detected pseudo defects; Module M2.6: Convert the point cloud data height value into RGB color value according to the color table, obtain the color strip image and transmit it to the defect classification module.

10. The strip steel surface defect detection system based on depth map according to claim 6, characterized in that: In the module M3: Provide a deep learning classification model, through which the category of defect images is determined. The deep learning classification model is trained based on a neural network algorithm and a large-scale training data set, and can automatically learn and extract features from defect images and classify them into different categories; The training process for deep learning defect classification using the ResNet18 model is carried out in the following steps: Data preparation: Prepare a dataset containing defect category annotations, including input samples and corresponding labels; resize the images in the dataset to 100x100 and perform preprocessing operations; Model construction: Use the PyTorch deep learning framework, import the ResNet18 model, and modify the last fully connected layer to adapt to the number of categories of the classification task. The structure of the ResNet18 model contains multiple convolutional layers, pooling layers, and fully connected layers; Loss function definition: Use the cross entropy loss function to measure the gap between the model prediction results and the true label; Data loading and training: Divide the prepared data set into a training set and a validation set, and use the data loader to input the data into the model. Use the training set to train the model, and update the model parameters by calculating the loss function and using the optimization algorithm. Use the learning rate scheduler to dynamically adjust the learning rate during the training process. Model evaluation: Use the validation set to evaluate the performance of the model and calculate indicators such as classification accuracy. If the performance of the model meets the requirements, stop training; otherwise, continue to adjust the model's hyperparameters or increase the number of training rounds; Model saving and deployment: After training is completed, the model is saved to disk and used and deployed in actual applications.

Images