A method and device for detecting the profile of a slab with surface water marks

Abstract

The application discloses a kind of slab contour detection methods and devices with surface water spot, method includes: step S1: obtaining intermediate slab image, construct sample data set and mark;Step S2: construct a convolutional neural network model, and use the data set marked to train, obtain the best feature extraction model;Step S3: wavelet transform is used to the pre-processing of slab image to be detected, and the best feature extraction model trained is used to the feature extraction of slab image to be detected after pre-processing, obtain the feature map of slab image to be detected;Step S4: the feature map of slab image to be detected is post-processed, complete the function of removing noise, smooth edge, the contour detection of slab image to be detected is completed by the method of extracting edge and finding maximum contour.The application can realize the real-time detection of intermediate slab contour, especially can be aimed at slab image with water spot, improve the precision of contour detection, and the method has strong robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of detection, in particular to a slab profile detection method and device with surface water stains. BACKGROUND

[0002] The slab side bending generated in the rough rolling stage has a great impact on production, which may cause excessive edge cutting, short length, reduced yield and sizing rate, and some visible defects such as edge creases, edge cracks, holes and scratches, affecting the plate shape and quality of the finished product and the rolling stability of the subsequent finishing process; or even fast stopping, scraping frame, causing steel stacking accidents at the finishing mill group, producing scrap steel, damaging the pusher and other equipment, and delaying a large amount of production time. At present, the industrial field generally uses a traditional image processing method to detect the rough rolling slab profile, which has low robustness and may cause detection errors under complex and variable working conditions. SUMMARY

[0003] The embodiment of the present application provides a slab profile detection method and device with surface water stains. The technical scheme is as follows:

[0004] In one aspect, a slab profile detection method with surface water stains is provided, which is realized by an electronic device, and the method comprises:

[0005] Step S1: acquiring an intermediate slab image, constructing a sample data set and marking;

[0006] Step S2: constructing a convolutional neural network model and training the labeled data set to obtain an optimal feature extraction model;

[0007] Step S3: pre-processing the to-be-detected slab image by wavelet transform, and extracting features of the pre-processed to-be-detected slab image by using the trained optimal feature extraction model to obtain a feature map of the to-be-detected slab image;

[0008] Step S4: post-processing the feature map of the to-be-detected slab image to complete the functions of removing noise and smoothing edges, and completing the profile detection of the to-be-detected slab image by extracting edges and finding the largest profile.

[0009] Optionally, the S1 specifically comprises:

[0010] S1-1: acquiring an RGB image of the intermediate slab with surface water stains by using an industrial CCD camera;

[0011] S1-2: cutting the effective area of the acquired slab image to obtain an effective area image;

[0012] S1-3, labeling the effective area image, selecting a profile label value set to constitute a sample data set;

[0013] S1-4, proportionally dividing the sample data set into a training set and a test set.

[0014] Optionally, the best feature extraction model in S2 is composed of seven edge detection modules combined with seven up-sampling modules. With the deepening of the network layers, deeper feature information in the image can be extracted, and the weak slab profile edge features covered by the water stains can be extracted. Finally, through the fusion of the seven side output images of the convolutional neural network, the weak edges affected by the water stains can also be extracted while obtaining obvious slab edge features.

[0015] The edge detection modules are fully connected through a 1x1 convolution kernel, and the side output results of the edge detection modules are used as the input of the up-sampling modules. Finally, the output feature maps of the seven up-sampling modules are fused to obtain the output results of the best feature extraction model.

[0016] The first edge detection module is composed of one convolution layer, the second edge detection module is composed of one convolution layer and one pooling layer, the third edge detection module is composed of two convolution layers and one pooling layer, the fourth edge detection module is composed of three convolution layers and one pooling layer, the fifth edge detection module is composed of three convolution layers, the sixth edge detection module is composed of three convolution layers, and the seventh edge detection module is composed of two convolution layers. The size of the convolution kernel of all edge detection modules is 3x3.

[0017] The calculation formula of the convolution layer is as follows:

[0018]

[0019] Where N represents the size of the output image after convolution, W represents the size of the input picture, F represents the size of the convolution kernel, P represents the number of pixels filled, and S represents the step length.

[0020] The calculation formula of the pooling layer is as follows:

[0021]

[0022] Where N represents the size of the output image after pooling, W represents the size of the input picture, F represents the size of the pooling kernel, P represents the number of pixels filled, S represents the step length, and G represents the control window element step.

[0023] The results output by the seven edge detection modules are respectively input into the up-sampling modules. The up-sampling modules use the bilinear interpolation method to up-sample the feature maps output by the edge detection modules.

[0024] Known Q 11 = (x1, y1), Q 12 = (x1, y2), Q 21 = (x2, y1), Q 22 = (x2, y2) four points, then the calculation formula of the bilinear interpolation is:

[0025]

[0026] Where f(x, y) represents the pixel value of the point P(x, y) to be solved;

[0027] The output feature maps of the seven up-sampling modules are image fused, and the weak slab profile features affected by noise are obtained by fusing the side output images of the seven different depths, and the obvious profile features of the slab not affected by noise are more prominently represented.

[0028] Optionally, the weighted cross-entropy is used as the loss function of the best feature extraction model;

[0029] The calculation formula of the weighted cross-entropy loss function is:

[0030] β = Y - / Y + + Y -

[0031] 1-β = Y + / Y + + Y -

[0032]

[0033]

[0034] Wherein, Y - and Y + respectively represent the edge and non-edge pixel points labeled in the image data set; n represents the number of edge detection modules; W represents all learning parameters in the model; w is the parameter corresponding to the n edge detection modules; σ() is a sigmoid function, which calculates the activation value of a pixel point; y j indicates whether the pixel j is marked as an edge; X represents the input image; δ represents the weight of each scale level; the sigmoid function can map a real number to the interval (0, 1), and its calculation formula is as follows:

[0035]

[0036] Optionally, the S3 specifically comprises:

[0037] S3-1, using wavelet transform to filter the to-be-detected slab image, filter out part of the background noise information, thereby improving the detection accuracy of the subsequent;

[0038] The to-be-detected slab image is a signal distributed along the space, which is a two-dimensional signal varying with the space axes x and y, and can be regarded as a group of digital arrays;

[0039] The wavelet transform is used to decompose the signal into a series of discrete approximation components and detail components, and the noise of the signal is mainly concentrated in the detail components. After removing the noise detail components greater than the preset threshold, the remaining detail components and approximation components are reconstructed by wavelet to obtain a smooth image;

[0040] S3-2, input the to-be-detected slab image filtered by the wavelet transform into the optimal feature extraction model to obtain a feature extraction image of the to-be-detected slab image.

[0041] Optionally, since the optimal feature extraction model fuses seven side output images of different depths, while obtaining deep and effective weak edge features of the slab profile, noise edges are also generated, the noise edges include background noise, water stain noise and side guide roller noise. Therefore, the feature image of the to-be-detected slab image needs to be post-processed to accurately extract the profile features of the slab, and the post-processing specifically includes:

[0042] S4-1, performing binarization processing on the feature image of the to-be-detected slab image to obtain a binary image; wherein the calculation formula of the binarization threshold is:

[0043] g = ω0ω1(u0-u1) 2

[0044] Wherein g is the binarization threshold, ω0 is the proportion of the target point number to the total image, u0 is the average gray value of the target point number, ω1 is the proportion of the background point number to the image, and u1 is the average gray value of the background point number;

[0045] S4-2, performing morphological opening operation on the obtained binary image to ensure that the slab peripheral profile is separated from the side guide roller noise on both sides of the rolling line. The opening operation is first erosion and then inflation;

[0046] The operation steps of erosion are to use a kernel element with a size of 3*3 and a square shape as a structural element to perform traversal operation on the binary image, and the part of the binary image traversed by the kernel element is subjected to AND operation with the kernel element. The operation steps of inflation are to use the kernel element to perform OR operation with the binary image;

[0047] The calculation formula of erosion is:

[0048] (f-b)(s, t) = min{f(s, x, t+y) - b(x, y) | (s+x), (t+y)EDf; (x, y)EDb b}

[0049] where D f and D b are the domain of f and b respectively, f-b represents the gray scale erosion of f by b;

[0050] The calculation formula of dilation is:

[0051]

[0052] where f(s, t) is the input image, b(x, y) is the structure element, D f and D b are the domain of f and b respectively, represents the gray scale dilation of f by b;

[0053] S4-3, calculate the area of each connected region of the image after morphological operation;

[0054] The calculation formula of the area of the connected region is:

[0055]

[0056] where the closed region D is surrounded by a piecewise smooth curve L, the functions P(x, y) and Q(x, y) have first-order continuous partial derivatives on D;

[0057] S4-4, fill the connected region with black pixels if the area of the connected region is less than half of the area of the largest connected region, so as to remove part of the noise region in the image and obtain a preliminary denoising image;

[0058] S4-5, adopt morphological operation on the preliminary denoising image to smooth the slab edge and fill the internal hole noise of the slab, so that the slab is independent and complete as a whole, and obtain a final processing image;

[0059] S4-6, extract all edges in the image by using the gradient amplitude and direction of the pixel points; the calculation formula of the gradient amplitude is:

[0060]

[0061] The calculation formula of the gradient direction is:

[0062]

[0063] wherein:

[0064] Gx = (f(x-1, y+1) + 2f(x, y+1) + f(x+1, y+1)) - (f(x-1, y-1) + 2f(x, y-1) + f(x+1, y-1))

[0065] Gy = (f(x+1, y-1) + 2f(x+1, y) + f(x+1, y+1)) - (f(x-1, y-1) + 2f(x-1, y) + f(x-1, y+1)

[0066] S4-7, calculate the contour perimeter of all edges, find the largest contour, wherein the contour perimeter is calculated by the following formula:

[0067]

[0068] wherein a, b are the range of the curve, x(t), y(t) are the parametric equation of the curve;

[0069] S4-8, finally display the largest contour to the original image, complete the detection.

[0070] Optionally, the specific method of full filling is:

[0071] finding the minimum circumscribed rectangle of the smaller area, and assigning all pixel values within the minimum circumscribed rectangle to 0.

[0072] In another aspect, a slab contour detection device for a surface with water stains is provided, the device comprising:

[0073] an acquisition module configured to acquire an intermediate slab image, construct a sample data set and label;

[0074] a construction module configured to construct a convolutional neural network model, and train the model with the labeled data set to obtain an optimal feature extraction model;

[0075] a feature extraction module configured to pre-process a to-be-detected slab image using wavelet transform, and extract features of the pre-processed to-be-detected slab image using the trained optimal feature extraction model to obtain a feature map of the to-be-detected slab image;

[0076] a post-processing module configured to post-process the feature map of the to-be-detected slab image to complete the functions of removing noise and smoothing edges, and complete contour detection of the to-be-detected slab image by extracting edges and finding the largest contour.

[0077] In another aspect, an electronic device is provided, the electronic device comprising a processor and a memory, the memory storing at least one instruction, the at least one instruction being loaded and executed by the processor to implement the above-mentioned slab contour detection method for a surface with water stains.

[0078] In another aspect, a computer-readable storage medium is provided, the storage medium having stored therein at least one instruction, the at least one instruction being loaded and executed by a processor to implement the slab profile detection method for slabs with surface water marks.

[0079] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0080] The optimal feature extraction model constructed based on the convolutional neural network extracts the profile features of the slab image, and after post-processing, the slab with surface water marks is efficiently and accurately detected. First, the constructed convolutional neural network is trained using the labeled industrial field images, so that the network is more conducive to the edge information of the slab with surface water marks. Then, before inputting the neural network, a wavelet transform is used to remove part of the background noise, so that the features extracted by the optimal feature extraction model are more focused on the edge features of the slab. Then, the feature map of the slab to be detected is obtained by inputting the trained network, and the profile of the slab is extracted after post-processing, thereby completing the profile detection of the slab image to be detected. The traditional image processing method can only complete the detection of a small part of the slab image profile due to its single method, and the detection error often occurs for the slab image with water marks. The present application solves the problem of profile detection of the slab, and is also suitable for images under normal conditions. The robustness of the present application in complex environments is higher, and the detection accuracy is further improved. The real-time online high-precision detection of the slab profile is realized, which is of great significance for the rough rolling intermediate slab quality detection. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical scheme in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0082] Figure 1 is a slab profile detection method flowchart provided by the embodiments of the present application for slabs with surface water marks;

[0083] Figure 2 is a schematic diagram of an intermediate slab image provided by the embodiments of the present application with water marks;

[0084] Figure 3 is a schematic diagram of the optimal feature extraction model structure provided by the embodiments of the present application;

[0085] Figure 4 is a schematic diagram of the slab image feature extraction provided by the embodiments of the present application;

[0086] Figure 5It is a schematic diagram of a slab image contour detection result to be detected provided by an embodiment of the present application.

[0087] Figure 6 It is a block diagram of a slab contour detection device with surface water stains provided by an embodiment of the present application.

[0088] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0089] To make the technical problems, technical solutions and advantages of the present application clearer, specific embodiments will be described in detail below with reference to the drawings.

[0090] An embodiment of the present application provides a slab contour detection method with surface water stains, which can be realized by an electronic device, which can be a terminal or a server. Figure 1 As shown in a slab contour detection method flow chart with surface water stains provided by an embodiment of the present application, the processing flow of the method can include the following steps:

[0091] Step S1: obtaining an intermediate slab image, constructing a sample data set and marking;

[0092] Step S2: constructing a convolutional neural network model and training it with the marked data set to obtain an optimal feature extraction model;

[0093] Step S3: pre-processing the slab image to be detected by wavelet transform, and extracting features of the pre-processed slab image to be detected by using the trained optimal feature extraction model, to obtain a feature map of the slab image to be detected;

[0094] Step S4: post-processing the feature map of the slab image to be detected to complete the functions of removing noise and smoothing edges, and completing the contour detection of the slab image to be detected by extracting edges and finding the largest contour.

[0095] An embodiment of the present application mainly aims at the problem of water stains on the surface of a slab. When the temperature is too high during rough rolling, the slab is cooled by side water spraying, and the surface of the intermediate slab after the rolling mill will have a large number of water stains, some of which will spread to the edges of the slab. In addition, when the rolled slab is wide, the slab is close to the side guide rollers on both sides. Since the surface temperature of the slab is high, the heat radiation is strong, which will light up the side guide rollers, resulting in side guide roller noise. That is, the noise in the embodiment of the present application mainly includes background noise, water stain noise and side guide roller noise.

[0096] Although the water stain noise and the background noise have a difference in gray value, the difference is small, and the gray values of the overall slab and the background and the water stain are quite different. In the absence of water stains, the gray value of the slab and the background is quite different, and the gray scale changes obviously, so that the edge of the slab can be easily extracted. However, in the presence of water stains, the gray value of the slab covered by the water stain and the background is quite small, and the gray scale change is not obvious. At this time, the traditional detection method will cause detection errors. For example, the gray value of the background area noise is 20, the gray value of the slab is 150, and the gray value of the slab area covered by the water stain is 50. When the edge is detected by a single gray scale change (gradient) threshold value of 150-20=130 in the traditional method, the gray scale change (gradient) of the water covered part of 50-20=30 is too small to be missed, and sometimes even the gray scale change (gradient) of 150-50=100 between the water covered part and the slab is regarded as the edge to cause detection errors. Therefore, the use of a single gray scale change threshold value to extract the edge in the traditional detection method will cause the profile of the water covered part of the slab edge to be missing.

[0097] The optimal feature extraction model constructed based on the convolutional neural network can extract the weak gray scale change (gradient) according to different gradient amplitudes and directions by deepening the network layers, using a 3*3 convolution kernel for feature extraction, and combining a 1*1 convolution kernel for edge detection module connection. All edge features can be extracted, including the edges of the background and the water covered part, the edges of the background and the slab, the edges of the water covered part and the slab, and the edges of the side guide roller and the background. Then, the feature map of the detected slab image output by the optimal feature extraction model is post-processed, the edges of the background and the water covered part of the slab and the edges of the background and the slab are detected, the noise edges of the water covered part and the slab and the noise edges of the side guide roller and the background are filtered out, and the profile of the slab is accurately detected.

[0098] The following will be described in detail Figures 2-5 A slab profile detection method for a surface with water stains is provided, including:

[0099] Step S1: obtaining an intermediate slab image, constructing a sample data set and marking;

[0100] Optionally, the S1 specifically includes:

[0101] S1-1, an industrial CCD camera is used to shoot and obtain an RGB image of an intermediate slab with water stains;

[0102] The data set image used in the experiment comes from the rough rolling intermediate slab image of a certain industrial site 1580 production line, and an industrial CCD camera is used to shoot to obtain an RGB image with a resolution of 4096*2160.

[0103] Of course, besides water stains on the surface, it could also be oil stains or other noises.

[0104] In an embodiment of the invention, the image of the intermediate blank with water stains is as follows: Figure 2 As shown.

[0105] S1-2, the effective area of ​​the acquired slab image is cropped to obtain the effective area image;

[0106] In this embodiment, the size of the cropped image is 1024*1024;

[0107] S1-3, Label the effective region image and select the label values ​​that match the contour annotation to form a sample dataset;

[0108] The label of the effective area is the actual outline information of the slab.

[0109] S1-4, the sample dataset is divided into training set and test set according to the proportion.

[0110] In this embodiment of the invention, the sample dataset is divided into a training set and a test set in an 8:2 ratio.

[0111] Step S2: Construct a convolutional neural network model and train it using a labeled dataset to obtain the optimal feature extraction model;

[0112] Optionally, such as Figure 3 As shown, the optimal feature extraction model in S2 consists of seven edge detection modules combined with seven upsampling modules. As the number of network layers increases, deeper feature information in the image will be extracted, and the weak slab outline edge features of the water-covered part will be extracted. Finally, by fusing the seven sides of the convolutional neural network to output the image, while obtaining obvious slab edge features, the weak edges affected by the water stains will also be extracted.

[0113] The edge detection modules are fully connected by a 1×1 convolution kernel. The side output of the edge detection module is used as the input of the upsampling module. Finally, the output feature maps of the seven upsampling modules are fused to obtain the output result of the optimal feature extraction model.

[0114] The first edge detection module consists of one convolutional layer, the second edge detection module consists of one convolutional layer and one pooling layer, the third edge detection module consists of two convolutional layers and one pooling layer, the fourth edge detection module consists of three convolutional layers and one pooling layer, the fifth edge detection module consists of three convolutional layers, the sixth edge detection module consists of three convolutional layers, and the seventh edge detection module consists of two convolutional layers. The kernel size of all edge detection modules is 3×3.

[0115] The calculation formula of the convolution layer is as follows:

[0116]

[0117] wherein N represents the image size after convolution, W represents the size of the input picture, F represents the size of the convolution kernel, P represents the number of pixels filled, and S represents the step length;

[0118] In the embodiment of the application, W=1024, F=3, P=1, and S=2, so the image size output after the convolution layer is calculated as 512;

[0119] The calculation formula of the pooling layer is as follows:

[0120]

[0121] wherein N represents the image size after pooling, W represents the size of the input picture, F represents the size of the pooling kernel, P represents the number of pixels filled, S represents the step length, and G represents the control window element step length;

[0122] In the embodiment of the application, W=1024, F=3, P=1, S=2, and G=1, so the image size output after the pooling layer is calculated as 512.

[0123] The results output by the seven edge detection modules are respectively input into the up-sampling modules, and the up-sampling modules use the bilinear interpolation method to up-sample the feature maps output by the edge detection modules;

[0124] Given the pixel values of four points Q 11 =(x1, y1), Q 12 =(x1, y2), Q 21 =(x2, y1), and Q 22 =(x2, y2), the calculation formula of the bilinear interpolation is as follows:

[0125]

[0126] wherein f(x, y) represents the pixel value of the point P(x, y) to be solved;

[0127] In the embodiment of the application, given the pixel values of four points Q 11 =(1, 1), Q 12 =(1, 3), Q 21 =(3, 1), and Q 22 =(3, 3) are 50, 100, 150, and 200 respectively, the calculation formula of the pixel bilinear interpolation of the point P(2, 2) to be solved is as follows:

[0128]

[0129] = 125

[0130] The output feature maps of the seven up-sampling modules are image fused, and through fusing the side output images of the seven different depths, the weak slab profile features affected by the noise are obtained, and the obvious profile features of the slab not affected by the noise are more prominently represented.

[0131] Optionally, the weighted cross-entropy is used as the loss function of the best feature extraction model.

[0132] The calculation formula of the weighted cross-entropy loss function is as follows:

[0133] β = |Y - | / |Y + +Y - |

[0134] 1-β = |Y + | / |Y + +Y - |

[0135]

[0136]

[0137] wherein Y - and Y + respectively represent the edges and non-edge pixel points labeled in the image data set; n represents the number of the edge detection module; W represents all learning parameters in the model; w is a parameter corresponding to the n edge detection modules (the weights of the modules are not shared); σ() is a sigmoid function, which calculates the activation value of a pixel point; y j represents whether the pixel j is marked as an edge; X represents the input image; δ represents the weight of each scale level; the sigmoid function can map a real number to the interval (0, 1), and its calculation formula is as follows:

[0138]

[0139] Optionally, the grid search method combined with the cross-validation method is used to optimize the model in the embodiment of the present application; in the embodiment, the network model training hyperparameters are set as follows: learningrate is 0.001, batchsize is 8, iterations is 150k, and the optimizer uses Adam for processing.

[0140] Optionally, the present embodiment sets appropriate model evaluation indexes, and in the embodiment of the present application, the determination coefficient and the mean square error are used for calculation:

[0141]

[0142] wherein R 2 represents a determination coefficient, f(x i ) represents predicted data, y i represents real data, and y represents a real data mean value;

[0143]

[0144] wherein MSE represents a mean square error, f(x i ) represents predicted data, y i represents real data, and n represents a data number.

[0145] The embodiment of the application saves a model that is optimal in the model evaluation index, and obtains the best feature extraction model.

[0146] Step S3: performing wavelet transform on the to-be-detected slab image for preprocessing, and performing feature extraction on the preprocessed to-be-detected slab image by using the best feature extraction model trained, to obtain a feature map of the to-be-detected slab image;

[0147] Optionally, the S3 specifically includes:

[0148] S3-1, performing filtering operation on the to-be-detected slab image by using wavelet transform, to filter out part of background noise information, thereby improving subsequent detection accuracy;

[0149] The to-be-detected slab image is a signal distributed along space, and is a two-dimensional signal varying with space axes x and y, and can be regarded as a digital array;

[0150] The wavelet transform is used to decompose the signal into a series of discrete approximation components and detail components, and noise of the signal is mainly concentrated in the detail components, and after removing the noise detail components greater than a preset threshold, the remaining detail components and approximation components are subjected to wavelet reconstruction, to obtain a smooth image;

[0151] S3-2, inputting the to-be-detected slab image filtered by the wavelet transform into the best feature extraction model, to obtain a feature extraction map of the to-be-detected slab image.

[0152] The feature extraction map of the slab image obtained in the embodiment of the application is shown in FIG. 2. Figure 4

[0153] Step S4: performing post-processing on the feature map of the to-be-detected slab image, to complete the functions of removing noise and smoothing edges, and completing contour detection of the to-be-detected slab image by extracting edges and finding the largest contour. ​

[0154] Optionally, since the optimal feature extraction model fuses seven different depth side output images, deep-level weak edge features of the slab profile are obtained, and noise edges including background noise, water stain noise and side guide roll noise are also generated, so post-processing is needed for the feature map of the slab image to be detected to accurately extract the slab profile features, and the post-processing specifically includes:

[0155] S4-1, the feature map of the slab image to be detected is binarized to obtain a binary image; wherein the calculation formula of the binarization threshold is:

[0156] g = ω0ω1(u0-u1) 2

[0157] Wherein g is the binarization threshold, ω0 is the proportion of target point number to total image, u0 is the average gray value of target point number, ω1 is the proportion of background point number to image, and u1 is the average gray value of background point number;

[0158] In the embodiment of the application, the target point number proportion is 0.38, the average gray value of the target point number is 95, the background point number proportion is 0.62, and the average gray value of the background point number is 76, so the binarization threshold of the embodiment of the application is 85, and the gray value of the pixel greater than the binarization threshold is assigned as 255, indicating full white; the gray value of the pixel less than or equal to the binarization threshold is assigned as 0, indicating full black.

[0159] S4-2, morphological opening operation is performed on the obtained binary image to ensure that the slab peripheral profile is separated from the side guide roll noise on both sides of the rolling line, and the opening operation is first erosion and then inflation;

[0160] The operation steps of erosion are to use a kernel element with a size of 3*3 and a square shape as a structural element to perform traversal operation on the binary image, and the part of the binary image traversed by the kernel element is ANDed with the kernel element (for example, the gray value of a point in the binary image is 255, and the gray value of the kernel element traversing to the point is 0, so after the erosion operation, the gray value of the point is 0); the operation steps of inflation are to use the kernel element to AND the binary image (for example, the gray value of a point in the binary image is 255, and the gray value of the kernel element traversing to the point is 0, so after the erosion operation, the gray value of the point is 255);

[0161] The calculation formula of erosion is:

[0162] (f-b)(s,t) = min{f(s,x,t+y)-b(x,y)|(s+x),(t+y)∈D f ;(x,y)∈D b}

[0163] where D f and D b are the definition domains of f and b respectively, f-b represents the gray scale erosion of f by b;

[0164] The calculation formula of the expansion is:

[0165]

[0166] where f(s, t) is the input image, b(x, y) is the structure element, D f and D b are the definition domains of f and b respectively, represents the gray scale expansion of f by b;

[0167] S4-3, calculate the area of each connected region of the image after the morphological operation;

[0168] The calculation formula of the area of the connected region is:

[0169]

[0170] where the closed region D is surrounded by a piecewise smooth curve L, the functions P(x, y) and Q(x, y) have first-order continuous partial derivatives on D;

[0171] S4-4, fill all the connected regions with black pixels whose area is lower than half of the area of the largest connected region, so as to remove part of the noise regions in the image and obtain a preliminary denoising image;

[0172] Optionally, the specific method of the filling is:

[0173] find the minimum circumscribed rectangle of the smaller region, and assign all the pixel values in the minimum circumscribed rectangle to 0.

[0174] S4-5, perform a morphological operation on the preliminary denoising image, so as to smooth the slab edge, fill the internal hole noise of the slab, and make the slab as a whole independent and complete, and obtain a final processing image;

[0175] S4-6, extract all the edges in the image by using the gradient amplitude and direction of the pixel points; the calculation formula of the gradient amplitude is:

[0176]

[0177] The calculation formula of the gradient direction is:

[0178]

[0179] wherein:

[0180] Gx=(f(x-1,y+1)+2f(x,y+1)+f(x+1,y+1))-(f(x-1,y-1)+2f(x,y-1)+f(x+1,y-1))

[0181] Gy=(f(x+1,y-1)+2f(x+1,y)+f(x+1,y+1))-

[0182] (f(x-1,y-1)+2f(x-1,y)+f(x-1,y+1)

[0183] In this embodiment of the invention, gradient magnitudes greater than 20 and gradient directions at the edge of (0,1) are extracted.

[0184] S4-7, calculate the perimeter of the contour formed by all edges, and find the largest contour. The formula for calculating the contour perimeter is:

[0185]

[0186] Where a and b are the ranges of the curve, and x(t) and y(t) are the parametric equations of the curve;

[0187] S4-8 ultimately displays the largest contour in the original image, completing the detection.

[0188] In this embodiment of the invention, the detection effect on the outline of a slab with water stains is as follows: Figure 5 As shown in the figure, the detected contour lines are marked with white lines. In this embodiment of the invention, the entire method takes 421ms, which meets the requirements of real-time detection in industrial settings. It can also be seen from the figure that this method effectively avoids the influence of water stains on the slab surface and the side guide plates on the contour detection, and accurately and efficiently solves the problem of slab contour detection.

[0189] like Figure 6 As shown, this embodiment of the invention also provides a slab contour detection device with water stains on the surface, the device comprising:

[0190] The acquisition module 610 is used to acquire intermediate billet images, construct sample datasets, and label them;

[0191] Module 620 is used to construct a convolutional neural network model and train it with a labeled dataset to obtain the best feature extraction model.

[0192] The feature extraction module 630 is used to preprocess the image of the slab to be detected using wavelet transform, and to extract features from the preprocessed image of the slab to be detected using the trained optimal feature extraction model to obtain the feature map of the image of the slab to be detected.

[0193] The post-processing module 640 is configured to post-process the feature map of the to-be-detected slab image, complete the functions of removing noise and smoothing edges, and complete the contour detection of the to-be-detected slab image by extracting edges and finding the largest contour.

[0194] The surface water-stained slab contour detection device provided by the embodiment of the present application has a function structure corresponding to the surface water-stained slab contour detection method provided by the embodiment of the present application, and thus will not be described here.

[0195] Figure 7 The electronic device 700 provided by the embodiment of the present application can have a large difference due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 701 and one or more memories 702, wherein the memory 702 stores at least one instruction, the at least one instruction is loaded and executed by the processor 701 to realize the steps of the surface water-stained slab contour detection method.

[0196] In the exemplary embodiment, a computer readable storage medium, for example, a memory including instructions, is also provided, and the instructions can be executed by a processor in a terminal to complete the surface water-stained slab contour detection method. For example, the computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0197] Those skilled in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or can be instructed by a program to complete relevant hardware, and the program can be stored in a computer readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, and the like.

[0198] The above only describes the preferred embodiments of the present application, and is not intended to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting the contour of a slab with water stains on its surface, characterized in that, Includes the following steps: Step S1: Obtain intermediate billet images, construct sample datasets, and label them; Step S2: Construct a convolutional neural network model and train it using a labeled dataset to obtain the optimal feature extraction model; Step S3: Wavelet transform is used to preprocess the image of the slab to be detected, and the trained optimal feature extraction model is used to extract features from the preprocessed image of the slab to be detected to obtain the feature map of the image of the slab to be detected. Step S4: Post-process the feature map of the slab image to be detected to remove noise and smooth edges. The contour detection of the slab image to be detected is completed by extracting edges and finding the maximum contour. The optimal feature extraction model in S2 consists of seven edge detection modules combined with seven upsampling modules; The edge detection modules are fully connected by a 1×1 convolution kernel. The side output of the edge detection module is used as the input of the upsampling module. Finally, the output feature maps of the seven upsampling modules are fused to obtain the output result of the optimal feature extraction model. The first edge detection module consists of one convolutional layer, the second edge detection module consists of one convolutional layer and one pooling layer, the third edge detection module consists of two convolutional layers and one pooling layer, the fourth edge detection module consists of three convolutional layers and one pooling layer, the fifth edge detection module consists of three convolutional layers, the sixth edge detection module consists of three convolutional layers, and the seventh edge detection module consists of two convolutional layers. The kernel size of all edge detection modules is 3×3. The outputs of the seven edge detection modules are each input into the upsampling module. The upsampling module uses bilinear interpolation to upsample the feature maps output by the edge detection modules. The output feature maps of the seven upsampling modules are fused together. By fusing the side output images at seven different depths, the weak slab contour features affected by noise are obtained. At the same time, the obvious contour features of the slab that are not affected by noise are represented more prominently. S3 specifically includes: S3-1 uses wavelet transform to filter the image of the slab to be detected, filtering out some background noise information, thereby improving the subsequent detection accuracy; The image of the slab to be detected is a signal distributed along space. It is a two-dimensional signal that varies with the spatial axes x and y, and can be regarded as a set of digital arrays. The signal is decomposed into a series of discrete approximate components and detail components using wavelet transform. The noise of the signal is mainly concentrated in the detail components. After removing the noise detail components that exceed a preset threshold, the remaining detail components and approximate components are then reconstructed using wavelet transform to obtain a smooth image. S3-2, Input the wavelet transform filtered image of the slab to be detected into the optimal feature extraction model to obtain the feature extraction map of the image of the slab to be detected.

2. The method according to claim 1, characterized in that, S1 specifically includes: S1-1 uses an industrial CCD camera to capture RGB images of intermediate blanks with water stains on the surface; S1-2, the effective area of ​​the acquired slab image is cropped to obtain the effective area image; S1-3, Label the effective region image and select the label values ​​that match the contour annotation to form a sample dataset; S1-4, the sample dataset is divided into training set and test set according to the proportion.

3. The method according to claim 1, characterized in that, The formula for calculating the convolutional layer is as follows: ； Where N1 represents the size of the output image after convolution, W1 represents the size of the input image, F1 represents the size of the convolution kernel, P1 represents the number of padding pixels, and S1 represents the stride. The formula for calculating the pooling layer is as follows: ； Where N2 represents the size of the output image after pooling, W2 represents the size of the input image, F2 represents the size of the pooling kernel, P2 represents the number of pixels to fill, S2 represents the stride, and G2 represents the stride of the elements in the control window. Given Q 11 = (x1, y1), Q 12 = (x1, y2), Q 21 = (x2, y1), Q 22 Given the pixel values ​​of points (x2, y2), the formula for bilinear interpolation is: ； Where f(x, y) represents the pixel value of the point P(x, y) to be calculated.

4. The method according to claim 1, characterized in that, Weighted cross-entropy is used as the loss function for the optimal feature extraction model; The formula for calculating the weighted cross-entropy loss function is: ； ； ； ； in, and These represent the labeled edge and non-edge pixels in the image dataset, respectively; n represents the number of the edge detection module. This represents all learned parameters in the model; These are the parameters corresponding to the n edge detection modules; It is the sigmoid function, which calculates the activation value of a pixel; Indicates whether pixel j is marked as an edge; X represents the input image; This represents the weight at each scale level; the sigmoid function maps a real number to the interval (0,1), and its calculation formula is as follows: 。 5. The method according to claim 1, characterized in that, Since the optimal feature extraction model fuses seven side output images at different depths, while obtaining effective weak edge features of the deep slab contour, it also generates noisy edges. These noisy edges include background noise, water stain noise, and side guide roller noise. Therefore, post-processing of the feature map of the slab image to be detected is necessary to achieve accurate extraction of the slab contour features. This post-processing specifically includes: S4-1, The feature map of the slab image to be detected is binarized to obtain a binary image; wherein the formula for calculating the binarization threshold is: ； Where g is the binarization threshold. The proportion of target points to the total image. The average gray value of the target number of points. The proportion of background dots to the image. The average grayscale value of the background pixels; S4-2, perform morphological opening operation on the obtained binary image to ensure that the outer contour of the slab is separated from the noise of the side guide rolls on both sides of the rolling line. The opening operation steps are erosion followed by expansion. The erosion operation involves using a kernel element with a 3×3 structuring element and a square shape to traverse the binary image, and then performing an AND operation between the portion of the binary image traversed by the kernel element and the kernel element itself. The dilation operation involves performing a union operation between the kernel element and the binary image. The formula for calculating corrosion is: ； In the formula, These are the domains of f and b, respectively. Represented by b Perform grayscale etching; The formula for calculating expansion is: ； In the formula, For the input image, As a structural element, These are the domains of f and b, respectively. Represented by b Perform grayscale expansion; S4-3, calculate the area of ​​each connected region in the image after morphological operations; The formula for calculating the area of ​​a connected region is: ； The closed region D is bounded by piecewise smooth curves L, and the functions P(x,y) and Q(x,y) have continuous first-order partial derivatives on D. S4-4: Fill connected regions with an area less than half the area of ​​the largest connected region with black pixels to remove some noise regions in the image and obtain a preliminary denoised image. S4-5. Morphological operations are performed on the image after initial denoising to smooth the slab edges and fill the noise in the slab's internal holes, making the slab independent and complete as a whole, and obtaining the final processed image. S4-6, extract all edges in the image using the gradient magnitude and direction of each pixel; the formula for calculating the gradient magnitude is: ； The formula for calculating the gradient direction is: ； in: ； ； S4-7, calculate the perimeter of the contour formed by all edges, and find the largest contour. The formula for calculating the contour perimeter is: ； Where a and b are the ranges of the curve, and x(t) and y(t) are the parametric equations of the curve; S4-8 ultimately displays the largest contour in the original image, completing the detection.

6. The method according to claim 5, characterized in that, The specific method for full filling is as follows: Find the smallest bounding rectangle of the smaller region, and set all pixel values ​​within the smallest bounding rectangle to 0.

7. A device for detecting the contour of a slab with water stains on its surface, characterized in that, The device includes: The acquisition module is used to acquire intermediate billet images, construct sample datasets, and label them; The construction module is used to construct a convolutional neural network model and train it with a labeled dataset to obtain the best feature extraction model; The feature extraction module is used to preprocess the image of the slab to be detected using wavelet transform, and to extract features from the preprocessed image of the slab to be detected using the trained optimal feature extraction model to obtain the feature map of the image of the slab to be detected. The post-processing module is used to post-process the feature map of the slab image to be detected, and to remove noise and smooth edges. It performs contour detection of the slab image to be detected by extracting edges and finding the maximum contour. The optimal feature extraction model consists of seven edge detection modules combined with seven upsampling modules; The edge detection modules are fully connected by a 1×1 convolution kernel. The side output of the edge detection module is used as the input of the upsampling module. Finally, the output feature maps of the seven upsampling modules are fused to obtain the output result of the optimal feature extraction model. The first edge detection module consists of one convolutional layer, the second edge detection module consists of one convolutional layer and one pooling layer, the third edge detection module consists of two convolutional layers and one pooling layer, the fourth edge detection module consists of three convolutional layers and one pooling layer, the fifth edge detection module consists of three convolutional layers, the sixth edge detection module consists of three convolutional layers, and the seventh edge detection module consists of two convolutional layers. The kernel size of all edge detection modules is 3×3. The outputs of the seven edge detection modules are each input into the upsampling module. The upsampling module uses bilinear interpolation to upsample the feature maps output by the edge detection modules. The output feature maps of the seven upsampling modules are fused together. By fusing the side output images at seven different depths, the weak slab contour features affected by noise are obtained. At the same time, the obvious contour features of the slab that are not affected by noise are represented more prominently. The feature extraction module is specifically used for: S3-1 uses wavelet transform to filter the image of the slab to be detected, filtering out some background noise information, thereby improving the subsequent detection accuracy; The image of the slab to be detected is a signal distributed along space. It is a two-dimensional signal that varies with the spatial axes x and y, and can be regarded as a set of digital arrays. The signal is decomposed into a series of discrete approximate components and detail components using wavelet transform. The noise of the signal is mainly concentrated in the detail components. After removing the noise detail components that exceed a preset threshold, the remaining detail components and approximate components are then reconstructed using wavelet transform to obtain a smooth image. S3-2, Input the wavelet transform filtered image of the slab to be detected into the optimal feature extraction model to obtain the feature extraction map of the image of the slab to be detected.

8. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the method for detecting the contour of a slab with water stains on its surface as described in any one of claims 1-6.

9. A computer-readable storage medium storing at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the method for detecting the contour of a slab with water stains on its surface as described in any one of claims 1-6.

Images