A visual analysis method and system based on real-time monitoring of steel plate profile

Abstract

The present application proposes a visualization analysis method and system based on real-time monitoring of steel plate contour, which belongs to the technical field of plate shape monitoring and visual analysis, wherein the method includes: obtaining a real-time motion image of a steel plate as an original image; extracting a steel plate image from the original image, and projecting the steel plate image; extracting the contour line of the steel plate image after projection transformation; calculating the contour bending coefficient of the contour line; matching each steel plate rolling pass with the corresponding contour bending coefficient, and outputting the contour bending coefficient corresponding to each steel plate rolling pass; performing three-dimensional reconstruction of the steel plate for the contour bending coefficient corresponding to each steel plate rolling pass, and displaying the bending degree of the steel plate in the three-dimensional reconstruction model of the steel plate. The present application does not require a large amount of data support, and can realize real-time monitoring and quantification of abnormal sickle bends of steel plates, and timely and effectively monitor the quality control level of thick plate rolling.

Description

Technical Field

[0001] The present invention belongs to the technical field of plate shape monitoring and visual analysis, and in particular relates to a visual analysis method and system based on real-time monitoring of steel plate contours. Background Art

[0002] In hot rolling production, the difference in the deformation of steel plates due to the following reasons can lead to asymmetry or abnormal profile of the slab: (1) Differences in equipment status, such as the equipment gap on the rolling mill is not completely tightened or the position of the hydraulic cylinder needs to be adjusted; (2) The center line of the steel plate is offset relative to the center line of the rolling mill during the rolling process, resulting in uneven rolling force on both sides of the roll; (3) The difference in temperature or toughness on both sides of the intermediate slab leads to different reductions, resulting in uneven extension of the intermediate slab in the length direction. Therefore, real-time monitoring and analysis of the generation of abnormal steel plate profiles during hot rolling has become a key way to improve product quality.

[0003] In the actual production process, due to the high temperature of the intermediate slab and the complex surface conditions, the traditional contact-type steel plate profile monitoring method is not only difficult to ensure accuracy, but also puts additional pressure on the production equipment and aggravates wear. In the existing research on the medium and thick plate rolling production process, the plate shape monitoring and abnormal factor analysis are mainly realized from the mechanism model of the rolling process and the data-driven analysis model. Mathematical and physical models are used to simulate and calculate various parameters and variables in the rolling production process. There are still problems such as the limitations of model assumptions, high requirements for parameter accuracy, and real-time and applicability. In addition, methods based on laser scanning monitoring, three-dimensional imaging technology, and ultrasonic monitoring technology, although they can provide higher monitoring accuracy to a certain extent, also have problems such as high cost, greater influence by ambient light, and greater influence by factors such as material properties and surface conditions.

[0004] In recent years, the non-contact characteristics and high-precision advantages of machine vision monitoring technology have brought more convenience to the production process. Traditional mathematical methods and monitoring methods based on machine learning are more common in this field. Traditional mathematical methods include pixel points, edge-based and template matching monitoring methods, which have problems such as high error rate, poor robustness, high signal-to-noise ratio requirements, high template selection requirements, and difficult maintenance. Machine learning methods based on support vector machines, neural networks, and decision trees have been widely used. For example, methods based on gray-level co-occurrence matrices and neural networks can achieve real-time monitoring and diagnosis of contour anomalies, and methods based on wavelet transforms have high monitoring accuracy and robustness. Among these methods, steel plate contour anomalies can be monitored by processing and analyzing image data during rolling. However, in actual industrial production environments, quality control in the thick plate rolling industry is difficult, and these methods have problems such as poor real-time performance, data imbalance, difficulty in parameter selection, overfitting and underfitting. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present application proposes a visual analysis method and system based on real-time monitoring of steel plate profile.

[0006] In the first aspect, the present application proposes a visual analysis method based on real-time monitoring of steel plate profile, comprising:

[0007] Acquire a real-time moving image of the steel plate as an original image;

[0008] Extracting a steel plate image from the original image, and performing a projection transformation on the steel plate image;

[0009] Extracting the contour line of the steel plate image after projection transformation;

[0010] Calculate the contour curvature coefficient of the contour line;

[0011] Match each steel plate rolling pass with the corresponding profile bending degree coefficient, and output the profile bending degree coefficient corresponding to each steel plate rolling pass;

[0012] According to the contour bending degree coefficient corresponding to each steel plate rolling pass, the steel plate is three-dimensionally reconstructed, and the bending degree of the steel plate is displayed in the three-dimensional reconstructed model of the steel plate.

[0013] The visualization analysis method based on real-time monitoring of steel plate contour also includes: outputting the contour line of the steel plate image after projection transformation and the corresponding original image in real time; and outputting the contour curvature coefficient of each contour line segment and the corresponding original image in real time.

[0014] The step of extracting a steel plate image from the original image and performing a projection transformation on the steel plate image includes:

[0015] Divide the original image into areas to be monitored;

[0016] The color channel image difference algorithm is used to process the monitored area to obtain the segmented steel plate foreground image of the monitored area and segment the steel plate image;

[0017] Calculate the color change curve of the steel plate image;

[0018] According to the color change curve, a steel plate position curve and a rate of change of the steel plate position are calculated;

[0019] The steel plate image is judged according to the steel plate position curve, the rate of change of the steel plate position and the corresponding threshold value to obtain the farthest steel plate image;

[0020] Perform projection transformation on the farthest steel plate image.

[0021] The step of extracting the contour line of the steel plate image after projection transformation comprises:

[0022] Filtering the steel plate image after projection transformation;

[0023] Calculating the transverse gradient of the filtered steel plate image to obtain a transverse gradient image;

[0024] In the transverse gradient image, pixel points whose absolute gradient values ​​are greater than or equal to the specified gradient value are selected to obtain the contour pixel points of the steel plate;

[0025] The contour line of the steel plate image is obtained by fitting the contour pixel points of all steel plates using a quadratic polynomial.

[0026] The calculating the contour curvature coefficient of the contour line comprises: normalizing the curvature coefficient of the contour line;

[0027] The normalized curvature coefficient is calculated by segment mapping to obtain the contour curvature coefficient after smoothing.

[0028] The normalization process of the contour curvature coefficient is calculated as follows:

[0029]

[0030] Among them, a is the contour curvature coefficient before normalization, and b is the contour curvature coefficient after normalization.

[0031] The normalized contour curvature coefficient is segmented and mapped to obtain the contour curvature coefficient after smoothing, so as to achieve smooth mapping of the contour coefficient in the range of 0 to 1. The calculation formula is as follows:

[0032]

[0033] Wherein, L(b) is the segment mapping function, abs(b) is the absolute value of the normalized contour curvature coefficient, sign(b) is the sign function value of the contour curvature coefficient, T1 is the threshold of the first segment, and T2 is the threshold of the second segment.

[0034] In the second aspect, the present application proposes a visual analysis system based on real-time monitoring of steel plate profile, including:

[0035] An image acquisition unit, used for acquiring a real-time moving image of the steel plate as an original image;

[0036] A projection transformation unit, used for extracting a steel plate image from the original image and performing projection transformation on the steel plate image;

[0037] A contour line extraction unit, used for extracting the contour line of the steel plate image after projection transformation;

[0038] A curvature coefficient calculation unit, used for calculating the contour curvature coefficient of the contour line;

[0039] A coefficient matching unit, used for matching each steel plate rolling pass with a corresponding profile bending degree coefficient, and outputting the profile bending degree coefficient corresponding to each steel plate rolling pass;

[0040] The three-dimensional reconstruction unit is used to perform three-dimensional reconstruction of the steel plate according to the contour bending degree coefficient corresponding to each steel plate rolling pass, and display the bending degree of the steel plate in the three-dimensional reconstruction model of the steel plate.

[0041] In a third aspect, the present application proposes an electronic device comprising: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the visual analysis method based on real-time monitoring of steel plate contour.

[0042] In a fourth aspect, the present application proposes a computer-readable storage medium storing executable instructions, which, when executed, enable a processor to execute the visualization analysis method based on real-time monitoring of steel plate contours.

[0043] In a fifth aspect, the present application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the visualization analysis method based on real-time monitoring of steel plate contour.

[0044] Beneficial effects:

[0045] This application proposes a visualization analysis method and system based on real-time monitoring of steel plate contours. It does not require a large amount of data support, has robustness and resistance to water mist interference, realizes real-time monitoring and quantification of abnormal sickle bend of steel plates, can timely and effectively monitor the quality control level of thick plate rolling, and can display real-time monitoring and quantitative statistical results in three dimensions from multiple perspectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a visual analysis method based on real-time monitoring of steel plate profile according to an embodiment of the present application;

[0047] Figure 2 The overall system architecture of the visual analysis based on real-time monitoring of steel plate profile in the embodiment of the present application;

[0048] Figure 3 Schematic diagram of the real-time monitoring result of the steel plate profile in the hot rolling process based on machine vision in an embodiment of the present application, (a) schematic diagram of the overall monitoring result and (b) schematic diagram of the enlarged quantitative statistical monitoring result;

[0049] Figure 4 Divide the image of the area to be monitored in the embodiment of the present application;

[0050] Figure 5 The steel plate image with the foreground portion segmented out after the difference calculation in the embodiment of the present application;

[0051] Figure 6 This is the farthest image of the embodiment of the present application;

[0052] Figure 7 The defiltered image of the embodiment of the present application;

[0053] Figure 8 This is a principle block diagram of a visual analysis system based on real-time monitoring of steel plate contours according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] The application and development of industrial Internet technology has accumulated a huge amount of data resources for enterprises. Industrial data often has the characteristics of large scale, complex structure, high dimension, and multi-source heterogeneity. Data visualization is one of the effective methods to mine the value behind data. It conveys data information through graphics to accelerate understanding and value discovery. A single graphic displays multi-dimensional attributes and effectively presents data patterns. At the same time, the accuracy, interpretability and flexibility of data information insights are improved through interactive visual combined with cognition and algorithms. Visual analysis methods are more intuitive and easy to understand. Through visualization tools, complex data can be converted into intuitive charts and graphs to help users quickly discover the relationship and pattern between data. In industrial production, visual analysis methods can intuitively show data changes in the production process, helping engineers and managers to detect abnormal situations in time and take corresponding measures. In the steel plate rolling process, a large amount of image data obtained through machine vision technology can be monitored and analyzed in real time through visual analysis methods, helping to improve production efficiency and product quality.

[0055] Patent document CN110947774B provides a strip shape prediction method considering rolling width, the method comprising: taking strip parameters, roll parameters and rolling parameters; introducing the transverse thickness difference change factor and width factor before and after rolling, and constructing a strip shape prediction model based on the coupling of strip shape, convexity and width; constructing a three-dimensional finite element model according to the strip parameters, roll parameters and rolling parameters, using the model to simulate the strip rolling and calculate the model parameters; and predicting the strip shape using the prediction model of the coupling of strip shape, convexity and width. However, the simulation model used in this method may require a lot of computing time, and it is impossible to predict the strip shape in real time, which may have certain shortcomings when the production parameters need to be adjusted in time.

[0056] Patent document CN115711893A discloses an online steel plate sickle bend measurement system and measurement method, the system includes: a mounting bracket installed above the steel plate conveyor roller, the mounting bracket is provided with a mounting seat, and the mounting seat is provided with a measuring device; the measuring device is located in a heat-insulating box, the heat-insulating box is connected to an air cooling device, and a dust cover is provided on the side facing the roller; the air cooling device is used for cooling measurement, and controls the opening of the dust cover, and provides positive pressure purge at the same time. The invention can accurately measure the sickle bend value of the steel plate online in real time. However, the measurement system in this method contains more components, which increases the complexity and maintenance cost of the system, and the lack of real-time visual display of the system may reduce the operator's ability to monitor and analyze the measurement results in real time.

[0057] Therefore, in order to solve the problem of real-time monitoring of steel plate contour, the present application proposes a visualization analysis method based on real-time monitoring of steel plate contour, which can not only display the contour line of the steel plate image, but also display the contour bending coefficient corresponding to each steel plate rolling pass, and can display the bending degree of the steel plate in the three-dimensional reconstruction model of the steel plate through three-dimensional reconstruction. At the same time, the above process meets the real-time requirements and does not require the addition of more components. It can realize real-time monitoring and quantification of abnormal sickle bend of steel plates, and can timely and effectively monitor the quality control level of thick plate rolling.

[0058] Embodiment 1:

[0059] This embodiment proposes a visual analysis method based on real-time monitoring of steel plate profile, such as Figure 1 As shown, including:

[0060] Step S1: acquiring a real-time motion image of a steel plate as an original image;

[0061] In this embodiment, the overall system architecture based on the visual analysis of the real-time monitoring of the steel plate profile is as follows: Figure 2 As shown, the system architecture includes: an information acquisition input module, a real-time monitoring module and a visual analysis module, wherein the real-time monitoring module includes: a video preprocessing unit and a perception and detection unit, and the visual analysis module includes: a contour curve quantification result display unit, and a statistical analysis display unit. This embodiment is illustrated by the industrial image data actually collected on the thick plate rolling production line of a domestic steel enterprise. The data of this embodiment is the image data of the complete hot rolling process of a thick plate produced in April 2023. The overall design based on the above system architecture implements a visual analysis method based on real-time monitoring of steel plate contours, such as Figure 3As shown, (a) is a schematic diagram of the overall monitoring results and (b) is an enlarged schematic diagram of the quantitative statistical monitoring results, including: using a high-precision camera installed at the industrial production site to collect the real-time motion image of the steel plate on the rolling mill during the hot rolling process as the original image, and transmitting the collected image to the real-time monitoring module of the system through the front-end and back-end and network technology. Figure 3 As shown in the A1 area, the real-time hot rolling production scene can be clearly observed.

[0062] Step S2: extracting a steel plate image from the original image, and performing a projection transformation on the steel plate image;

[0063] Step S3: extracting the contour line of the steel plate image after projection transformation;

[0064] In a specific implementation, the contour line of the steel plate image after projection transformation and the corresponding original image can also be output and displayed in real time.

[0065] Step S4: Calculate the contour curvature coefficient of the contour line;

[0066] In a specific implementation, the contour curvature coefficient of the contour line and the corresponding original image may be output and displayed in real time.

[0067] Step S5: Match each steel plate rolling pass with the corresponding profile bending degree coefficient, and output the profile bending degree coefficient corresponding to each steel plate rolling pass;

[0068] Step S6: for each steel plate rolling pass corresponding to the contour bending coefficient, the steel plate is three-dimensionally reconstructed, and the bending degree of the steel plate is displayed in the three-dimensional reconstructed model of the steel plate.

[0069] In step S2, the steel plate image is extracted from the original image, and the steel plate image is projected and transformed, including:

[0070] Step S2.1: Divide the original image into areas to be monitored;

[0071] Step S2.2: using a color channel image difference algorithm to process the area to be monitored, obtaining a segmented steel plate foreground image of the area to be monitored, and segmenting a steel plate image;

[0072] Step S2.3: Calculate the color change curve of the steel plate image;

[0073] Step S2.4: Calculate the steel plate position curve and the rate of change of the steel plate position according to the color change curve;

[0074] Step S2.5: judging the steel plate image according to the steel plate position curve, the rate of change of the steel plate position and the corresponding threshold value, and obtaining the farthest steel plate image;

[0075] Step S2.6: Perform a projection transformation on the steel plate image on the farthest image.

[0076] In this embodiment, the original image is divided into areas to be monitored, and then the color channel difference calculation is performed on the area to be monitored, so as to segment the steel plate image as the foreground part, and then the calculation is performed on the middle position of the steel plate. The color change curve is calculated to obtain the steel plate position curve, so as to calculate the changes in the position, speed, and acceleration of the steel plate. By judging whether the slope sign of the steel plate position curve changes, the farthest position, acceleration, and rolling pass of the steel plate are found, so as to judge whether the steel plate has reached the farthest position from the rolling mill, that is, the image when the steel plate is closest to the camera and clearest. The obtained steel plate picture with an inclined angle is transformed into a picture with a vertical perspective, and the projection transformation of the farthest image is completed. The processed image is transmitted to the system projection transformation module in parallel through the front-end and back-end and network technology. This step is specifically implemented as follows:

[0077] Let I represent the original input image, I R represents the red channel image of I, I G represents the green channel image of I, I B represents the blue channel image of I, x represents the horizontal coordinate of the image, and y represents the vertical coordinate of the image. Let C(x, y) represent the difference image of the red channel minus the blue channel, and its expression is:

[0078] C(x, y) = I R (x,y)-I B (x,y)

[0079] Assume that in the y-axis direction, the longitudinal range of the steel plate is [y min ,y max ], then the expression of the steel plate position curve P(x) is:

[0080]

[0081] Among them, the slope of the steel plate position curve can represent the rate of change of the steel plate position. curve[k] :

[0082]

[0083] Among them, x k is the horizontal coordinate value when the video image frame count is k, τ is the time delay parameter, P(x k ) is the steel plate position curve of the k-th frame video image, P(x k-τ) is the steel plate position curve of the k-τth frame video curve.

[0084] According to P(x k )The change in the slope sign can be used to find the farthest position of the steel plate. Assuming position max is the threshold value of the steel plate reaching the farthest end, position min is the minimum threshold for determining the position of the steel plate, speed max is the maximum threshold of speed. Then the condition for judging whether the steel plate reaches the farthest position is:

[0085] {P(x k )≥position min ,|speed curve[k] <speed max , k>τ}

[0086] In order to achieve the correction of the image perspective, the process of projecting the farthest image is completed, and the steel plate image with an inclined angle is transformed into a vertical angle image. The specific implementation of this step is as follows:

[0087] (1) Define four points P1, P2, P3 and P4 on the rolling roller table located on the rectangle to determine the source and target coordinates of the projection transformation:

[0088] P1=[x1,y1]

[0089] P2=[x2,y2]

[0090] P3=[x3,y3]

[0091] P4 = [x4, y4]

[0092] (2) Calculate the projection transformation matrix A based on the coordinates of the four points on the rolling roller that are located on the rectangle:

[0093]

[0094] Among them, the calculation formula of each element in the projection transformation matrix A is as follows:

[0095]

[0096] a 31 =0,a 32 =0

[0097] (3) Perform projection transformation on the steel plate image, and scale and project the image:

[0098] I out (u,v)=I in (x,y)

[0099] in:

[0100]

[0101] Among them, u, v represent the transformed image coordinates, I in Represents the original input image, I out represents the image after projection transformation, h and w represent the height and width of the image respectively.

[0102] Through the above calculations, the simulation results are as follows: the original image is divided into the area to be monitored, such as Figure 4 As shown, the color channel difference calculation is then performed to segment the steel plate as the foreground part, as shown in Figure 5 As shown, the middle position of the steel plate is calculated. The color change curve is calculated to obtain the steel plate position curve, so as to calculate the changes in the position, speed, and acceleration of the steel plate in the previous and next frame images. By judging whether the slope sign of the steel plate position curve has changed, the farthest position, acceleration, and rolling pass of the steel plate can be found, so as to judge whether the steel plate has reached the farthest position from the rolling mill, that is, the image when the steel plate is closest to the camera and clearest, as shown in FIG. Figure 6 As shown. The obtained steel plate image with an inclined angle is transformed into a vertical angle image to complete the projection transformation of the farthest image. The processed image is transmitted to the system projection transformation module in parallel through the front-end and back-end and network technology, as shown in Figure 3 As shown in area A2.

[0103] In step S3, extracting the contour line of the steel plate image after projection transformation includes:

[0104] Step S3.1: filtering the steel plate image after projection transformation;

[0105] Step S3.2: Calculate the transverse gradient of the filtered steel plate image to obtain a transverse gradient image;

[0106] Step S3.3: In the transverse gradient image, select pixel points whose absolute gradient value is greater than or equal to the specified gradient value to obtain the contour pixel points of the steel plate;

[0107] Step S3.4: Use a quadratic polynomial to fit the contour pixels of all steel plates to obtain the contour line of the steel plate image.

[0108] In this embodiment, after the projection transformation, the steel plate image after the projection transformation is filtered to remove interference factors such as water mist, and then the lateral gradient of the processed image is obtained, the obtained contour points are fitted with contour curves, and the fitted contour curves are superimposed and displayed on the projection transformed image, and transmitted to the system fitting curve module in parallel through the front-end and back-end and network technologies. The specific implementation is as follows:

[0109] Based on the idea of ​​median filter, the spatial distance and gray value similarity between pixels are considered at the same time, and the weighted median filter with edge preservation is used on the image to reduce noise interference and smooth the steel plate contour. In order to achieve better noise suppression effect (such as roller highlight), Gaussian weighting is also introduced on the basis of weighted median filter. At the same time, the filter effect and processing speed can be balanced by adjusting the standard deviation parameter.

[0110] Weighted Median Filter is a sorting-based image filtering method that can effectively remove noise and retain edge information. The basic idea is to sort the area of ​​each pixel in the image and then perform weighted averaging on the sorting results. Weighted Median Filter uses different weights for different areas of the image to improve the filtering effect.

[0111] Specifically, for a given pixel position (i, j), a filter window of size (2r+1)×(2r+1) is extracted with the pixel as the center, and the grayscale values ​​of all pixels in the window are sorted by size. The sorted result is recorded as Assuming the weighting function is w(x), the output of the weighted median filter is:

[0112]

[0113] Where z is the z-th pixel, WMF(i, j) is the weighted median filter output at pixel position (i, j), and W is the weight matrix;

[0114] A common choice for the weighting function is the Gaussian function:

[0115]

[0116] Where m is the median and σ is the standard deviation. Since weighted median filtering is a computationally intensive algorithm, a fast weighted median filtering algorithm is usually used to speed up the calculation.

[0117] The Sobel operator and other methods are used to extract the horizontal gradient of the image. The filtered image is first split by channel, and then the horizontal gradient of the two channels I1 and I2 is extracted to obtain the horizontal significant contour information. Among them, I1 is the difference image obtained by subtracting the blue channel from the green channel of the filtered image, and I2 is the difference image obtained by subtracting the green channel from the blue channel of the filtered image. Finally, the two gradient images are spliced ​​to obtain the return image formed by the gradient splicing of the difference image of the green channel and the blue channel of the filtered image.

[0118] Suppose the image obtained after weighted median filtering is Iw , take two of the channels as I1 and I2 respectively. Assume that the size of I is M×N and the lateral Sobel operator is S x , the gradient calculation result is G. G 1，x It refers to the image I1 applying the Sobel operator S in the horizontal direction. x The gradient image obtained after that. 2，x It refers to the image I2 applying the Sobel operator S in the horizontal direction. x The resulting gradient image.

[0119] Calculate the gradient of the lateral Sobel operator for each image:

[0120] G 1,x =I1*S x

[0121] G 2,x =I2*S x

[0122] Where * represents the convolution operation. Concatenate the two difference images:

[0123] G x =[G 1，x , G 2,x ]

[0124] Convolve the spliced ​​image with the lateral Sobel operator:

[0125] G=G x *S x

[0126] The final G is the final lateral gradient image.

[0127] The steel plate contour curve is fitted using a weighted quadratic curve fitting algorithm so that the curve passes through a given set of data points as much as possible, and a quadratic function expression of the steel plate contour curve is obtained to facilitate subsequent quantitative analysis and abnormality monitoring. The quadratic curve fitting method proposed in this embodiment can effectively overcome the interference of the side contour and significantly reduce the influence of the contour recognition accuracy on the influence of the dense water mist.

[0128] After obtaining the lateral gradient of the image, the pixels whose absolute gradient value is greater than or equal to the specified gradient value are extracted to obtain the contour pixels on both sides of the steel plate. The weight of the sampling point depends on the gradient size corresponding to the sampling point. Then, a quadratic polynomial is used to fit these pixels.

[0129] The goal of fitting is to find a quadratic curve that passes through a given set of data points as closely as possible. In order to better fit the gradient points, a weighted quadratic curve fitting method is used, that is, for each gradient point (xi ,y i ), given a weight w i , solve the coefficients a, b, and c by the least squares method to minimize the residual sum of squares.

[0130] Assume that there are n steel plate contour data points (x i ,y i , w i ), where w i is the weight of data point i. The goal is to fit a quadratic curve model f(x) = ax 2 +bx+c, so that the model minimizes the weighted residual sum of squares:

[0131]

[0132] This problem can be transformed into a linear least squares problem, that is, the parameter vector p = [a, b, c] T Written as a matrix product:

[0133]

[0134] For each data point (x i ,y i , w i ) is represented as an extended vector u i , then the weighted residual of the data point can be written as:

[0135]

[0136] The ultimate goal is to find the method that minimizes the weighted residual sum of squares, that is:

[0137]

[0138] This is a linear least squares problem that can be solved using the standard least squares method. That is, the optimal solution that minimizes the sum of squared residuals is obtained by solving the following normal equations:

[0139] (X T WX)p=X T Wy

[0140] Where X is an n×3 matrix, each row is an expansion vector u i y is a combination of all y i The n×1 vector represents the y coordinate of the data point. W is an n×n diagonal matrix, and the elements on the diagonal are the weights w of the data points. i .

[0141] After solving the equations and obtaining the parameter vector p, we can get the coefficients a, b, and c of the quadratic curve. The final fitted quadratic curve model is f(x) = ax 2 +bx+c, so that the curve passes through all the key data points as much as possible, thereby obtaining the contour curve of the steel plate.

[0142] The simulation results are as follows: After the projection transformation, the image is filtered to remove interference factors such as water mist, such as Figure 7 As shown, the processed image is subjected to lateral gradient acquisition, the acquired contour points are subjected to contour curve fitting, the fitted contour curve is superimposed and displayed on the image after projection transformation, and is transmitted to the system fitting curve module in parallel through the front-end and back-end and network technologies, as shown in FIG. Figure 3 As shown in area A3.

[0143] In step S4, the contour curvature coefficient of the contour line is calculated, including:

[0144] Step S4.1: normalizing the curvature coefficient of the contour line;

[0145] Step S4.2: Perform segment mapping calculation on the normalized curvature coefficient to obtain the contour curvature coefficient after smoothing.

[0146] In this embodiment, according to the quadratic function expression of the steel plate contour curve, the extracted steel plate contour information (i.e., the quadratic term coefficient) is quantized to the interval [-1,1] by combining Sigmoid and piecewise linear mapping. In the interface, the degree of bending of the steel plate contour is distinguished and prompted by the green, yellow, and red colors of the real-time circle indicator lights in combination with process production experience, thereby assisting the monitoring of abnormal sickle bends during the steel plate rolling process. A method combining normalized Sigmoid mapping and piecewise linear mapping is adopted: wherein, piecewise linear mapping can set segmented intervals (the mapping result is also 0 when there is almost no sickle bend), but the transition is not smooth, and it is easy to produce a small sickle bend that produces a large value change after being disturbed by noise: the normalized Sigmoid mapping transition is relatively smooth, but the fluctuation is large in the range of 0 (almost no sickle bend), and the curvature coefficient of the contour line is normalized, and the calculation formula is as follows:

[0147]

[0148] Among them, a is the contour curvature coefficient before normalization, and b is the contour curvature coefficient after normalization.

[0149] The normalized curvature coefficient is segmented and mapped to obtain the contour curvature coefficient after smoothing, so as to achieve smooth mapping of the contour coefficient in the range of 0 to 1. The calculation formula is as follows:

[0150]

[0151] Wherein, L(b) is a segmented mapping function, abs(b) is the absolute value of the normalized contour curvature coefficient, sign(b) is the sign function value of the contour curvature coefficient, T1 is the threshold of the first segment, and T2 is the threshold of the second segment. The combination of the above two mapping functions can achieve the best effect. In a specific embodiment, even if a combined image of the contour line of the steel plate and the original image is given, it is not easy for a technician in this field to see the problem of the steel plate with the naked eye. However, this embodiment does not require the use of historical data and can give a contour curvature coefficient, which not only meets the real-time requirements but also provides a reliable basis for technicians in this field to judge the curvature of the steel plate.

[0152] Simulation results: The coefficient representing the degree of contour curvature in the fitting curve and the transmitted monitoring screen are transmitted to the system front end and displayed in the contour curve quantization result display unit. The fitting coefficient interval representing the contour quantization result is distinguished by a significant color prompt, such as Figure 3 As shown in the B2 area. According to the quadratic function expression of the steel plate contour curve, the extracted steel plate contour information (i.e., the quadratic term coefficient) is quantized to the [-1,1] interval using a combination of Sigmoid and piecewise linear mapping. The interface combines process production experience to distinguish and prompt the degree of bending of the steel plate contour through real-time circle indicator lights (in the actual production process, for example: green indicates that the steel plate contour of this pass is normal, yellow indicates that the steel plate contour of this pass is slightly bent, and red indicates that the steel plate contour of this pass is greatly bent), thereby assisting the monitoring of abnormal sickle bending during the steel plate rolling process.

[0153] In step S5, each steel plate rolling pass is matched with a corresponding profile bending degree coefficient, and the profile bending degree coefficient corresponding to each steel plate rolling pass is output;

[0154] In the specific implementation, during the above calculation of the motion state of the steel plate, the steel plate rolling passes are counted, the above profile quantification results are matched with the pass judgment results, and transmitted to the statistical analysis and display unit in real time. The rolling situation of the entire steel plate is summarized through a bar chart and the contour quantification results of each pass are prominently displayed. The proportion of the interval where the quantification results of each pass are located is summarized through a pie chart. The specific implementation of pass judgment is as follows:

[0155] Let run_pass represent the pass number. The condition for setting the pass number to 1 is: when the first steel plate is detected to enter the farthest end of the video frame, that is, the steel plate to be rolled has just entered the camera screen, run_pass is set to 1.

[0156] The judgment condition for adding one pass count when the steel plate moves toward the rolling mill is: when the steel plate position curve is smaller than the rolling mill position, that is, the steel plate enters the other end of the rolling mill and disappears from the screen, the run_pass count is increased by 1.

[0157] When the steel plate moves away from the rolling mill, the pass count is increased by one, that is, when the steel plate reaches the farthest end, the run_pass count is increased by 1. The basis for this judgment has been given in the previous step.

[0158] In the above process of calculating the motion state of the steel plate, the steel plate rolling passes are counted, and the above profile quantification results are matched with the pass judgment results and transmitted to the statistical analysis and display unit in real time. The rolling situation of the entire steel plate is summarized through a bar chart and the profile quantification results of each pass are prominently displayed. The proportion of the interval where the quantification results of each pass are located is summarized through a pie chart, such as Figure 3 As shown in area B1.

[0159] In step S6, the steel plate is three-dimensionally reconstructed according to the contour bending degree coefficient corresponding to each steel plate rolling pass, and the bending degree of the steel plate is displayed in the three-dimensional reconstructed model of the steel plate.

[0160] In this embodiment, the steel plate model is 3D reconstructed according to the real-time quantification result of the profile (i.e., the profile bending degree coefficient), providing a multi-view real-time observation of the bending degree of the steel plate during the hot rolling process, such as Figure 3 As shown in the middle C area. According to the simulation results of the method of the embodiment, the left side profile of the steel plate is in a normal (green) state after the 4th to 15th rolling passes, accounting for 80% of all passes; the left side profile of the steel plate is not slightly bent during the rolling passes; the left side profile of the steel plate is bent more during the 1st to 3rd passes, accounting for 20% of all passes; the right side profile of the steel plate is in a normal (green) state after the 4th to 14th rolling passes, accounting for 73% of all passes; the right side profile of the steel plate is slightly bent during the 1st to 3rd passes, accounting for 20%; the right side profile of the steel plate is bent more during the 15th pass, accounting for 7% of all passes. In the hot rolling production process, the quantitative results of the rolling passes with relatively abnormal profiles are prompted, and the steel plate model is rebuilt in the three-dimensional view.

[0161] In order to solve the problems that the existing technical methods are complex, difficult to maintain and apply, have no quantitative data accumulation, and do not support multi-angle display of steel plate contour data in hot rolling production, the present application provides a real-time monitoring and quantification method and system for steel plate contour anomalies in the hot rolling process based on machine vision. The method and system do not require a large amount of data support, have robustness and resistance to water mist interference, and can achieve real-time monitoring and quantification of steel plate sickle bend anomalies. The quality control level of thick plate rolling can be monitored timely and effectively. A user-friendly visual design form is adopted to display real-time monitoring and quantitative statistical results from multiple angles, allowing users to explore in layers. It provides solutions to problems that are difficult to quantify based on manual judgment and complex problems of contact plate shape monitoring equipment, and provides data support for the hot rolling control process.

[0162] Embodiment 2:

[0163] This embodiment proposes a visual analysis system based on real-time monitoring of steel plate profiles, such as Figure 8 As shown, it includes: an image acquisition unit, a projection transformation unit, a contour line extraction unit, a bending coefficient calculation unit, a coefficient matching unit and a three-dimensional reconstruction unit;

[0164] The image acquisition unit, the projection transformation unit, the contour line extraction unit, the bending coefficient calculation unit, the coefficient matching unit and the three-dimensional reconstruction unit are connected in sequence;

[0165] An image acquisition unit, used for acquiring a real-time moving image of the steel plate as an original image;

[0166] A projection transformation unit, used for extracting a steel plate image from the original image and performing projection transformation on the steel plate image;

[0167] A contour line extraction unit, used for extracting the contour line of the steel plate image after projection transformation;

[0168] A curvature coefficient calculation unit, used for calculating the contour curvature coefficient of the contour line;

[0169] A coefficient matching unit, used for matching each steel plate rolling pass with a corresponding profile bending degree coefficient, and outputting the profile bending degree coefficient corresponding to each steel plate rolling pass;

[0170] The three-dimensional reconstruction unit is used to perform three-dimensional reconstruction of the steel plate according to the contour bending degree coefficient corresponding to each steel plate rolling pass, and display the bending degree of the steel plate in the three-dimensional reconstruction model of the steel plate.

[0171] It can be understood that the image acquisition unit inputs the information collection input module in the system architecture, the projection transformation unit, the contour extraction unit, the bending coefficient calculation unit, and the coefficient matching unit belong to the real-time monitoring module in the system architecture, and the three-dimensional reconstruction unit belongs to the perception and detection unit in the system architecture.

[0172] Embodiment 3:

[0173] This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the visualization analysis method based on real-time monitoring of steel plate contour.

[0174] The electronic device may be a mobile phone, a computer or a tablet computer, etc., including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the visual analysis method based on real-time monitoring of the steel plate profile as described in the embodiment is implemented. It is understood that the electronic device may also include an input / output (I / O) interface and a communication component.

[0175] The processor is used to execute all or part of the steps in the visual analysis method based on real-time monitoring of steel plate profile as described in the above embodiment. The memory is used to store various types of data, which may include instructions of any application or method in the electronic device, as well as data related to the application.

[0176] The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the visual analysis method based on real-time monitoring of steel plate profile described in the above embodiments.

[0177] Embodiment 4:

[0178] This embodiment provides a computer-readable storage medium storing executable instructions. When the instructions are executed and implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0179] The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the visual analysis method based on real-time monitoring of steel plate contours described in each embodiment of the present application.

[0180] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (for example, SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR abbreviation, memory data register) memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, CD, server, APP (Application, abbreviation of application software) application store and other media that can store program verification codes, on which computer programs are stored. When the computer program is executed by the processor, the various steps of the above-mentioned visual analysis method based on real-time monitoring of steel plate contour can be implemented.

[0181] Embodiment 5:

[0182] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the visualization analysis method based on real-time monitoring of steel plate contours.

[0183] Based on such understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a computer program product.

[0184] The various embodiments in the present application are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0185] The protection scope of the present application is not limited to the above-mentioned embodiments. Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the scope and spirit of the present disclosure. If these changes and modifications fall within the scope of the claims of the present disclosure and their equivalents, the intention of the present disclosure also includes these changes and modifications.

Claims

1. A visual analysis method based on real-time monitoring of steel plate profile, characterized in that: include: Acquire a real-time moving image of the steel plate as an original image; Extracting a steel plate image from the original image, and performing a projection transformation on the steel plate image; Extracting the contour line of the steel plate image after projection transformation; Calculate the contour curvature coefficient of the contour line; Match each steel plate rolling pass with the corresponding profile bending degree coefficient, and output the profile bending degree coefficient corresponding to each steel plate rolling pass; According to the contour bending coefficient corresponding to each steel plate rolling pass, the steel plate is reconstructed in three dimensions, and the bending degree of the steel plate is displayed in the three-dimensional reconstruction model of the steel plate; The step of extracting a steel plate image from the original image and performing a projection transformation on the steel plate image includes: Divide the original image into areas to be monitored; The color channel image difference algorithm is used to process the monitored area to obtain the segmented steel plate foreground image of the monitored area and segment the steel plate image; Calculate the color change curve of the steel plate image; According to the color change curve, a steel plate position curve and a rate of change of the steel plate position are calculated; The steel plate image is judged according to the steel plate position curve, the rate of change of the steel plate position and the corresponding threshold value to obtain the farthest end image; On the farthest image, the steel plate image is projected and transformed; The step of extracting the contour line of the steel plate image after projection transformation comprises: Filtering the steel plate image after projection transformation; Calculating the transverse gradient of the filtered steel plate image to obtain a transverse gradient image; In the transverse gradient image, pixel points whose absolute gradient values ​​are greater than or equal to the specified gradient value are selected to obtain the contour pixel points of the steel plate; The contour pixel points of all steel plates are fitted using a quadratic polynomial to obtain the contour line of the steel plate image; The calculation of the contour curvature coefficient of the contour line includes: Calculate the contour curvature coefficient of the contour line, and perform normalization processing on the contour curvature coefficient; The normalized curvature coefficient is calculated by segment mapping to obtain the contour curvature coefficient after smoothing; The normalized curvature coefficient is subjected to segment mapping calculation to obtain the contour curvature coefficient after smoothing, and the calculation formula is as follows: Wherein, L(b) is the segment mapping function, abs(b) is the absolute value of the normalized contour curvature coefficient, sign(b) is the sign function value of the contour curvature coefficient, T1 is the threshold of the first segment, and T2 is the threshold of the second segment.

2. The visualization analysis method based on real-time monitoring of steel plate profile according to claim 1 is characterized in that: The visualization analysis method based on real-time monitoring of steel plate contour also includes: outputting the contour line of the steel plate image after projection transformation and the corresponding original image in real time; and outputting the contour curvature coefficient of each contour line segment and the corresponding original image in real time.

3. A visual analysis system based on real-time monitoring of steel plate profile, characterized in that: include: An image acquisition unit, used for acquiring a real-time moving image of the steel plate as an original image; A projection transformation unit, used for extracting a steel plate image from the original image and performing projection transformation on the steel plate image; A contour line extraction unit, used for extracting the contour line of the steel plate image after projection transformation; A curvature coefficient calculation unit, used for calculating the contour curvature coefficient of the contour line; A coefficient matching unit, used for matching each steel plate rolling pass with a corresponding profile bending degree coefficient, and outputting the profile bending degree coefficient corresponding to each steel plate rolling pass; A three-dimensional reconstruction unit is used to perform three-dimensional reconstruction of the steel plate according to the contour bending degree coefficient corresponding to each steel plate rolling pass, and display the bending degree of the steel plate in the three-dimensional reconstruction model of the steel plate; The step of extracting a steel plate image from the original image and performing a projection transformation on the steel plate image includes: Divide the original image into areas to be monitored; The color channel image difference algorithm is used to process the monitored area to obtain the segmented steel plate foreground image of the monitored area and segment the steel plate image; Calculate the color change curve of the steel plate image; According to the color change curve, a steel plate position curve and a rate of change of the steel plate position are calculated; The steel plate image is judged according to the steel plate position curve, the rate of change of the steel plate position and the corresponding threshold value to obtain the farthest end image; On the farthest image, the steel plate image is projected and transformed; The step of extracting the contour line of the steel plate image after projection transformation comprises: Filtering the steel plate image after projection transformation; Calculating the transverse gradient of the filtered steel plate image to obtain a transverse gradient image; In the transverse gradient image, pixel points whose absolute gradient values ​​are greater than or equal to the specified gradient value are selected to obtain the contour pixel points of the steel plate; The contour pixel points of all steel plates are fitted using a quadratic polynomial to obtain the contour line of the steel plate image; The calculation of the contour curvature coefficient of the contour line includes: Calculate the contour curvature coefficient of the contour line, and perform normalization processing on the contour curvature coefficient; The normalized curvature coefficient is calculated by segment mapping to obtain the contour curvature coefficient after smoothing; The normalized curvature coefficient is subjected to segment mapping calculation to obtain the contour curvature coefficient after smoothing, and the calculation formula is as follows: Wherein, L(b) is the segment mapping function, abs(b) is the absolute value of the normalized contour curvature coefficient, sign(b) is the sign function value of the contour curvature coefficient, T1 is the threshold of the first segment, and T2 is the threshold of the second segment.

4. An electronic device, characterized in that: include: One or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the visualization analysis method based on real-time monitoring of steel plate contour as described in any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that: It stores executable instructions, which, when executed, enable the processor to execute the visualization analysis method based on real-time monitoring of steel plate profile as described in any one of claims 1 to 2.

6. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the visualization analysis method based on real-time monitoring of the steel plate profile as described in any one of claims 1 to 2 is implemented.

Images