Continuous casting automatic casting method and system based on intelligent visual inspection

Abstract

The invention provides a continuous casting automatic casting method and system based on intelligent visual inspection, and the method comprises the steps: S1, collecting video data of a process that molten steel flows to a tundish through an industrial camera; s2, cutting and positioning a to-be-identified target area, and further preprocessing video data of the target area; s3, on the basis of the video data of the target area, molten steel splashing characteristics are extracted, the probability that whether steel flowers appear in the target area or not is judged through depth separable convolution, if yes, the step S4 is executed, and if not, processing is not carried out; s4, steel flowers are detected in a preset time window, whether the occurrence probability and duration of the steel flowers reach preset thresholds at the same time or not is judged, if yes, a pouring starting signal is triggered, and if not, the step S1 is executed again. According to the method, intelligent upgrading of casting operation of the continuous casting machine is achieved, molten steel temperature loss caused by casting delay is avoided, and smooth proceeding of continuous casting production of special steel types is guaranteed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent visual recognition, and particularly relates to a continuous casting automatic pouring method and system based on intelligent visual detection. BACKGROUND

[0002] The continuous casting machine pouring operation is a key link in the whole continuous casting process, specifically refers to that the molten steel reaches the pouring platform from the ladle and is injected into the crystallizer, and each step needs to be completed quickly and stably to reduce the temperature loss of the molten steel and prevent equipment failure during pouring.

[0003] At present, the ladle weighing auxiliary manual pouring mode is generally used in China. Due to the influence of high temperature environment, the detection accuracy of the weighing sensor is limited, the real-time performance is poor, and the tonnage delay of the pouring time is easy to cause, which seriously affects the accuracy of different steel types and the quality of the cast slab.

[0004] Under the background of pursuing the production efficiency improvement of high-quality steel, it is crucial to accurately and timely determine the ladle pouring time.

[0005] The patent document CN 110340322 B introduces a complex machine learning model and multi-dimensional data input, integrates the molten steel level, the intermediate ladle nozzle opening, the steel type, the molten steel temperature and the drawing speed and other multiple key parameters. The scheme optimizes the multi-parameter collaborative control, introduces the field device information to reduce the false alarm caused by the brightness change of the non-pouring time, and realizes simple and low cost, and can achieve 99.9% of the accuracy of the pouring time. The traditional method of relying only on weight change to determine the pouring time is improved, which is more easily modified and deployed on the basis of existing facilities, and is especially suitable for the automation upgrade of the steel plant. The scheme needs to integrate the data of five types of sensors such as molten steel level, nozzle opening, temperature and other sensors, and any sensor failure will cause system misjudgment; the model needs to be retrained for different steel types, and the model generalization ability is limited; the existing weighing system needs to be modified and temperature sensor array needs to be installed, which has high implementation cost.

[0006] The patent document CN117830904A discloses a method for accurately determining the pouring time of a ladle based on image recognition. The method obtains a video stream by installing a camera hardware in the pouring area, extracts single-frame images from the obtained video stream for recognition, intercepts a picture of the tundish impact area directly below the long nozzle of the ladle from the image, removes noise points in the selected area using Gaussian blur, performs binaryzation processing on the selected area, detects the target of the steel flower and brightness in the binaryzation picture, compares the brightness and steel flower of the current frame with those of the previous frame, and determines the pouring time of the ladle when the increase amplitude exceeds the threshold value and lasts for 0.5 seconds. The method can accurately determine the pouring time of the ladle in the continuous casting process, significantly improves the accuracy of slab separation under the condition of multi-continuous casting, and brings substantial economic benefits to the enterprise. However, the method cannot significantly reduce the dependence on artificial experience parameters, has low accuracy, and has poor anti-interference performance. This problem needs to be solved. SUMMARY

[0007] In view of the defects in the prior art, the purpose of the present application is to provide a continuous casting automatic pouring method and system based on intelligent visual detection.

[0008] According to the present application, a continuous casting automatic pouring method based on intelligent visual detection is provided, which comprises:

[0009] Step S1: Collect video data of the process of molten steel flowing to the tundish through an industrial camera;

[0010] Step S2: Crop and locate the target area to be recognized, and then preprocess the video data of the target area;

[0011] Step S3: Based on the video data of the target area, extract the molten steel splashing feature, judge whether the probability of the appearance of the steel flower in the target area through deep separable convolution, if the result is yes, execute step S4, if the result is no, do not process;

[0012] Step S4: Detect the steel flower in a preset time window, and judge whether the appearance probability and the duration of the steel flower reach the preset threshold value at the same time, if the result is yes, trigger the pouring signal, if the result is no, re-execute step S1.

[0013] Preferably, it further comprises: Step S0: Real-time monitor and judge whether the ladle is rotated to the target position and the bushing installation of the continuous casting machine is completed, if the result is yes, open the ladle sliding nozzle to make the molten steel flow out, and then execute step S1, if the result is no, re-execute step S0;

[0014] In the step S2, the target area, i.e. the region of interest, is abbreviated as ROI;

[0015] Let (ROI x,ROI y ) are the horizontal and vertical coordinates of the upper left corner of ROI, (ROI w ,ROI h ) are the width and height of ROI respectively, then the mathematical expression of ROI clipping is:

[0016] I ROI (x′,y′)=I(x+ROI x ,y+ROI y ),x′∈[0,ROI w ],y′∈[0,ROI h ]

[0017] Among them, I ROI (x′, y′) represents the cropping result, that is, the target area; x′ represents the horizontal coordinate of the target area, y′ represents the vertical coordinate of the target area, I represents the input original image, x represents the horizontal coordinate of the original image, and y represents the vertical coordinate of the original image.

[0018] Preferably, in step S2, the preprocessing includes denoising and sharpening;

[0019] The denoising process is to median filter the image of the video data;

[0020] The mathematical expression of the gray value g′(x,y) of the pixel at point (x,y) after median filtering is:

[0021] g′(x,y)=Median{f(xw,yh),...,f(x+w,y+h)}

[0022] Where f(x,y) represents the grayscale value of the original image at point (x,y), the Median function represents the median calculation of the grayscale values ​​of all pixels in the neighborhood around point (x,y), and (w,h) represents the width and height of the neighborhood, that is, the size of the filter window.

[0023] The image sharpening is performed using the Laplace operator; the mathematical expression of the Laplace operator is:

[0024]

[0025] in, Represents the Laplace operator, I(x,y) is the grayscale value or color channel value of the image at the coordinate position (x,y), and and Represent the second-order partial derivatives of the image in the x-direction and y-direction respectively.

[0026] Preferably, the step S3 includes:

[0027] Step S3.1: extracting and obtaining a multi-dimensional feature map of the video data of the target region through a convolutional neural network;

[0028] Step S3.2: extracting a one-dimensional feature vector of the multi-dimensional feature map through a fully connected layer of the MobileNetV2 model, and then mapping to a category space;

[0029] Step S3.3: converting the one-dimensional feature vector into a normalized probability distribution through a Softmax layer of the MobileNetV2 model; outputting a probability of 'with splashing' and a probability of 'without splashing';

[0030] Step S3.4: judging whether the probability of splashing with steel flowers, i.e., the probability of 'with splashing', exceeds a preset threshold, and if yes, executing Step S4, and if no, not processing;

[0031] In the step S3.1, the convolutional neural network comprises: a depthwise convolution and a pointwise convolution;

[0032] The depthwise convolution has a mathematical expression as follows:

[0033]

[0034] wherein, I represents an input feature map, C in is a channel number of the input feature map, w i is an i-th depthwise convolution kernel, O d is an output feature map after the depthwise convolution, DepthwiseConv represents a channel-wise spatial convolution, I i represents an i-th channel slice of the input feature map I; b represents a bias term;

[0035] The pointwise convolution has a mathematical expression as follows:

[0036]

[0037] wherein, C dw is an output channel number after the depthwise convolution, C out is a channel number of an output of the pointwise convolution, w ij is a pointwise convolution kernel, O di is an i-th channel of the depthwise convolution output feature map, b j is a bias term of a j-th output channel, O p is a final output feature map after the pointwise convolution, PointwiseConv represents the pointwise convolution; the symbol * represents a product;

[0038] In the step S3.2, the one-dimensional feature vector has a mathematical expression as follows:

[0039] z = W fc · X flatten + b fc

[0040] wherein z represents a one-dimensional feature vector, W fc represents a weight matrix, b fc represents a bias term, the symbol X flatten represents a flattened feature input, wherein, is a real set, c is the number of classes, and d is the dimension of the flattened vector;

[0041] In the step S3.3, the mathematical expression of the normalized probability distribution is:

[0042]

[0043] wherein p i represents the probability that the input image belongs to the i-th class, i.e., the probability of 'with splashes' and the probability of 'without splashes', satisfying i.e., p i ∈ [0, 1], i = 1, 2, wherein i = 1 corresponds to the probability of 'with splashes', and i = 2 corresponds to the probability of 'without splashes'; represents the exponential operation on the score of the i-th classification; represents the exponential operation on the score of the j-th classification.

[0044] Preferably, in the step S4, it is judged whether the probability of splashes is greater than a preset probability threshold and the duration of splashes reaches a preset time threshold within a preset number of consecutive seconds; if the result is yes, a casting signal is sent, and an event timestamp is recorded synchronously; if the result is no, the step S1 is re-executed;

[0045] The probability of splashes is 0.9;

[0046] The preset time threshold is 1 second.

[0047] According to the application, an automatic continuous casting casting system based on intelligent visual detection is provided, which comprises:

[0048] An imaging module: through an industrial camera, video data of the process of molten steel flowing to a tundish is collected;

[0049] An image processing module: a target region to be recognized is cropped and positioned, and video data of the target region is preprocessed;

[0050] Image recognition module: based on the video data of the target area, extract molten steel splashing features, judge whether the probability of molten steel splashing in the target area appears through deep separable convolution, the result is yes, then execute step S4, the result is no, then do not process;

[0051] Control unit module: detect the molten steel splashing in the preset time window, and judge whether the molten steel splashing probability and duration reach the preset threshold value at the same time, the result is yes, then trigger the casting signal, the result is no, then trigger the imaging module to work again.

[0052] Preferably, it also includes: preparation module: real-time monitoring and judging whether the ladle rotates to the target position and the bushing installation of the continuous casting machine is completed, the result is yes, then open the ladle slide gate, make the molten steel flow out, and then trigger the imaging module to work, the result is no, then trigger the preparation module to work again;

[0053] In the image recognition module, the target area, i.e. the region of interest, is abbreviated as ROI;

[0054] Let (ROI y ,ROI w ) be the horizontal coordinate and vertical coordinate of the upper left corner of ROI respectively, (ROI h ,ROI ROI ) be the width and height of ROI respectively, and the mathematical expression of the ROI clipping is:

[0055] I x (x′,y′)=I(x+ROI y ,y+ROI w ),x′∈[0,ROI h ],y′∈[0,ROI ROI ]

[0056] Where I in (x′,y′) represents the clipping result, i.e. the target area; x′ represents the horizontal coordinate of the target area, y′ represents the vertical coordinate of the target area, I represents the input original image, x represents the horizontal coordinate of the original image, and y represents the vertical coordinate of the original image.

[0057] Preferably, in the image processing module, the preprocessing includes denoising processing and sharpening processing;

[0058] The denoising processing, i.e. median filtering, filters the image of the video data;

[0059] The mathematical expression of the gray value g′(x,y) of the pixel at point (x,y) after median filtering is:

[0060] g′(x,y)=Median{f(x-w,y-h),...,f(x+w,y+h)}

[0061] where f(x, y) represents the gray value of the original image at point (x, y), the Median function represents the median value calculation of all pixel gray values in the neighborhood around point (x, y), and (w, h) represents the width and height of the neighborhood, i.e. the size of the filter window;

[0062] The image is sharpened, i.e. image sharpening is performed using a Laplacian operator; the Laplacian operator has a mathematical expression as follows:

[0063]

[0064] wherein, represents the Laplacian operator, I(x, y) is the gray value or color channel value of the image at coordinate position (x, y), and and respectively represent the second-order partial derivatives of the image in the x direction and the y direction.

[0065] Preferably, in the image recognition module, comprising:

[0066] Image recognition unit one: extracting and obtaining the multi-dimensional feature map of the video data of the target region through a convolutional neural network;

[0067] Image recognition unit two: extracting a one-dimensional feature vector of the multi-dimensional feature map through the fully connected layer of the MobileNetV2 model, and then mapping to the category space;

[0068] Image recognition unit three: converting the one-dimensional feature vector into a normalized probability distribution through the Softmax layer of the MobileNetV2 model; outputting the probability of 'with splashing' and the probability of 'without splashing';

[0069] Image recognition unit four: judging whether the probability of splashing with steel flowers, i.e. the probability of 'with splashing', exceeds a preset threshold, and if the result is yes, executing step S4, and if the result is no, not processing;

[0070] In the image recognition unit one, the convolutional neural network comprises: a deep convolution and a point-by-point convolution;

[0071] The deep convolution has a mathematical expression as follows:

[0072]

[0073] wherein I represents an input feature map, C in is the number of channels of the input feature map, w i is the i-th deep convolution kernel, O d is the output feature map after deep convolution, DepthwiseConv represents a channel-by-channel spatial convolution, and Ii represents the i-th channel slice of the input feature map I; b represents a bias term;

[0074] The mathematical expression of the pointwise convolution is:

[0075]

[0076] wherein C dw is the number of output channels after deep convolution, C out is the number of channels of the pointwise convolution output, w ij is the pointwise convolution kernel, O di is the i-th channel of the deep convolution output feature map, b j is the bias term of the j-th output channel, O p is the final output feature map after pointwise convolution, and PointwiseConv represents pointwise convolution; the symbol * represents multiplication;

[0077] In the image recognition unit two, the mathematical expression of the one-dimensional feature vector is:

[0078] z = W fc · X flatten + b fc

[0079] wherein z represents a one-dimensional feature vector, W fc represents a weight matrix, b fc represents a bias term, the symbol X flatten represents a flattened feature input, wherein is a real set, c is the number of categories, and d is the dimension of the flattened vector;

[0080] In the image recognition unit three, the mathematical expression of the normalized probability distribution is:

[0081]

[0082] wherein p i represents the probability that the input image belongs to the i-th category, i.e., the probability of 'with splashes' and the probability of 'without splashes', which satisfies i.e., p i ∈ [0, 1], i = 1, 2, wherein i = 1 corresponds to the probability of 'with splashes', and i = 2 corresponds to the probability of 'without splashes'; represents the exponential operation on the score of the i-th classification; represents the exponential operation on the score of the j-th classification.

[0083] Preferably, in the control unit module, it is judged whether all the steel flower splashing probabilities are greater than a probability preset threshold and the duration of the steel flower splashing reaches a time preset threshold within a continuous preset number of seconds; if yes, a casting start signal is sent and an event time stamp is recorded synchronously; if no, the imaging module is triggered again.

[0084] The steel flower splashing probability is 0.9.

[0085] The time preset threshold is 1 second.

[0086] Compared with the prior art, the present application has the following beneficial effects:

[0087] 1. The present application uses intelligent visual recognition technology to replace the traditional method of determining the casting start based on the weight change of the tundish, realizes the accurate and real-time detection of the casting start time of the continuous casting machine, significantly improves the automation level and safety of the continuous casting production, and solves the problems of chaotic splitting and quality fluctuation of the cast slab caused by weighing delay.

[0088] 2. The present application uses intelligent visual recognition technology to replace the traditional weighing method, solves the problem of misjudgment of the casting start time caused by low precision and poor real-time performance of the weighing equipment in a high-temperature environment, and improves the one-time success rate of the casting operation and the accuracy of the cast slab splitting.

[0089] 3. The present application realizes the intelligent upgrading of the casting operation of the continuous casting machine, avoids the loss of molten steel temperature caused by delayed casting, and ensures the smooth progress of the continuous casting production of special steel grades. BRIEF DESCRIPTION OF DRAWINGS

[0090] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the attached drawings:

[0091] Figure 1 A continuous casting machine casting start work schematic diagram is provided for the present application.

[0092] Figure 2 A work flow schematic diagram is provided for the present application. DETAILED DESCRIPTION

[0093] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These all belong to the protection scope of the present application.

[0094] The present application aims to provide a new automatic pouring method based on intelligent visual detection, so as to make up for the error of traditional weighing and judgment of pouring time, and improve the one-time pouring success rate and the accuracy of separation.

[0095] According to the present application, an automatic continuous casting pouring method based on intelligent visual detection is provided, and the flow of the software system thereof comprises:

[0096] Step 0: Step 0: monitor and judge whether the ladle is rotated to the target position and the bushing installation of the continuous casting machine is completed, and the result is yes, then open the ladle slide nozzle, make the molten steel flow out, and then execute step 1, and the result is no, then re-execute step 0;

[0097] Step 1: collect the video data of the molten steel flowing to the tundish through an industrial camera;

[0098] Step 2: crop and position the target area to be identified, and then preprocess the video data of the target area;

[0099] Step 3: analyze the video data of the target area based on a convolutional neural network, judge whether there is a molten steel splashing phenomenon, the result is yes, then execute step 4, the result is no, then do not process;

[0100] Step 4: judge whether there is a continuous molten steel splashing event in the target area, the result is yes, then send a pouring signal, the result is no, then re-execute step 2.

[0101] Step S1: the control module monitors the ladle rotation to the position signal in real time, and confirms the completion of the bushing installation, the result is yes, then execute step S2, the result is no, then re-execute step S1.

[0102] Step S2: the control module accepts the ladle slide nozzle opening instruction, the result is yes, then execute step S3, the result is no, then re-execute step S2.

[0103] Step S3: the industrial camera starts video collection, continuously shoots the dynamic process of molten steel flowing to the tundish;

[0104] Step S4: position the tundish port boundary, crop and position the target area to be identified, and then preprocess the video data of the target area;

[0105] Step S5: extract the molten steel splashing feature based on the MobileNetV2 model, and judge whether the molten steel splashing probability appears in the target area through depth separable convolution;

[0106] Step S6: detect the molten steel splashing in a preset time window, trigger the pouring signal when the molten steel splashing probability and the duration time reach the threshold value at the same time, record the event timestamp and detection parameters synchronously, and the result is no, then re-execute step S3.

[0107] In the step S2, the target region to be identified is cropped and positioned, that is, the boundary of the middle package opening is accurately identified by using the edge detection Canny algorithm, and then the region of interest ROI is determined.

[0108] Let (ROI x ,ROI y ) be the horizontal coordinate and the vertical coordinate of the upper left corner of the ROI respectively, and (ROI w ,ROI h ) be the width and the height of the ROI respectively, and the ROI cropping formula is:

[0109] I ROI (x′,y′)=I(x+ROI x ,y+ROI y ),forx′∈[0,ROI w ],y′∈[0,ROI h ]

[0110] wherein I ROI (x′,y′) represents the cropping result, that is, the target region; x′ represents the horizontal coordinate of the target region, y′ represents the vertical coordinate of the target region, I represents the input original image, x represents the horizontal coordinate of the original image, and y represents the vertical coordinate of the original image.

[0111] Specifically, in the step S2, the preprocessing includes denoising processing and sharpening processing.

[0112] The denoising processing, that is, the denoising is performed by using a two-dimensional median filter. The median filter is a nonlinear filtering method, which can effectively remove salt and pepper noise and speckle noise without affecting the image edge details.

[0113] The original image is often accompanied by noise, including quantum noise, readout noise, etc.

[0114] The mathematical expression of the gray value g′(x,y) of the pixel at point (x,y) after median filtering is:

[0115] g′(x,y)=Median{f(x-w,y-h),...,f(x+w,y+h)}

[0116] wherein f(x,y) represents the gray value of the original image at point (x,y), the Median function represents the median value calculation of all pixel gray values in the neighborhood around point (x,y), and (w,h) represents the width and height of the neighborhood, that is, the size of the filtering window.

[0117] The image sharpening, detail enhancement, i.e. image sharpening with Laplacian operator to increase the edge definition of the steel flower in the steel flower splashing area. Laplacian operator, expression as follows:

[0118]

[0119] Wherein, represents the Laplacian operator, I(x, y) is the gray value or color channel value of the image at the coordinate position (x, y), and and respectively represent the second-order partial derivative of the image in the x direction and the y direction. In order to be sensitive to the change in the diagonal direction, an extended convolution kernel is used:

[0120]

[0121] Specifically, in the step S3, MobileNetV2 deep learning model is used as the core algorithm for image feature extraction and recognition. The MobileNetV2 model is built on the basis of depth separable convolution, which is a light and computationally efficient convolution form, aiming to reduce the number of model parameters and maintain high recognition accuracy.

[0122] Depth separable convolution consists of two parts: depth convolution and pointwise convolution. Depth convolution performs convolution operation independently on each input channel, and the mathematical expression is:

[0123]

[0124] Wherein, I represents the input feature map, C in is the number of input feature map channels, w i is the i-th depth convolution kernel, O d is the output feature map after depth convolution, DepthwiseConv represents pointwise spatial convolution, I i represents the i-th channel slice of the input feature map I; b represents the bias term;

[0125] Then, pointwise convolution is performed, which integrates the information of all channels together, and its formula is expressed as:

[0126]

[0127] Wherein, C dw is the number of output channels after depth convolution, C out is the number of channels output by pointwise convolution, w ij is the pointwise convolution kernel, O di is the i-th channel of the depth convolution output feature map, b j is the bias term of the j-th output channel, Op is the final output feature map after pointwise convolution, and PointwiseConv represents pointwise convolution.

[0128] The MobileNetV2 model also introduces a learnable shortcut connection on the basis of the original. Through a series of stacks of depth separable convolution, bottleneck layer and shortcut connection layer, effective extraction and recognition of steel flower splashing features are realized. The intelligent recognition module inputs the processed image features into the model, and the model finally outputs the probability distribution of two categories of 'with steel flower' and 'without steel flower' through the full connection layer and the Softmax layer, so as to determine whether there is a steel flower splashing phenomenon and trigger the pouring instruction.

[0129] Specifically, a convolutional neural network is followed by a full connection layer and a softmax layer. The probability distribution of whether the steel flower splashes is obtained.

[0130] Specifically, it is converted into a normalized probability distribution. In combination with 'the steel flower splashing probability of each model output exceeds the threshold value' in the detailed description below, it is judged whether there is a steel flower.

[0131] In the embodiment, the probability of having a steel flower is 0.8, and the probability of not having a steel flower is 0.2.

[0132] The full connection layer flattens the multi-dimensional feature map extracted by the convolutional network into a one-dimensional feature vector, and maps it to the class space through linear transformation:

[0133] z=W fc ·X flatten +b fc

[0134] wherein z represents a one-dimensional feature vector, represents a weight matrix, represents a bias term, and the symbol represents the flattened feature input, wherein c is the number of classes and d is the dimension of the flattened vector; the symbol · represents dot product;

[0135] The Softmax layer converts the full connection layer output z into a normalized probability distribution:

[0136]

[0137] wherein the output p i ∈[0,1] represents the probability that the input image belongs to the i-th class, and satisfies

[0138] Specifically, the preset threshold of the steel flower splashing probability is the probability of the 'with steel flower' category, and the threshold is set to 0.9; when the 'with steel flower' probability is continuously detected to be greater than or equal to 0.9, it is considered as an effective splashing event;

[0139] The duration of the steel flower spatter is usually 1 second, and if the valid spatter event is continuously detected for a duration within the range, it is determined that the pouring has started.

[0140] The two thresholds are manually adjusted according to the field process and imaging conditions.

[0141] The control unit first monitors the rotation of the ladle to the position and confirms the completion of the sleeve, and then receives the ladle opening instruction. The camera starts capturing video of the molten steel flowing to the tundish. The image recognition module analyzes the video stream in real time according to the preset logic, and when it finds that the target area has a sustained bright flame that meets the pouring characteristics, the control module immediately responds, sends a pouring signal, and stores the event record in the designated business database.

[0142] The intelligent visual detection system includes an imaging module, an image processing module, an image recognition module, and a control unit.

[0143] Imaging module: Use existing pouring area industrial cameras to capture image data of the ladle pouring moment at key nodes during continuous casting.

[0144] Image processing module: Real-time analysis of the received continuous video stream, accurate cropping of the tundish opening area, and positioning of the target area to be identified.

[0145] Image recognition module: Through convolutional neural network analysis of the cropped image segments, it identifies whether there is a sustained steel flower spatter phenomenon, which serves as the basis for determining the ladle pouring. Specifically, in the intelligent visual recognition automatic pouring system based on image recognition technology, the MobileNetV2 deep learning model is used as the core algorithm for image feature extraction and recognition. The MobileNetV2 model is built on the basis of depth separable convolution, which is a lightweight and computationally efficient convolution form designed to reduce the number of model parameters while maintaining high recognition accuracy.

[0146] Control unit: Receive visual recognition results, when the model output changes from no bright flame to sustained bright flame, trigger the system to automatically determine the ladle pouring time, and send relevant information to the continuous casting production line control system to guide the stable start of the subsequent process.

[0147] Imaging module, in the intelligent visual recognition automatic pouring system based on image recognition technology, uses existing pouring area industrial cameras to capture image data of the ladle pouring moment during continuous casting. The camera module first faces the problem of hardware selection. Considering the harsh environment of high temperature, dust, and strong electromagnetic interference in the continuous casting workshop, the selected camera should have the following characteristics:

[0148] The camera should have high temperature resistance; specifically, the camera shell material needs to withstand long-term high temperature environment, usually using metal or special plastic material, to ensure that it can still work normally under high temperature.

[0149] The camera can meet the optical performance; specifically, the lens needs to have high light transmittance and resolution, with a resolution not less than 1080P, to ensure the clarity of the captured image is sufficient for subsequent image processing and recognition.

[0150] The camera can meet the low latency performance; specifically, the camera design supports low latency mode, ensuring that the delay in the process of data acquisition to network output is not more than 40 milliseconds, ensuring the immediacy and accuracy of the judgment of the continuous casting machine pouring time.

[0151] The camera can be dustproof and waterproof; specifically, the camera should have IP65 protection level to ensure that it is not damaged in a dusty environment.

[0152] The image preprocessing module in the continuous casting machine intelligent visual recognition automatic pouring system based on image recognition technology adopts multiple key technologies to process the original image captured by the industrial camera in the imaging module, to optimize the image quality, highlight the feature information and prepare for the subsequent intelligent recognition.

[0153] To better extract the characteristics of steel flower splashing, the image preprocessing module converts the original RGB color space to HSV color space. The brightness information in the HSV color space can better reflect the difference between the steel flower and the background. The following is the key formula in the conversion process:

[0154] Standardize the RGB value from [0, 255] to [0, 1], the mathematical expression is:

[0155]

[0156] Calculate C max 、C min and Δ, the mathematical expressions from top to bottom are:

[0157] C max = max(R', G', B')

[0158] C min = min(R', G', B')

[0159] Δ = C max -C min

[0160] Where C max represents the maximum color value, C minRepresents the minimum color value, and Δ represents the chromaticity span between the maximum and minimum values ​​of the three channels (R', G', B') in the RGB color space.

[0161] Calculate the hue, which is determined by the maximum color value. The mathematical expression is:

[0162]

[0163] Here, H represents hue.

[0164] Calculate the saturation, the mathematical expression is:

[0165]

[0166] Among them, S represents saturation;

[0167] Calculate the brightness, which is determined by the maximum color value. The mathematical expression is:

[0168] V=C max

[0169] Where V represents brightness.

[0170] In the continuous casting machine intelligent visual recognition automatic pouring system, the control unit adopts the following strategy to determine the pouring start time:

[0171] 1. Continuous monitoring of recognition results: The control unit continuously receives and processes the steel flower splash probability sequence output by the image recognition module for continuous image frames, which is recorded as: P 钢花 (t),t=1,2,…

[0172] 2. Set the recognition threshold: Set a preset threshold θ. Only when the probability of steel spatter identified by the model exceeds this threshold, the model is considered to have recognized the possible steel spatter phenomenon: P 钢花 (t)>θ

[0173] 3. Time window judgment: Set a time window T. If the probability of steel spatter output by the model exceeds the threshold θ every time within T seconds, it is determined that there is continuous steel spatter during this period: P 钢花 (t)>θ; where middle, and They represent existence and arbitrariness respectively.

[0174] 4. Determine the casting start time: when the condition that the steel flower splashing probability continuously exceeds the threshold value in the above continuous time period is met, define the time point t0 when the condition is first met as the casting start time. At this time, the control unit takes this time point as the key time node for triggering the ladle casting operation of the continuous casting machine, and records it to the real-time database on site.

[0175] According to the application, an automatic continuous casting casting start system based on intelligent visual detection is provided, which comprises: a control unit that monitors the ladle rotation to position signal in real time and confirms the completion of the sleeve, and monitors the ladle nozzle opening signal, and starts the automatic continuous casting casting start system when receiving the ladle nozzle opening signal.

[0176] After the imaging module receives the control unit start signal, it continuously shoots continuous video data of the continuous casting process by using the existing pouring area industrial camera.

[0177] The image processing module analyzes the continuous video stream shot by the imaging module in real time, accurately crops the tundish opening area, locates the target area to be recognized, and processes the image to improve the image quality.

[0178] The image recognition module analyzes the processed image segments through a convolutional neural network to identify whether there is a steel flower splashing phenomenon, and outputs the category information and probability of the judgment.

[0179] The control unit judges whether the condition for triggering the casting start is reached according to the steel flower splashing feature information and its duration fed back by the recognition module, and records the casting start time to the real-time database on site.

[0180] The application also provides an automatic continuous casting casting start system based on intelligent visual detection, which can be realized by executing the process steps of the automatic continuous casting casting start method based on intelligent visual detection, that is, those skilled in the art can understand the automatic continuous casting casting start method based on intelligent visual detection as the preferred embodiment of the automatic continuous casting casting start system based on intelligent visual detection.

[0181] According to the application, an automatic continuous casting casting start system based on intelligent visual detection is provided, which comprises:

[0182] Imaging module: collect video data of the process of molten steel flowing to the tundish;

[0183] Image processing module: crop and locate the target area to be recognized, and then preprocess the video data of the target area;

[0184] Image recognition module: based on the video data of the target area, extract the molten steel splashing feature, judge whether the steel flower appears in the target area through a deep separable convolution, and if the result is yes, execute step S4, and if the result is no, do not process.

[0185] The control unit module detects the steel flower in a preset time window, and judges whether the steel flower occurrence probability and the duration reach the preset threshold value at the same time, and if yes, a casting start signal is triggered, and if no, the imaging module is triggered again.

[0186] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module and unit thereof in a pure computer readable program code manner, the system provided by the present application and each device, module and unit thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps to achieve the same functions. Therefore, the system provided by the present application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for achieving various functions can also be considered as structures in the hardware component; the devices, modules and units for achieving various functions can also be considered as both software modules for implementing methods and structures in the hardware component.

[0187] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be combined with each other in any manner without conflict.

Claims

1. A continuous casting automatic pouring method based on intelligent visual detection, characterized in that: include: Step S1: collecting video data of the process of molten steel flowing into the tundish; Step S2: cropping and locating the target area to be identified, and then preprocessing the video data of the target area; Step S3: Based on the video data of the target area, the molten steel splash features are extracted, and the probability of whether steel splashes appear in the target area is determined by depthwise separable convolution. If the result is yes, step S4 is executed; if the result is no, no processing is performed; Step S4: Detect steel flowers within the preset time window and determine whether the probability of steel flower occurrence and duration reach the preset threshold at the same time. If the result is yes, trigger the pouring start signal; if the result is no, re-execute step S1.

2. The method for automatic continuous casting based on intelligent visual detection according to claim 1, characterized in that: The process also includes: step S0: real-time monitoring and determining whether the ladle, that is, whether the ladle has rotated to the target position and the sleeve of the continuous casting machine has been installed. If so, the sliding nozzle of the ladle is opened to allow the molten steel to flow out, and step S1 is then executed. If not, step S0 is executed again. In the step S2, the target region is the region of interest, ROI for short; Order (ROI x ,ROI y ) are the horizontal and vertical coordinates of the upper left corner of ROI, (ROI w ,ROI h ) are the width and height of ROI respectively, then the mathematical expression of ROI clipping is: Yo ROI (x′,y′)=I(x+ROI x ,and+ROI y ),x′∈[0,ROI w ], and′∈[0,ROI h ] Among them, I ROI (x′, y′) represents the cropping result, that is, the target area; x′ represents the horizontal coordinate of the target area, y′ represents the vertical coordinate of the target area, I represents the input original image, x represents the horizontal coordinate of the original image, and y represents the vertical coordinate of the original image.

3. The method for automatic continuous casting based on intelligent visual detection according to claim 2, characterized in that: In the step S2, the pre-processing includes denoising and sharpening; The denoising process is to median filter the image of the video data; The mathematical expression of the gray value g′(x,y) of the pixel at point (x,y) after median filtering is: g′(x,y)=Median{f(xw,yh),...,f(x+w,y+h)} Where f(x,y) represents the grayscale value of the original image at point (x,y), the Median function represents the median calculation of the grayscale values ​​of all pixels in the neighborhood around point (x,y), and (w,h) represents the width and height of the neighborhood, that is, the size of the filter window. The image sharpening is performed using the Laplace operator; the mathematical expression of the Laplace operator is: in, Represents the Laplace operator, I(x,y) is the grayscale value or color channel value of the image at the coordinate position (x,y), and and Represent the second-order partial derivatives of the image in the x-direction and y-direction respectively.

4. The method for automatic continuous casting based on intelligent visual detection according to claim 3, characterized in that: In the step S3, it includes: Step S3.1: extracting and obtaining a multi-dimensional feature map of the video data of the target area through a convolutional neural network; Step S3.2: Extract the one-dimensional feature vector of the multi-dimensional feature map through the fully connected layer of the MobileNetV2 model, and then map it to the category space; Step S3.3: Convert the one-dimensional feature vector into a normalized probability distribution through the Softmax layer of the MobileNetV2 model; output the probability of "splash" and "no splash"; Step S3.4: Determine whether the probability of steel sparks flying, i.e., the probability of "sparks flying", exceeds a preset threshold. If yes, execute step S4; if no, do not process. In the step S3.1, the convolutional neural network includes: depthwise convolution and point-by-point convolution; The mathematical expression of the depth convolution is: Among them, I represents the input feature map, C in is the number of channels of the input feature map, w i is the i-th depth convolution kernel, O d It is the output feature map after depth convolution, DepthwiseConv represents channel-by-channel spatial convolution, I i represents the i-th channel slice of the input feature map I; b represents the bias term; The mathematical expression of the point-by-point convolution is: Among them, C dw is the number of output channels after depthwise convolution, C out is the number of channels of the point-by-point convolution output, w ij is the point-by-point convolution kernel, O di is the i-th channel of the depthwise convolution output feature map, b j is the bias term of the jth output channel, O p It is the final output feature map after point-by-point convolution. PointwiseConv represents point-by-point convolution. The symbol * represents the product. In step S3.2, the mathematical expression of the one-dimensional feature vector is: z=W fc ·X flatten +b fc Among them, z represents a one-dimensional feature vector, W fc represents the weight matrix, b fc represents the bias term, Symbol X flatten represents the flattened feature input, in, is a set of real numbers; c is the number of categories, and d is the dimension of the vector after flattening; In step S3.3, the mathematical expression of the normalized probability distribution is: Among them, p i represents the probability that the input image belongs to the i-th category, that is, the probability of 'with splash' and the probability of 'without splash', satisfying That is, p i ∈[0,1], i=1,2, where i=1 corresponds to the probability of 'splashing' and i=2 corresponds to the probability of 'no splashing'; Indicates the exponential operation of the score of the i-th category; Indicates the exponential operation on the score of the j-th category.

5. The method for automatic continuous casting based on intelligent visual detection according to claim 4, characterized in that: In step S4, it is determined whether the probability of steel sparks flying is greater than a preset probability threshold within a continuous preset number of seconds, and whether the duration of steel sparks flying reaches a preset time threshold; if the result is yes, a pouring start signal is issued and the event timestamp is recorded synchronously; if the result is no, step S1 is executed again; The probability of steel sparks splashing is 0.9; The preset time threshold is 1 second.

6. A continuous casting automatic pouring system based on intelligent visual detection, characterized in that: include: Imaging module: collects video data of the process of molten steel flowing into the tundish; Image processing module: cropping and locating the target area to be identified, and then preprocessing the video data of the target area; Image recognition module: extracts the molten steel splash features based on the video data of the target area, and determines the probability of steel splashes appearing in the target area through depthwise separable convolution. If the result is yes, step S4 is executed; if the result is no, no processing is performed; Control unit module: detects steel flowers within the preset time window, and determines whether the probability of steel flowers appearing and the duration reach the preset threshold at the same time. If the result is yes, it triggers the pouring start signal; if the result is no, it re-triggers the imaging module to work.

7. The continuous casting automatic pouring system based on intelligent visual detection according to claim 6 is characterized in that: The system also includes: a preparation module: real-time monitoring and judging whether the ladle, that is, whether the ladle has rotated to the target position and the sleeve of the continuous casting machine has been installed. If the result is yes, the sliding nozzle of the ladle is opened to allow the molten steel to flow out, thereby triggering the operation of the imaging module; if the result is no, the preparation module is re-triggered; In the image recognition module, the target region is the region of interest, referred to as ROI; Order (ROI x ,ROI y ) are the horizontal and vertical coordinates of the upper left corner of ROI, (ROI w ,ROI h ) are the width and height of ROI respectively, then the mathematical expression of ROI clipping is: Yo ROI (x′,y′)=I(x+ROI x ,and+ROI y ),x′∈[0,ROI w ], and′∈[0,ROI h ] Among them, I ROI (x′, y′) represents the cropping result, that is, the target area; x′ represents the horizontal coordinate of the target area, y′ represents the vertical coordinate of the target area, I represents the input original image, x represents the horizontal coordinate of the original image, and y represents the vertical coordinate of the original image.

8. The continuous casting automatic pouring system based on intelligent visual detection according to claim 7 is characterized in that: In the image processing module, the preprocessing includes denoising and sharpening; The denoising process is to median filter the image of the video data; The mathematical expression of the gray value g′(x,y) of the pixel at point (x,y) after median filtering is: g′(x,y)=Median{f(xw,yh),...,f(x+w,y+h)} Where f(x,y) represents the grayscale value of the original image at point (x,y), the Median function represents the median calculation of the grayscale values ​​of all pixels in the neighborhood around point (x,y), and (w,h) represents the width and height of the neighborhood, that is, the size of the filter window. The image sharpening is performed using the Laplace operator; the mathematical expression of the Laplace operator is: in, Represents the Laplace operator, I(x,y) is the grayscale value or color channel value of the image at the coordinate position (x,y), and and Represent the second-order partial derivatives of the image in the x-direction and y-direction respectively.

9. The continuous casting automatic pouring system based on intelligent visual detection according to claim 8 is characterized in that: The image recognition module includes: Image recognition unit 1: extracts and obtains a multi-dimensional feature map of the video data of the target area through a convolutional neural network; Image Recognition Unit 2: Extract one-dimensional feature vectors from multidimensional feature maps through the fully connected layer of the MobileNetV2 model and then map them to the category space; Image recognition unit 3: Converts the one-dimensional feature vector into a normalized probability distribution through the Softmax layer of the MobileNetV2 model; outputs the probability of "splash" and "no splash"; Image recognition unit 4: determines whether the probability of steel sparks flying, that is, whether the probability of "sparks flying" exceeds a preset threshold. If the result is yes, step S4 is executed; if the result is no, no processing is performed; In the image recognition unit 1, the convolutional neural network includes: depth convolution and point-by-point convolution; The mathematical expression of the depth convolution is: Among them, I represents the input feature map, C in is the number of channels of the input feature map, w i is the i-th depth convolution kernel, O d It is the output feature map after depth convolution, DepthwiseConv represents channel-by-channel spatial convolution, I i represents the i-th channel slice of the input feature map I; b represents the bias term; The mathematical expression of the point-by-point convolution is: Among them, C dw is the number of output channels after depthwise convolution, C out is the number of channels of the point-by-point convolution output, w ij is the point-by-point convolution kernel, O di is the i-th channel of the depthwise convolution output feature map, b j is the bias term of the jth output channel, O p It is the final output feature map after point-by-point convolution. PointwiseConv represents point-by-point convolution. The symbol * represents the product. In the second image recognition unit, the mathematical expression of the one-dimensional feature vector is: z=W fc ·X flatten +b fc Among them, z represents a one-dimensional feature vector, W fc represents the weight matrix, b fc represents the bias term, Symbol X flatten represents the flattened feature input, in, is a set of real numbers; c is the number of categories, and d is the dimension of the vector after flattening; In the image recognition unit three, the mathematical expression of the normalized probability distribution is: Among them, p i represents the probability that the input image belongs to the i-th category, that is, the probability of 'with splash' and the probability of 'without splash', satisfying That is, p i ∈[0,1], i=1,2, where i=1 corresponds to the probability of 'splashing' and i=2 corresponds to the probability of 'no splashing'; Indicates the exponential operation of the score of the i-th category; Indicates the exponential operation on the score of the j-th category.

10. The continuous casting automatic pouring system based on intelligent visual detection according to claim 9 is characterized in that: In the control unit module, it is determined whether the probability of all steel sparks flying is greater than a preset probability threshold within a continuous preset number of seconds, and whether the duration of the steel sparks flying reaches a preset time threshold; if the result is yes, a pouring start signal is issued and the event timestamp is recorded synchronously; if the result is no, the imaging module is re-triggered to work; The probability of steel sparks splashing is 0.9; The preset time threshold is 1 second.

Images