Method and system for measuring mud ramming amount of blast furnace mud gun machine based on computer vision

Abstract

The invention relates to a method and a system for measuring the mud ramming amount of a blast furnace mud gun machine based on computer vision. The method comprises the following steps: collecting a motion video data set of the mud gun machine, and carrying out data labeling on a mud gun machine scale device in a picture; a target detection model is constructed, the target detection model is obtained by defining a network structure, a loss function and a back propagation algorithm, and the target detection model is used for detecting a scale device on the clay gun machine; using a straight line detection algorithm in a scale device area to screen out a slide block straight line and a reference straight line; a mapping function of the sliding block position and the mud ramming amount of the blast furnace mud gun machine is constructed, and the mud ramming amount is calculated in real time through the function; no additional monitoring equipment is needed, no physical device needs to be added to the clay gun machine, the measurement precision is high based on the machine vision technology, the clay beating amount can be continuously calculated in real time, and database reading and writing are carried out.

Description

Technical Field

[0001] The invention relates to the technical field of visual recognition, and is a method and system for measuring the mud volume of a blast furnace mud gun machine based on computer vision. Background Art

[0002] The blast furnace mud gun machine plays an important role in the blast furnace smelting process; it is responsible for pressing the gun mud into the blast furnace iron mouth, and maintaining the iron mouth by controlling the amount of addition to ensure the stable operation of the blast furnace and the longevity of the furnace; the purpose of measuring the amount of mud added by the blast furnace mud gun machine is to ensure the stable and effective distribution of the gun mud near the iron mouth on the inner wall of the blast furnace.

[0003] At present, manual measurement is often used to obtain the amount of mud produced by the blast furnace mud gun machine. The operator estimates the depth of the mud material pressed into the gun by observing the travel of the pointer on the gun machine. However, this method has the following obvious defects: (1) The measurement results are easily affected by the operator's subjective judgment and personal experience. Different operators may have different judgment standards and measurement methods, resulting in inconsistent and inaccurate results. (2) Operators are required to observe and measure at close range, which may pose safety risks. Harsh working conditions such as high temperature increase the complexity and difficulty of measurement. (3) Visual inspection results are usually recorded manually, which is not accurate and standardized enough, and data collection is not timely, which cannot meet the needs of real-time analysis.

[0004] Although in order to improve the accuracy and efficiency of measurement, some studies have proposed auxiliary measurement methods based on mechanical devices; however, most of the above methods optimize the mud cannon machine by improving the convenience of personnel observation and the accuracy of measurement, and still cannot overcome the problems of large subjectivity of manual measurement and difficulty in recording and analysis; in addition, some studies have proposed automatic measurement devices based on technologies such as sensors, but the above methods all require additional devices, which are relatively expensive. In addition, since the main iron ditch is located below the mud cannon machine, which is a high-temperature area, the impact vibration is strong when the mud cannon machine is blocked, which makes the related devices easy to be damaged and difficult to use stably for a long time.

[0005] In order to obtain the amount of mud driven by the blast furnace mud gun machine, most of the current methods require the operator to closely observe the scale where the pointer on the mud gun machine is located to measure the driving depth of the mud gun machine during the mud gun operation. This method has the disadvantages of harsh working environment, high operating risk, large measurement error, and untimely data collection. Although some studies have proposed auxiliary measurement methods based on sensors, mechanical devices, etc. to improve the accuracy and efficiency of measurement; however, the above methods require the installation of additional devices, which are relatively costly. In addition, the reliability and stability of the devices cannot be guaranteed under high temperature and harsh working conditions, making them difficult to promote and apply in actual production. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and to provide a method and system for measuring the mud volume of a blast furnace mud gun machine based on computer vision. There is no need to add additional monitoring equipment or any physical device to the mud gun machine. The measurement accuracy is high based on machine vision technology, and the mud volume can be calculated in real time and continuously, and database reading and writing can be performed.

[0007] The present invention relates to a method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision, comprising the following steps: S110, collecting a video data set of a mud cannon machine movement, and annotating data on a scale device of the mud cannon machine in the picture; S120, constructing a target detection model, including defining a network structure, a loss function, and a back propagation algorithm to obtain the target detection model, wherein the target detection model is used to detect a scale device on a mud gun machine; S130, using a straight line detection algorithm in the area of ​​the ruler device to screen out a slider straight line and a reference straight line; S140, constructing a mapping function between the position of the slider and the mud amount of the blast furnace mud gun machine, and using the function to calculate the mud amount in real time; In step S140, the area where the mud gun machine is located can be detected by step S120. In this area, the straight line of the slider on the mud gun machine and the reference straight line information are screened out by straight line detection according to step S130. Based on the relative position of the slider in the image, the coordinates are converted to calculate the mud volume of the mud gun machine in the real environment. The specific calculation method is as follows: Among them, D is the actual mud volume of the current mud gun machine, k and b are proportional constants calculated based on the actual data and image data, and pval is the relative position of the current slider in the mud gun machine; The relative position pval of the current slider in the mud gun machine in the above formula is obtained by the following formula: Among them, MCX is the x-coordinate of the left endpoint of the slider straight line detected, and MCY is the y-coordinate of the left endpoint of the slider straight line; RPX is the x-coordinate of the left endpoint of the reference straight line detected, and RPY is the y-coordinate of the left endpoint of the reference straight line; that is, we calculate the linear distance between the edge point of the current slider and the edge point of the reference object, thereby obtaining the relative position of the current slider in the mud gun machine; The constants k and b in the above are calculated by the following formula: k=(R1-R2) / (pval1-pval2) b=R1-k*sub1 Among them, R1 is the first measured mud amount collected in the early stage, pval1 is the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to the first measured mud amount value, R2 is the second measured mud amount collected in the early stage, pval2 is the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to the second measured mud amount value, and the k value is substituted into the formula to obtain the constant value b; Mud amount Mud amount Further, the step S110 includes, in the blast furnace front monitoring system, selecting a monitoring angle position where the operation of the blast furnace mud cannon machine can be observed, obtaining the mud making video of the blast furnace mud cannon machine at this angle, and cutting the video, and dividing the obtained video into picture sets according to the number of frames.

[0008] Furthermore, the step S110 also includes selecting a target detection model, and dividing the labeled images into a training set, a test set and a validation set, inputting the training set into the target detection model and training iteratively for a set number of times to obtain a preliminary detection model, and verifying the preliminary detection model using the validation set and the test set, and finally, selecting the preliminary detection model with the best effect in the training stage as the trained target detection model.

[0009] Furthermore, in the step S130, it includes using a straight line detection algorithm to screen and detect the area where the mud gun machine is located, and according to the features of the slider and the mud gun machine in the image, the straight line with the smallest endpoint x coordinate (the straight line is the edge of the slider) is screened out from the obtained straight line set to obtain the slider straight line, and the straight line with the smallest endpoint y coordinate (the straight line is the upper edge of the mud gun machine) is screened out from the obtained straight line set to obtain the reference straight line; further, the target detection model is a combination of one or more of SSD, R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN, MaskR-CNN, FCN, R-FCN, and YOLO.

[0010] Furthermore, the network structure of the YOLO model includes an input layer, a convolutional layer, a feature fusion layer, a detection layer, and a non-maximum suppression layer: Input layer: YOLO's input is an image, which is processed by dividing the image into fixed-size grids; Convolutional layer: YOLO uses convolutional layers to extract features of images, using a pre-trained convolutional neural network as a feature extractor; Feature fusion layer: In order to obtain target features of different scales, YOLO inserts multiple feature fusion layers in the network. These layers fuse feature maps from different levels to simultaneously detect targets of different sizes. Detection layer: The detection layer of YOLO is the last layer of the network, which is responsible for detecting and locating the target in the image. The detection layer maps the feature map to a grid of bounding box predictions and generates the coordinates and confidence scores of the prediction boxes. Each prediction box usually contains confidence scores of multiple categories to determine the category of the target. Non-maximum suppression layer: Since YOLO's detection layer generates a large number of candidate boxes, in order to remove redundant detection results, non-maximum suppression is usually used to filter out the best target box. The non-maximum suppression layer filters the detection results according to the confidence score and overlap to retain the most accurate target box.

[0011] Furthermore, the usage of the YOLO model includes: After building the YOLO network architecture, input the labeled training data into the YOLO model and use the back propagation algorithm to optimize the network parameters; During the training process, the position loss and category loss of the prediction box are calculated and added together to get the total loss. The error between the model prediction result and the actual position of the manually labeled mud gun machine or slider is calculated through the loss function, and the network parameters are updated through back propagation to reduce the error. The YOLO model is optimized by using one or more of hyperparameter configuration and loss function adjustment to improve the performance and robustness of the model. Finally, the trained YOLO model is deployed in practical applications, and the monitoring video or image of the mud cannon machine to be detected is input into the trained target detection model to obtain the area where the mud cannon machine is located.

[0012] A system for measuring the amount of mud produced by a blast furnace mud gun machine using computer vision, characterized in that it includes an input module, a model building module, a screening module and a calculation module; The input module is used to collect the mud cannon machine motion video data set and to annotate the mud cannon machine scale device in the picture; The model building module is used to build a target detection model, including defining a network structure, a loss function and a back-propagation algorithm to obtain a target detection model, and the target detection model is used to detect a scale device on a mud gun machine; The screening module is used to use a straight line detection algorithm in the area of ​​the ruler device to screen out the slider straight line and the reference object straight line; The calculation module is used to construct a mapping function between the position of the slider of the mud gun machine and the mud discharge amount of the blast furnace mud gun machine, and use the function to calculate the mud discharge amount in real time.

[0013] The invention is beneficial in that: the target detection technology is used to identify the monitoring video of the blast furnace mud gun machine to obtain the information of the area where the mud gun machine is located; the straight line detection technology is used to obtain the slider straight line and the reference straight line information in the area where the mud gun machine is located; the mapping relationship between the calculated relative position information and the mud gun machine mud amount information is used to construct a conversion function to obtain the current mud gun machine mud amount in real time; finally, the final mud amount is obtained, the detection process is completed, and the data is stored in a database in real time; A method for measuring the amount of mud produced by blast furnace iron mouth mud gun based on computer vision technology is designed, which can calculate and display the amount of mud produced by the mud gun machine in real time, significantly improving the objectivity of measurement and the real-time nature of data recording. In actual application, only cameras are used for monitoring, without the need for any physical devices attached to the mud gun machine. This solves the monitoring problems caused by the harsh environment in front of the blast furnace that are limited by traditional manual inspection and physical measurement, and effectively reduces costs.

[0014] In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is a schematic diagram of the steps of a method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision.

[0017] Figure 2 It is the algorithm flow chart of the target detection model.

[0018] Figure 3 It is the flow chart of the line detection algorithm.

[0019] Figure 4 This is a marked schematic diagram of a mud cannon machine.

[0020] Figure 5 Detection results of the scale device for the YOLO model.

[0021] Figure 6 Detection results of the scaler device of Fast R-CNN.

[0022] Figure 7 This is the FLD straight line detection result diagram.

[0023] Figure 8 Schematic diagram of the slider line and reference line selected from the FLD results.

[0024] Fig. 9 This is the Hough line detection result diagram.

[0025] Fig.10 Schematic diagram of the slider line and reference line filtered out from the Hough result.

[0026] Fig.11 This is a real-time detection result chart of mud volume. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0028] Reference Figure 1 In a preferred embodiment of the present invention, a method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision comprises the following steps: S110, collecting a video data set of a mud cannon machine movement, and annotating data on a scale device of the mud cannon machine in the picture; S120, constructing a target detection model, including defining a network structure, a loss function, and a back propagation algorithm to obtain the target detection model, wherein the target detection model is used to detect a scale device on a mud gun machine; S130, using a straight line detection algorithm in the area of ​​the ruler device to screen out a slider straight line and a reference straight line; S140, constructing a mapping function between the relative position of the slider and the mud amount of the blast furnace mud gun machine, and using the function to calculate the mud amount in real time; In step S140, the area where the mud gun machine is located can be detected by using step S120. In this area, the straight line of the slider on the mud gun machine and the reference straight line information are screened out according to step S130 by using straight line detection. Based on the relative position of the slider in the image, the coordinates are converted to calculate the mud volume of the mud gun machine in the real environment. The specific calculation method is as follows: Formula 1 Among them, D is the actual mud volume of the current mud gun machine, k and b are proportional constants calculated based on actual data and image data, and pval is the relative position of the current slider in the mud gun machine.

[0029] The relative position pval of the current slider in the mud gun machine in the above formula is obtained by formula 2: Formula 2 Among them, MCX is the x-coordinate of the left endpoint of the detected slider straight line, and MCY is the y-coordinate of the left endpoint of the slider straight line; RPX is the x-coordinate of the left endpoint of the detected reference straight line, and RPY is the y-coordinate of the left endpoint of the reference straight line; that is, we calculate the straight-line distance between the edge point of the current slider and the edge point of the reference object, and thus we can get the relative position of the current slider in the mud gun machine.

[0030] The constants k and b in the above are calculated by formula 3 and formula 4: k=(R1-R2) / (pval1-pval2) Formula 3 b=R1-k*sub1 Formula 4 R1 is the first measured mud production amount collected in the early stage, pval1 is the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to the first measured mud production value, R2 is the second measured mud production amount collected in the early stage, pval2 is the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to the second measured mud production value, and the k value is substituted into the formula to obtain the constant value b; Mud production amount In the above embodiment, the step S110 includes selecting a monitoring angle position at which the operation of the blast furnace mud cannon machine can be observed in the blast furnace front monitoring system, obtaining a mud production video of the blast furnace mud cannon machine at this angle, and cutting the video, and dividing the obtained video into picture sets according to the number of frames.

[0031] In the above embodiment, the step S110 also includes selecting a target detection model, and dividing the annotated pictures into a training set, a test set and a verification set. After the training set is input and the target detection model is iterated for a set number of training, a preliminary detection model is obtained, and the preliminary detection model is verified by using the verification set and the test set. Finally, the preliminary detection model with the best effect in the training stage is selected as the trained target detection model. In one embodiment, 5 videos of the working process of the mud gun machine are obtained, and these videos are divided into picture sets. Finally, 8,000 pictures are selected for annotation to construct a VOC data set. The annotation examples are as follows: Figure 4 As shown in the figure, the constructed VOC dataset is divided into training set, validation set and test set in a ratio of 6:2:2.

[0032] In the above embodiment, in the step S130, a straight line detection algorithm is used to screen and detect the area where the mud gun machine is located. According to the features of the slider and the mud gun machine in the image, the straight line with the smallest endpoint x coordinate (the straight line is the edge of the slider) is screened out from the obtained straight line set to obtain the slider straight line, and the straight line with the smallest endpoint y coordinate (the straight line is the upper edge of the mud gun machine) is screened out from the obtained straight line set to obtain the reference straight line.

[0033] The mainstream algorithms for target detection include SSD, R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN, MaskR-CNN, FCN, R-FCN, YOLO, etc.; taking YOLO technology as an example, the core process of this target detection is introduced; YOLO is an object recognition and positioning algorithm based on deep neural network; now YOLO has developed to version v8, although each version is slightly different, but its basic framework is similar, and each new version is continuously improved and evolved on it, and is suitable for solving this problem; therefore, this patent does not distinguish between specific versions, but only explains the core technical solution process of this patent based on the YOLO core technology framework; it should be emphasized that the solution idea of ​​the present invention is to obtain the location information of the mud gun machine based on target detection technology, and the specific target detection technology can be any existing target detection technology or various future target detection algorithms.

[0034] Reference Figure 2 In the above embodiment, the network structure of the YOLO model includes an input layer, a convolution layer, a feature fusion layer, a detection layer, and a non-maximum suppression layer: Input layer: YOLO's input is an image, which is processed by dividing the image into fixed-size grids; Convolutional layer: YOLO uses convolutional layers to extract features of images, using a pre-trained convolutional neural network as a feature extractor; Feature fusion layer: In order to obtain target features of different scales, YOLO inserts multiple feature fusion layers in the network. These layers fuse feature maps from different levels to simultaneously detect targets of different sizes. Detection layer: The detection layer of YOLO is the last layer of the network, which is responsible for detecting and locating the target in the image. The detection layer maps the feature map to a grid of bounding box predictions and generates the coordinates and confidence scores of the prediction boxes. Each prediction box usually contains confidence scores of multiple categories to determine the category of the target. Non-maximum suppression layer: Since YOLO's detection layer generates a large number of candidate boxes, in order to remove redundant detection results, non-maximum suppression is usually used to filter out the best target box. The non-maximum suppression layer filters the detection results according to the confidence score and overlap to retain the most accurate target box.

[0035] In the above embodiment, the method of using the YOLO model includes: After building the YOLO network architecture, input the labeled training data into the YOLO model and use the back propagation algorithm to optimize the network parameters; During the training process, the position loss and category loss of the prediction box are calculated and added together to get the total loss. The error between the model prediction result and the actual position of the manually labeled mud gun machine is calculated through the loss function, and the network parameters are updated through back propagation to reduce the error. The YOLO model is optimized by using one or more of hyperparameter configuration and loss function adjustment to improve the performance and robustness of the model. Finally, the trained YOLO model is deployed in practical applications, and the monitoring video or image of the mud cannon machine to be detected is input into the trained target detection model to obtain the area where the mud cannon machine is located.

[0036] In one embodiment, the YOLO model is selected as the target detection model to improve the detection speed and accuracy. After 150 iterations of training, the map_50 (a performance indicator for measuring the target detection algorithm) has reached 93%. In order to improve the detection accuracy, the validation set and the test set are used for verification during the training process. Finally, the model with the best effect in the training stage is selected. The image or video to be detected is input into this model to obtain the target area of ​​the mud gun machine, such as Figure 5 shown.

[0037] In another embodiment, the Fast R-CNN model is selected as the target detection model. After 150 iterations of training, the map_50 obtained is 95%. The image or video to be detected is input into this model to obtain the target area of ​​the mud gun machine, such as Figure 6 As shown; by comparing the effects of the two embodiments, it can be seen that the existing target detection models can accurately obtain the mud gun machine target and meet the design requirements of this patent.

[0038] A system for measuring the amount of mud produced by a blast furnace mud gun machine using computer vision, characterized in that it includes an input module, a model building module, a screening module and a calculation module; The input module is used to collect the mud cannon machine motion video data set and to annotate the mud cannon machine scale device in the picture; The model building module is used to build a target detection model, including defining a network structure, a loss function and a back-propagation algorithm to obtain a target detection model, and the target detection model is used to detect a scale device on a mud gun machine; The screening module is used to use a straight line detection algorithm in the area of ​​the ruler device to screen out the slider straight line and the reference object straight line; The calculation module is used to construct a mapping function between the position of the slider and the mud discharge amount of the blast furnace mud gun machine, and use the function to calculate the mud discharge amount in real time.

[0039] In the actual implementation process, in this example, the FLD line detection algorithm is used to screen and detect the area where the mud gun machine is located; according to the characteristics of the slider and the mud gun machine in the image, the straight line with the smallest endpoint x coordinate (the straight line is the edge of the slider) is selected from the obtained straight line set to obtain the slider straight line, and the straight line with the smallest endpoint y coordinate (the straight line is the upper edge of the mud gun machine) is selected from the obtained straight line set to obtain the reference straight line; although the position of the mud gun machine can be obtained through target detection, the above position is still not accurate enough; in order to accurately calculate the amount of mud, the precise moving position of the slider must be accurately obtained; by observation, It can be seen that the edge of the slider on the mud gun machine has a typical straight line feature, and the slider is often located at the far left in the monitoring image. Therefore, the straight line detection technology can be used to output a straight line set in the slider target, and the straight line with the smallest endpoint x coordinate in the straight line set can be screened out to obtain the slider edge; further, in order to facilitate the calculation of the slider stroke, it is necessary to determine the reference point; the reference point can be any fixed point on the mud gun machine. For example, the straight line detection technology can be used to output a straight line set in the mud gun machine target, and the left endpoint of the straight line with the smallest y coordinate (i.e., the upper edge of the mud gun machine) can be screened out in the output straight line set to obtain the reference point.

[0040] It should be emphasized that the solution of the present invention is to obtain the accurate position information of the slider and the reference object based on the straight line detection technology. The specific straight line detection technology can be any existing straight line detection technology or various future straight line detection algorithms. Now, taking the FLD technology as an example, the core process of target detection is introduced. FLD (Fast Line Detection) is a fast straight line detection algorithm that aims to efficiently detect straight lines in an image with low computational complexity. The algorithm is based on the characteristics of the straight line in the polar coordinate space and realizes the detection and positioning of the straight line by extracting the angle and distance information of the straight line. The FLD algorithm can provide a faster straight line detection speed while maintaining a high accuracy by reducing the amount of calculation and simplifying the straight line detection process. The basic steps of the FLD algorithm are as follows: Figure 3 shown.

[0041] Edge detection: First, edge detection is performed on the input image, usually using a pixel-level edge detection algorithm such as Canny edge detection; edge detection can extract edge information in the image and provide input for subsequent straight line detection.

[0042] Polar coordinate transformation: The FLD algorithm transforms edge points from rectangular coordinates to polar coordinates, which is a difference from the traditional Hough transform algorithm. In polar coordinates, each edge point is represented as (ρ, θ), where ρ is the distance from the origin and θ is the angle between the edge point and the polar coordinate axis.

[0043] Line Detection: In polar coordinate space, the FLD algorithm detects lines through a series of fast and simple calculation steps; it uses a method called AC-ELMD (Angular Coherence and Extreme Line Micro-Detection), which detects straight line segments by calculating the angular differences between edge points; the AC-ELMD method does not need to traverse the entire polar coordinate space, so it has lower computational complexity.

[0044] Line parameter estimation: After detecting a line segment, the FLD algorithm estimates the line parameters, such as the starting point, the end point, and the angle of the line, by statistical and optimization methods.

[0045] Line detection is performed in the target area of ​​the mud gun machine to screen out the required slider line and reference line; by analyzing the information such as the slider edge point and the reference object edge point, it is substituted into S140 to obtain the current mud volume in real time.

[0046] An example of step S130 is Figure 7 and Figure 8 As shown in FIG. 1 , the FLD method is used to perform straight line detection on the target area of ​​the mud gun machine. The detected straight line set is shown in FIG. Figure 7 As shown; the straight line with the smallest endpoint x coordinate is selected from the straight line set to obtain the slider straight line, and the straight line with the smallest y coordinate is obtained as the reference straight line, as shown Figure 8 As shown; it can be seen that the slider straight line can accurately mark the edge of the slider, while the reference straight line accurately marks the edge of the mud gun machine.

[0047] Another example is Fig. 9 and Fig.10 As shown in FIG. 1 , the Hough line method is used to perform line detection on the target area of ​​the mud gun machine. The detected line set is as follows: Fig. 9 As shown in the figure, the filtered slider line and reference line are as follows Fig.10 shown; contrast Figure 8 and Fig.10 It can be seen that the above methods can accurately mark the edges of the slider and the mud gun machine, indicating that the currently commonly used straight line detection method can meet the design requirements of this patent.

[0048] In order to obtain the mud hitting value when the slider is in this position, coordinate transformation is required; in the example, the mud hitting value when the slider is in two different positions of the mud gun machine is first obtained, and then the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to these two positions is obtained. The mapping coefficients k=0.0003 and b=0.12 can be obtained by Formula 3 and Formula 4.

[0049] During the mud-beating process of the mud gun machine, the slider straight line and the reference straight line are detected in real time, and the left endpoints of the slider straight line and the reference straight line are obtained; the relative position of the current slider in the mud gun machine can be obtained by formula 2, and the relative position of the slider is substituted into formula 1 to calculate the mud-beating value; for example Fig.11 In this example, the pixel values ​​of the left endpoints of the current slider line and the reference line are (37, 276) and (123, 32) respectively. The relative position of the slider can be calculated as , further according to the mapping coefficient, the depth of the mud before can be calculated as 0.0003*258.71+0.12=0.2 meters; finally, according to the bore diameter of 0.5 meters and the density of the mud of 2.2 tons / cubic meter, the actual mud volume can be calculated as Kilograms; at the same time, we also use drawing methods to record the real-time data values ​​of the mud volume produced by the mud gun machine during its working process.

[0050] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the scope of the specific implementation methods. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision, characterized in that: The following steps are involved: S110, collecting a video data set of a mud cannon machine movement, and annotating data on a scale device of the mud cannon machine in the picture; S120, constructing a target detection model, including defining a network structure, a loss function, and a back propagation algorithm to obtain the target detection model, wherein the target detection model is used to detect a scale device on a mud gun machine; S130, using a straight line detection algorithm in the scale device area to screen out a slider straight line and a reference straight line; S140, constructing a mapping function between the position of the slider and the mud amount of the blast furnace mud gun machine, and using the function to calculate the mud amount in real time; In step S140, the area where the mud gun machine scale device is located can be detected by step S120. In this area, the slider straight line and reference straight line information on the mud gun machine are screened out by straight line detection according to step S130, and the coordinates are converted based on the slider position to calculate the mud hitting depth (mud gun machine piston movement distance) in the real environment. Finally, the mud hitting amount is calculated according to the mud hitting depth, channel radius, and mud bubble density. The specific calculation method is as follows: Where D is the actual mud volume of the current mud gun machine, Π is pi, d is the bore diameter, ρ is the mud density, k and b are proportional constants calculated based on actual data and image data, pval is the relative position of the current slider in the mud gun machine, and k×pval +b reflects the actual mud depth; The relative position pval of the current slider in the mud gun machine in the above formula is obtained by the following formula: Among them, MCX is the x-coordinate of the left endpoint of the slider straight line detected, and MCY is the y-coordinate of the left endpoint of the slider straight line; RPX is the x-coordinate of the left endpoint of the reference straight line detected, and RPY is the y-coordinate of the left endpoint of the reference straight line; that is, we calculate the linear distance between the edge point of the current slider and the edge point of the reference object, thereby obtaining the relative position of the current slider in the mud gun machine; The constants k and b in the above are calculated by the following formula: k=(R1-R2) / (pval1-pval2) b=R1-k*pval1 Among them, R1 is the first measured mud depth collected in the early stage, pval1 is the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to the first measured mud depth, R2 is the second measured mud depth collected in the early stage, pval2 is the distance between the left end point of the slider straight line and the left end point of the reference straight line in the image corresponding to the second measured mud depth, and the k value is substituted into the formula to obtain the constant value b.

2. Mud-making amount The method for measuring the mud-making amount of a blast furnace mud gun machine based on computer vision according to claim 1 is characterized in that: The step S110 includes selecting a monitoring angle position where the operation of the blast furnace mud cannon machine can be observed in the blast furnace front monitoring system, obtaining a mud beating video of the blast furnace mud cannon machine at the angle position, cutting the video, and dividing the obtained video into picture sets according to the number of frames.

3. The method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision according to claim 2 is characterized in that: The step S110 also includes selecting a target detection model, dividing the labeled images into a training set, a test set and a validation set, inputting the training set into the target detection model, obtaining a preliminary detection model after a set number of iterative trainings, verifying the preliminary detection model using the validation set and the test set, and finally, selecting the preliminary detection model with the best effect in the training phase as the trained target detection model.

4. The method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision according to claim 2 is characterized in that: In the step S130, a straight line detection algorithm is used to screen and detect the area where the mud gun machine is located. According to the features of the slider and the mud gun machine in the image, the straight line with the smallest endpoint x coordinate (the straight line is the slider edge) is screened out from the obtained straight line set to obtain the slider straight line, and the straight line with the smallest endpoint y coordinate (the straight line is the upper edge of the mud gun machine) is screened out from the obtained straight line set to obtain the reference object edge point.

5. The method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision according to claim 3 is characterized in that: The target detection model is a combination of one or more of SSD, R-CNN, SPP-Net, Fast R-CNN, Faster R-CNN, MaskR-CNN, FCN, R-FCN, and YOLO.

6. The method for measuring the amount of mud produced by a blast furnace mud cannon machine based on computer vision according to claim 4, characterized in that: The network structure of the YOLO model includes input layer, convolution layer, feature fusion layer, detection layer, and non-maximum suppression layer: Input layer: YOLO's input is an image, which is processed by dividing the image into fixed-size grids; Convolutional layer: YOLO uses convolutional layers to extract features of images, using a pre-trained convolutional neural network as a feature extractor; Feature fusion layer: In order to obtain target features of different scales, YOLO inserts multiple feature fusion layers in the network. These layers fuse feature maps from different levels to simultaneously detect targets of different sizes. Detection layer: The detection layer of YOLO is the last layer of the network, which is responsible for detecting and locating the target in the image. The detection layer maps the feature map to a grid of bounding box predictions and generates the coordinates and confidence scores of the prediction boxes. Each prediction box usually contains confidence scores of multiple categories to determine the category of the target. Non-maximum suppression layer: Since YOLO's detection layer generates a large number of candidate boxes, in order to remove redundant detection results, non-maximum suppression is usually used to filter out the best target box. The non-maximum suppression layer filters the detection results according to the confidence score and overlap to retain the most accurate target box.

7. The method for measuring the amount of mud produced by a blast furnace mud gun machine based on computer vision according to claim 6 is characterized in that: The usage of the YOLO model includes: After building the YOLO network architecture, input the labeled training data into the YOLO model and use the back propagation algorithm to optimize the network parameters; During the training process, the position loss and category loss of the prediction box are calculated and added together to get the total loss. The error between the model prediction result and the actual position of the manually labeled mud gun machine is calculated through the loss function, and the network parameters are updated through back propagation to reduce the error. The YOLO model is optimized by using one or more of hyperparameter configuration, data enhancement technology, and loss function adjustment to improve the performance and robustness of the model. Finally, the trained YOLO model is deployed in practical applications, and the monitoring video or image of the mud cannon machine to be detected is input into the trained target detection model to obtain the area where the mud cannon machine is located.

8. A system using the computer vision-based method for measuring the amount of mud produced by a blast furnace mud cannon machine according to any one of claims 1 to 7, characterized in that: It includes input module, model building module, screening module and calculation module; The input module is used to collect the mud cannon machine motion video data set and to annotate the mud cannon machine scale device in the picture; The model building module is used to build a target detection model, including defining a network structure, a loss function and a back-propagation algorithm to obtain a target detection model, and the target detection model is used to detect a scale device on a mud gun machine; The screening module is used to use a straight line detection algorithm in the area of ​​the ruler device to screen out the slider straight line and the reference object straight line; The calculation module is used to construct a mapping function between the position of the slider and the mud discharge amount of the blast furnace mud gun machine, and use the function to calculate the mud discharge amount in real time.

Images