Method, system and equipment for judging splashing duration of blast furnace tap hole and medium

Abstract

The invention provides a method, a system, equipment and a medium for judging splashing time of a blast furnace tap hole, and belongs to the technical field of blast furnace ironmaking. The method comprises the following steps: continuously acquiring video data of a tapping process through an industrial camera deployed at a specific position of a blast furnace tapping hole; performing video frame extraction, effective area cutting and image enhancement operation processing on the video data in the tapping process to generate an enhanced image data set; labeling the enhanced image data set based on a pre-constructed expert knowledge base to form a labeled data set; training a deep learning model by using the labeled data set; converting the trained deep learning model into an ONNX format, performing accelerated optimization by utilizing TensorRT, deploying the model to edge computing equipment, performing frame-level prediction on the real-time video stream, and generating a frame-level prediction result; and adopting a multi-frame voting strategy to carry out time sequence smoothing processing on the frame-level prediction result, and calculating the starting and ending time and the total duration of the splashing state based on the smoothing processing result.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of blast furnace ironmaking, and more particularly relates to a method, system, device and medium for judging the duration of a blast furnace tap hole spatter. BACKGROUND

[0002] In the process of blast furnace ironmaking, tap hole spatter is a common abnormal condition, which is caused by the imbalance between the pressure system and the fluid dynamics in the furnace. When the coal gas pressure in the furnace abnormally rises or the flowability of molten iron and slag changes suddenly, the high-temperature melt will abnormally gush out from the tap hole in the form of spatter. This phenomenon not only causes waste of valuable molten iron resources and damage to tapping equipment, but also may cause serious safety accidents, directly threatening the safety of production personnel and the stable operation of equipment.

[0003] Currently, the monitoring of this problem mainly relies on traditional methods. Among them, manual visual monitoring is the most common way, which depends on the experience of the operator. However, this method has obvious limitations: on the one hand, the human eye observation is easily disturbed by environmental factors such as smoke and light, and the accuracy of judgment is difficult to guarantee; on the other hand, manual recording inevitably has a time delay, and cannot accurately capture the start and end time of spatter, resulting in significant errors in the duration statistics. In addition, long-term high-intensity observation of the high-temperature tap hole also has an adverse effect on the occupational health of the operator.

[0004] In order to overcome the shortcomings of manual monitoring, some blast furnaces use sensor monitoring schemes to indirectly judge by installing temperature, pressure and other detection elements. However, this method has exposed new problems in practice: the reliability of sensors decreases in the harsh environment of high temperature and dust, the service life is greatly shortened, and the maintenance cost is high; at the same time, a single sensor can only reflect the change of local parameters, and it is difficult to accurately distinguish spatter from other abnormal conditions, resulting in a high false positive rate.

[0005] In recent years, the deep learning visual recognition technology has provided a new solution to this problem. This technology can directly analyze the real-time video data of the tap hole and capture the visual features of spatter. However, the existing technical solutions still have room for improvement: first, the effective integration of field expert experience in the model training process is lacking, resulting in a disconnection between feature learning and actual working conditions; second, the original video data contains a lot of redundant information, which reduces the processing efficiency; third, insufficient attention is paid to the sample imbalance problem of small probability events such as spatter, which affects the recognition sensitivity of the model. SUMMARY

[0006] To address the above problems, the present invention aims to provide a method, system, device, and medium for judging the duration of splashing at the blast furnace taphole. By integrating visual feature definitions from an expert knowledge base, using a ResNet34 model optimized with a weighted loss function, and combining TensorRT to accelerate inference and multi-frame voting post-processing technology, the invention achieves automated, high-precision real-time identification and accurate duration statistics of splashing behavior at the blast furnace taphole.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a method for determining the duration of splashing at the blast furnace taphole, including: Industrial cameras deployed at specific locations at the blast furnace tapping outlet continuously collect video data of the tapping process; Video frame extraction, effective region cropping, and image enhancement operations are performed on the video data of the iron tapping process to generate an enhanced image dataset. The augmented image dataset is annotated based on a pre-built expert knowledge base to form an annotated dataset; A deep learning model was trained using a labeled dataset. The deep learning model used ResNet34 as the backbone network and was optimized using a weighted loss function. The trained deep learning model is converted into ONNX format and accelerated and optimized using TensorRT. It is then deployed to an edge computing device to perform frame-level prediction on real-time video streams and generate frame-level prediction results. A multi-frame voting strategy is used to perform temporal smoothing on the frame-level prediction results, and the start and end times and total duration of the splash state are calculated based on the smoothing results.

[0008] In one optional implementation, the continuous acquisition of video data of the tapping process by an industrial camera deployed at a specific location at the blast furnace taphole includes: An industrial camera is deployed 8-10 meters directly in front of the blast furnace taphole, and the camera parameters are set to a resolution of 2048×2448 pixels and a frame rate of 25fps. Continuous acquisition of video data of the iron tapping process using industrial cameras; The collected video data of the iron tapping process is transmitted in real time to the central control room via optical fiber, stored in MP4 format with a bit rate of no less than 10Mbps, forming the original video dataset.

[0009] In an optional implementation, the step of performing video frame extraction, effective region cropping, and image enhancement operations on the iron tapping process video data to generate an enhanced image dataset includes: The original video dataset is split into individual images frame by frame; Each image is cropped by a valid pixel region to generate a cropped image; the valid pixel region is a rectangular area with starting coordinates (224, 624), a width of 1600 pixels, and a height of 1200 pixels; The cropped image is augmented with data operations including at least one of the following: rotation ±5°, translation ±10 pixels, Gaussian blur, color jitter, and random cropping, which increases the size of the original video dataset by 5 times to generate an enhanced image dataset.

[0010] In an optional implementation, the annotation of the enhanced image dataset based on a pre-built expert knowledge base to form an annotated dataset includes: Based on the experience of experts in blast furnace ironmaking, a mapping relationship between visual features and splashing states is established, and an expert knowledge base is constructed. The visual features include morphological features, motion features, temperature features, and spatiotemporal features. Morphological features include the height of molten iron jet exceeding the set threshold at the tapping spout, and the slag and molten iron mixing in a splashing distribution. Motion features include abnormal fluctuations in the molten iron stream and the splashed material exhibiting multi-directional splashing trajectories. Temperature features include the core area of ​​the splash appearing as a high-temperature bright white area in thermal imaging. Spatiotemporal features include the spatiotemporal characteristics of the splashing phenomenon exhibiting suddenness and continuity. The enhanced image dataset is independently labeled by at least two data labeling servers based on a constructed expert knowledge base, with splashed image data labeled as 1 and non-splashed image data labeled as 0. By using data augmentation techniques, the ratio of data samples labeled as splashing states to data samples labeled as non-splashing states is adjusted to a set ratio, forming a balanced labeled dataset.

[0011] In an optional implementation, the step of training a deep learning model using a labeled dataset, wherein the deep learning model employs ResNet34 as the backbone network and is optimized using a weighted loss function, includes: The labeled dataset was divided into training, validation, and test sets in a 7:2:1 ratio. A deep learning model is constructed using ResNet34 as the backbone network. A weighted mean squared error loss function is set for model training, and the weighted mean squared error loss function includes:

[0012] in, For the sample size, For real labels, These are the model's predicted values. For sample weights, when =1 =2, when =0 =1; Set the training parameters as follows: set the initial learning rate to 1e-4, set the learning rate decay strategy to decay to 0.8 times the original value every 5 training cycles, set the training batch size to 16, and set the training epochs to 50. The Adam optimizer is used to train and optimize the model, generating a splash state prediction model.

[0013] In an optional implementation, the step of converting the trained deep learning model to ONNX format and accelerating it using TensorRT, deploying it to an edge computing device, performing frame-level prediction on the real-time video stream, and generating frame-level prediction results includes: Convert the splash state prediction model to ONNX format; The TensorRT acceleration engine is used to optimize the splash state prediction model in the ONNX format to improve inference speed; Deploy the optimized splash state prediction model to edge computing devices; The deployed splash state prediction model is used to perform frame-level prediction on the real-time video stream, outputting the splash state prediction value for each frame.

[0014] In an optional implementation, the step of using a multi-frame voting strategy to perform temporal smoothing on the frame-level prediction results, and calculating the start and end times and total duration of the splash state based on the smoothing results, includes: Obtain the splash state prediction value of N consecutive frames of images. If the splash state prediction value of any frame of image is not lower than 0.8, then the frame of image is determined to be a candidate splash frame. Count the number of candidate splash frames in N frames of images. If the number of candidate splash frames is not less than N / 2, mark the corresponding time period of the image as splash state. Based on the marking results, record the start frame number Fstart and end frame number Fend that are marked as splashing state. Calculate the splashing duration T based on the camera frame rate of 25fps, using the following formula:

[0015] Where T represents the spray duration in seconds.

[0016] Secondly, embodiments of this application also provide a system for determining the duration of splashing at the blast furnace taphole, including: The data acquisition module is used to continuously acquire video data of the iron tapping process using industrial cameras deployed at specific locations at the blast furnace tapping spout. The data preprocessing module is used to perform video frame extraction, effective region cropping, and image enhancement operations on the iron tapping process video data to generate an enhanced image dataset. The data annotation module is used to annotate the augmented image dataset based on a pre-built expert knowledge base, forming an annotated dataset; The deep learning model training module is used to train a deep learning model using a labeled dataset. The deep learning model uses ResNet34 as the backbone network and is optimized using a weighted loss function. The model deployment and inference module is used to convert the trained deep learning model into ONNX format and use TensorRT for acceleration and optimization, deploy it to edge computing devices, perform frame-level prediction on real-time video streams, and generate frame-level prediction results. The post-processing and splash duration calculation module is used to perform temporal smoothing on the frame-level prediction results using a multi-frame voting strategy, and calculate the start and end times and total duration of the splash state based on the smoothing results.

[0017] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for determining the splashing duration at the blast furnace taphole as described in any of the above.

[0018] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, wherein when the computer program is executed by a processor, it implements the steps of the method for determining the splashing duration at the blast furnace taphole as described in any of the above embodiments.

[0019] As can be seen from the above technical solutions, the present invention has the following advantages: The method for determining the duration of blast furnace taphole splashing provided in this application achieves automation, high precision, and real-time performance by integrating expert experience and deep learning technology. This effectively overcomes the shortcomings of traditional manual visual monitoring, such as strong subjectivity, recording lag, and the vulnerability of sensors and high false alarm rates. The method guides data annotation by constructing an expert knowledge base that incorporates multi-dimensional splashing visual features, ensuring the model's learned features align with actual operating conditions. Targeted data preprocessing and sample balancing strategies improve model training efficiency and generalization ability. Weighted loss function optimization significantly reduces the missed detection rate of splashing states. Furthermore, by combining multi-frame voting post-processing and frame-level duration calculation, instantaneous interference is eliminated, achieving accurate capture of splashing start and end times and high-precision duration statistics, providing reliable technical support for safe blast furnace production.

[0020] This application constructs an end-to-end automatic identification system by deploying industrial cameras and edge computing devices, which completely replaces the traditional method of relying on manual observation and recording, realizes full-process automation of splash monitoring, and significantly reduces labor costs and safety risks.

[0021] This application guides model training by constructing an expert knowledge base that integrates multi-dimensional visual features and uses a weighted loss function to solve the sample imbalance problem, enabling the model to accurately learn the essential features of splashing, thereby effectively improving the accuracy of splashing state recognition and effectively reducing the rate of missed and false judgments.

[0022] This application significantly improves the model inference speed by converting the model to ONNX format and using TensorRT for acceleration and optimization, enabling it to perform smooth frame-level analysis of 25fps real-time video streams, thus meeting the needs of production sites for real-time monitoring and immediate early warning.

[0023] This application constructs a large-scale and high-quality enhanced image dataset by performing targeted and effective region cropping and diverse image enhancement operations on the original video data. This enables the trained model to adapt to complex iron tapping environments under different furnace conditions, lighting conditions, and smoke interference, thus ensuring the robustness of the system.

[0024] This application introduces a multi-frame voting strategy to perform temporal smoothing on the prediction results of consecutive frames, effectively filtering out instantaneous interference. Based on high frame rate video, it accurately calculates the start and end times of splashing, ensuring the statistical accuracy of splashing duration and providing precise data support for process analysis and optimization. Attached Figure Description

[0025] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating the method for determining the duration of splashing at the blast furnace taphole provided in this application.

[0027] Figure 2 A schematic diagram of the blast furnace taphole splash duration determination system provided in this application.

[0028] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0029] The various embodiments of this disclosure will be described more fully in the detailed steps of the method for determining the splashing duration at the blast furnace taphole, which will be described in more detail below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0030] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Please see Figure 1 The diagram shows a flowchart of a method for determining the splashing duration at the taphole of a blast furnace, as described in a specific embodiment. The method includes: S1: Continuously collect video data of the iron tapping process using industrial cameras deployed at specific locations at the blast furnace tapping outlet.

[0033] In this specific implementation, a professional industrial camera system is carefully deployed in a safe area 8-10 meters directly in front of the blast furnace taphole. This system uses a high-temperature resistant, dustproof industrial camera with a temperature tolerance of at least 800℃ and an IP67 protection rating to ensure stable operation in harsh, high-temperature, and dusty environments. The camera parameters are precisely configured, with a resolution of 2048×2448 pixels and a fixed frame rate of 25fps. This configuration clearly captures the subtle features of the splashing process, providing a high-quality image data foundation for subsequent analysis.

[0034] A high-speed data transmission channel is established between the industrial camera and the central control room host via armored fiber optic cable, enabling real-time and stable transmission of video data. The video stream is encoded and stored in MP4 format, with the bitrate strictly controlled above 10Mbps to ensure image quality meets the training requirements of the deep learning model. The entire acquisition process continuously covers the complete iron tapping cycle, systematically collecting complete video data including normal iron tapping and splashing conditions, forming a sufficiently large raw video dataset.

[0035] S2: Perform video frame extraction, effective region cropping, and image enhancement operations on the video data of the iron tapping process to generate an enhanced image dataset.

[0036] In a specific implementation, the acquired raw video data undergoes systematic preprocessing. First, the continuous video stream is split into independent frame images according to time sequence, using a standard of "one image per frame." Based on statistical analysis of over 10,000 frames of images from the taphole, the effective monitoring area is precisely defined as a rectangular region with a starting point at coordinates (224, 624), a width of 1600 pixels, and a height of 1200 pixels, ending at coordinates (1824, 1824). This area completely covers the taphole and the potential impact range of splashes. Precise cropping effectively removes invalid black pixels from image edges, reducing the data volume by over 40% and significantly improving subsequent processing efficiency.

[0037] To address the diverse characteristics of the blast furnace tapping environment, a multi-level data augmentation strategy was employed to improve dataset quality and generate an enhanced dataset. Specific implementations included: random rotation within a ±5° range to simulate the angular deviation of actual camera installation; ±10 pixel translation transformation to accommodate slight changes in the tapping nozzle position; application of Gaussian blur with a standard deviation of 0.5-1.0 to effectively simulate smoke obstruction; color jitter of ±15% brightness and ±10% contrast to simulate different lighting conditions; and random cropping of 1500×1100 pixel sub-images within the effective area to enhance the model's sensitivity to local splash features. After this comprehensive augmentation process, the original dataset was systematically expanded by five times, ensuring the model possesses good generalization ability.

[0038] S3: Annotate the augmented image dataset based on a pre-built expert knowledge base to form an annotated dataset.

[0039] In a specific implementation, a structured expert knowledge system is constructed by integrating the splashing judgment experience of senior operators and furnace managers with over 10 years of experience in blast furnace ironmaking, forming an expert knowledge base. Specifically, by deeply refining the key visual characteristics of splashing states, a comprehensive "visual characteristic-splashing state" mapping relationship is established. This system clearly defines the judgment criteria for splashing: the height of molten iron jet exceeds 10cm from the taphole, the slag and molten iron mix to exhibit a typical "splashing" distribution, and the temperature radiation of the splashing area exhibits a bright white characteristic, providing authoritative technical basis for data annotation.

[0040] Based on the established expert knowledge base, two or more senior operators independently labeled the preprocessed image dataset using a labeling server. The labeling process strictly adhered to a double-blind principle, with operators independently judging the operational status of each frame of image based on the visual feature-splashing state mapping relationship. An image consistently identified as being in a splashing state was labeled "1," while a non-splashing state was labeled "0." For cases with inconsistent judgments, the blast furnace manager conducted a final review to ensure the accuracy of the labeling results.

[0041] Finally, through targeted sample augmentation techniques, the ratio of splashed samples to non-splashed samples was precisely adjusted to 1:3, effectively avoiding the sample imbalance problem and forming a high-quality labeled dataset.

[0042] S4: Train a deep learning model using a labeled dataset. The deep learning model uses ResNet34 as the backbone network and is optimized using a weighted loss function.

[0043] In the specific implementation, the labeled dataset is first scientifically divided into training, validation, and test sets in a 7:2:1 ratio. ResNet34 is selected as the backbone network to construct the deep learning model. This network effectively addresses the depth bottleneck problem of traditional convolutional networks through residual connection mechanisms. In terms of network structure design, the shallow layers (first 10 layers) focus on learning low-level features such as the edges and textures of the taphole; the middle layers (layers 11-20) focus on extracting the flow morphology features of molten iron and slag; and the deep layers (layers 21-34) specifically learn high-level features such as high-temperature radiation and splash trajectory, meeting the need for fine-grained recognition of splash states.

[0044] Then, to address the real-world problem of imbalance between splashed and non-splashed samples, a weighted mean squared error loss function is used instead of the traditional loss function. The mathematical expression for this loss function is:

[0045] in, For the sample size, For real labels, These are the model's predicted values. These are sample weights, and the weight coefficients can be dynamically adjusted according to the sample category. =1 =2, when =0 =1; This weight allocation strategy effectively guides the model to pay more attention to feature learning of the splash state.

[0046] The model training process employed the Adam optimizer with an initial learning rate of 1e-4 and a strict learning rate decay strategy, reducing the learning rate to 0.8 times its original value every 5 training epochs. The batch size was set to 16, and the total number of training epochs was 50. An early stopping strategy was implemented during training, and the performance changes on the validation set were monitored in real time to prevent overfitting. After systematic training, the final model achieved excellent performance metrics on the test set, including a splash state recognition accuracy of no less than 95%, a false negative rate of no more than 3%, and a false positive rate of no more than 2%.

[0047] Finally, the trained model is used as a splash state prediction model.

[0048] S5: Convert the trained deep learning model into ONNX format and use TensorRT for acceleration and optimization. Deploy it to edge computing devices to perform frame-level prediction on real-time video streams and generate frame-level prediction results.

[0049] In a specific implementation, the splash state prediction model is converted to the ONNX open format to ensure model compatibility across different hardware platforms. The ONNX model is deeply optimized using the TensorRT acceleration engine, employing techniques such as layer fusion and kernel tuning to improve model inference speed by 3-5 times, fully meeting the performance requirements of real-time monitoring.

[0050] The optimized model was deployed to edge computing devices at the iron tapping site to build a complete engineering application system. Real-time communication links were established between the edge computing devices and industrial cameras and the central control room host, enabling an end-to-end processing flow of "image acquisition - model inference - result feedback." The deployed model performed frame-by-frame analysis of the 25fps real-time video stream, outputting predicted splash states for each frame, providing a reliable data foundation for subsequent time-series analysis.

[0051] S6: Employ a multi-frame voting strategy to perform temporal smoothing on the frame-level prediction results, and calculate the start and end times and total duration of the splash state based on the smoothing results. In a specific implementation, to address the issue that the inference results of a single frame image may be affected by transient interference, a "multi-frame voting" strategy is adopted for post-processing optimization. The model prediction values ​​of 30 consecutive frames are systematically analyzed. When a prediction value of a frame is ≥0.8, that frame is determined to be a "candidate splashing frame." The cumulative number of "candidate splashing frames" in the 30 frames is counted. When the number reaches or exceeds 15 frames, the time period is determined to be in a "splashing" state; otherwise, it is determined to be in a "non-splashing" state.

[0052] Finally, based on the optimized frame state determination results, the start frame number Fstart and end frame number Fend of each splashing event are accurately recorded, ensuring that Fend ≥ Fstart. Using the camera's fixed frame rate of 25fps, the splashing duration is calculated using the formula T = (Fend - Fstart + 1) / 25, where T represents the splashing duration in seconds. This calculation method ensures that the duration statistics achieve a single-frame interval accuracy, i.e., a time resolution not exceeding 0.04 seconds, providing precise data support for blast furnace process optimization.

[0053] In this embodiment, a visual feature knowledge base integrating expert experience is constructed to guide model training. The sample imbalance problem is solved by combining weighted loss function optimization. TensorRT is used to accelerate real-time frame-level analysis, and a multi-frame voting strategy is used to eliminate instantaneous interference. Ultimately, the system achieves the beneficial effects of fully automating the splash monitoring process, significantly improving the identification accuracy, providing excellent real-time response capability, strong environmental adaptability, and achieving single-frame level accuracy in duration statistics. This completely overcomes the technical defects of traditional manual monitoring, which is highly subjective, and sensor monitoring, which is unreliable.

[0054] like Figure 2 As shown, the following is an embodiment of the blast furnace taphole splash duration determination system provided in this disclosure. This system and the blast furnace taphole splash duration determination method of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the blast furnace taphole splash duration determination system, please refer to the embodiments of the above blast furnace taphole splash duration determination method.

[0055] A system for determining the duration of splashing at the taphole of a blast furnace, comprising: The data acquisition module is used to continuously acquire video data of the iron tapping process using industrial cameras deployed at specific locations at the blast furnace tapping outlet.

[0056] The data preprocessing module is used to perform video frame extraction, effective region cropping, and image enhancement operations on the iron tapping process video data to generate an enhanced image dataset.

[0057] The data annotation module is used to annotate the augmented image dataset based on a pre-built expert knowledge base, forming an annotated dataset.

[0058] The deep learning model training module is used to train a deep learning model using a labeled dataset. The deep learning model uses ResNet34 as the backbone network and is optimized using a weighted loss function.

[0059] The model deployment and inference module is used to convert the trained deep learning model into ONNX format and use TensorRT for acceleration and optimization, deploy it to edge computing devices, perform frame-level prediction on real-time video streams, and generate frame-level prediction results.

[0060] The post-processing and splash duration calculation module is used to perform temporal smoothing on the frame-level prediction results using a multi-frame voting strategy, and calculate the start and end times and total duration of the splash state based on the smoothing results.

[0061] The blast furnace taphole splash duration judgment system provided in this embodiment achieves a significant breakthrough in blast furnace taphole splash monitoring by integrating an expert knowledge base and deep learning technology: It establishes an expert knowledge system based on multi-dimensional visual features to guide the model in accurately learning the essential characteristics of splash; it optimizes the ResNet34 network using a weighted loss function to effectively overcome the sample imbalance problem; and it combines TensorRT to accelerate inference and a multi-frame voting strategy to ensure real-time analysis and anti-interference capabilities. Ultimately, this forms a complete automated solution that significantly improves the accuracy, real-time performance, and duration statistics of splash recognition, fundamentally solving the industry problems of strong subjectivity in traditional manual monitoring and insufficient reliability of sensor monitoring.

[0062] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0063] The method for determining the duration of blast furnace taphole splashing provided in this application embodiment can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0064] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0065] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0066] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0067] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0068] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0069] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0070] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0071] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0072] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0073] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0074] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0075] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0076] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0077] The aforementioned electronic device realizes the beneficial effects of the blast furnace taphole splash duration judgment method of this application by integrating expert knowledge base and deep learning technology, constructing multi-dimensional visual feature mapping relationship, optimizing ResNet34 model with weighted loss function, and combining TensorRT to accelerate inference and multi-frame voting post-processing strategy. It achieves the benefits of realizing automatic splash behavior recognition, significantly improving recognition accuracy, ensuring real-time monitoring capability and realizing accurate duration statistics.

[0078] The storage medium provided in this application stores a program product capable of implementing a method for determining the duration of splashing at the blast furnace taphole.

[0079] Methods for determining the duration of splashing at the blast furnace taphole include: Industrial cameras deployed at specific locations at the blast furnace tapping outlet continuously collect video data of the tapping process; Video frame extraction, effective region cropping, and image enhancement operations are performed on the video data of the iron tapping process to generate an enhanced image dataset. The augmented image dataset is annotated based on a pre-built expert knowledge base to form an annotated dataset; A deep learning model was trained using a labeled dataset. The deep learning model used ResNet34 as the backbone network and was optimized using a weighted loss function. The trained deep learning model is converted into ONNX format and accelerated and optimized using TensorRT. It is then deployed to an edge computing device to perform frame-level prediction on real-time video streams and generate frame-level prediction results. A multi-frame voting strategy is used to perform temporal smoothing on the frame-level prediction results, and the start and end times and total duration of the splash state are calculated based on the smoothing results.

[0080] In some possible implementations, the method for determining the splashing duration at the blast furnace taphole of this disclosure can be implemented as a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to perform the steps described in the "Exemplary Methods" section above, according to various exemplary embodiments of this disclosure.

[0081] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining the duration of splashing at the taphole of a blast furnace, characterized in that, include: Industrial cameras deployed at specific locations at the blast furnace tapping outlet continuously collect video data of the tapping process; Video frame extraction, effective region cropping, and image enhancement operations are performed on the video data of the iron tapping process to generate an enhanced image dataset. The augmented image dataset is annotated based on a pre-built expert knowledge base to form an annotated dataset; A deep learning model was trained using a labeled dataset. The deep learning model used ResNet34 as the backbone network and was optimized using a weighted loss function. The trained deep learning model is converted into ONNX format and accelerated and optimized using TensorRT. It is then deployed to an edge computing device to perform frame-level prediction on real-time video streams and generate frame-level prediction results. A multi-frame voting strategy is used to perform temporal smoothing on the frame-level prediction results, and the start and end times and total duration of the splash state are calculated based on the smoothing results.

2. The method for determining the duration of splashing at the blast furnace taphole according to claim 1, characterized in that, The method of continuously acquiring video data of the iron tapping process through industrial cameras deployed at specific locations at the blast furnace tapping outlet includes: An industrial camera is deployed 8-10 meters directly in front of the blast furnace taphole, and the camera parameters are set to a resolution of 2048×2448 pixels and a frame rate of 25fps. Continuous acquisition of video data of the iron tapping process using industrial cameras; The collected video data of the iron tapping process is transmitted in real time to the central control room via optical fiber, stored in MP4 format with a bit rate of no less than 10Mbps, forming the original video dataset.

3. The method for determining the duration of splashing at the blast furnace taphole according to claim 2, characterized in that, The process of extracting video frames, cropping effective regions, and enhancing images from the iron tapping process video data to generate an enhanced image dataset includes: The original video dataset is split into individual images frame by frame; Each image is cropped by a valid pixel region to generate a cropped image; the valid pixel region is a rectangular region with starting coordinates (224, 624), a width of 1600 pixels, and a height of 1200 pixels; The cropped image is augmented with data operations including at least one of the following: rotation ±5°, translation ±10 pixels, Gaussian blur, color jitter, and random cropping, which increases the size of the original video dataset by 5 times to generate an enhanced image dataset.

4. The method for determining the duration of splashing at the blast furnace taphole according to claim 3, characterized in that, The augmented image dataset is annotated based on a pre-built expert knowledge base to form an annotated dataset, including: Based on the experience of experts in blast furnace ironmaking, a mapping relationship between visual features and splashing states is established, and an expert knowledge base is constructed. The visual features include morphological features, motion features, temperature features, and spatiotemporal features. Morphological features include the height of molten iron jet exceeding the set threshold at the tapping spout, and the slag and molten iron mixing in a splashing distribution. Motion features include abnormal fluctuations in the molten iron stream and the splashed material exhibiting multi-directional splashing trajectories. Temperature features include the core area of ​​the splash appearing as a high-temperature bright white area in thermal imaging. Spatiotemporal features include the spatiotemporal characteristics of the splashing phenomenon exhibiting suddenness and continuity. The enhanced image dataset is independently labeled by at least two data labeling servers based on a constructed expert knowledge base, with splashed image data labeled as 1 and non-splashed image data labeled as 0. By using data augmentation techniques, the ratio of data samples labeled as splashing states to data samples labeled as non-splashing states is adjusted to a set ratio, forming a balanced labeled dataset.

5. The method for determining the duration of splashing at the blast furnace taphole according to claim 4, characterized in that, The process of training a deep learning model using a labeled dataset, wherein the deep learning model employs ResNet34 as the backbone network and is optimized using a weighted loss function, includes: The labeled dataset was divided into training, validation, and test sets in a 7:2:1 ratio. A deep learning model is constructed using ResNet34 as the backbone network. A weighted mean squared error loss function is set for model training, and the weighted mean squared error loss function includes: in, For the sample size, For real labels, These are the model's predicted values. For sample weights, when =1 =2, when =0 =1; Set the training parameters as follows: set the initial learning rate to 1e-4, set the learning rate decay strategy to decay to 0.8 times the original value every 5 training cycles, set the training batch size to 16, and set the training epochs to 50. The Adam optimizer is used to train and optimize the model, generating a splash state prediction model.

6. The method for determining the duration of splashing at the blast furnace taphole according to claim 5, characterized in that, The process of converting the trained deep learning model to ONNX format and accelerating it using TensorRT, then deploying it to an edge computing device to perform frame-level prediction on real-time video streams and generate frame-level prediction results includes: Convert the splash state prediction model to ONNX format; The TensorRT acceleration engine is used to optimize the splash state prediction model in the ONNX format to improve inference speed; Deploy the optimized splash state prediction model to edge computing devices; The deployed splash state prediction model is used to perform frame-level prediction on the real-time video stream, outputting the splash state prediction value for each frame.

7. The method for determining the duration of splashing at the blast furnace taphole according to claim 6, characterized in that, The step of employing a multi-frame voting strategy to perform temporal smoothing on the frame-level prediction results, and calculating the start and end times and total duration of the splash state based on the smoothing results, includes: Obtain the splash state prediction value of N consecutive frames of images. If the splash state prediction value of any frame of image is not lower than 0.8, then the frame of image is determined to be a candidate splash frame. Count the number of candidate splash frames in N frames of images. If the number of candidate splash frames is not less than N / 2, mark the corresponding time period of the image as splash state. Based on the marking results, record the start frame number Fstart and end frame number Fend that are marked as splashing state. Calculate the splashing duration T based on the camera frame rate of 25fps, using the following formula: Where T represents the spray duration in seconds.

8. A system for determining the duration of splashing at the taphole of a blast furnace, characterized in that, The system employs the method for determining the splashing duration at the blast furnace taphole as described in any one of claims 1 to 7; The system includes: The data acquisition module is used to continuously acquire video data of the iron tapping process using industrial cameras deployed at specific locations at the blast furnace tapping spout. The data preprocessing module is used to perform video frame extraction, effective region cropping, and image enhancement operations on the iron tapping process video data to generate an enhanced image dataset. The data annotation module is used to annotate the augmented image dataset based on a pre-built expert knowledge base, forming an annotated dataset; The deep learning model training module is used to train a deep learning model using a labeled dataset. The deep learning model uses ResNet34 as the backbone network and is optimized using a weighted loss function. The model deployment and inference module is used to convert the trained deep learning model into ONNX format and use TensorRT for acceleration and optimization, deploy it to edge computing devices, perform frame-level prediction on real-time video streams, and generate frame-level prediction results. The post-processing and splash duration calculation module is used to perform temporal smoothing on the frame-level prediction results using a multi-frame voting strategy, and calculate the start and end times and total duration of the splash state based on the smoothing results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for determining the splashing duration at the blast furnace taphole as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for determining the splashing duration at the blast furnace taphole as described in any one of claims 1 to 7.

Images