Method and device for identifying high-temperature red ore on belt based on machine vision technology

Abstract

The invention relates to a method and a device for identifying high-temperature red ores on a belt based on a machine vision technology, the machine vision technology is applied, the high-temperature red ores on the running belt are identified in real time, a camera in a belt vestibule is used for collecting images during running of the belt, and the images are extracted and identified through the vision technology. After the red ore is detected, alarm output and image storage are carried out, so that the risk that personnel do not find the red ore can be effectively eliminated, and the dangerous working condition that materials which are not completely cooled enter the belt is prevented through an automatic alarm mechanism. The machine vision technology is applied to achieve real-time safety early warning of the belt, and the defects that a traditional supervision mode, namely real-time checking, monitoring and patrolling of personnel are combined, cameras are installed in a production workshop, operators on duty in a monitoring room monitor operation of the belt for 24 hours, the labor intensity of the operators on duty is large, and the operators on duty are prone to fatigue and leak inspection can be effectively overcome.

Description

Technical Field

[0001] This patent application belongs to the field of safety intelligent early warning technology for sintering in iron and steel metallurgy. It is a method and device for heating powdered materials to their melting point to form blocks and cooling them to ensure temperature monitoring on the conveyor belt and taking measures to prevent overheating. More specifically, it relates to a method and device for identifying high-temperature red ore on a conveyor belt based on machine vision technology. Background Technology

[0002] Enterprise safety is the lifeline of every enterprise and a red line that cannot be crossed. Therefore, safe production is the top priority for every enterprise.

[0003] Machine vision technology is one of the fastest-growing and most cutting-edge technologies in recent years. Currently, video image recognition technology is widely used in transportation (license plate recognition), medical (medical imaging), banking, and factory manufacturing, both domestically and internationally. It has advantages such as enabling non-contact inspection, long-term operation, reducing labor intensity for workers, and high efficiency and reliability.

[0004] With the deepening of digital transformation in factories and workshops, the rapid and continuous operation of safety monitoring is becoming increasingly important. How to achieve rapid detection and post-event investigation of high-temperature red ore during conveyor belt operation has become an urgent need.

[0005] Traditional monitoring methods combine real-time personnel observation, surveillance, and patrols. Cameras are installed in the production workshop, and monitoring personnel in the control room monitor the conveyor belt operation 24 hours a day. However, this method is labor-intensive, prone to fatigue, and prone to missing checks. Applying machine vision technology to achieve conveyor belt safety early warning can effectively overcome these shortcomings.

[0006] The conveyor belt transports the finished ore from the sintering machine. If the ore is not completely cooled, hot red ore may fall onto the conveyor belt. The hot red ore can burn the belt, which can easily cause fires and belt breakage, affecting production. Summary of the Invention

[0007] The technical problem this invention aims to solve is to provide a method and device for identifying high-temperature red ore on a conveyor belt based on machine vision technology. The purpose is to identify high-temperature red ore on a running conveyor belt in real time. A camera in the conveyor belt corridor captures images of the belt during operation. Visual technology is used to extract and identify the ore. Upon detection of red ore, an alarm is output and the image is stored. This eliminates the risk of personnel failing to detect the red ore and prevents the dangerous situation of incompletely cooled material entering the conveyor belt through an automatic alarm mechanism. The invention automatically stores photos and videos of detected red ore and the time of its occurrence, providing support for subsequent analysis of the production process and operating conditions.

[0008] To solve the above problems, the technical solution adopted by the present invention is as follows: A method for identifying high-temperature red ore on a conveyor belt based on machine vision technology includes the following steps: S1. Industrial video stream processing: The video stream signal from the network camera in the sintering machine belt is acquired using the FFmpeg decoding algorithm, and then quickly decoded to generate and save an image that can be processed by OpenCVSharp. S2, Sample Training: Accumulate images of high-temperature red ore in the scene as training samples for the neural network. Through image sample collection and labeling, generate a sample set that the YOLO model can train. S3. Industrial System Integration: Through YOLO model training, an intelligent model capable of recognizing high-temperature red ore is generated. Based on a multi-threaded pipeline architecture, video acquisition programs, image processing programs, intelligent recognition programs, and alarm storage programs are developed to realize intelligent alarm functions.

[0009] Furthermore, in S1, the problem of dust and vibration interference needs to be solved when processing industrial video streams. The specific solutions are as follows: based on FFmpeg, a customized multi-protocol decoding engine is used to integrate hardware acceleration (NVDEC) to achieve real-time decoding of 4 channels of 1080P (latency <50ms). Combined with dark channel defogging, Retinex illumination compensation and IMU image stabilization technology, the problem of dust and vibration interference is solved.

[0010] Furthermore, in S2, the YOLO model integrates visible light and thermal imaging data, designs a dynamic attention mechanism and a scale-adaptive detection head, enhances the high-temperature characteristics of high-temperature red ore and the robustness to human posture, supports incremental learning with small samples, and addresses the potential for new target categories or changes in the distribution of old target categories in high-temperature red ore scenarios (such as changes in the morphology of high-temperature areas). With only a small number of labeled samples (such as 10-30 samples for each new category), the YOLO model can quickly adapt to new tasks without retraining all the data, while maintaining the detection performance of the original targets. in: The core of the dynamic attention mechanism is to enable the YOLO model to automatically focus on the most critical regions and features for identifying "high-temperature red ore" (such as the thermal signal of the high-temperature region and the outline edge of the red ore), while suppressing irrelevant backgrounds (such as belt texture and environmental noise). It is especially suitable for scenarios where the contrast between red ore and background is low and high-temperature features are easily interfered with in belt transportation. The scale-adaptive detection head is designed to adapt to the multi-scale variations of high-temperature red ore. Red ore transported by conveyor belt exhibits significant scale differences: foreground red ore occupies a large proportion of the image (large scale), while background or scattered red ore occupies a small proportion (small scale), and large, medium, and small-scale red ore may coexist within the same frame. The scale-adaptive detection head enables the model to stably detect red ore at different scales, avoiding missed detections of small ore blocks or misjudgments of large ore blocks.

[0011] Furthermore, the most critical regions and features include: thermal signals in high-temperature areas and the outline edges of high-temperature red ore; irrelevant backgrounds include belt texture and environmental noise.

[0012] Furthermore, in S3, video capture programs, image processing programs, intelligent recognition programs, and alarm storage programs are developed using C#.

[0013] Furthermore, in S3, the throughput of the multi-threaded pipeline architecture is >30fps, and the alarm storage program interfaces with the PLC through the OPC UA protocol to realize the intelligent alarm function.

[0014] Furthermore, the alarm storage program employs a multi-level alarm strategy to automatically save the alarm evidence chain. This multi-level alarm strategy includes two aspects: confidence level classification and spatiotemporal false alarm filtering. Confidence level refers to dividing the "high-temperature red ore" target detection by the intelligent model into four levels (0-1) to correspond to different alarm intensities and processing logic, and to adapt to the degree of danger of red ore (such as high temperature value, accumulation amount). These four different levels are high confidence level, medium confidence level, low confidence level, and filtering level. High confidence level, confidence level ≥ 0.85: The intelligent model has a very high degree of confidence in determining the target as "high temperature red ore". It immediately captures the current frame image, stores it to the local hard drive, and pushes alarm information to the monitoring terminal. Medium confidence level, 0.6 ≤ confidence < 0.85: The target features are somewhat ambiguous, but still conform to the core features of high-temperature red ore. Store 3 consecutive frames of images (including the preceding and following frames for context verification), mark them as "medium priority", and prompt "pending confirmation" on the terminal. No manual intervention is required, but alarm logs are recorded. Low confidence level, 0.4 ≤ confidence < 0.6: target features are blurry or there is suspected interference. Store a single frame image and feature data (such as high temperature point coordinates and temperature values) and mark it as "low priority". Filtering level (confidence level < 0.4): Direct filtering, no storage or alarm, only recording "suspected target" in the log; Spatiotemporal false alarm filtering refers to further filtering out noise by verifying the temporal continuity and spatial correlation of the target, including both temporal and spatial dimension filtering.

[0015] Time-dimensional filtering: For the "high-temperature red ore" target detected in a single frame, lightweight target tracking is initiated. If the same type of target is detected in the same area (IOU≥0.5) for 3 consecutive frames (interval≤500ms) and the confidence level is ≥ the corresponding level threshold, the alarm is confirmed to be valid. If it only appears in a single frame, it is judged as transient interference, and the alarm is filtered (only low-priority logs are recorded). If the same fixed area (such as a section of belt joint) frequently triggers low-confidence / medium-confidence alarms (≥20 times) within 10 consecutive minutes, it is judged as a fixed interference source (such as belt joint reflection), and an "outlet" is automatically generated. Subsequent low-confidence alarms in this area are directly filtered (manual confirmation is required for permanent outletting).

[0016] Spatial dimension filtering: Preset "effective detection area" (such as the effective width range of belt conveyor, excluding the background area outside the belt edge). If the detected "high temperature target" is located outside the "effective detection area" (such as the wall or equipment at the edge of the camera), regardless of the confidence level, an alarm will be filtered (marked as "area abnormal" in the log).

[0017] A machine vision-based identification device for high-temperature red ore on a conveyor belt, used to implement the above method, includes a camera installed in the conveyor belt corridor, a gateway connected to the camera, a server connected to the gateway, and a client connected to the server. The camera is used to capture images; The captured images are transmitted to the server via the gateway; The server uses model algorithms to perform calculations and identifications, and finally displays the results on the client side. When the server detects high-temperature red ore, it issues an alarm in the alarm area and stores the image.

[0018] Due to the adoption of the above technical solution, the beneficial effects achieved by this invention are: This invention effectively eliminates / avoids the drawbacks of traditional monitoring methods, which combine real-time personnel viewing, surveillance, and patrols. Traditional methods involve installing cameras in the production workshop and having monitoring personnel in the control room monitor the conveyor belt operation 24 hours a day. These methods suffer from the disadvantages of high labor intensity, fatigue, and the risk of missed checks. This invention utilizes machine vision technology to achieve real-time safety warnings for conveyor belts, offering higher accuracy and wider applicability. Attached Figure Description

[0019] Figure 1 This is a network architecture diagram of the present invention.

[0020] Figure 2 This is a diagram of the client interface of the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the embodiments.

[0022] A method for identifying high-temperature red ore on a conveyor belt based on machine vision technology includes the following steps: S1. Industrial video stream processing: The video stream signal from the network camera in the sintering machine belt is acquired using the FFmpeg decoding algorithm, and then quickly decoded to generate and save an image that can be processed by OpenCVSharp. S2, Sample Training: Accumulate images of red mines in the scene as training samples for the neural network. Through image sample collection and labeling, generate a sample set that the YOLO model can train. S3. Industrial System Integration: Through YOLO model training, an intelligent model capable of recognizing red ore is generated. Based on a multi-threaded pipeline architecture, video acquisition programs, image processing programs, intelligent recognition programs, and alarm storage programs are developed to realize intelligent alarm functions.

[0023] In S1, the processing of industrial video streams requires addressing issues such as dust and vibration interference. Specifically, it involves using a customized multi-protocol decoding engine based on FFmpeg, integrating hardware acceleration (NVDEC) to achieve real-time decoding of 4 channels of 1080P (latency <50ms), and combining dark channel defogging, Retinex illumination compensation, and IMU image stabilization technology to solve the problems of dust and vibration interference.

[0024] In S2, the YOLO model integrates visible light and thermal imaging data, designs a dynamic attention mechanism and a scale-adaptive detection head, enhances the robustness of red mine high-temperature features and personnel posture, supports incremental learning with few samples, and addresses potential new target categories or changes in the distribution of old target categories in red mine scenes (such as changes in the morphology of high-temperature areas) by using only a small number of labeled samples (such as 10-30 samples for each new category). This allows the model to quickly adapt to new tasks without retraining all the data, while maintaining the detection performance of the original targets.

[0025] The core of the dynamic attention mechanism is to enable the model to automatically focus on the most critical areas and features for identifying "high-temperature red ore" (such as the thermal signal of the high-temperature area and the outline edge of the red ore), while suppressing irrelevant backgrounds (such as belt texture and environmental noise). It is especially suitable for scenarios where the contrast between red ore and background is low and high-temperature features are easily disturbed in belt transportation.

[0026] Scale-adaptive detection head: Adapts to multi-scale variations in red ore. Red ore transported by conveyor belt exhibits significant scale differences: foreground red ore occupies a large proportion of the image (large scale), while background or scattered red ore occupies a small proportion (small scale), and large, medium, and small-scale red ore may coexist within the same frame. The function of the scale-adaptive detection head is to enable the model to stably detect red ore at different scales, avoiding missed detections of small ore blocks or misjudgments of large ore blocks.

[0027] In S3, video capture, image processing, intelligent recognition, and alarm storage programs were developed using C#. The multi-threaded pipeline architecture achieves a throughput of >30fps, and the OPC UA protocol interfaces with a PLC to implement intelligent alarm functionality.

[0028] The alarm storage program employs a multi-level alarm strategy (confidence level + spatiotemporal false alarm filtering) to automatically save the alarm evidence chain.

[0029] The model classifies the detection confidence level (0-1) of the "high-temperature red ore" target into four levels. Different levels correspond to different alarm intensities and processing logics, which are adapted to the degree of danger of red ore (such as high temperature value and accumulation amount).

[0030] High confidence level, confidence level ≥ 0.85: The model's determination of the target as "high temperature red ore" is highly reliable. It immediately captures the current frame image, stores it on the local hard drive, and pushes alarm information to the monitoring terminal. Medium confidence level, 0.6 ≤ confidence < 0.85: The target features are somewhat ambiguous, but still conform to the core features of Red Mine. Store 3 consecutive frames of images (including the preceding and following frames for context verification), mark them as "medium priority", and prompt "pending confirmation" on the terminal. No manual intervention is required, but alarm logs are recorded. Low confidence level, 0.4 ≤ confidence < 0.6: target features are blurry or there is suspected interference. Store a single frame image and feature data (such as high temperature point coordinates and temperature values) and mark it as "low priority". Filtering level (confidence level < 0.4): Direct filtering, no storage or alarm, only recording "suspected target" in the log.

[0031] Spatiotemporal false alarm filtering: combining verification of temporal continuity and spatial correlation.

[0032] Relying solely on confidence levels may lead to false alarms due to transient interference (such as flying insects passing by the mirror or sudden changes in lighting). Spatiotemporal filtering further filters out noise by verifying the "temporal stability" and "spatial rationality" of the target.

[0033] Time-dimensional filtering: For the "high-temperature red ore" target detected in a single frame, lightweight target tracking is initiated. If the same type of target is detected in the same area (IOU≥0.5) for 3 consecutive frames (interval ≤500ms) and the confidence level is ≥ the corresponding level threshold, the alarm is confirmed to be valid. If it only appears in a single frame, it is judged as transient interference, and the alarm is filtered (only low-priority logs are recorded). If the same fixed area (such as a section of belt joint) frequently triggers low / medium confidence alarms (≥20 times) within 10 consecutive minutes, it is judged as a fixed interference source (such as belt joint reflection), and an "outlet" is automatically generated. Subsequent low confidence alarms in this area are directly filtered (manual confirmation is required for permanent outletting).

[0034] Spatial dimension filtering: Preset "effective detection area" (such as the effective width range of belt conveyor, excluding the background area outside the belt edge). If the detected "high temperature target" is located outside the area (such as the wall or equipment at the edge of the camera), regardless of the confidence level, an alarm will be filtered (marked as "area abnormal" in the log).

[0035] Meanwhile, the present invention also discloses an identification device for high-temperature red ore on a conveyor belt based on machine vision technology, used to implement the above method, characterized in that: it includes a camera installed in the conveyor belt corridor, a gateway connected to the camera, a server connected to the gateway, and a client connected to the server; The camera is used to capture images; The captured images are transmitted to the server via the gateway; The server uses model algorithms to perform calculations and identifications, and finally displays the results on the client side. When the server detects high-temperature red ore, it issues an alarm in the alarm area and stores the image.

[0036] Figure 1 The images captured by the camera are transmitted to the server through the gateway. The server uses model algorithms to perform calculations and recognition, and then displays and outputs the results to the client. Figure 2 The client outputs and displays the video. After detecting high-temperature red ore, it issues an alarm in the alarm area and stores the image. Figure 2 The left side includes detection classification, model address, input address, inference device (CPU), inference score (0.1), nonmaximum suppression (0.5), number of camera rows (2), number of camera columns (2), and alarm messages at the bottom.

[0037] The inference score (0.1) means: When the YOLO model detects a candidate target (such as high-temperature red ore on a belt) in an image, it outputs a "confidence score", which represents the probability that the YOLO model believes that "a target exists" and "is correctly classified" within the candidate box; 0.1 is a screening threshold, that is: only when the confidence score of a candidate target is ≥0.1 will it be retained for subsequent processing; candidate targets with a confidence score below 0.1 will be directly filtered out and will not participate in the final output; Nonmaximum suppression means that during YOLO model detection, the same target (such as a red mineral) may be captured by multiple candidate boxes at the same time (these boxes are close in position and have a high degree of overlap). NMS determines whether they are the same target by calculating the "Intersection over Union (IOU)" between candidate boxes, retaining the box with the highest confidence and deleting other redundant boxes.

[0038] Function of the parameter: The NMS threshold (0.5) is the standard for judging whether two candidate boxes are the same target. That is, when the IOU of two candidate boxes is greater than or equal to 0.5, they are considered to belong to the same target, and only the one with higher confidence is retained; if IOU is less than 0.5, they are considered to be different targets, and both are retained.

[0039] The video stream signal from the network camera on the sintering belt of the sintering machine is acquired using the FFmpeg decoding algorithm and decoded quickly to generate and save images that can be processed by OpenCVSharp. Images of red ore in the scene are accumulated as training samples for the neural network. Through image sample acquisition and labeling, a sample set that can be trained by the YOLO framework is generated. An intelligent model that can identify red ore is generated through YOLO model training. The video acquisition program, image processing program, intelligent recognition program, and alarm storage program are developed in C# to realize the intelligent alarm function.

[0040] To make the technical problem, technical solution, and beneficial effects of this invention clearer, the following describes a method for detecting high-temperature red ore on a conveyor belt, in conjunction with the accompanying drawings and embodiments. Images captured by a camera are transmitted to a server via a gateway. The server uses a model algorithm for calculation and recognition, and the results are displayed and output through a client. The specific technical solution is as follows: Industrial video stream processing: Based on a customized multi-protocol decoding engine of FFmpeg, it integrates hardware acceleration (NVDEC) to achieve real-time decoding of 4 channels of 1080P (latency <50ms), and combines dark channel defogging, Retinex illumination compensation and IMU image stabilization technology to solve dust and vibration interference.

[0041] Dedicated YOLO model: It integrates visible light and thermal imaging data, designs a dynamic attention mechanism and scale-adaptive detection head, enhances the robustness of red ore high-temperature characteristics and personnel posture, and supports incremental learning with small samples.

[0042] C# industrial system integration: multi-threaded pipeline architecture (throughput > 30fps), OPC UA protocol interface with PLC, multi-level alarm strategy (confidence level + spatiotemporal false alarm filtering), automatic saving of alarm evidence chain.

Claims

1. A method for identifying high-temperature red ore on a conveyor belt based on machine vision technology, characterized in that: Includes the following steps: S1. Industrial video stream processing: The video stream signal from the network camera in the sintering machine belt is acquired using the FFmpeg decoding algorithm, and then quickly decoded to generate and save an image that can be processed by OpenCVSharp. S2, Sample Training: Accumulate images of high-temperature red ore in the scene as training samples for the neural network. Through image sample collection and labeling, generate a sample set that the YOLO model can train. S3. Industrial System Integration: Through YOLO model training, an intelligent model capable of recognizing high-temperature red ore is generated. Based on a multi-threaded pipeline architecture, video acquisition programs, image processing programs, intelligent recognition programs, and alarm storage programs are developed to realize intelligent alarm functions.

2. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 1, characterized in that: In S1, the problem of dust and vibration interference needs to be solved when processing industrial video streams. The specific solution is to use a customized multi-protocol decoding engine based on FFmpeg, integrate hardware acceleration to achieve real-time decoding of 4 channels of 1080P, and combine dark channel defogging, Retinex illumination compensation and IMU image stabilization technology to solve the problem of dust and vibration interference.

3. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 1, characterized in that: In S2, the YOLO model integrates visible light and thermal imaging data, designs a dynamic attention mechanism and a scale-adaptive detection head, enhances the high-temperature characteristics of high-temperature red ore and the robustness to personnel posture, and supports incremental learning with small samples. in: The dynamic attention mechanism allows the YOLO model to automatically focus on the most critical regions and features for identifying "high-temperature red ore" while suppressing irrelevant background. The scale-adaptive detection head is designed to adapt to the multi-scale changes of high-temperature red ore, avoiding missed detection of small ore blocks or misjudgment of large ore blocks.

4. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 3, characterized in that: The most critical areas and features include: thermal signals in high-temperature areas and the outline edges of high-temperature red ore; irrelevant backgrounds include belt texture and environmental noise.

5. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 1, characterized in that: In S3, video capture programs, image processing programs, intelligent recognition programs, and alarm storage programs are developed using C#.

6. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 1, characterized in that: In S3, the throughput of the multi-threaded pipeline architecture is >30fps, and the alarm storage program interfaces with the PLC through the OPC UA protocol to realize the intelligent alarm function.

7. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 1, characterized in that: The alarm storage program employs a multi-level alarm strategy to automatically save the alarm evidence chain. This multi-level alarm strategy includes two aspects: confidence level classification and spatiotemporal false alarm filtering. Confidence level refers to dividing the confidence level of the "high-temperature red ore" target detection by the intelligent model into four levels, which are used to correspond to different alarm intensities and processing logics to adapt to the danger level of red ore. These four different levels are high confidence level, medium confidence level, low confidence level, and filtering level. High confidence level, confidence level ≥ 0.85: The intelligent model has a very high degree of confidence in determining the target as "high temperature red ore". It immediately captures the current frame image, stores it to the local hard drive, and pushes alarm information to the monitoring terminal. Medium confidence level, 0.6 ≤ confidence < 0.85: The target features are somewhat ambiguous, but still conform to the core features of high-temperature red ore. Store 3 consecutive frames of images, mark them as "medium priority", and the terminal prompts "pending confirmation". No manual intervention is forced, but alarm logs are recorded. Low confidence level, 0.4 ≤ confidence < 0.6: target features are blurry or there is suspected interference. Store a single frame image and feature data, and mark it as "low priority". Filtering level, confidence level < 0.4: Direct filtering, no storage or alarm, only recording "suspected target" in the log; Spatiotemporal false alarm filtering refers to further filtering out noise by verifying the temporal continuity and spatial correlation of the target, including both temporal and spatial dimension filtering.

8. The method for identifying high-temperature red ore on a conveyor belt based on machine vision technology according to claim 7, characterized in that: Time-dimensional filtering: For the "high-temperature red ore" target detected in a single frame, lightweight target tracking is initiated. If the same type of target is detected in the same area for 3 consecutive frames and the confidence level is ≥ the corresponding level threshold, the alarm is confirmed to be effective. If it occurs only in a single frame, it is determined to be transient interference and the alarm is filtered out; if low-confidence / medium-confidence alarms are frequently triggered in the same fixed area within 10 consecutive minutes, it is determined to be a fixed interference source, and a "shielded area" is automatically generated. Subsequent low-confidence alarms in this area are directly filtered out. Spatial dimension filtering: A preset "effective detection area" is set. If the detected "high temperature target" is located outside the "effective detection area", an alarm will be triggered regardless of the confidence level.

9. A device for identifying high-temperature red ore on a conveyor belt based on machine vision technology, used to implement the method described in any one of claims 1 to 8, characterized in that: This includes cameras installed in the belt conveyor corridor, gateways connected to the cameras, servers connected to the gateways, and clients connected to the servers; The camera is used to capture images; The captured images are transmitted to the server via the gateway; The server uses model algorithms to perform calculations and identifications, and finally displays the results on the client side. When the server detects high-temperature red ore, it issues an alarm in the alarm area and stores the image.

Images