Belt conveying fire early warning method based on image processing

Abstract

The present application relates to the technical field of image data processing, and more particularly to a belt material conveying fire early warning method based on image processing. First, a reference RGB image of a conveyor belt when normal and not on fire is obtained, and an ROI region is extracted therefrom; then, an actual scene image of the same shooting angle position is obtained, smoke analysis is performed on the ROI region of the actual scene image, smoke confidence of each ROI pixel point in the ROI region is obtained, fire early warning identification is performed on the actual scene image based on the smoke confidence of each ROI pixel point, so that whether the conveyor belt will catch fire is detected. By comparing the fire smoke generated earlier for detection, the fire signs can be found earlier, and measures can be taken earlier to cope with the fire.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to a belt conveying fire early warning method based on image processing. BACKGROUND

[0002] In the mine, coal is usually transported by using a belt conveyor. When the belt conveyor has electrical faults, carrier roller faults, coal spontaneous combustion and the like, the conveyor belt may catch fire. Since the space in the mine is narrow and there are more flammable gases, once the belt catches fire, the coal may be directly ignited, the fire spreads to the nearby equipment and buildings, causing equipment damage, building burning, and even causing the loss of life of the mine workers. In order to deal with the fire, production must be suspended and repair work must be carried out, which will cause the delay of the production plan and directly cause economic losses to the enterprise.

[0003] The existing fire early warning system mainly studies flame recognition, but when the flame appears, the fire has already occurred. Therefore, the fire early warning method of this kind of flame detection cannot achieve early warning and avoid the occurrence of fire. SUMMARY

[0004] The technical problem to be solved by the present application is that the existing fire early warning system cannot achieve early warning and avoid the occurrence of fire. In view of the defects in the prior art, a belt conveying fire early warning method based on image processing is provided.

[0005] In order to solve the above technical problems, the present application provides a belt conveying fire early warning method based on image processing, comprising:

[0006] An image of a normal non-fire conveyor belt is collected to obtain a reference RGB image, and an ROI region is extracted from the reference RGB image;

[0007] An actual scene image of the conveyor belt is collected at the same shooting angle position as the reference RGB image;

[0008] Smoke analysis is performed on the ROI region of the actual scene image to obtain the smoke confidence of each ROI pixel point in the ROI region;

[0009] Based on the smoke confidence of each ROI pixel point, fire early warning identification is performed on the actual scene image.

[0010] Optionally, the ROI region is extracted from the reference RGB image, comprising:

[0011] The reference RGB image is converted into a reference grayscale image;

[0012] Sobel operator edge detection is performed on the reference gray image to obtain an edge profile in the reference gray image;

[0013] Hough line transformation is performed on the edge profile to obtain each straight line segment in the reference gray image and coordinates of each pixel point on the straight line segment;

[0014] The number of pixel points on each straight line segment is counted as the length of the straight line segment, and two straight line segments with the largest length are selected as edge straight lines;

[0015] Edge end points of the two edge straight lines are obtained;

[0016] The edge end points located at the same image boundary are connected and extended to both sides to obtain a boundary line, and an end point perpendicular line passing through the edge end points and perpendicular to the boundary line is drawn;

[0017] The largest region surrounded by the boundary line and the end point perpendicular line is taken as an ROI region.

[0018] Optionally, an actual scene image of the conveyor belt is collected at the same shooting angle position as the reference RGB image, comprising:

[0019] An actual RGB image of the conveyor belt is collected at the same shooting angle position as the RGB image by using a CMOS camera;

[0020] The actual RGB image is preprocessed by using median filtering to obtain an actual scene image;

[0021] The actual scene image is converted into an actual scene gray image.

[0022] Optionally, smoke analysis is performed on the ROI region of the actual scene image to obtain a smoke confidence of each ROI pixel point in the ROI region, comprising:

[0023] The ROI region of the actual scene gray image is extracted;

[0024] A neighborhood sliding window is constructed for the ROI pixel points in the ROI region;

[0025] Smoke features of the ROI region are extracted by using the neighborhood sliding window to obtain a smoke probability factor of the ROI pixel point;

[0026] The clarity of each ROI pixel point is calculated based on the neighborhood sliding window;

[0027] The smoke clarity diffusion degree of each ROI pixel point is calculated based on the clarity of each ROI pixel point;

[0028] calculate a smoke confidence of each of the ROI pixel points based on the smoke probability factor and the clear diffusion degree of smoke.

[0029] Optionally, smoke features of the ROI region are extracted by using the neighborhood sliding window to obtain the smoke probability factor of the ROI pixel points, including:

[0030] The gray level co-occurrence matrix of each of the ROI pixel points in the ROI region is obtained by using the neighborhood sliding window;

[0031] The contrast and entropy of the gray level co-occurrence matrix are calculated as the neighborhood contrast and neighborhood entropy of the ROI pixel points;

[0032] The texture richness of the ROI pixel points is calculated according to the neighborhood contrast and the neighborhood entropy;

[0033] For the ROI pixel points with the texture richness less than or equal to a texture threshold, the neighborhood gray level variance of the neighborhood sliding window corresponding to the ROI pixel points is calculated;

[0034] The smoke probability factor of the ROI pixel points is calculated based on the texture richness and the neighborhood gray level variance.

[0035] Optionally, the texture richness is calculated according to the following formula:

[0036]

[0037] In the formula, is the texture richness; is the neighborhood contrast; is the neighborhood entropy.

[0038] Optionally, the smoke probability factor is calculated according to the following formula:

[0039]

[0040] In the formula, is the smoke probability factor; is the normalized texture richness, is the neighborhood gray level variance; is the texture threshold.

[0041] Optionally, the clear degree of each of the ROI pixel points is calculated based on the neighborhood sliding window, including:

[0042] The gradient value of each of the ROI pixel points is calculated;

[0043] Harris corner detection is performed on the ROI region, and the number of corners in the neighborhood sliding window corresponding to each ROI pixel is counted.

[0044] Calculate the Euclidean distance from each non-center ROI pixel to the center ROI pixel within the neighborhood sliding window corresponding to each ROI pixel;

[0045] The sharpness of the ROI pixel at the center is calculated based on the gradient value of each ROI pixel within the neighborhood sliding window, the number of corner points, and the Euclidean distance from the non-center ROI pixel to the center ROI pixel.

[0046] Alternatively, the sharpness can be calculated according to the following formula:

[0047]

[0048] In the formula, It is the first one within the neighborhood sliding window Gradient values ​​of the non-centered ROI pixels; It is the gradient value of the ROI pixel at the center within the neighborhood sliding window; It is the first one within the neighborhood sliding window The Euclidean distance from each non-central ROI pixel to the central ROI pixel; It is the sum of all Euclidean distances within the neighborhood sliding window; It is the number of corner points within the neighborhood sliding window.

[0049] Optionally, the smoke clarity diffusion degree of each ROI pixel is calculated based on the clarity of each ROI pixel, including:

[0050] The sharpness of each ROI pixel within the neighborhood sliding window is constructed into a sharpness matrix according to the arrangement of the ROI pixels.

[0051] Calculate the mean sharpness of each row in the sharpness matrix and the Euclidean distance between different rows;

[0052] The smoke clarity diffusion of the ROI pixel centered on the row is calculated based on the mean clarity of each row and the Euclidean distance between different rows.

[0053] Optionally, the smoke clarity diffusion can be calculated according to the following formula:

[0054]

[0055] In the formula, It refers to the clarity and diffusion of smoke; and are the first and row of the sharpness matrix, and ; is the Euclidean distance between the first and row of the sharpness matrix.

[0056] Optionally, the smoke confidence of each of the ROI pixel points is calculated based on the smoke probability factor and the smoke clear diffusion degree, comprising:

[0057] The ROI pixel points with the smoke probability factor greater than or equal to a smoke probability threshold value are divided into high-probability smoke points, and the ROI pixel points with the smoke probability factor less than the smoke probability threshold value are divided into non-high-probability smoke points.

[0058] The smoke confidence of each of the high-probability smoke points is calculated based on the smoke probability factor and the smoke clear diffusion degree, and the smoke confidence of each of the non-high-probability smoke points is calculated based on the smoke probability factor.

[0059] Optionally, the smoke confidence is calculated according to the following formula:

[0060]

[0061] In the formula, is the smoke confidence; is the normalized smoke probability factor; is the normalized smoke clear diffusion degree; is the smoke probability threshold value.

[0062] Optionally, fire warning identification is performed on the actual scene image based on the smoke confidence of each of the ROI pixel points, comprising:

[0063] Threshold segmentation is performed on each of the ROI pixel points of the actual scene image according to the smoke confidence, and the ROI pixel points with the smoke confidence greater than or equal to a confidence threshold value are divided into smoke points.

[0064] The number of the smoke points in the actual scene image is counted, and when the number of the smoke points exceeds a smoke threshold value, it is determined that a fire occurs on the conveyor belt.

[0065] Optionally, threshold segmentation is performed according to the following formula:

[0066]

[0067] In the formula, is a coordinate point is a binary value of the corresponding ROI pixel point, and the binary value of 0 indicates that the ROI pixel point is the smoke point, and the binary value of 1 indicates that the ROI pixel point is a non-smoke point; is a coordinate point is a normalized smoke confidence of the corresponding ROI pixel point; is the confidence threshold value.

[0068] The belt material conveying fire early warning method based on image processing has the following beneficial effects:

[0069] Unlike the traditional flame detection fire early warning method, the present application detects the fire smoke generated earlier than the flame during the fire, so that the signs of fire can be found early and measures can be taken early. The present application improves the image threshold segmentation, no longer uses the gray threshold value for image segmentation, but combines the specific scene, analyzes the smoke feature such as texture richness and smoke probability factor classification of the ROI region, and then combines the characteristics that the smoke caused by the conveyor belt on fire will spread, constructs the smoke clear diffusion degree, and thus segments the actual scene image, improves the time and accuracy of fire early warning. In combination with the actual conveyor belt on fire scene, the ROI region is extracted according to the features of the conveyor belt in the actual scene image, and only the ROI region is analyzed, thereby improving the detection and early warning efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0070] Figure 1 is a schematic diagram of the main steps of a belt material conveying fire early warning method based on image processing according to an embodiment of the present application;

[0071] Figure 2 is a schematic diagram of the overall logic of a belt material conveying fire early warning method based on image processing according to an embodiment of the present application;

[0072] Figure 3 is a flowchart of image preprocessing of a belt material conveying fire early warning method based on image processing according to a reference embodiment of the present application;

[0073] Figure 4 is a schematic diagram of the ROI region according to an embodiment of the present application;

[0074] Figure 5 is a flowchart of constructing a smoke clear diffusion degree of a belt material conveying fire early warning method based on image processing according to a reference embodiment of the present application;

[0075] Figure 6 is a flowchart of threshold segmentation of a belt material conveying fire early warning method based on image processing according to a reference embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0077] As shown in Figure 1 The fire warning method based on image processing provided by the embodiment of the present application mainly includes the following steps:

[0078] Step 101: Collecting images of a normal non-ignited conveyor belt to obtain a reference RGB image, and extracting an ROI region from the reference RGB image.

[0079] Different from the traditional flame detection fire warning method, the fire warning method of the embodiment of the present application is a fire smoke detection fire warning method. Smoke is generated earlier than flame during a fire. Detecting fire smoke can discover fire signs early and take measures to cope with the fire earlier. The embodiment of the present application monitors the conveyor belt in real time through machine vision technology and discovers fire signs in time. In order to monitor the conveyor belt, visual data of a normal non-ignited conveyor belt, i.e., a reference RGB image, needs to be obtained in advance. It should be noted that the reference RGB image can be obtained by a CMOS camera.

[0080] The CMOS camera is a camera that converts optical images into digital signals for transmission using a complementary metal oxide semiconductor (CMOS sensor). The RGB image is an image based on a red-green-blue color model in the industry. It is obtained by changing the three color channels of red (R), green (G) and blue (B) and superimposing them on each other to get various colors. RGB represents the three channels of red, green and blue. This color model includes almost all colors that can be perceived by human vision, and is one of the most widely used color systems. In addition, bilateral filtering is a nonlinear filtering method, and its specific implementation process can refer to existing technical solutions or use existing computer programs to implement. The embodiment of the present application will not be described again.

[0081] In the embodiment of the present application, the step of extracting the ROI region from the reference RGB image can be implemented in the following manner: converting the reference RGB image into a reference grayscale image; performing Sobel operator edge detection on the reference grayscale image to obtain edge contours in the reference grayscale image; performing Hough line transformation on the edge contours to obtain each straight line segment in the reference grayscale image and the coordinates of each pixel point on the straight line segment; counting the number of pixel points on each straight line segment as the length of the straight line segment, and selecting two straight line segments with the largest length as edge straight lines; obtaining edge end points of the two edge straight lines; connecting the edge end points located at the same image boundary and extending to both sides to obtain a boundary line; drawing end point perpendicular lines through the edge end points and perpendicular to the boundary line, respectively; and taking the largest region enclosed by the boundary line and the end point perpendicular lines as the ROI region.

[0082] The Sobel operator is an important processing method in the field of computer vision, which is to obtain the weighted difference of the gray values of the upper, lower, left and right four fields of each pixel in the image, so as to detect the edge where the extreme value is reached. The Sobel operator is mainly used to obtain the first-order gradient of a digital image, and the common application and physical meaning is edge detection. The specific implementation process of the Sobel operator can refer to the existing technical solutions or use the existing computer program, and the embodiment of the present application will not be repeated here.

[0083] In the actual scene image of the conveyor belt, the target to be detected is the belt, which is relatively straight in normal circumstances, and the edge straight line is obvious, and the belt edge is greatly different from the surrounding environment. The Sobel operator edge detection is performed on the grayscale image of the reference RGB image to obtain the edge contours in the reference grayscale image, and then the Hough line transformation is used to obtain each straight line segment in the reference grayscale image, and the coordinates of each pixel point on each straight line segment are obtained. The number of pixel points on each straight line segment is counted, that is, the length of the straight line segment. The edge straight lines of the belt are the longest in the reference grayscale image, and the lengths of all straight line segments are arranged in descending order, and the two longest straight line segments are the edge straight lines of the belt. At this time, by comparing the horizontal and vertical coordinates of all points in the edge line segment, the two end points of each edge line segment are found, and a total of four end points are obtained. The edge end points located at the same image boundary are connected and extended to both sides to obtain a boundary line, and end point perpendicular lines are drawn through the four end points and perpendicular to the boundary line, respectively, to obtain four end point perpendicular lines. The region enclosed by the two outermost end point perpendicular lines and the boundary line is the largest ROI (region of interest) region, which is used for subsequent analysis, and the remaining two end point perpendicular lines are contained in the ROI region. The ROI region is the region that needs to be actually analyzed, and the pixel points in the ROI region are called ROI pixel points.

[0084] Step 102, collect the actual scene image of the conveyor belt at the same shooting angle position as the reference RGB image.

[0085] The embodiment of the present application uses fixed camera shooting for image collection, so that the ROI region in each frame image or each image is unchanged, that is, the ROI region of the reference RGB image and the actual scene image is the same, so as to extract the ROI region in each frame image for analysis. It should be noted that the actual scene image can be a single frame image extracted from real-time monitoring image at fixed frequency timing or can be a single frame image extracted from real-time monitoring image at fixed frequency timing.

[0086] In the embodiment of the present application, step 102 can be implemented in the following manner: the actual RGB image of the conveyor belt is collected by a CMOS camera at the same shooting angle position as the RGB image at fixed frequency timing; the actual RGB image is preprocessed by median filtering to obtain the actual scene image; and the actual scene image is converted into an actual scene gray image.

[0087] The actual RGB image can also be collected by a CMOS camera, and the collection frequency can be adjusted as needed, which is not limited by the embodiment of the present application. All actual RGB images are preprocessed to eliminate interference caused by the device itself and the external environment, and to improve the accuracy of subsequent analysis and operation. As a preferred embodiment, the actual RGB image can be preprocessed by median filtering, and all actual RGB images can be converted into gray images. In addition, other denoising methods can also be used, and the method is not limited to median filtering.

[0088] Step 103, smoke analysis is performed on the ROI region of the actual scene image to obtain the smoke confidence of each ROI pixel point in the ROI region.

[0089] Since the conveyor belt is a chemical material, which is an organic mixture, combustion will produce black smoke, so the smoke appearing in the early stage of the belt fire is a mixture composed of gas, solid or particles produced by belt combustion. At the same time, a large amount of heat and smoke is generated in the early stage of the fire, resulting in a large thermal expansion and contraction of the air, and a large amount of hot gas rising from the fire source. These hot gases gather together to form a relatively stable air column, and the surrounding air is also drawn to flow into the air column, thereby forming an upward columnar smoke in the early stage of the fire. Based on this, the smoke confidence of each ROI pixel point is obtained by smoke analysis of the ROI region in this step.

[0090] In the embodiment of the present application, step 103 can be implemented in the following way: a neighborhood sliding window is constructed for the ROI pixel points in the ROI region; the smoke features of the ROI region are extracted by using the neighborhood sliding window to obtain the smoke probability factor of the ROI pixel points; the definition of each ROI pixel point is calculated based on the neighborhood sliding window; the smoke clear diffusion degree of each ROI pixel point is calculated based on the definition of each ROI pixel point; and the smoke confidence of each ROI pixel point is calculated based on the smoke probability factor and the smoke clear diffusion degree.

[0091] In the actual scene gray image, the pixel points in the ROI region (i.e. ROI pixel points) are the objects of analysis, and the neighborhood sliding window is used to obtain the smoke features of the ROI pixel points in the embodiment of the present application. As a preferred implementation, the size of the neighborhood sliding window is a 5*5 square, and the step is 1, i.e. each ROI pixel point corresponds to a 5*5 neighborhood sliding window with the ROI pixel point as the center.

[0092] In the embodiment of the present application, the smoke features of the ROI region are extracted by using the neighborhood sliding window to obtain the smoke probability factor of the ROI pixel points, which can be implemented in the following way: the gray level co-occurrence matrix of each ROI pixel point in the ROI region is obtained by using the neighborhood sliding window; the contrast and entropy of the gray level co-occurrence matrix are calculated as the neighborhood contrast and neighborhood entropy of the ROI pixel point; the texture richness of the ROI pixel point is calculated according to the neighborhood contrast and neighborhood entropy; for the ROI pixel point with the texture richness less than or equal to the texture threshold, the neighborhood gray level variance of the neighborhood sliding window corresponding to the ROI pixel point is calculated; and the smoke probability factor of the ROI pixel point is calculated based on the texture richness and the neighborhood gray level variance.

[0093] For the ROI region of the actual scene gray image in the mine environment, there are four kinds of substances, the first kind is the black coal transported by the belt, the sizes of these coals are different after mining and breaking, and the textures are rich; the second kind is the belt body, the surface of the belt is extremely smooth and almost has no texture; the third kind is the black smoke generated after the belt is on fire, the belt on fire will generate a large number of particles, so that the smoke contains some particulate matters and has less texture, but far less than the texture richness of the coal; and the fourth kind is other substances of the belt at the conveyer, including the roller and the railing part of the conveyer, and this part is in disorder and has rich texture. According to the richness of the textures of the substances, the four kinds of substances can be divided into two categories, the first category is the coal, the roller and the railing part with high texture richness, and the second category is the belt and the smoke with low texture richness.

[0094] As a preferred implementation, the texture richness of the ROI pixel point is constructed by using the neighborhood contrast and the neighborhood entropy corresponding to the ROI pixel point, and the texture richness can be calculated according to the following formula:

[0095]

[0096] wherein, is the texture richness, is the neighborhood contrast, is the neighborhood entropy. Wherein, the neighborhood contrast is used to measure the contrast or difference degree of the gray value in the neighborhood sliding window, which reflects the depth of the image texture, the shallower the texture groove, the greater the contrast, indicating that the texture is more rich. The neighborhood entropy reflects the uniformity and randomness of the gray value distribution of the neighborhood pixel points, the greater the difference between the gray values of each ROI pixel point, the more chaotic and random the distribution, the greater, indicating that the texture is more rich. It should be noted that the texture richness calculated is normalized, with a value range of [0, 1], denoted as . Using the normalized texture richness , the coal, roller or railing part with high texture richness and the belt or smoke with low texture richness can be distinguished. The first type is the coal, roller or railing part with high texture richness, which satisfies , while the second type of material, the belt or smoke part with texture deficiency, satisfies . is the texture threshold, which can be determined according to the needs, and is preferably 0.3.

[0097] However, the texture richness of the belt and the smoke is relatively low, and if only the texture threshold is used, they cannot be distinguished, which will cause them to be mixed together. Through specific scene research and investigation, it is found that the surface of the normal unignited belt is smooth, the gray value transformation between each ROI pixel point in the neighborhood sliding window is uniform, and the difference between the gray values of each ROI pixel point is not large. When the belt is on fire, it will release various gases, particles and combustion products, including fine soot particles. The soot particles are not uniformly distributed in the smoke, the greater the concentration of a certain area, the darker the area, and the different concentrations of the area will cause the gray value of the smoke to change greatly. The gray value of each ROI pixel point in the smoke changes more significantly than the belt, so according to the uniformity of the gray value change of the belt and the smoke, the neighborhood gray variance in each neighborhood sliding window is calculated and denoted as . The neighborhood gray variance is used to distinguish the belt and the black smoke. When the neighborhood gray variance is small, it means that the gray value transformation in the neighborhood sliding window is uniform, which may be the belt, and when the neighborhood gray variance is large, it means that the gray value transformation in the neighborhood sliding window is chaotic, which may be a smoke point.

[0098] The normalized texture richness It can distinguish between coal, idler rollers, or railing sections with high texture richness and belts or smoke with low texture richness, and then use neighborhood grayscale variance. It can distinguish between belts and smoke. As a preferred implementation, texture richness (TR) and neighborhood grayscale variance are used. The smoke probability factor can be constructed using the following formula:

[0099]

[0100] In the formula, It is the smoke probability factor. It is the normalized texture richness. It is the neighborhood gray-level variance; It is the texture threshold. Where, when When the texture richness is high, it indicates that the texture of the ROI pixel is relatively rich. The higher the texture richness, the higher the smoke probability factor of the ROI pixel. The smaller. When At that time, the TR of the ROI pixels was relatively small. Compared to the belt, the neighborhood gray-level variance of the smoke was larger. The neighborhood gray-level variance within the sliding window of the ROI pixel was... The larger the value, the greater the smoke probability factor for that ROI pixel. The larger it is. It should be noted that the same applies to the smoke probability factor. Normalization is performed, with a range of [0,1], denoted as .

[0101] In this embodiment of the invention, the sharpness of each ROI pixel is calculated based on a neighborhood sliding window, which can be achieved as follows: calculate the gradient value of each ROI pixel; perform Harris corner detection on the ROI region and count the number of corners in the neighborhood sliding window corresponding to each ROI pixel; calculate the Euclidean distance from each non-center ROI pixel to the center ROI pixel in the neighborhood sliding window corresponding to each ROI pixel; and calculate the sharpness of the center ROI pixel based on the gradient value, the number of corners, and the Euclidean distance from the non-center ROI pixel to the center ROI pixel in the neighborhood sliding window.

[0102] When a belt catches fire, it releases various gases, particles, and combustion products, including fine carbon soot particles that produce black smoke. In the early stages of a fire, these particles rise and diffuse. Below the smoke, the particles are more numerous and concentrated, resulting in richer smoke texture and more corner points, making the smoke clearer in the image. However, above the smoke, the particles are fewer and more dispersed, suspended in the air and interacting with light, scattering and absorbing it. This causes light scattering within the smoke, resulting in a blurrier image with less smoke texture and fewer corner points. Therefore, the sharpness of ROI pixels is constructed and calculated based on a neighborhood sliding window.

[0103] In sharp images, distinct object boundaries and textures cause drastic changes in pixel values, resulting in large gradient values. In contrast, in blurry, smoky images, due to the blurring of boundaries and textures, pixel value changes are relatively gradual, leading to smaller gradient values. Therefore, the gradient value represents the rate of change of pixel grayscale values ​​in an image. By calculating the gradient value at each point in the image, the degree of change of each pixel can be obtained, thus reflecting the image's sharpness. Larger gradient values ​​indicate sharper edges and textures, resulting in a relatively sharp image; while smaller gradient values ​​indicate blurred edges and textures, resulting in a relatively blurry image. Based on this, the gradient values ​​of all pixels within the Region of Interest (ROI) are calculated, and Harris corner detection is performed simultaneously. The core of Harris corner detection is to move a local window across the image and determine whether there are significant changes in grayscale values. If the grayscale values ​​within the window (on the gradient map) all show significant changes, then a corner exists in the area where the window is located. The specific implementation process of Harris corner detection can refer to existing technical solutions or utilize existing computer programs; this embodiment will not elaborate further.

[0104] As a preferred implementation method, the sharpness can be calculated according to the following formula:

[0105]

[0106] In the formula, It's about clarity; It is the first in the neighborhood sliding window Gradient values ​​of non-centered ROI pixels; It is the gradient value of the ROI pixel at the center of the neighborhood sliding window; It is the first in the neighborhood sliding window The Euclidean distance from a non-centered ROI pixel to a centered ROI pixel; It is the sum of all Euclidean distances within the neighborhood sliding window; This represents the number of corner points within the neighborhood sliding window. A smaller value indicates the Euclidean distance between a non-centered ROI pixel and a centered ROI pixel. The smaller the value, the greater the influence of the non-center ROI pixel on the center ROI pixel. A larger value indicates a gradient value for a non-centered ROI pixel. The larger the difference between the pixel value and the central ROI pixel value, the more significant the gradient change within that neighborhood sliding window, and the sharper the central ROI pixel value. The higher, This refers to the number of corner points within the neighborhood sliding window. A larger number of corner points indicates a more dramatic and obvious change in grayscale values ​​within the neighborhood sliding window, affecting the sharpness of the central ROI pixel. The higher.

[0107] The final result is the sharpness of all ROI pixels within the ROI region of the grayscale image of the actual scene. The target being detected is the smoke generated by the burning belt. Smoke has a unique characteristic of spreading upwards, and the smoke below has richer texture and higher sharpness compared to the smoke above. The accuracy will be relatively high. For any ROI pixel within the ROI region, there is a sharpness of each ROI pixel within the neighborhood sliding window. Based on this, the smoke sharpness diffusion of the ROI pixel is further calculated.

[0108] In this embodiment of the invention, the smoke clarity diffusion degree of each ROI pixel is calculated based on the clarity of each ROI pixel, which can be achieved as follows: the clarity of each ROI pixel within the neighborhood sliding window is constructed into a clarity matrix according to the arrangement of ROI pixels; the mean clarity value of each row in the clarity matrix and the Euclidean distance between different rows are calculated; the smoke clarity diffusion degree of the ROI pixel as the center is calculated based on the mean clarity value of each row and the Euclidean distance between different rows.

[0109] Sharpness can be constructed into a sharpness matrix by arranging the pixels of the ROI. Clarity of different lines There are differences between them, and the greater the distance, the greater the difference. (Clarity matrix) It can be represented as:

[0110]

[0111] Calculate the average sharpness of each line and record it as: , , , , The mean sharpness of different rows is compared pairwise to ensure that no two rows are compared repeatedly. There are 10 possible pairwise combinations. The distance between different rows is denoted as . The clear diffusion of smoke is calculated using the following formula:

[0112]

[0113] In the formula, It refers to the clarity and diffusion of smoke. and These are the first and second resolutions of the resolution matrix. and Mean line sharpness (where, and ); For the first part of the resolution matrix and The Euclidean distance between rows. The larger the value, the greater the difference in the mean sharpness between different lines, the more intense the smoke diffusion, and the greater the smoke sharpness diffusion (b). This is a distance-related factor. The greater the distance between rows, the more pronounced the change in image sharpness and the more intense the smoke diffusion, resulting in a larger smoke sharpness diffusion degree, b. It's important to note that the smoke sharpness diffusion degree, b, is also normalized, with a value range of [0,1], denoted as B.

[0114] In this embodiment of the invention, the smoke confidence level of each ROI pixel is calculated based on the smoke probability factor and the smoke clarity diffusion level, which can be achieved as follows: ROI pixels with a smoke probability factor greater than or equal to the smoke probability threshold are classified as high-probability smoke points; ROI pixels with a smoke probability factor less than the smoke probability threshold are classified as non-high-probability smoke points; the smoke confidence level of each high-probability smoke point is calculated based on the smoke probability factor and the smoke clarity diffusion level, and the smoke confidence level of each non-high-probability smoke point is calculated based on the smoke probability factor.

[0115] Because the clear diffusion of smoke is a unique characteristic of smoke, which other substances do not possess, for high-probability smoke points ( In a real-world grayscale image, whether a high-probability smoke point is caused by a burning belt depends on the smoke probability factor SP and the smoke clarity diffusion B of the corresponding ROI pixel; for non-high-probability smoke points ( In real-world grayscale images, a non-high-probability smoke point is more likely to be a non-smoke point. Whether this non-high-probability smoke point is smoke from a burning belt depends solely on the smoke probability factor SP of the corresponding ROI pixel. The smoke probability threshold... The value of can be determined as needed, and is preferably 0.8.

[0116] As a preferred embodiment, the smoke confidence level can be expressed as:

[0117]

[0118] In the formula, is the smoke confidence; is the normalized smoke probability factor; is the normalized smoke clearness diffusivity. Wherein, for the ROI pixel point , whether it belongs to the smoke point only depends on the smoke probability factor SP of the ROI pixel point, the greater the smoke probability factor, the greater the smoke confidence of the ROI pixel point; for the ROI pixel point , whether it belongs to the smoke point not only depends on the smoke probability factor SP of the ROI pixel point, but also depends on the unique feature of the smoke, the greater the smoke clearness diffusivity B of the ROI pixel point, the greater the smoke confidence of the ROI pixel point. It should be noted that the smoke confidence is also normalized, and the value range is [0, 1], which is recorded as , is the coordinate corresponding to the ROI pixel point.

[0119] It should be noted that the texture richness, the smoke probability factor, the smoke clearness diffusivity and the smoke confidence calculated by the embodiment of the present application all need to be normalized before being used for subsequent calculation, and the specific implementation process can refer to the existing technical solution or be realized by using the existing computer program, and the embodiment of the present application will not be repeated here.

[0120] Step 104, fire warning identification is performed on the actual scene image based on the smoke confidence of each ROI pixel point.

[0121] Through the smoke confidence of each ROI pixel point, whether the ROI region of the entire actual scene image appears fire signs can be analyzed, so as to realize the fire warning identification of the conveyor belt, so as to give a warning before the fire occurs.

[0122] In the embodiment of the present application, step 104 can be realized in the following way: according to the smoke confidence, threshold segmentation is performed on each ROI pixel point of the actual scene image, and the ROI pixel point with the smoke confidence greater than or equal to the confidence threshold is divided into a smoke point; the number of smoke points in the actual scene image is counted, and when the number of smoke points exceeds the smoke threshold, it is determined that the conveyor belt has a fire.

[0123] This step employs threshold segmentation. The smoke confidence score of all ROI pixels in the actual scene image is used as input. The smoke confidence score of each ROI pixel is binarized, outputting a binarized image. Segmentation is achieved by comparing the score with a confidence threshold during binarization. When the smoke confidence score is greater than or equal to the threshold, the binarized value of the ROI pixel is 0; when the smoke confidence score is less than the threshold, the binarized value is 1. In this binarized image, points with a value of 0 represent smoke points, and points with a value of 1 represent non-smoke points. The confidence threshold value can be determined as needed, but is preferably 0.9.

[0124] When the number of zero points (i.e., smoke points) in the binarized image Exceeding the smoke threshold 4 o'clock ( This indicates that the number of ROI pixels belonging to smoke points in the actual scene image exceeds the smoke threshold. At this point, the conveyor belt shows signs of fire and there is a high probability of a fire, requiring fire warning. This can be achieved by issuing a fire warning signal through an intelligent system or triggering alarm devices to remind staff to take appropriate action. (Smoke threshold) The value of 4 can be determined as needed, but 16 is preferred.

[0125] As a preferred implementation, threshold segmentation can be performed according to the following formula:

[0126]

[0127] In the formula, Coordinates The corresponding binarized value of the ROI pixel: a binarized value of 0 indicates that the ROI pixel is a smoke point, and a binarized value of 1 indicates that the ROI pixel is not a smoke point; Coordinates The normalized smoke confidence level of the corresponding ROI pixel; It is the confidence threshold.

[0128] The belt material conveying fire early warning method based on image processing of the embodiment of the present application can monitor the conveyor belt in real time and find fire signs in time through machine vision technology, so as to give early warning and minimize potential risks and hazards. Moreover, the machine vision system can continuously monitor the fire signs of the conveyor belt during transportation, without the need for continuous manual observation, thereby improving work efficiency and reducing the burden on workers. Through machine vision technology, the fire signs monitored by the system can be displayed in the form of images or videos, so that the operator can intuitively understand the fire situation and make decisions and take measures quickly. Finally, the machine vision system can record and store the fire monitoring data for subsequent analysis and evaluation, and these data can be used for investigation and analysis of fire incidents to improve safety precautions.

[0129] As shown in Figure 2 The overall logic of the belt material conveying fire early warning method based on image processing of the embodiment of the present application includes the following three steps:

[0130] I. Obtain the conveyor belt image through the CMOS camera and pre-process the image.

[0131] First, collect the image of the normal actual scene without fire to obtain the reference RGB image I1. Then continue to collect the image of the actual scene at the same shooting position to obtain the actual RGB images I2, I3, …, pre-process all the actual RGB images to eliminate the interference caused by the device itself and the external environment, and improve the accuracy of subsequent analysis and operation. The embodiment of the present application uses median filtering to pre-process the actual RGB image to obtain the actual scene image, and finally converts all the actual scene images into actual scene gray images.

[0132] II. According to the characteristics of the conveyor belt, extract the ROI region from the reference RGB image, and analyze the smoke and non-smoke substances in the actual scene gray image to construct the smoke clear diffusion degree.

[0133] Sobel edge detection is performed on the grayscale image of the reference RGB image I1 to obtain the edge contours in the reference grayscale image. Then, Hough line transform is used to obtain the line segments in the reference grayscale image, and the coordinates of each pixel on each line segment are obtained. The number of pixels on each line segment is counted, which is the length of the line segment. The edge line of the belt is the longest in the reference grayscale image. The lengths of all line segments are sorted in descending order and denoted as a1, a2, ..., an. The two longest line segments in the image (a1 and a2) are the two edge lines of the belt. At this time, by comparing the x and y coordinates of all points within the edge line segments, the coordinates of each endpoint of a1 and a2 are found and denoted as a1(x1,y1), a1(x2,y2), a2(x3,y3), and a2(x4,y4), respectively. Combining the image boundary and the four endpoints, the Region of Interest (ROI) can be obtained.

[0134] For ROI pixels in a grayscale image of a real-world scene, a 5x5 neighborhood sliding window with a step size of 1 is constructed. The gray-level co-occurrence matrix (GLCM) corresponding to each ROI pixel is obtained. The contrast and entropy of the GLCM are calculated as the neighborhood contrast (con) and entropy (ent) of the ROI pixel. The con and ent of this point are used to construct the texture richness (tr). The texture richness is normalized to a value range of [0,1], denoted as TR. For ROI pixels with a texture richness less than or equal to a texture threshold, the gray-level variance of each neighborhood is calculated and denoted as... Utilizing texture richness (TR) and neighborhood grayscale variance Construct the smoke probability factor sp. Normalize the smoke probability factor sp, with a value range of [0,1], and denote it as SP.

[0135] At this point, all ROI pixels in the actual scene grayscale image have a smoke probability factor SP. Based on the magnitude of the SP value, all ROI pixels can be divided into two categories: the first category is high-probability smoke pixels. The second category is non-high probability smoke points ( ), ).

[0136] Calculate the gradient values ​​of all ROI pixels within the ROI region, and perform Harris corner detection simultaneously. Within a 5x5 neighborhood sliding window, the gradient value of the ROI pixel centered at the center is... , No. The gradient value of each non-centered ROI pixel is , No. The Euclidean distance from each non-centered ROI pixel to the central ROI pixel is: The sum of all Euclidean distances is The number of corner points in the neighborhood sliding window is recorded as r. Thus, based on the gradient value of each ROI pixel point in the neighborhood sliding window, the number of corner points, and the Euclidean distance from the non-central ROI pixel point to the central ROI pixel point, the definition of the clarity of the central ROI pixel point is calculated .

[0137] The clarity of all ROI pixel points in the ROI region is finally obtained. For any 5*5 neighborhood sliding window in the ROI region, the clarity of each ROI pixel point is different between different rows, and the farther the distance, the greater the difference. The average clarity of each row is calculated and recorded as , , , , The average clarity of the neighborhood is compared in pairs, and there are 10 cases. The distance between different rows is recorded as . Based on the average clarity of each row and the Euclidean distance between different rows, the smoke clarity diffusion degree b of the central ROI pixel point is calculated. The smoke clarity diffusion degree b is normalized, and the value range is [0, 1], recorded as B.

[0138] For the ROI pixel point , the ROI pixel point belongs to a non-high-probability smoke point. Whether the ROI pixel point is smoke depends only on the smoke probability factor SP of the ROI pixel point. The greater the smoke probability factor, the greater the smoke confidence s of the ROI pixel point. For the ROI pixel point , the ROI pixel point belongs to a high-probability smoke point. Whether the ROI pixel point is smoke depends not only on the smoke probability factor SP of the ROI pixel point, but also on the unique feature of smoke, i.e., the smoke clarity diffusion degree B. The greater the smoke clarity diffusion degree B of the ROI pixel point, the greater the smoke confidence s of the ROI pixel point. Based on this, the smoke confidence s of each ROI pixel point is calculated. The smoke confidence s is normalized, and the value range is [0, 1], recorded as .

[0139] III. According to the smoke confidence of each ROI pixel point in the actual scene gray image, the actual scene gray image is threshold segmented to detect whether the conveyor belt has a fire.

[0140] Image threshold segmentation is adopted. The smoke confidence of all ROI pixel points in the actual scene image is input . The smoke confidence of all ROI pixel points in the actual scene image is input ​The binarization processing is performed, and a binarization image is output. When the smoke confidence is greater than or equal to a confidence threshold, the binarization value of the ROI pixel point is 0, that is, the ROI pixel point is a smoke point; when the smoke confidence is less than the confidence threshold, the binarization value of the ROI pixel point is 1, that is, the ROI pixel point is a non-smoke point.

[0141] When the number of 0-value points in the binarization image exceeds a smoke threshold of 4, , the conveyor belt has shown signs of fire, and a fire warning signal can be issued by an intelligent system or an alarm device is triggered to remind the staff to take corresponding measures.

[0142] As shown in the application embodiment of the belt conveying fire warning method based on image processing, the image preprocessing can refer to the following flow: Figure 3

[0143] 301. Image acquisition is performed on a normal non-ignited conveyor belt to obtain a reference RGB image, and the reference RGB image is converted into a reference gray image;

[0144] 302. Sobel operator edge detection is performed on the reference gray image to obtain the edge profile in the reference gray image;

[0145] 303. Hough line transformation is performed on the edge profile to obtain each straight line segment in the reference gray image and the coordinates of each pixel point on the straight line segment;

[0146] 304. The number of pixel points on each straight line segment is counted as the length of the straight line segment, and the two longest straight line segments are selected as the edge straight lines;

[0147] 305. The edge endpoints of the two edge straight lines are obtained;

[0148] 306. The edge endpoints located at the same image boundary are connected and extended to both sides to obtain a boundary line;

[0149] 307. Endpoint perpendicular lines passing through the edge endpoints and perpendicular to the boundary line are drawn respectively;

[0150] 308. The largest region surrounded by the boundary line and the endpoint perpendicular line is taken as the ROI region;

[0151] 309. The actual RGB image of the conveyor belt is collected by a CMOS camera at the same shooting angle position as the RGB image;

[0152] 310. The actual RGB image is preprocessed by median filtering to obtain an actual scene image.

[0153] As​​Figure 4 As shown, after the gray image of the reference RGB image is processed by Sobel operator edge detection, Hough line transformation and the like according to step 101, two edge straight lines of the belt (i.e., the belt edge segment a1 and the belt edge segment a2) are obtained, and then by comparing the horizontal and vertical coordinates of all points in the edge straight lines, the end point coordinates of the belt edge segment a1 and the belt edge segment a2 are found, respectively recorded as a1(x1, y1), a1(x2, y2), a2(x3, y3), and a2(x4, y4), the edge end points located at the same image boundary (i.e., a1(x1, y1) and a2(x3, y3), a1(x2, y2) and a2(x4, y4)) are connected and then extended to both sides to obtain the boundary line (the horizontal dashed line in the figure and the image boundary), and then the end point perpendiculars passing through the four end points and perpendicular to the boundary line are drawn, respectively, to obtain a total of four end point perpendiculars, and the two outermost end point perpendiculars (the vertical dashed line in the figure) have the largest area surrounded by the boundary line, which can be used as the ROI region.

[0154] As shown in the application embodiment of the belt material conveying fire early warning method based on image processing, the following process can be referred to when calculating the smoke confidence: Figure 5

[0155] 501, extract the ROI region of the actual scene gray image;

[0156] 502, construct a 5*5 neighborhood sliding window for the ROI pixel points in the ROI region;

[0157] 503, obtain the gray level co-occurrence matrix of each ROI pixel point by using the neighborhood sliding window;

[0158] 504, calculate the texture richness of the ROI pixel points:

[0159] The texture richness is calculated according to the following formula ;

[0160] In the formula, is the neighborhood contrast, which is obtained by calculating the contrast of the gray level co-occurrence matrix; is the neighborhood entropy, which is obtained by calculating the entropy of the gray level co-occurrence matrix. The texture richness is normalized, and the value range is [0, 1], which is recorded as .

[0161] For the ROI pixel points with a texture richness greater than 0.3, step 505 is executed; for the ROI pixel points with a texture richness less than or equal to 0.3, step 506 is executed;

[0162] 505, calculate the smoke probability factor of the ROI pixel point according to the texture richness;

[0163] ​506. Calculate the neighborhood grayscale variance of the neighborhood sliding window corresponding to the ROI pixel, and calculate the smoke probability factor of the ROI pixel based on the texture richness and the neighborhood grayscale variance.

[0164] 507. Calculate the sharpness of each ROI pixel:

[0165] Calculate sharpness using the following formula. ;

[0166] In the formula, It is the first in the neighborhood sliding window Gradient values ​​of non-centered ROI pixels It is the gradient value of the ROI pixel at the center of the neighborhood sliding window. It is the first in the neighborhood sliding window The Euclidean distance from a non-centered ROI pixel to a centered ROI pixel It is the sum of all Euclidean distances within the neighborhood sliding window. It represents the number of corner points within the neighborhood sliding window.

[0167] 508. Construct a sharpness matrix by arranging the sharpness of each ROI pixel within the neighborhood sliding window according to the ROI pixel layout:

[0168]

[0169] Calculate the smoke sharpness diffusion of the ROI pixel centered on the neighborhood sliding window. ;

[0170] In the formula, It refers to the clarity and diffusion of smoke. and These are the first and second resolutions of the resolution matrix. and Mean line sharpness ( and ), For the first part of the resolution matrix and The Euclidean distance between rows. The smoke clarity diffusion b is normalized to a value in the range [0,1], denoted as B.

[0171] 509. ROI pixels with a smoke probability factor greater than or equal to 0.8 are classified as high-probability smoke points;

[0172] 510. Calculate the smoke confidence level for each high-probability smoke point based on the smoke probability factor and smoke clarity diffusion.

[0173] 511. ROI pixels with a smoke probability factor less than 0.8 are classified as non-high probability smoke points;

[0174] 512、calculate the smoke confidence of each non-high-probability smoke point based on the smoke probability factor.

[0175] In the embodiment of the present application, the texture threshold is set to 0.3, and the smoke probability threshold is set to 0.8.

[0176] As shown in the application of the belt conveying fire early warning method based on image processing in the embodiment of the present application, the following process can be referred to: Figure 6

[0177] 601、perform threshold segmentation on each ROI pixel point of the actual scene image;

[0178] divide the ROI pixel point with the smoke confidence less than 0.9 into a non-smoke point;

[0179] divide the ROI pixel point with the smoke confidence greater than or equal to 0.9 into a smoke point;

[0180] 602、count the number of smoke points in the actual scene image;

[0181] if the number of smoke points is greater than or equal to 16, determine that the conveyor belt is on fire.

[0182] if the number of smoke points is less than 16, determine that the conveyor belt is normal.

[0183] In the embodiment of the present application, the confidence threshold is set to 0.9, and the smoke threshold is set to 16.

[0184] In summary, the belt conveying fire early warning method based on image processing in the embodiment of the present application is different from the traditional flame detection fire early warning method. The present application detects the fire smoke generated earlier than the flame during the fire, so that the fire signs can be found early and measures can be taken early. The present application improves the image threshold segmentation, no longer uses the gray threshold for image segmentation, but analyzes the smoke features such as texture richness and smoke probability factor classification of the ROI region in combination with the specific scene, and then constructs the smoke clear diffusion degree in combination with the characteristics that the smoke caused by the conveyor belt on fire will spread, so as to segment the actual scene image, improve the time and accuracy of fire early warning. In combination with the actual conveyor belt on fire scene, the ROI region is extracted according to the features of the conveyor belt in the actual scene image, and only the ROI region is analyzed, so that the detection and early warning efficiency is improved.

[0185] ​It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A fire early warning method for belt conveyors based on image processing, characterized in that, include: Image acquisition was performed on a normal, non-flammable transport aircraft belt to obtain a reference RGB image, and the ROI region was extracted from the reference RGB image; The actual scene image of the transport conveyor belt was acquired at the same shooting angle and position as the reference RGB image; Smoke analysis is performed on the ROI region of a real-world scene image to obtain the smoke confidence level of each ROI pixel within the ROI region; Fire early warning recognition is performed on real-world scene images based on the smoke confidence scores of each ROI pixel. Smoke analysis is performed on the Region of Interest (ROI) of a real-world scene image to obtain the smoke confidence score for each ROI pixel, including: Extract the Region of Interest (ROI) from a grayscale image of a real-world scene; Construct a neighborhood sliding window for each ROI pixel within the ROI region; Smoke features of the ROI region are extracted using a neighborhood sliding window to obtain the smoke probability factor of the ROI pixel. The sharpness of each ROI pixel is calculated based on a neighborhood sliding window. Calculate the smoke clarity diffusion degree of each ROI pixel based on the clarity of each ROI pixel; The smoke confidence level of each ROI pixel is calculated based on the smoke probability factor and the smoke clarity diffusion. Smoke features of the Region of Interest (ROI) are extracted using a neighborhood sliding window to obtain the smoke probability factor of ROI pixels. This includes: obtaining the gray-level co-occurrence matrix (GLCM) of each ROI pixel using a neighborhood sliding window; calculating the contrast and entropy of the GLCM as the neighborhood contrast and entropy of the ROI pixel; calculating the texture richness of the ROI pixel based on the neighborhood contrast and entropy; for ROI pixels with texture richness less than or equal to a texture threshold, calculating the neighborhood gray-level variance of the corresponding neighborhood sliding window; and calculating the smoke probability factor of the ROI pixel based on the texture richness and neighborhood gray-level variance. The texture richness is calculated using the following formula: In the formula, It refers to texture richness; It is the neighborhood contrast. It is the neighborhood entropy; The smoke probability factor is calculated using the following formula: In the formula, It is the smoke probability factor; It is the normalized texture richness. It is the neighborhood gray-level variance; It is the texture threshold; The sharpness of each ROI pixel is calculated using a neighborhood sliding window, including: calculating the gradient value of each ROI pixel; performing Harris corner detection on the ROI region and counting the number of corners within the neighborhood sliding window corresponding to each ROI pixel; calculating the Euclidean distance from each non-center ROI pixel to the center ROI pixel within the neighborhood sliding window corresponding to each ROI pixel; and calculating the sharpness of the center ROI pixel based on the gradient value of each ROI pixel within the neighborhood sliding window, the number of corners, and the Euclidean distance from non-center ROI pixels to the center ROI pixel; wherein, the sharpness is calculated according to the following formula: In the formula, It is the first in the neighborhood sliding window Gradient values ​​of non-centered ROI pixels; It is the gradient value of the ROI pixel at the center of the neighborhood sliding window; It is the first in the neighborhood sliding window The Euclidean distance from a non-centered ROI pixel to a centered ROI pixel; It is the sum of all Euclidean distances within the neighborhood sliding window; It represents the number of corner points within the neighborhood sliding window; The smoke clarity diffusion degree of each ROI pixel is calculated based on the sharpness of each ROI pixel, including: constructing a sharpness matrix by arranging the sharpness of each ROI pixel within the neighborhood sliding window according to the ROI pixel arrangement; calculating the mean sharpness of each row in the sharpness matrix and the Euclidean distance between different rows; calculating the smoke clarity diffusion degree of the ROI pixel as the center based on the mean sharpness of each row and the Euclidean distance between different rows; wherein, the smoke clarity diffusion degree is calculated according to the following formula: In the formula, It refers to the clarity and diffusion of smoke; and These are the first and second resolutions of the resolution matrix. and Mean line sharpness and ; For the first part of the resolution matrix and Euclidean distance between rows; The smoke confidence score for each ROI pixel is calculated based on the smoke probability factor and the smoke clarity diffusion, including: classifying ROI pixels with a smoke probability factor greater than or equal to the smoke probability threshold as high-probability smoke points; classifying ROI pixels with a smoke probability factor less than the smoke probability threshold as low-probability smoke points; calculating the smoke confidence score for each high-probability smoke point based on the smoke probability factor and the smoke clarity diffusion; and calculating the smoke confidence score for each low-probability smoke point based on the smoke probability factor. The smoke confidence level is calculated using the following formula: In the formula, It is the confidence level of the smoke; It is the normalized smoke probability factor; It is the normalized smoke clarity and diffusion. It is the smoke probability threshold.

2. The image processing-based belt conveyor fire early warning method according to claim 1, characterized in that, Extract the Region of Interest (ROI) from the reference RGB image, including: Convert the reference RGB image to a reference grayscale image; Sobel operator edge detection is performed on a reference grayscale image to obtain the edge contours in the reference grayscale image; Perform Hough line transform on the edge contour to obtain each line segment in the reference grayscale image and the coordinates of each pixel on the line segment; The number of pixels on each line segment is counted as the length of the line segment, and the two line segments with the longest length are selected as the edge lines; Get the edge endpoints of the two edge lines; Connect the edge endpoints located on the same image boundary and extend them to both sides to obtain the boundary lines; draw the endpoint perpendicular lines that pass through the edge endpoints and are perpendicular to the boundary lines; The largest area enclosed by the boundary line and the perpendicular lines to the endpoints is defined as the ROI region.

3. The image processing-based belt conveyor fire early warning method according to claim 1, characterized in that, Actual scene images of the transport conveyor belt were acquired at the same shooting angle as the reference RGB image, including: A CMOS camera was used to periodically capture actual RGB images of the transport belt at the same shooting angle as the RGB images. Median filtering is used to preprocess the actual RGB image to obtain the actual scene image; Convert real-world scene images into grayscale images of the real-world scene.

4. The image processing-based belt conveyor fire early warning method according to claim 1, characterized in that, Fire early warning recognition is performed on real-world scene images based on the smoke confidence scores of each ROI pixel, including: Based on the smoke confidence level, threshold segmentation is performed on each ROI pixel in the actual scene image, and ROI pixels with a smoke confidence level greater than or equal to the confidence level threshold are classified as smoke points. The number of smoke points in the actual scene image is counted. When the number of smoke points exceeds the smoke threshold, it is determined that the conveyor belt has caught fire.

5. The image processing-based belt conveyor fire early warning method according to claim 4, characterized in that, Threshold segmentation is performed according to the following formula: In the formula, Coordinates The corresponding binarized value of the ROI pixel: a binarized value of 0 indicates that the ROI pixel is a smoke point, and a binarized value of 1 indicates that the ROI pixel is not a smoke point; Coordinates The normalized smoke confidence level of the corresponding ROI pixel; It is the confidence threshold.

Images