Method and system for detecting motion state of coal flow

Abstract

The invention relates to a coal flow motion state detection method, which belongs to the field of coal flow detection, and comprises the following steps: (1) video preprocessing; (2) motion and static detection; (3) processing a threshold value; (4) contour analysis; and (5) state judgment: judging whether the coal flow is in a static state or not according to a contour analysis result and a preset threshold value. The coal flow monitoring system has the advantages that the coal flow is monitored in real time by utilizing a visual analysis technology, manpower is not consumed, and the motion state of the coal flow can be accurately detected. Compared with a traditional detection method, the method not only improves the detection accuracy, but also reduces the labor cost, thereby improving the coal generation efficiency. In addition, other scene tasks needing coal flow motion static detection can be detected more accurately by calling the system. Through the network service mode, the system provided by the invention has wider application prospect and practical value.

Description

Technical Field

[0001] The invention relates to a method for detecting the movement state of coal flow, and belongs to the field of coal flow detection. Background Art

[0002] In the coal production process, coal transportation is a crucial link. If the coal cannot be transported to the designated location accurately and on time, the coal production efficiency will be seriously reduced and the production cost will increase.

[0003] The motion state of coal flow plays a key role in other scene task detection, such as funnel blockage. The traditional method of detecting the motion and static state of coal flow usually determines the state of coal flow by judging whether the machine transporting coal is turned on. If the machine is turned on, the coal flow is considered to be in motion, otherwise it is considered to be static. However, this method is not only cumbersome, but also prone to false detection.

[0004] In 2023, Shandong Energy Group and Huawei jointly proposed a large mine model to monitor multiple scenarios of coal production, thereby effectively improving coal production efficiency. However, when using large models to detect some complex scenarios, relying solely on large models may result in recognition errors. Taking the belt coal pile scenario as an example, if only the large model is used to detect whether coal piles appear, when the coal flow is large, incorrect detection results will occur. Summary of the invention

[0005] In order to overcome the defects of the prior art, the present invention provides a method for detecting the movement state of coal flow. The technical solution of the present invention is:

[0006] A method for detecting the movement state of coal flow comprises the following steps:

[0007] (1) Video preprocessing: Use a camera to collect the coal flow video stream data of the working face in real time, convert the collected video stream into a grayscale image, and divide the region of interest to reduce the amount of calculation;

[0008] (2) Motion and stillness detection: The motion state of the coal flow is determined by calculating the Laplace variance and difference image of consecutive video frames;

[0009] (3) Threshold processing: Threshold processing is performed on the difference image to reduce noise interference;

[0010] (4) Contour analysis: Contour analysis is performed on the threshold processed image to determine the movement state of the coal flow;

[0011] (5) State judgment: Based on the profile analysis results and the preset threshold, determine whether the coal flow is in a static state.

[0012] The step (1) is specifically as follows: reading the camera RTSP video stream, converting the first frame image into a grayscale image; drawing an area of ​​interest on the first frame image, performing motion and stillness detection on the coal flow in the area of ​​interest, and evenly dividing the area of ​​interest into 9 squares of equal area.

[0013] The step (2) is specifically as follows: taking the first frame image read as a template, continuing to read the second frame image, and comparing the difference with the first frame image used as a template to determine whether there is motion, then setting the second frame image as a template, comparing the difference between the third frame image read and the second frame image, and detecting in sequence;

[0014] 2.1 Calculate the Laplace variance of the current frame image. The Laplace variance of the image represents the blur degree of the image. The calculation method is shown in the following formula:

[0015] in, represents the Laplace of image f, and Represent the second-order partial derivatives of the image with respect to the x and y coordinates respectively;

[0016] Then perform Gaussian blur processing on the image to reduce the noise in the image and smooth the details of the image; 2.2 Read the second frame of the image, convert the current frame into a grayscale image, calculate the Laplace variance of the frame image and the difference with the previous frame, and save the difference value as a difference image. The calculation method is shown in the following formula: Wherein, I(t) and I(t-1) are the images of the current moment and the previous frame respectively, and T is the threshold set when the differential image is binarized.

[0017] In order to reduce the impact of small differences on recognition, step (3) performs threshold processing on the difference image, setting pixels less than 7 to 0 and pixels greater than 7 to 255; specifically:

[0018] Input: difference image frame_diff;

[0019] Output: image thresh_frame after threshold processing;

[0020] Traverse each pixel of the image matrix: if pixel>7, set pixel to 255; otherwise: set pixel to 0; end after returning thresh_frame.

[0021] The step (4) is specifically as follows:

[0022] 4.1 Perform a dilation operation on the difference image to expand the white area in the image, thereby eliminating the small black area and finding the contour in the image. The mathematical expression of the dilation operation is expressed as follows: Among them, B' represents the binary image after expansion, B represents the original binary image, and S represents the structural element; ⊕ represents the expansion operator, which translates the structural element S and covers it on the image B. If the pixels in the structural element are all white, the pixels at the corresponding position are also marked as white;

[0023] 4.2 Loop through the contours found in step 4.1. If the contour area is larger than the set threshold, record the coordinate points of the contour, otherwise discard it;

[0024] 4.3 Calculate the number of overlaps between the contour saved in step 4.2 and the 9-grid area in the region of interest, set a list of [0,0,0,0,0,0,0,0,0], and set the 0 in the list to 1 if there is overlap with the 9-grid area;

[0025] 4.4 Count the total number of overlaps, that is, count the number of 1s in the list in step 6, recorded as move_area_num; count the total number of overlaps without the edge and center points, that is, subtract the numbers at the 2nd, 5th, and 8th positions in the list, recorded as edge_move_area_num;

[0026] 4.5 Calculate the Laplace variance of the current frame image. If the Laplace variance of the image is greater than the set maximum stop state threshold, set move_area_num and edge_move_area_num to 0; if the Laplace variance of the image is less than the set minimum motion state threshold, reduce the speedup_change threshold for judging whether to move to half of the original value;

[0027] 4.6 Update the number of changes. If edge_move_area_num and edge_move_area_num meet the set thresholds and the current frame state is static, cont_change_num is set to the maximum value between 1 and speedup_change, otherwise cont_change_num is set to 0;

[0028] If edge_move_area_num and edge_move_area_num do not meet the set threshold, if the previous frame state is motion, cont_change_num is set to the maximum value between 1 and speedup_change, otherwise cont_change_num is set to 0.

[0029] The step (5) is specifically as follows:

[0030] 5.1 Compare the relationship between cont_change_num and speedup_change; if cont_change_num is greater than speedup_change, it is considered to be in motion, otherwise it is in a stationary state;

[0031] 5.2 Save the current frame status, set the current frame as a template, and continue to detect the next frame.

[0032] A system for realizing a method for detecting a coal flow movement state, comprising:

[0033] The video preprocessing module is used to collect the coal flow video stream data of the working face in real time and convert the collected video stream into a grayscale image;

[0034] State detection module: used to determine the movement state of coal flow by calculating the Laplace variance and difference image of continuous video frames;

[0035] The threshold processing module is used to perform threshold processing on the difference image to reduce noise interference; the contour analysis module is used to perform contour analysis on the image after threshold processing to determine the movement state of the coal flow;

[0036] The state judgment module is used to judge whether the coal flow is in a static state based on the profile analysis results and the preset threshold value.

[0037] The advantages of the present invention are:

[0038] The use of visual analysis technology to monitor coal flow in real time does not require manpower and can accurately detect the movement state of coal flow. Compared with traditional detection methods, this method not only improves the accuracy of detection, but also reduces labor costs, thereby improving coal generation efficiency. In addition, other scene tasks that require static detection of coal flow movement can achieve more accurate detection by calling this system. Through this network service method, the system of the present invention can have a wider application prospect and practical value.

[0039] By adding the method of detecting the motion and stillness of coal flow, when the coal flow is in motion, it is not considered that coal pile occurs; and when the coal flow is in a still state, it is considered that coal pile occurs. The detection method proposed by the present invention can effectively improve the accuracy of belt coal pile detection, thereby ensuring the safe and stable operation of coal mine production and further improving the overall efficiency of coal mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the detection method of the present invention.

[0041] Figure 2 It is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with specific embodiments, and the advantages and features of the present invention will become clearer as the description proceeds. However, these embodiments are exemplary only and do not constitute any limitation to the scope of the present invention. It should be understood by those skilled in the art that the details and forms of the technical solution of the present invention may be modified or replaced without departing from the spirit and scope of the present invention, but these modifications and replacements all fall within the scope of protection of the present invention.

[0043] See also Figure 1 and Figure 2 The present invention relates to a method for detecting the movement state of coal flow, comprising the following steps:

[0044] (1) Video preprocessing: Use a camera to collect the coal flow video stream data of the working face in real time, convert the collected video stream into a grayscale image, and divide the region of interest to reduce the amount of calculation;

[0045] (2) Motion and stillness detection: The motion state of the coal flow is determined by calculating the Laplace variance and difference image of consecutive video frames;

[0046] (3) Threshold processing: Threshold processing is performed on the difference image to reduce noise interference;

[0047] (4) Contour analysis: Contour analysis is performed on the threshold processed image to determine the movement state of the coal flow;

[0048] (5) State judgment: Based on the profile analysis results and the preset threshold, determine whether the coal flow is in a static state.

[0049] The step (1) is specifically as follows: reading the camera RTSP video stream, converting the first frame image into a grayscale image, reducing the amount of image data, and improving computing efficiency; drawing a region of interest (ROI) on the first frame image, performing motion and stillness detection on the coal flow in the region of interest, and dividing the region of interest into 9 squares of equal area.

[0050] The step (2) is specifically as follows: taking the first frame image read as a template, continuing to read the second frame image, and comparing the difference with the first frame image used as a template to determine whether there is motion, then setting the second frame image as a template, comparing the difference between the third frame image read and the second frame image, and detecting in sequence;

[0051] 2.1 Calculate the Laplace variance of the current frame image. The Laplace variance of the image represents the blur degree of the image. The calculation method is shown in the following formula:

[0052] in, represents the Laplace of image f, and Represent the second-order partial derivatives of the image with respect to the x and y coordinates respectively;

[0053] Then perform Gaussian blur processing on the image to reduce the noise in the image and smooth the details of the image; 2.2 Read the second frame of the image, convert the current frame into a grayscale image, calculate the Laplace variance of the frame image and the difference with the previous frame, and save the difference value as a difference image. The calculation method is shown in the following formula: Wherein, I(t) and I(t-1) are the images of the current moment and the previous frame respectively, and T is the threshold set when the differential image is binarized. In the present invention, the threshold is set to 7, D(x,y)=1 represents the foreground, and D(x,y)=0 represents the background.

[0054] In order to reduce the impact of small differences on recognition, step (3) performs threshold processing on the difference image, setting pixels less than 7 to 0 and pixels greater than 7 to 255; specifically:

[0055] Input: difference image frame_diff;

[0056] Output: image thresh_frame after threshold processing;

[0057] Traverse each pixel of the image matrix: if pixel>7, set pixel to 255; otherwise: set pixel to 0; end after returning thresh_frame.

[0058] The step (4) is specifically as follows:

[0059] 4.1 Perform a dilation operation on the difference image to expand the white area in the image, thereby eliminating the small black area and finding the contour in the image. The mathematical expression of the dilation operation is expressed as follows: Among them, B' represents the binary image after expansion, B represents the original binary image, and S represents the structural element; ⊕ represents the expansion operator, which translates the structural element S and covers it on the image B. If the pixels in the structural element are all white, the pixels at the corresponding position are also marked as white;

[0060] 4.2 Loop through the contours found in step 4.1. If the contour area is larger than the set threshold, record the coordinate points of the contour, otherwise discard it;

[0061] 4.3 Calculate the number of overlaps between the contour saved in step 4.2 and the 9-grid area in the region of interest, set a list of [0,0,0,0,0,0,0,0,0], and set the 0 in the list to 1 if there is overlap with the 9-grid area;

[0062] 4.4 Count the total number of overlaps, that is, count the number of 1s in the list in step 6, recorded as move_area_num; count the total number of overlaps without the edge and center points, that is, subtract the numbers at the 2nd, 5th, and 8th positions in the list, recorded as edge_move_area_num;

[0063] 4.5 Calculate the Laplace variance of the current frame image. If the Laplace variance of the image is greater than the set maximum stop state threshold, set move_area_num and edge_move_area_num to 0; if the Laplace variance of the image is less than the set minimum motion state threshold, reduce the speedup_change threshold for judging whether to move to half of the original value;

[0064] 4.6 Update the number of changes. If edge_move_area_num and edge_move_area_num meet the set thresholds and the current frame state is static, cont_change_num is set to the maximum value between 1 and speedup_change, otherwise cont_change_num is set to 0;

[0065] If edge_move_area_num and edge_move_area_num do not meet the set threshold, if the previous frame state is motion, cont_change_num is set to the maximum value between 1 and speedup_change, otherwise cont_change_num is set to 0.

[0066] The step (5) is specifically as follows:

[0067] 5.1 Compare the relationship between cont_change_num and speedup_change; if cont_change_num is greater than speedup_change, it is considered to be in motion, otherwise it is in a stationary state;

[0068] 5.2 Save the current frame status, set the current frame as a template, and continue to detect the next frame.

[0069] like Figure 2 As shown, the present invention also relates to a system for realizing a method for detecting a coal flow movement state, comprising:

[0070] The video preprocessing module 1 is used to collect the coal flow video stream data of the working face in real time and convert the collected video stream into a grayscale image;

[0071] State detection module 2: used to determine the movement state of the coal flow by calculating the Laplace variance and difference image of continuous video frames;

[0072] A threshold processing module 3 is used to perform threshold processing on the difference image to reduce noise interference;

[0073] The contour analysis module 4 is used to perform contour analysis on the image after threshold processing to determine the movement state of the coal flow;

[0074] The state judgment module 5 is used to judge whether the coal flow is in a static state according to the profile analysis result and the preset threshold value.

[0075] In the present invention, the working principle of the present invention is based on video image processing and computer vision technology, and the movement state (movement or stillness) of the coal flow is automatically detected by real-time analysis of the coal flow video stream data collected by the camera. The following is the detailed working principle of the present invention:

[0076] Video preprocessing:

[0077] Use cameras to collect coal flow video stream data at the working face in real time.

[0078] Converting the captured video stream into a grayscale image can reduce the amount of image data and improve the computational efficiency of subsequent processing.

[0079] Regions of interest (ROIs) are divided on the first frame image, and only these regions are analyzed to further reduce the amount of calculation.

[0080] The area of ​​interest is evenly divided into 9 squares of equal area to analyze the coal flow state in more detail.

[0081] Motion and stillness detection:

[0082] The first frame image is used as a template, and then subsequent frame images are read continuously and compared with the previous frame image to detect differences.

[0083] Calculate the Laplace variance of the current frame image, which indicates the blurriness of the image and is used to determine whether the image content has changed significantly.

[0084] Gaussian blur the image to reduce noise and smooth image details.

[0085] Thresholding:

[0086] The calculated difference image is thresholded, and the pixels with a value less than 7 are set to 0 (indicating the background), and the pixels with a value greater than 7 are set to 255 (indicating the foreground) to reduce noise interference.

[0087] Profile Analysis:

[0088] Dilation is performed on the thresholded image to enlarge white areas, eliminate small black areas, and find contours in the image.

[0089] Traverse the found contours and record the coordinate points of the contours whose area is greater than the set threshold.

[0090] The number of overlaps between these contours and the nine small grid areas divided within the region of interest is calculated to determine the movement state of the coal flow.

[0091] Status judgment:

[0092] Based on the results of the profile analysis and the preset threshold, it is determined whether the coal flow is in a stationary state.

[0093] If the Laplace variance of the current frame image is greater than the set maximum stop state threshold, the coal flow is considered to be stationary; if it is less than the set minimum motion state threshold, the coal flow is considered to be moving, and the threshold for determining whether it is moving is updated.

[0094] According to the number of overlaps between the contour and the small grid area, the number of changes is updated to determine the movement state of the coal flow.

[0095] System structure:

[0096] It includes a video preprocessing module, a state detection module, a threshold processing module, a contour analysis module and a state judgment module. These modules work together to realize automatic detection of the movement state of coal flow.

[0097] The advantage of the present invention is that it can utilize visual analysis technology to monitor coal flow in real time without manual intervention, thereby improving detection accuracy and coal production efficiency.

[0098] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for detecting the movement state of coal flow, characterized in that: The following steps are involved: (1) Video preprocessing: Use a camera to collect the coal flow video stream data of the working face in real time, convert the collected video stream into a grayscale image, and divide the region of interest to reduce the amount of calculation; (2) Motion and stillness detection: The motion state of the coal flow is determined by calculating the Laplace variance and difference image of consecutive video frames; (3) Threshold processing: Threshold processing is performed on the difference image to reduce noise interference; (4) Contour analysis: Contour analysis is performed on the threshold processed image to determine the movement state of the coal flow; (5) State judgment: Based on the profile analysis results and the preset threshold, determine whether the coal flow is in a static state.

2. The method for detecting the movement state of coal flow according to claim 1, characterized in that: The step (1) is specifically as follows: reading the camera RTSP video stream, converting the first frame image into a grayscale image; drawing an area of ​​interest on the first frame image, performing motion and stillness detection on the coal flow in the area of ​​interest, and evenly dividing the area of ​​interest into 9 squares of equal area.

3. The method for detecting the movement state of coal flow according to claim 1 or 2, characterized in that: The step (2) is specifically as follows: taking the first frame image read as a template, continuing to read the second frame image, and comparing the difference with the first frame image used as a template to determine whether there is motion, then setting the second frame image as a template, comparing the difference between the third frame image read and the second frame image, and detecting in sequence; 2.1 Calculate the Laplace variance of the current frame image. The Laplace variance of the image represents the blur degree of the image. The calculation method is shown in the following formula: in, represents the Laplace of image f, and Represent the second-order partial derivatives of the image with respect to the x and y coordinates respectively; Then perform Gaussian blur processing on the image to reduce the noise in the image and smooth the details of the image; 2.2 Read the second frame of the image, convert the current frame into a grayscale image, calculate the Laplace variance of the frame image and the difference with the previous frame, and save the difference value as a difference image. The calculation method is shown in the following formula: Wherein, I(t) and I(t-1) are the images of the current moment and the previous frame respectively, and T is the threshold set when the differential image is binarized.

4. The method for detecting the movement state of coal flow according to claim 3, characterized in that: In order to reduce the impact of small differences on recognition, step (3) performs threshold processing on the difference image, setting pixels less than 7 to 0 and pixels greater than 7 to 255; specifically: Input: difference image frame_diff; Output: image thresh_frame after threshold processing; Traverse each pixel of the image matrix: if pixel>7, set pixel to 255; otherwise: set pixel to 0; end after returning thresh_frame.

5. The method for detecting the movement state of coal flow according to claim 4, characterized in that: The step (4) is specifically as follows: 4.1 Perform a dilation operation on the difference image to expand the white area in the image, thereby eliminating the small black area and finding the contour in the image. The mathematical expression of the dilation operation is expressed as follows: Among them, B' represents the binary image after expansion, B represents the original binary image, and S represents the structural element; represents the dilation operator, which translates the structure element S and covers it on the image B. If the pixels in the structure element are all white, the pixels at the corresponding position are also marked as white; 4.2 Loop through the contours found in step 4.

1. If the contour area is larger than the set threshold, record the coordinate points of the contour, otherwise discard it; 4.3 Calculate the number of overlaps between the contour saved in step 4.2 and the 9-grid area in the region of interest, set a list of [0,0,0,0,0,0,0,0,0], and set the 0 in the list to 1 if there is overlap with the 9-grid area; 4.4 Count the total number of overlaps, that is, count the number of 1s in the list in step 6, recorded as move_area_num; count the total number of overlaps without the edge and center points, that is, subtract the numbers at the 2nd, 5th, and 8th positions in the list, recorded as edge_move_area_num; 4.5 Calculate the Laplace variance of the current frame image. If the Laplace variance of the image is greater than the set maximum stop state threshold, set move_area_num and edge_move_area_num to 0; if the Laplace variance of the image is less than the set minimum motion state threshold, reduce the speedup_change threshold for judging whether to move to half of the original value; 4.6 Update the number of changes. If edge_move_area_num and edge_move_area_num meet the set thresholds and the current frame state is static, cont_change_num is set to the maximum value between 1 and speedup_change, otherwise cont_change_num is set to 0; If edge_move_area_num and edge_move_area_num do not meet the set threshold, if the previous frame state is motion, cont_change_num is set to the maximum value between 1 and speedup_change, otherwise cont_change_num is set to 0.

6. The method for detecting the movement state of coal flow according to claim 5, characterized in that: The step (5) is specifically as follows: 5.1 Compare the relationship between cont_change_num and speedup_change; if cont_change_num is greater than speedup_change, it is considered to be in motion, otherwise it is in a stationary state; 5.2 Save the current frame status, set the current frame as a template, and continue to detect the next frame.

7. A system for implementing the method for detecting the movement state of coal flow as claimed in any one of claims 1 to 6, characterized in that: include: The video preprocessing module is used to collect the coal flow video stream data of the working face in real time and convert the collected video stream into a grayscale image; State detection module: used to determine the movement state of coal flow by calculating the Laplace variance and difference image of continuous video frames; A threshold processing module is used to perform threshold processing on the difference image to reduce noise interference; A contour analysis module is used to perform contour analysis on the image after threshold processing to determine the movement state of the coal flow; The state judgment module is used to judge whether the coal flow is in a static state based on the profile analysis results and the preset threshold value.

Images