Roller rotating speed detection method and system based on optical flow method

Abstract

The invention discloses a roller rotating speed detection method and system based on an optical flow method. According to the scheme, on the basis of organic fusion of two-path image optical flow analysis and spatial geometry modeling, the detection of the rotating speed of a roller is realized under the condition of not depending on a physical sensor or any physical mark; non-contact detection is carried out on the rotating speed of the coal mining machine roller in the underground coal mine directly based on the operation video image of the coal mining machine roller, and the efficiency of high robustness and friendly visualization can be achieved.

Description

Technical Field

[0001] This invention relates to intelligent coal mining technology, specifically to real-time monitoring technology for the operating status of coal mining machines. Background Technology

[0002] With the increasing demand for intelligent and unmanned operations in coal mines, the real-time monitoring of the operating status of coal mining machines, as the core of fully mechanized coal mining equipment, is particularly crucial. In particular, drum speed, as an important indicator for judging coal mining intensity, load status, and equipment wear, is of great significance for ensuring production safety and optimizing scheduling.

[0003] Traditional methods for detecting the rotational speed of a coal mining machine drum mainly include installing speed sensors (such as encoders or Hall sensors) for detection, indirectly inferring the rotational speed through motor current and voltage, or obtaining data through manual timed observation or based on a PLC system.

[0004] Traditional methods for detecting the rotational speed of coal mining machine drums are relatively stable in surface environments or laboratory conditions. However, in complex underground working conditions (such as high dust levels, poor lighting, confined spaces, and easily damaged equipment), the installation and maintenance costs of sensors are high, their stability is poor, and it is difficult to achieve non-contact, visual, and intelligent detection.

[0005] With the development of computer vision and artificial intelligence technologies, a non-contact drum speed measurement solution based on image analysis has been developed to address the needs and challenges of detecting the drum speed of coal mining machines.

[0006] Existing image analysis-based non-contact roller speed measurement solutions generally suffer from the following problems in practical applications:

[0007] (1) It relies on feature templates or markers and has poor adaptability;

[0008] Existing non-contact drum speed measurement solutions based on image analysis require marking points or stickers to be pasted on the drum surface to assist image tracking. However, in actual coal mine operations, these marking solutions are very easy to fall off or be covered by dust, leading to detection failure.

[0009] (2) Ignoring spatial configuration and estimating linear velocity solely based on pixel motion results in large errors;

[0010] Existing non-contact roller speed measurement solutions based on image analysis generally employ optical flow analysis methods used for planar motion analysis of pedestrian and vehicle traffic. These methods lack modeling of the equipment's rotation axis structure and the distance between points, making it impossible to directly convert the data into angular velocity data. This results in low detection accuracy and large errors.

[0011] (3) It is sensitive to noise and its angular velocity fluctuates drastically;

[0012] Video acquisition in coal mine operating environments is easily affected by changes in lighting, dust obstruction, and coal block interference, causing optical flow disturbances. Existing non-contact drum speed measurement solutions based on image analysis lack robust outlier detection and data smoothing mechanisms, resulting in unstable speed measurement data that cannot be used for high-frequency regulation or linkage control.

[0013] (4) Failure to distinguish between the roller area and the background motion

[0014] Existing non-contact drum speed measurement schemes based on image analysis generally analyze the entire frame of the image without constructing a specific analysis path for the drum's rotation area (ROI) within the image, resulting in significant noise interference and computational redundancy.

[0015] (5) Lack of visual interaction and synchronized image output;

[0016] Most existing non-contact roller speed measurement solutions based on image analysis only support the output of numerical data and lack an interactive visualization interface for on-site personnel; at the same time, they lack a mechanism for synchronous recording of images and angular velocity curves, which is not conducive to later review and diagnosis.

[0017] As can be seen from the above, how to effectively improve the accuracy and reliability of non-contact coal mining machine drum speed measurement based on video analysis is an urgent problem to be solved in this field. Summary of the Invention

[0018] To address the issues of accuracy and stability in existing image analysis-based non-contact drum speed measurement schemes, the present invention aims to provide a drum speed detection scheme based on optical flow, which can perform non-contact and robust drum angular velocity monitoring in complex downhole environments without relying on feature templates or sticker markings.

[0019] To achieve the above objectives, the present invention provides a drum rotation speed detection method based on optical flow. The method first acquires video images of the coal mining machine drum operation and extracts the ROI region corresponding to the drum region in the video frame image.

[0020] Next, sparse optical flow is calculated based on the ROI regions in the extracted operation video frame images, and a drum rotation axis model is constructed accordingly.

[0021] Next, dense optical flow is calculated based on the ROI regions in the extracted operation video frame images. Pixels with significant motion amplitudes are extracted from the dense optical flow, and their vertical distance to the rotation axis is calculated based on the constructed drum rotation axis model.

[0022] Next, valid motion points that meet the requirements for motion amplitude and distance are selected, and after the number of valid motion points meets the quantity range requirements, the overall roller angular velocity is calculated accordingly.

[0023] Furthermore, when the detection method extracts the ROI region corresponding to the roller region in the video frame image, it first selects and determines the ROI region corresponding to the roller region in the video frame image for the first time, and then automatically updates the ROI position in consecutive frames to track the roller movement in real time.

[0024] Furthermore, when calculating sparse optical flow for the ROI region, the detection method selects local feature points within the ROI region and performs sparse optical flow tracking to obtain the displacement information of the feature points between consecutive frames.

[0025] Furthermore, in the detection method, based on the sparse optical flow results, a set of feature points with small motion amplitudes is extracted to determine the position of the roller rotation center, and a roller rotation axis model is further established according to the preset direction of the roller.

[0026] Furthermore, when calculating dense optical flow for the ROI region, the detection method performs dense optical flow calculation on all pixels in the ROI region, determines the motion vector of each pixel, and calculates the motion amplitude and orientation angle of each pixel.

[0027] Furthermore, after the number of effective moving points meets the quantity area requirement, the detection method first divides the ROI region into several small segments in the horizontal direction, calculates the average radius and average linear velocity in each segment, and derives the local angular velocity of each segment from this. Then, several values ​​of the local angular velocities are selected and averaged to form a preliminary estimate of the overall roller angular velocity.

[0028] Furthermore, the detection method also includes outlier detection and data balancing steps in the overall drum angular velocity calculation process.

[0029] To achieve the above objectives, the present invention provides a drum rotation speed detection system based on optical flow method, the system comprising:

[0030] The ROI region extraction module extracts the ROI region corresponding to the drum area from the video frame image of the coal mining machine drum operation.

[0031] The optical flow calculation module calculates sparse optical flow based on the ROI regions in the job video frame image extracted by the ROI region extraction module; the optical flow calculation module also calculates dense optical flow based on the extracted ROI regions in the job video frame image, and extracts pixels with significant motion amplitude in the dense optical flow.

[0032] The cylinder rotation axis model establishment module establishes the cylinder rotation axis model based on the sparse optical flow calculation results of the optical flow calculation module.

[0033] A vertical distance calculation module calculates the vertical distance from pixels with significant motion amplitude in the dense optical flow extracted by the optical flow calculation module to the rotation axis based on the constructed drum rotation axis model.

[0034] The roller angular velocity calculation module filters out effective motion points in the dense optical flow that meet the requirements of motion amplitude and distance based on the calculation results of the vertical distance calculation module, and performs overall roller angular velocity calculation after the number of effective motion points meets the quantity area requirements.

[0035] Furthermore, when the ROI region extraction module extracts the ROI region corresponding to the roller region in the video frame image, it first selects and determines the ROI region corresponding to the roller region in the video frame image, and then automatically updates the ROI position in consecutive frames to track the roller movement in real time.

[0036] Furthermore, when calculating sparse optical flow for the ROI region, the optical flow calculation module selects local feature points within the ROI region and performs sparse optical flow tracking to obtain the displacement information of the feature points between consecutive frames.

[0037] Furthermore, when calculating dense optical flow for the ROI region, the optical flow calculation module performs dense optical flow calculation on all pixels in the ROI region, determines the motion vector of each pixel, and calculates the motion amplitude and direction angle of each pixel.

[0038] Furthermore, the cylinder rotation axis model establishment module extracts a set of feature points with small motion amplitudes based on sparse optical flow results, thereby determining the position of the cylinder rotation center and further establishing a cylinder rotation axis model according to the preset direction of the cylinder.

[0039] Furthermore, when calculating the overall drum angular velocity, the drum angular velocity calculation module divides the ROI region into several small segments in the horizontal direction, calculates the average radius and average linear velocity in each segment, and derives the local angular velocity of each segment. Based on this, several values ​​of the local angular velocities are selected and averaged to form a preliminary estimate of the overall drum angular velocity.

[0040] Furthermore, the drum speed detection system also includes an outlier smoothing module, which can perform outlier detection and data balancing processing when the drum angular velocity calculation module performs overall drum angular velocity calculation.

[0041] The roller rotation speed detection scheme based on optical flow method presented in this invention has the following superior effects compared with the prior art:

[0042] (1) This solution can obtain the drum angular velocity based on visual images without relying on physical sensors or sticker markings, making it particularly suitable for harsh underground environments; it can reduce equipment installation and maintenance costs and improve operational reliability;

[0043] (2) This scheme has a robust angular velocity estimation mechanism. It uses local spatial segmentation and axis distance to construct an angular velocity model. Combined with median filtering and dynamic outlier correction algorithm, it significantly improves the stability and anti-interference ability of velocity measurement data and can effectively cope with complex underground working conditions such as coal block interference, dust obstruction and light changes.

[0044] (3) This solution can realize a closed loop of visualization and data storage, output angular velocity curves in real time, and dynamically display fluctuations through a scrolling window; it supports exporting the original and smoothed angular velocity data, which is convenient for subsequent analysis, modeling and predictive maintenance. Attached Figure Description

[0045] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 This is a flowchart illustrating the construction of smooth curves based on linear and cubic splines in this invention.

[0047] Figure 2 This is an example of a return curve formed by interpolation fitting based on cubic splines;

[0048] Figure 3 This is an example diagram of the yield curve constructed by the smooth curve construction system based on linear and cubic splines provided in this invention. Detailed Implementation

[0049] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0050] In response to the harsh underground environment, this invention achieves non-contact detection of the rotational speed of a coal mining machine drum in underground mines by organically integrating two-way image optical flow analysis and spatial geometric modeling, without relying on physical sensors or any physical markers, and can achieve high efficiency with strong robustness and user-friendly visualization.

[0051] Accordingly, this invention provides a method for detecting drum rotation speed based on optical flow, combined with... Figure 1 As shown, this drum speed detection method mainly includes the following steps:

[0052] (1) First, obtain video images of the coal mining machine drum operation and extract the ROI region corresponding to the drum area in the video frame image.

[0053] (2) Calculate sparse optical flow based on the ROI region in the extracted operation video frame image, and construct a drum rotation axis model accordingly;

[0054] (3) Calculate dense optical flow based on the ROI region in the extracted operation video frame image, extract the pixel points with significant motion amplitude in the dense optical flow, and calculate the vertical distance from the roller rotation axis to the rotation axis based on the constructed roller rotation axis model, as the basis for local radius estimation.

[0055] (4) Select effective motion pixels that meet the requirements of motion amplitude and distance, and calculate the overall roller angular velocity based on the number of effective motion pixels after the number of effective motion pixels meets the requirements of the quantity area.

[0056] Regarding this drum speed detection method, the implementation scheme of each step in the method and the technical effect achieved are further explained below.

[0057] In step (1), before extracting the ROI region from the video image of the coal mining machine drum operation, the method first acquires the video image of the coal mining machine drum operation, performs grayscale conversion, and then performs preprocessing. This reduces computational complexity, improves processing efficiency, highlights structural features, suppresses color interference, enhances key area features, improves robustness, and improves the accuracy of subsequent ROI selection and axis modeling.

[0058] Specifically, when performing grayscale conversion and preprocessing on video images of coal mining machine drum operation, this method first calculates local features (local variance) by calculating the variance of image grayscale values ​​within a local window (e.g., 3×3 or 5×5).

[0059]

[0060] Where, μ local (x,y) represents the average gray value within the local window.

[0061] Next, the standard deviation of the Gaussian filter is adaptively determined, and σ is determined based on the magnitude of the local variance.

[0062] For regions with high local variance (rich texture and detail), a smaller σ is used;

[0063] For regions with small local variance (smooth regions, where there may be more noise), a larger σ is used.

[0064] And specifically use the following mapping relationship:

[0065]

[0066] in:

[0067] σ max ,σ minThese represent the allowed ranges of σ;

[0068] V max V min These are the maximum and minimum values ​​of the overall local variance of the image, used to standardize the local variance.

[0069] Based on this, further constraints can be imposed to improve stability:

[0070] σ gaussian (x,y)=clip(σ gaussian (x,y),σ min ,σ max );

[0071] Next, adaptive Gaussian filtering is performed, for each pixel, to determine the local σ. gaussian Perform Gaussian convolution on (x,y):

[0072] I blurred (x,y)=I gray (x,y)*G(x,y,σ gaussian (x,y)).

[0073] In step (1), when extracting the ROI region corresponding to the drum area in the video frame image, the method initially selects the initial ROI region corresponding to the drum on the pre-processed coal mining machine drum operation video image through human-computer interaction (such as manual selection). Then, the KCF tracker is used to track and process the subsequent coal mining machine drum operation video images to track and update the ROI region in real time, thereby suppressing background black noise interference.

[0074] In step (2), when calculating sparse optical flow for the ROI region, this method selects local feature points within the extracted ROI region and uses the Lucas-Kanade method to track sparse optical flow, thereby obtaining the displacement information of the feature points between consecutive frames.

[0075] The sparse optical flow here is the coarse optical flow. The corresponding feature points are calculated based on the Lucas-Kanade algorithm and used for subsequent drum center point calculation. This algorithm can adapt to noise conditions such as changes in lighting, dust interference, and coal block obstruction in the downhole environment, avoiding the hardship of overall tracking failure due to local obstruction.

[0076] As further explanation, the feature points to be extracted by this method are preferably distributed in areas with obvious texture changes, such as the structure or rust patterns on the surface of the roller, so as to be accurately tracked in consecutive frames; at the same time, background or textureless area features are avoided to improve the accuracy and stability of optical flow matching; in conjunction with this, the video acquisition frame rate is moderate, so that the roller movement produces a moderate and detectable displacement between adjacent frames, avoiding the loss of feature points due to being too fast, or the lack of obvious movement due to being too slow.

[0077] In step (2), when constructing the roller rotation axis model, this method analyzes the slowly moving feature points near the central region based on the sparse optical flow results (i.e., the local feature points selected within the ROI region and the displacement information of the feature points between consecutive frames) and the amount of motion of the feature points in consecutive frames (i.e., the displacement information of the feature points between consecutive frames), determines the rotation center, and then performs rotation axis modeling.

[0078] This method specifically extracts a set of feature points with small motion amplitudes based on sparse optical flow results. Then, it determines the position of the roller's rotation center by averaging the geometric centers, resulting in two rotation centers. Further, it establishes a roller rotation axis model based on a preset roller orientation, which may include a horizontal direction or a slight tilt angle.

[0079] As further explanation, in the specific implementation, this method first selects the k feature points with the smallest optical flow (the smallest optical flow amplitude) and determines the center coordinates, where x and y are the coordinates:

[0080]

[0081] Next, based on the overall motion direction of the feature points, this method estimates the rotation angle θ of the roller (the angle with the horizontal line), and then constructs the rotation axis direction vector:

[0082]

[0083] Finally, based on the center of rotation and the direction vector, a mathematical model of the axis of rotation is constructed. This mathematical model of the axis of rotation can be a line segment or a straight line equation. The two endpoints of the axis are determined through the following mathematical model, thereby constructing the mathematical model of the axis of rotation:

[0084] (x1,y1)=(x c -Lcosθ,y c -Lsinθ);

[0085] (x2,y2)=(x c +Lcosθ,y c +Lsinθ);

[0086] The rotation direction angle θ can be set to 0 degrees, or calculated and determined based on the two rotation centers.

[0087] In step (3), when calculating dense optical flow for the ROI region, this method performs dense optical flow calculation on all pixels in the ROI region, determines the motion vector of each pixel, and calculates the motion amplitude and orientation angle of each pixel, thereby extracting pixels with significant motion amplitude in the dense optical flow.

[0088] The motion vector for each pixel determined here includes both horizontal and vertical displacement.

[0089] Specifically, this method preferably uses the Farneback method to perform dense optical flow calculations on all pixels, obtaining the optical flow vector for each pixel:

[0090] flow(x,y)=(u(x,y),v(x,y));

[0091] Based on this, the motion frame values ​​(i.e., velocity) of the optical flow are calculated using cv2.cartToPolar decomposition:

[0092]

[0093] Calculate the optical flow direction angle using cv2.cartToPolar decomposition:

[0094] ang(x,y)=arctan2(v(x,y),u(x,y));

[0095] Such dense optical flow calculation can cover the entire ROI region without relying on feature points, and has stronger robustness to low-texture or cornerless regions.

[0096] Based on this, this method further filters motion frame values, specifically setting a threshold δ to retain only optical flow points where mag(x,y)≥δ:

[0097]

[0098] The value of δ depends on the situation, but it is generally greater than 2 pixels, which is used to remove minor jitter and noise.

[0099] After completing the dense optical flow in step (3), this method will calculate the distance from the optical flow point to the axis based on the constructed roller rotation axis model (i.e. the corresponding fixed axis equation) for the effective optical flow points (i.e., effective moving pixels) extracted by screening.

[0100] Specifically, for each effective moving pixel, the vertical distance to the roller's rotation axis is calculated:

[0101]

[0102] Where A, B, and C are the general correlation coefficients of the axes.

[0103] In step (4), this method can use the calculation results of step (3) to filter out effective motion pixels that meet the amplitude and distance requirements based on the aforementioned motion frame value filtering scheme.

[0104] Furthermore, in step (4), this method further counts the number of valid motion points that meet the requirements for motion amplitude and distance, and compares it with a preset value to determine whether the number of valid motion points is sufficient.

[0105] If the number of effective motion points is insufficient, i.e. the motion information is sparse, return to step (1) to recalculate the dense optical flow and sparse optical flow until the number of effective motion points is sufficient.

[0106] If there are enough effective moving points, continue with the subsequent calculation of the roller angular velocity.

[0107] Specifically, after the number of effective motion points is sufficient (i.e., the quantity range requirement is met), this method mainly achieves the calculation of the overall roller angular velocity through the following steps:

[0108] Step (4-1) First, the effective optical flow points within the ROI region are divided into several small segments along the x-direction (e.g., the horizontal direction), and spatial partitioning is performed according to the positional structure. This structured processing can effectively improve the consistency of spatial regions and the ability to filter out artifact interference.

[0109] As an example, the ROI region is divided into 10-15 segments horizontally, with each segment containing a number of effective moving pixels, which are then assigned to their respective sub-regions based on their X coordinates.

[0110] Based on this structured division, it is possible to calculate each segment independently and then fuse them together in the subsequent angular velocity calculation, thus combining local estimation with global modeling.

[0111] Furthermore, based on such a structured partition, not only is the local consistency of angular velocity estimation improved, but the global estimation error caused by image noise, artifacts or local occlusion can also be effectively suppressed, which has strong robustness and practical engineering applicability.

[0112] Step (4-2) Next, calculate the average radius and average linear velocity within each ROI segment, and derive the local angular velocity of each ROI segment from this.

[0113] Specifically, in this step, for each region, all effective moving pixels are traversed, and the coordinates of the small ROI region are transformed to convert the effective optical flow points (x,y) in its local coordinate system to the global coordinates (x+x_i,y+y_i) of the image.

[0114] Where (x_i, y_i) is the offset of the top left corner of the current ROI segment in the original image, used to maintain spatial consistency and ensure accurate subsequent calculations.

[0115] Next, based on this, the average radius r_avg of all effective moving pixels in each region is calculated.

[0116] Here, based on the vertical distance to the rotation axis calculated for each effective moving pixel in step (3) (i.e., the rotation radius of the current effective moving pixel), the vertical distances to the rotation axis of all effective moving pixels in each segment are counted, and the average radius r_avg of all effective moving pixels in each segment is calculated accordingly, which is used as the average radius of each segment:

[0117]

[0118] Next, calculate the average linear velocity v_avg of all effective moving pixels within each segment.

[0119] This method uses the optical flow amplitude (i.e., pixel displacement modulus) of each effective moving pixel as the linear velocity. Based on this, the average linear velocity v_avg is obtained by statistically analyzing the optical flow amplitude of all effective moving pixels in each region and averaging the values.

[0120] Finally, based on the calculated average radius r_avg and average linear velocity v_avg, the local angular velocity of each ROI segment is calculated and derived.

[0121] This step uses the following formula to derive the local angular velocity:

[0122] ω = v / r;

[0123] The local angular velocity ω_i of each segment is calculated based on its v_avg and r_avg.

[0124] Finally, in step (4-3), several values ​​of the local angular velocities are selected and averaged to form a preliminary estimate of the overall drum angular velocity. Here, the top-5 maximum values ​​of the local angular velocities are preferred for averaging.

[0125] Specifically, the local angular velocities calculated from all the small ROI regions formed by horizontal segmentation are arranged from largest to smallest, and the five largest values ​​are then averaged and fused to obtain the final roller speed.

[0126] The average fusion calculation here is as follows:

[0127]

[0128] On this machine, it is further converted into a time unit (frame):

[0129]

[0130] This method further includes outlier detection and data balancing steps in the overall drum angular velocity calculation process to achieve smooth output of angular velocity data.

[0131] This method first performs median filtering, which is accomplished using the following model:

[0132] ω i ′ =median(ω i- h,...,ω i+h ).

[0133] Next, outliers are removed by combining neighborhood averaging detection.

[0134] Here, the average and standard deviation of the angular velocities of adjacent regions are used as a reference to identify and remove outlier data that exceeds a preset threshold. The specific calculation model for detecting the average difference in the detection neighborhood is as follows:

[0135]

[0136] in:

[0137]

[0138] Furthermore, when outliers are detected, the outlier data is not discarded directly. Instead, based on historical information, the data within the sliding window is used to effectively fill in the redundancy, ensuring data continuity and reliability.

[0139] Specifically, when this method detects outliers in a small segment of local angular velocity, it does not discard the data directly. Instead, it constructs a sliding time window containing historical valid estimates and uses methods such as weighted averaging, median, or exponential smoothing to compensate for the outliers, replacing the original outliers and allowing them to participate in the subsequent overall angular velocity fusion process. This redundancy compensation effectively ensures the continuity and integrity of angular velocity data, improves the robustness and estimation accuracy of the entire system in complex environments, and has high engineering practicality and deployment adaptability.

[0140] The drum rotation speed detection method based on the optical flow method developed by the above scheme effectively solves the problems of slight camera drift and small target displacement in coal mining machine operations by using dynamic estimation of the drum rotation center and adaptive axis modeling based on sparse optical flow. This scheme introduces a dynamic estimation method for the rotation center based on sparse optical flow, and fits it with feature points with small motion amplitudes in the current image frame to deduce the position of the rotation center in the image in real time. Simultaneously, an adaptive axis model is constructed based on this rotation center to achieve dynamic modeling of the actual spatial rotation axis of the drum. This allows the scheme to effectively offset the effects of slight camera drift, image jitter, or small target displacement, significantly improving the robustness and spatial geometric consistency of angular velocity estimation, avoiding error accumulation caused by fixed model assumptions, and possessing good engineering adaptability and innovation. It is particularly suitable for the real-time analysis and processing of rolling motion characteristics in complex underground operating environments.

[0141] This drum rotation speed detection method further utilizes a combination of dense optical flow and geometric modeling of the rotation axis to calculate local angular velocity, effectively improving the accuracy and robustness of drum rotation speed detection.

[0142] In this method for detecting the rotational speed of a drum, the effective optical flow points within the ROI region are further segmented, and the local angular velocity is derived from the average radius and linear velocity of the small segments, thereby achieving stable measurement under complex motion fields.

[0143] This drum rotation speed detection method further introduces a multi-segment linear velocity screening and fusion mechanism (taking the average of the top-5 maximum angular velocities) to improve the overall angular velocity estimation's ability to suppress local abnormal motion.

[0144] This drum rotation speed detection method further incorporates median filtering and neighborhood mean surge detection, and designs a robust method for removing outlier angular velocity values ​​and smoothing data to ensure the continuity and stability of angular velocity output.

[0145] The resulting optical flow-based drum rotation speed detection method is adaptable to complex downhole lighting, dust, and obstruction environments, and requires no additional sensors, achieving non-contact rotation speed monitoring entirely based on video images.

[0146] The optical flow-based roller speed detection method provided in this invention can be configured into a corresponding software program to form a roller speed detection system based on the optical flow method. When running, this software program will execute the aforementioned roller speed detection method and implement the corresponding functions, while storing the results in a suitable storage medium for the processor to retrieve and execute.

[0147] See Figure 2The resulting roller rotation speed detection system 100 based on optical flow mainly includes the following functions: ROI region extraction module 110, optical flow calculation module 120, roller rotation axis model establishment module 130, vertical distance calculation module 140, roller angular velocity calculation module 150, and outlier smoothing module 160.

[0148] The ROI region extraction module 110 in this system is used to preprocess the acquired video images of the coal mining machine drum operation and extract the ROI region corresponding to the drum region in the video frame image.

[0149] The optical flow calculation module 120 in this system is configured to interact with the ROI region extraction module 110, enabling dual-path optical flow calculations of sparse and dense optical flow. The optical flow calculation module 120 can calculate sparse optical flow based on the ROI regions in the work video frame image extracted by the ROI region extraction module 110; simultaneously, the optical flow calculation module 120 further calculates dense optical flow based on the extracted ROI regions in the work video frame image, extracting pixels with significant motion amplitude from the dense optical flow.

[0150] The cylinder rotation axis model establishment module 130 in this system is configured to interact with the optical flow calculation module 120, and can establish the cylinder rotation axis model based on the sparse optical flow calculation results of the optical flow calculation module 130.

[0151] The vertical distance calculation module 140 in this system is configured to interact with the cylinder rotation axis model establishment module 130 and the optical flow calculation module 120. It can calculate the vertical distance from the pixel point with significant motion amplitude in the dense optical flow extracted by the optical flow calculation module to the rotation axis based on the constructed cylinder rotation axis model.

[0152] The drum angular velocity calculation module 150 in this system is configured to interact with the vertical distance calculation module 140. It can filter out effective motion points in the dense optical flow that meet the requirements of motion amplitude and distance based on the calculation results of the vertical distance calculation module 140. After the number of effective motion points meets the quantity area requirements, the overall drum angular velocity is calculated accordingly.

[0153] The outlier smoothing module 160 in this system is configured to perform outlier detection and data balancing when the drum angular velocity calculation module 150 performs overall drum angular velocity calculation.

[0154] The following describes the specific implementation scheme of each functional module in this system.

[0155] When the ROI region extraction module 110 in this system extracts the ROI region corresponding to the roller region in the video frame image, it first selects and determines the ROI region corresponding to the roller region in the video frame image, and then automatically updates the ROI position in consecutive frames to track the roller movement in real time.

[0156] When calculating sparse optical flow for a Region of Interest (ROI), the optical flow calculation module 120 in this system selects local feature points within the ROI and performs sparse optical flow tracking to obtain the displacement information of the feature points between consecutive frames.

[0157] When calculating dense optical flow for a region of interest (ROI), the optical flow calculation module 120 performs dense optical flow calculation on all pixels in the ROI region, determines the motion vector of each pixel, and calculates the motion amplitude and orientation angle of each pixel.

[0158] The cylinder rotation axis model establishment module 130 in this system extracts a set of feature points with small motion amplitudes based on sparse optical flow results, thereby determining the position of the cylinder rotation center and further establishing a cylinder rotation axis model according to the preset direction of the cylinder.

[0159] In this system, the drum angular velocity calculation module 150 divides the ROI region into several small segments in the horizontal direction when calculating the overall drum angular velocity. It calculates the average radius and average linear velocity in each segment and derives the local angular velocity of each segment. Based on this, it selects several values ​​of the local angular velocities and performs mean fusion to form a preliminary estimate of the overall drum angular velocity.

[0160] The resulting optical flow-based drum rotation speed detection system can achieve non-contact rotation speed monitoring entirely based on video images in complex downhole lighting, dust, and obstruction environments, without the need for additional sensors or any physical markers.

[0161] The following examples illustrate this further.

[0162] In this example, the drum speed detection software system based on the optical flow method provided by the present invention is first deployed. The system can be deployed on the coal mining machine working underground on site, or it can be deployed in the back-end control room.

[0163] Furthermore, video monitoring equipment capable of acquiring the operating status of the coal mining machine drum is deployed on-site. The specific deployment plan and equipment configuration can be determined according to actual needs. This video monitoring equipment can transmit the acquired video images of the coal mining machine drum's operation to the drum speed detection software system in real time.

[0164] Based on the aforementioned deployment, the implementation process for detecting the drum rotation speed in this example is as follows: Figure 3 As shown:

[0165] (1) System initialization, loading video acquisition module, and preparing image processing environment.

[0166] (2) Next, read the video frames of the coal mining machine and extract the ROI;

[0167] In this step, continuous video frames of the coal mining machine operation site are acquired, and the drum area (ROI) is determined by manual selection at the beginning. Then, the ROI position is automatically updated in the continuous frames using a KCF tracker to track the drum movement in real time and suppress background interference.

[0168] (3) Calculate dense optical flow;

[0169] Within the ROI region, the Farneback method is used to perform dense optical flow calculations on all pixels to obtain the motion vector (horizontal displacement u(x,y) and vertical displacement v(x,y)) for each pixel, and the motion amplitude mag(x,y) and orientation angle ang(x,y) are calculated.

[0170] (4) Calculate the sparse optical flow;

[0171] Meanwhile, local feature points are selected within the ROI region, and sparse optical flow tracking is performed using the Lucas-Kanade method to obtain the displacement information of the feature points between consecutive frames.

[0172] (5) Modeling of the roller axis and estimation of the center of rotation;

[0173] First, based on the sparse optical flow results, a set of feature points with small motion amplitudes is extracted, and the position of the roller rotation center is determined by averaging the geometric centers. Then, a roller axis rotation model is established according to the preset direction of the roller (horizontal or slightly tilted).

[0174] (6) Calculate the distance from the optical flow point to the axis;

[0175] Pixels with significant motion amplitudes are extracted from the dense optical flow, and their vertical distances to the rotation axis are calculated as the basis for local radius estimation. Valid motion points that meet the amplitude and distance requirements are further selected to prepare for subsequent angular velocity calculations.

[0176] (7) Determine whether a certain number of regions exist;

[0177] Count the number of valid motion points: If the number of valid points is insufficient (i.e., the motion information is sparse), backtrack and recalculate the dense optical flow and sparse optical flow; if the number of valid points is sufficient, continue with subsequent processing.

[0178] (8) Calculate the effective mean and smooth out the outliers

[0179] First, the effective optical flow points within the ROI region are divided into several small segments in the horizontal direction. The average radius and average linear velocity are calculated in each segment, and the local angular velocity of the segment is derived. Then, the top-5 maximum values ​​of the local angular velocities are selected and averaged to form a preliminary estimate of the overall roller angular velocity.

[0180] During this process, median filtering is applied simultaneously, and outliers are removed by neighborhood averaging detection, thus achieving smooth output of angular velocity data.

[0181] (9) End

[0182] The output provides a stable and continuous data stream of drum angular velocity, providing a reliable basis for subsequent coal mining machine status monitoring and intelligent control.

[0183] In response to the optical flow-based drum speed detection scheme provided in this invention, this embodiment of the invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the steps of the above-described optical flow-based drum speed detection method.

[0184] This invention also provides a processor for running a program, wherein the program executes the steps of the above-described roller speed detection method based on optical flow.

[0185] This invention also provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. The program code is loaded and executed by the processor to implement the steps of the above-described roller speed detection method based on optical flow.

[0186] The present invention also provides a computer program product, which, when executed on a data processing device, is adapted to perform the steps of the above-described roller speed detection method based on optical flow.

[0187] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0188] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0189] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0190] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0192] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0193] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0194] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0195] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0196] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0197] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0198] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting drum rotation speed based on optical flow, characterized in that, The method first acquires video images of the coal mining machine drum operation, and then extracts the ROI region corresponding to the drum area from the video frame image; Next, sparse optical flow is calculated based on the ROI regions in the extracted operation video frame images, and a drum rotation axis model is constructed accordingly. Next, dense optical flow is calculated based on the ROI regions in the extracted operation video frame images. Pixels with significant motion amplitudes are extracted from the dense optical flow, and their vertical distance to the rotation axis is calculated based on the constructed drum rotation axis model. Next, effective motion points that meet the requirements for motion amplitude and distance are selected, and after the number of effective motion points meets the quantity range requirements, the overall roller angular velocity is calculated accordingly.

2. The drum rotation speed detection method based on optical flow method according to claim 1, characterized in that, When extracting the ROI region corresponding to the roller region in the video frame image, the detection method first initially selects and determines the ROI region corresponding to the roller region in the video frame image, and then automatically updates the ROI position in consecutive frames to track the roller movement in real time.

3. The drum rotation speed detection method based on optical flow method according to claim 1, characterized in that, When calculating sparse optical flow for the ROI region, the detection method selects local feature points within the ROI region and performs sparse optical flow tracking to obtain the displacement information of the feature points between consecutive frames.

4. The drum rotation speed detection method based on optical flow method according to claim 1, characterized in that, The detection method extracts a set of feature points with small motion amplitudes based on sparse optical flow results, thereby determining the position of the roller rotation center and further establishing a roller rotation axis model based on the preset direction of the roller.

5. The drum rotation speed detection method based on optical flow method according to claim 1, characterized in that, When calculating dense optical flow for the ROI region, the detection method performs dense optical flow calculation on all pixels in the ROI region, determines the motion vector of each pixel, and calculates the motion amplitude and orientation angle of each pixel.

6. The drum rotation speed detection method based on optical flow method according to claim 1, characterized in that, After the number of effective moving points meets the requirements of the number of regions, the detection method first divides the ROI region into several small segments in the horizontal direction, calculates the average radius and average linear velocity in each segment, and derives the local angular velocity of each segment from this. Then, several values ​​of the local angular velocities are selected and averaged to form a preliminary estimate of the overall roller angular velocity.

7. The drum rotation speed detection method based on optical flow method according to claim 1, characterized in that, The detection method also includes outlier detection and data balancing steps in the overall drum angular velocity calculation process.

8. A drum rotation speed detection system based on optical flow method, characterized in that, The system includes: The ROI region extraction module extracts the ROI region corresponding to the drum area from the video frame image of the coal mining machine drum operation. The optical flow calculation module calculates sparse optical flow based on the ROI regions in the job video frame image extracted by the ROI region extraction module; the optical flow calculation module also calculates dense optical flow based on the extracted ROI regions in the job video frame image, and extracts pixels with significant motion amplitude in the dense optical flow. The cylinder rotation axis model establishment module establishes the cylinder rotation axis model based on the sparse optical flow calculation results of the optical flow calculation module. A vertical distance calculation module calculates the vertical distance from pixels with significant motion amplitude in the dense optical flow extracted by the optical flow calculation module to the rotation axis based on the constructed drum rotation axis model. The roller angular velocity calculation module filters out effective motion points in the dense optical flow that meet the requirements of motion amplitude and distance based on the calculation results of the vertical distance calculation module, and performs overall roller angular velocity calculation after the number of effective motion points meets the quantity area requirements.

9. The drum speed detection system based on optical flow method according to claim 8, characterized in that, When the ROI region extraction module extracts the ROI region corresponding to the roller region in the video frame image, it first selects the ROI region corresponding to the roller region in the video frame image, and then automatically updates the ROI position in consecutive frames to track the roller movement in real time.

10. The drum rotation speed detection system based on optical flow method according to claim 8, characterized in that, When calculating sparse optical flow for a Region of Interest (ROI), the optical flow calculation module selects local feature points within the ROI and performs sparse optical flow tracking to obtain the displacement information of the feature points between consecutive frames.

11. The drum rotation speed detection system based on optical flow method according to claim 8, characterized in that, When calculating dense optical flow for a Region of Interest (ROI), the optical flow calculation module performs dense optical flow calculation on all pixels in the ROI, determines the motion vector of each pixel, and calculates the motion amplitude and orientation angle of each pixel.

12. The drum rotation speed detection system based on optical flow method according to claim 8, characterized in that, The cylinder rotation axis model establishment module extracts a set of feature points with small motion amplitudes based on sparse optical flow results, thereby determining the position of the cylinder rotation center and further establishing a cylinder rotation axis model according to the preset direction of the cylinder.

13. The drum rotation speed detection system based on optical flow method according to claim 8, characterized in that, When calculating the overall drum angular velocity, the drum angular velocity calculation module divides the ROI region into several small segments in the horizontal direction, calculates the average radius and average linear velocity in each segment, and derives the local angular velocity of each segment. Based on this, several values ​​of the local angular velocities are selected and averaged to form a preliminary estimate of the overall drum angular velocity.

14. The drum rotation speed detection system based on optical flow method according to claim 8, characterized in that, The drum rotation speed detection system also includes an outlier smoothing module, which can perform outlier detection and data balancing when the drum angular velocity calculation module performs overall drum angular velocity calculation.

Images