Mine operation behavior visual monitoring system based on machine vision
By constructing a local structural tensor and a dense optical flow field, and combining the target monitoring area with the core axis of danger, the convergence potential energy is calculated, and the potential energy threshold is adaptively adjusted, thus solving the problem of false alarms in the underground environment and realizing safe and reliable mine operation monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JINAN FUSHEN HINGGAN TECH CO LTD
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-05
AI Technical Summary
Existing visual monitoring technologies cannot effectively distinguish between the actual displacement of a solid human body and the random disturbance of fluid dust in underground coal mine environments, leading to frequent false alarms and failing to provide effective safety guarantees.
By constructing a local structural tensor, calculating dense optical flow field and coherent motion tension, and combining the target monitoring area with the core axis of danger, the convergence potential energy is calculated, and the potential energy activation threshold is adaptively adjusted to generate explosion-proof interlock control commands.
It effectively distinguishes between physical movement and environmental disturbances, reduces false alarm rates, provides safe and reliable monitoring of mining operations, ensures that equipment can be shut down when necessary, and avoids large-scale shutdowns caused by false alarms.
Smart Images
Figure CN122157166A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of motion capture technology. More specifically, this invention relates to a machine vision-based visual monitoring system for mining operations. Background Technology
[0002] In the field of underground mine safety production, machine vision-based work behavior monitoring systems are widely deployed to detect whether workers have illegally entered dangerous machinery operating areas.
[0003] Existing visual monitoring technologies typically employ background subtraction algorithms or basic dense optical flow algorithms for on-site moving target detection: acquiring continuous monitoring image sequences, establishing a static background distribution model in time using a Gaussian mixture model, then calculating the pixel grayscale difference between the newly introduced image frame and the established background distribution model; if the calculated pixel grayscale difference is greater than a preset difference threshold, the system marks the pixel as a foreground moving target, further merging the scattered pixels into connected components through morphological erosion and dilation operations, and finally determining whether to trigger a line-crossing alarm based on the area of the connected components.
[0004] However, in the actual physical environment of underground coal mines, the aforementioned existing technologies have serious limitations: the enclosed underground roadway space is filled with high concentrations of suspended coal dust particles year-round, and the only lighting source at the working face is the directional headlamps worn by the workers and the spotlights carried by the mining equipment; the high concentration of suspended coal dust will produce Mie scattering physical effects on the penetrating light, forming optical haze with random spatial fluctuations and alternating brightness in the field of view of the camera; moving point light sources will produce changing reflective highlights and shadows on the rough coal walls and metal equipment surfaces; the underlying logic of traditional background difference algorithms and basic optical flow algorithms is based on the assumption of constant optical brightness, which cannot cope with the global pixel fluctuations caused by nonlinear physical scattering and light source displacement; the system is very prone to misidentifying the random flow scattering of dust, the flickering of lights, and the movement of equipment shadows as dangerous movements of actual workers, resulting in the final extracted foreground connected regions being fragmented in the image and having a drastically increased total area. Because existing technology lacks the physical ability to distinguish between the actual displacement of a solid human body and the random disturbance of fluid dust, and cannot adapt to the dynamic changes in dust concentration, the explosion-proof control box connected to it in the mine receives erroneous interlocking shutdown signals extremely frequently, causing large-scale false alarms. Ultimately, this causes the visual safety monitoring system to fail in the harsh underground environment, and it cannot provide substantial safety guarantees for mining operations. Summary of the Invention
[0005] To address the technical problem of existing technologies lacking the ability to physically distinguish between the actual displacement of solid human bodies and the random disturbance of fluid dust, leading to large-scale false alarms, this invention provides a machine vision-based visual monitoring system for mining operations. The system includes the following modules: a multi-dimensional physical feature acquisition module, used to acquire continuous image frames of the underground mining face; constructing a local structure tensor for each pixel based on the spatial gradient distribution within the neighborhood of each pixel in the current image frame; performing eigenvalue decomposition on the local structure tensor and extracting the maximum and minimum eigenvalues; obtaining the dense optical flow field of the current image frame to obtain the optical flow velocity components in the horizontal and vertical directions for each pixel; and a coherent motion tension derivation module, used to utilize the horizontal and vertical optical flow velocity components... The flow velocity component is used to obtain the divergence at each pixel, and the coherent motion tension of each pixel is obtained based on the maximum and minimum eigenvalues and the divergence. A convergence potential energy calculation module is used to calibrate the target monitoring area and the hazardous core axis corresponding to the polygonal mask in the two-dimensional pixel coordinate system of the reference image frame under dust-free conditions. Based on the spatial Euclidean distance from each pixel within the target monitoring area to the hazardous core axis and the coherent motion tension of each pixel, the convergence potential energy formed by the superposition of the entire target monitoring area is obtained. An explosion-proof interlocking command generation module is used to determine a dynamic potential energy activation threshold based on the total number of pixels within the target monitoring area. The convergence potential energy is compared and verified with the potential energy activation threshold, and an explosion-proof interlocking control command is generated when the convergence potential energy is greater than the potential energy activation threshold.
[0006] Preferably, the step of constructing the local structure tensor of each pixel based on the spatial gradient distribution in the neighborhood of each pixel in the current image frame includes: extracting the spatial gradient in the horizontal direction and the spatial gradient in the vertical direction of each pixel in the current image frame; and performing local weighted integration on the spatial gradient in the horizontal direction and the spatial gradient in the vertical direction based on a preset Gaussian smoothing window to construct the local structure tensor.
[0007] Preferably, the formula for calculating the local structure tensor is: In the formula, any pixel in the current image frame is taken as the target pixel. It is the local structure tensor of the target pixel in the current image frame; This represents the spatial gradient of the neighboring pixels of the target pixel in the current image frame in the horizontal direction. This represents the spatial gradient of the neighboring pixels of the target pixel in the current image frame in the vertical direction. These are the weighting coefficients for the Gaussian smoothing window; This represents a weighted integral summation operation performed within the neighborhood of the target pixel.
[0008] Preferably, the divergence at each pixel is obtained using the horizontal and vertical optical flow velocity components, including: calculating the partial derivative of the horizontal optical flow velocity component of the pixel in the horizontal direction along the abscissa; calculating the partial derivative of the vertical optical flow velocity component of the pixel in the vertical direction along the ordinate; and summing the two partial derivatives to obtain the divergence at the pixel.
[0009] Preferably, obtaining the coherent motion tension of each pixel based on the maximum eigenvalue, minimum eigenvalue, and divergence includes: In the formula, The coherent motion tension of pixels in the current image frame; The maximum feature value of the pixel; The minimum feature value of a pixel; To prevent extremely small constants with a denominator of zero; It is a natural constant; This represents the absolute value operation; This represents the divergence at a pixel.
[0010] Preferably, the process of calibrating the target monitoring area and the hazardous core axis corresponding to the polygonal mask in the two-dimensional pixel coordinate system of the reference image frame under dust-free conditions includes: during the system edge computing deployment phase, controlling the mine explosion-proof camera to acquire the reference image frame under dust-free conditions; in the two-dimensional pixel coordinate system of the reference image frame, calibrating multiple pixels as vertices to close and generate a polygonal mask based on the physical visual contour of the on-site mining equipment, defining the set of all pixels covered by the polygonal mask as the target monitoring area; calibrating the starting and ending pixels along the physical extension direction of the rotating component, and fitting a straight line analytical expression based on the coordinates of the two pixels. Define it as the dangerous core axis. , , The linear analytical coefficients of the axis of the dangerous core are denoted as . The x-coordinate of the pixel; y is the ordinate of the pixel.
[0011] Preferably, the step of obtaining the convergent potential energy formed by the superposition of the entire target monitoring area based on the spatial Euclidean distance from each pixel within the target monitoring area to the dangerous core axis and the coherent motion tension of each pixel includes: In the formula, To gather potential energy across the entire target monitoring area; Indicates the target monitoring area Iterate through and sum all pixels within the range; The coherent motion tension of the pixel; It is a natural constant; The spatial Euclidean distance from the pixel to the axis of the danger core; This represents the maximum diagonal length of the region.
[0012] Preferably, determining the dynamic potential energy activation threshold based on the total number of pixels in the target monitoring area includes: counting the total number of pixels contained in the target monitoring area; calculating the product of a preset baseline potential energy density and the total number of pixels to obtain the dynamic potential energy activation threshold; the value range of the baseline potential energy density is 0.15 to 0.35.
[0013] Preferably, the method for obtaining the dense optical flow field of the current image frame includes: using the Farneback dense optical flow algorithm based on polynomial expansion to analyze the neighborhood relative displacement of each pixel between the current image frame and the adjacent previous image frame, and calculating the dense optical flow field of the current image frame, wherein the dense optical flow field includes the optical flow velocity component in the horizontal direction and the optical flow velocity component in the vertical direction of each pixel.
[0014] Preferably, generating an explosion-proof interlock control command when the converged potential energy is greater than the potential energy activation threshold includes: determining that a dangerous unauthorized intrusion has occurred when the converged potential energy is greater than the potential energy activation threshold, generating the explosion-proof interlock control command; and sending the explosion-proof interlock control command to the corresponding explosion-proof control equipment to cut off the main power supply circuit of the target mining equipment and achieve power outage and shutdown.
[0015] The beneficial effects of this invention are as follows:
[0016] This invention constructs a local structural tensor by extracting the spatial gradient of pixels in the spatial domain and performing eigenvalue decomposition to capture the eigenvalues characterizing the saliency of rigid body edges. At the same time, it calculates the divergence of the dense optical flow field in the time domain to reflect the fluid's visual expansion rate, and then merges the two to derive coherent motion tension. This operation establishes a multi-dimensional physical filtering mechanism at the underlying data level, effectively distinguishing between the real displacement of the human body that maintains the solid boundary and the suspended dust and light and shadow disturbances that exhibit fluid divergence characteristics. Thus, it suppresses false motion noise caused by complex downhole environments while preserving the characteristics of physical motion.
[0017] Furthermore, this invention obtains the target monitoring area and the core axis of danger, and fuses the spatial Euclidean distance from the pixel to the core axis of danger with the aforementioned coherent motion tension to determine the convergence potential energy. This process introduces spatial defense depth perception, endowing spatial distance with a gravitational attenuation effect, so that the substantial physical motion closer to the core of the mechanical danger source can generate a larger potential energy disturbance, while the motion characteristics in the outer safe area are effectively weakened. This constructs a gradient defense force field with the danger source as the absolute gravitational center, solving the problem of danger misjudgment caused by the indiscriminate attention to traditional global features.
[0018] Finally, this invention adaptively determines a dynamic potential energy activation threshold based on the total number of pixels within the target monitoring area. It automatically accommodates and offsets the additional environmental noise potential energy introduced by differences in camera deployment angle or monitoring area. The converged potential energy of the entire area is compared and verified with this dynamic potential energy activation threshold to generate explosion-proof interlock control commands. This adjudication mechanism forcibly intercepts and filters out the slight potential energy fluctuations caused by residual noise from normal equipment oscillations. It only triggers shutdown intervention when personnel actually violate the physical safety threshold, completing a closed-loop safety defense, eliminating the interference of harsh physical environments on visual monitoring, reducing the frequent false alarm rate, and providing safety assurance for mining operations. Attached Figure Description
[0019] Figure 1 This is a schematic diagram illustrating the system block diagram of the machine vision-based visual monitoring system for mining operations in this invention. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0022] This invention provides a machine vision-based visual monitoring system for mining operations. For example... Figure 1 As shown, the machine vision-based visual monitoring system for mining operations includes a multi-dimensional physical feature acquisition module 100, a coherent motion tension derivation module 200, a convergent potential energy calculation module 300, and an explosion-proof interlock command generation module 400, which are described in detail below.
[0023] Multidimensional physical feature acquisition module 100: used to acquire continuous image frames, construct local structure tensors and extract feature values, and simultaneously calculate dense optical flow field to obtain temporal displacement vectors.
[0024] It should be noted that the underground mining face in coal mines is a harsh environment, filled with high-density suspended coal dust, and the surfaces of large mining equipment exhibit highly irregular metallic textures and reflective properties. In such a complex physical scene, traditional single-pixel grayscale features are easily submerged by background optical noise and cannot truly reflect the existence state of physical entities. This invention reconstructs the underlying data features from both spatial and temporal dimensions. In the spatial domain, gradient information in orthogonal directions is extracted for each pixel to construct a local structure tensor, and the edge structure saliency unique to physical entities is captured by matrix eigenvalue decomposition. At the same time, a dense optical flow field is calculated in the temporal domain to capture the true dynamic displacement vector of pixels. This operation transforms the chaotic two-dimensional image stream into a multi-dimensional physical data space containing structural tension and motion tendency, laying a data foundation for subsequent differentiation of entity behavior and environmental noise.
[0025] Specifically, video streams are collected in real time by explosion-proof cameras deployed around the underground working face and transmitted to an edge computing analysis server via a mining industrial Ethernet ring network. These streams are then decoded to obtain continuous image frames. For the current image frame within the continuous image frames, the spatial gradient of each pixel in the horizontal direction is extracted. and spatial gradient in the vertical direction .
[0026] Furthermore, based on a preset Gaussian smoothing window, the spatial gradient in the horizontal direction corresponding to all neighboring pixels within the neighborhood of each pixel is calculated. and spatial gradient in the vertical direction Perform local weighted integration to construct the local structure tensor for each pixel in the current image frame. The formula for its calculation is:
[0027]
[0028] In the formula, It is the local structure tensor of the target pixel in the current image frame; This represents the spatial gradient of the neighboring pixels of the target pixel in the current image frame in the horizontal direction. This represents the spatial gradient of the neighboring pixels of the target pixel in the current image frame in the vertical direction. These are the weighting coefficients for the Gaussian smoothing window; This represents a weighted integral summation operation performed within the neighborhood of the target pixel.
[0029] In this embodiment, the preset size of the Gaussian smoothing window is 5×5, and the size of the neighborhood of each pixel is the same as the preset size of the Gaussian smoothing window.
[0030] Furthermore, a local structure tensor is constructed. Then, the local structure tensor Perform eigenvalue decomposition to extract the maximum eigenvalue corresponding to the pixel. with minimum eigenvalue .
[0031] Among them, the local structure tensor By performing a weighted integral on the spatial gradient in orthogonal directions, discrete pixel edge information is transformed into matrix parameters with directionality and structural tension; when a pixel is located at the edge of a rigid texture on the surface of the mining equipment, the local structure tensor... Large eigenvalues are produced after matrix decomposition. When a pixel is located in an irregular area of suspended coal dust, its gradients in all directions are weak and symmetrical, resulting in the largest eigenvalue being decomposed. with minimum eigenvalue The values are all relatively small, and the discrete distribution of these eigenvalues highlights the salience of rigid body structures in space.
[0032] Furthermore, a Farneback dense optical flow algorithm based on polynomial expansion is employed to analyze the relative displacement of each pixel in the neighborhood between the current image frame and the adjacent previous image frame, thereby calculating the dense optical flow field of the current image frame. The dense optical flow field Includes the optical flow velocity component in the horizontal direction for each pixel. Optical flow velocity component in the vertical direction .
[0033] Coherent motion tension derivation module 200: Used to calculate the divergence of dense optical flow field and derive the coherent motion tension of each pixel by combining eigenvalues.
[0034] It should be noted that, under the combined influence of strong light flashes and pervasive coal dust underground, the optical scattering of dust appears visually as an irregularly radiating fluid expansion. Meanwhile, the workers, as solid physical objects, maintain strict solid rigid body boundaries in their limb extensions and torso displacements. Faced with these two motion phenomena that highly overlap in numerical amplitude but are drastically different in physical form, this invention analyzes the fluid dynamics of the moving matter, introducing divergence—a partial derivative feature—to reflect the visual expansion rate of the optical flow field. This divergence is then fused with the proportion of characteristic values representing the salience of rigid body edges to derive a coherent motion tension index. This operation establishes a filter at the pixel level to attenuate and suppress the divergence of fluid expansion while accurately preserving the characteristic parameters representing rigid body motion, thus avoiding false motion caused by environmental disturbances.
[0035] Specifically, for dense optical flow fields Each pixel in the image is analyzed using its horizontal optical flow velocity component. Optical flow velocity component in the vertical direction Calculate spatial partial derivatives to obtain dense optical flow field divergence at this pixel :
[0036]
[0037] In the formula, Dense optical flow field The divergence at the target pixel in the current image frame; This represents the optical flow velocity component of the target pixel in the horizontal direction. The optical flow velocity component of the target pixel in the vertical direction; The x-coordinate of the target pixel; The ordinate of the target pixel; For the partial derivative operator performed with respect to the horizontal axis; This is the partial derivative operator performed with respect to the vertical axis.
[0038] Furthermore, based on the maximum feature value of each pixel... Minimum eigenvalue and divergence Calculate the coherent motion tension of each pixel. :
[0039]
[0040] In the formula, The coherent motion tension of the target pixel in the current image frame; The maximum feature value of the target pixel; The minimum feature value of the target pixel; To prevent extremely small constants with a denominator of zero, in this embodiment, ; It is a natural constant; This represents the absolute value operation; This represents the divergence at the target pixel.
[0041] Among them, the maximum eigenvalue of the current pixel in the fractional term. with minimum eigenvalue The ratio represents the probability that the pixel belongs to the edge of a sharp rigid body, with an exponential decay term. As a filter: when the target is a dense solid, its motion on the two-dimensional image plane is mainly translation, with weak local area deformation, and the divergence during the motion is... Approaching 0, causing the exponentially decaying term to... The calculated value approaches 1, at which point the value calculated by the fractional term is proportionally mapped to the coherent motion tension. When the target is optical scattering of dust and light flickering, its fluid expansion characteristics produce a large absolute value of divergence, causing the exponential decay term to decay rapidly at this point and approach 0, thereby forcibly suppressing the coherent motion tension at this pixel to zero; thus achieving fidelity in the solid motion and suppression of fluid noise.
[0042] Converging potential energy calculation module 300: Used to establish the target monitoring area and the core axis of danger, and calculate the convergent potential energy within the area by combining spatial Euclidean distance and coherent motion tension.
[0043] It should be noted that the spaces in underground coal mine electromechanical chambers or fully mechanized mining faces are extremely narrow and confined. The wide-angle field of view of a single camera often simultaneously includes legitimate patrol actions in the outer safety passages, as well as high-risk intrusion behaviors close to high-speed rotating machinery. If all motion features in the entire image are treated equally, the system will lose its perception of the spatial defense depth and confuse safe operations with dangerous violations. In order to reshape the spatial defense hierarchy of the visual image, this invention establishes a target monitoring area and a core danger axis in a two-dimensional pixel plane, introduces a physical potential energy field, calculates the spatial Euclidean distance from each pixel to the mechanical hazard source, and combines this spatial distance as a gravity attenuation factor with coherent motion tension to deduce the convergence potential energy. This operation allows substantial physical movements closer to the core danger to generate greater potential energy disturbances, while peripheral safety actions are weakened, constructing a gradient defense force field with the hazard source as the absolute center of gravity.
[0044] Specifically, during the system edge computing deployment phase, the explosion-proof mining camera is controlled to acquire a reference image frame under dust-free conditions. In the two-dimensional pixel coordinate system of the reference image frame, based on the physical visual contour of the on-site mining equipment, multiple pixels are marked as vertices to form a polygonal mask. The set of all pixels covered by this polygonal mask is defined as the target monitoring area. ; Mark the start and end points of two pixels along the physical extension direction of the rotating component, and fit an analytical expression of a straight line based on the coordinates of these two pixels. Define it as the dangerous core axis .
[0045] Furthermore, targeting the monitoring area For each pixel within the core, calculate the distance from that pixel to the critical core axis. Spatial Euclidean distance :
[0046]
[0047] In the formula, The axis from the target pixel to the dangerous core. Spatial Euclidean distance; , , To calibrate the fitted critical core axis The coefficients of the linear analytical expression; The x-coordinate of the target pixel; The vertical coordinate of the target pixel.
[0048] Furthermore, by combining the calculated coherent motion tension of each pixel, the convergent potential energy of the entire target monitoring area is calculated. The formula for its calculation is:
[0049]
[0050] In the formula, To gather potential energy across the entire target monitoring area; Indicates the target monitoring area Iterate through and sum all pixels within the range; The coherent motion tension of the target pixel; It is a natural constant; The axis from the target pixel to the dangerous core. Spatial Euclidean distance; The maximum diagonal length of the area is equal to the defined target monitoring area. The maximum diagonal pixel span in pixel coordinates.
[0051] Among them, the exponent term Each pixel is assigned a spatial Euclidean distance. Gravity collapse effect: When the pixel point where personnel movement occurs is far from the axis of the danger core. When they are close together, the spatial Euclidean distance between corresponding pixels As the exponent term approaches 0, its value approaches 1, resulting in coherent motion tension at that pixel. Completely preserved and included in the convergence potential energy When movement occurs within the target monitoring area When dealing with edge pixels, due to the spatial Euclidean distance... The increase in its impact on convergence potential energy The numerical contribution decreases exponentially, thus constructing a protective field with a gradient distribution.
[0052] Explosion-proof interlock command generation module 400: used to compare and verify the converged potential energy with the potential energy activation threshold, and generate explosion-proof interlock control commands based on the decision result.
[0053] It should be noted that the converged potential energy obtained through multi-dimensional fusion calculation in the spatiotemporal domain is essentially still a simulated continuous parameter that fluctuates continuously with mechanical vibrations and weak airflow disturbances on site. However, the execution unit of the mine's bottom-level safety interlocking system must receive a switch signal with clear instructions and distinct boundaries. Therefore, this invention introduces a potential energy activation threshold with double-ended constraints as a safety red line, performs numerical comparison and verification of the converged potential energy across the entire domain, forcibly intercepts and filters out the slight potential energy fluctuations caused by residual noise from normal equipment vibrations, and sets an explosion-proof interlocking control command that is only triggered when the potential energy bursts and breaks through this physical critical red line, thus completing a closed loop of safety defense.
[0054] Specifically, the target monitoring area is marked in the reference image frame. Then, the control system statistically analyzes the target monitoring area. Total number of pixels contained The preset reference potential energy density within the calculation system. With the target monitoring area Total number of pixels contained The product of these values yields the dynamic potential energy activation threshold from the current deployment perspective. This calculation formula constructs the potential energy activation threshold. With the target monitoring area Linear mapping relationship of pixel area: when the target monitoring area The increased area occupied in the image leads to a higher total number of pixels. When the potential energy activation threshold increases, This is proportionally increased, thus automatically accommodating and offsetting the additional environmental noise potential energy introduced by the increased area; when the target monitoring area... When the area shrinks, the potential energy activation threshold The system is then proportionally reduced to ensure it remains highly sensitive to even the slightest human intrusion; this fundamentally eliminates the interference of differences in camera installation angles on the final security decision.
[0055] Wherein, the reference potential energy density This is a constant parameter characterizing the maximum safe noise floor that a unit pixel can withstand. If this value is set too small, the unavoidable minor air pressure disturbances in the underground environment will cause the tension of individual pixels to suddenly exceed the limit, resulting in frequent false shutdowns of the mine explosion-proof control box. If this value is set too large, the system's tolerance for substantial personnel intrusion will be too high, leading to a lag in the safety defense line. Therefore, this invention sets its value range to 0.15 to 0.35, and in this embodiment, it is locked at 0.25 to ensure a sensitive physical interception response to real personnel intrusion while cutting off the complex noise floor transmission path.
[0056] Furthermore, within each calculation cycle, the control system summarizes the convergence potential energy obtained from the current image frame. With the calculated potential energy activation threshold Perform real-time numerical comparison and verification; if the comparison and verification result is convergent potential energy... Less than or equal to the potential energy activation threshold If the system determines that the current monitoring area is in a compliant operating state, the system will not output any intervention commands; if the verification result is convergence potential energy Greater than the potential energy activation threshold When the system determines that a dangerous or unauthorized intrusion has occurred, it immediately generates and outputs an explosion-proof interlock control command. This command is then sent to the corresponding explosion-proof control box via the mining industrial Ethernet ring network, cutting off the main power supply circuit of the target mining equipment and shutting down the mechanical components.
Claims
1. A machine vision-based visual monitoring system for mining operations, characterized in that, include: The multi-dimensional physical feature acquisition module is used to acquire continuous image frames of the underground working face in the mine; Based on the spatial gradient distribution in the neighborhood of each pixel in the current image frame, a local structure tensor for each pixel is constructed. The local structure tensor is then decomposed into eigenvalues, and the maximum and minimum eigenvalues are extracted. Obtain the dense optical flow field of the current image frame to obtain the optical flow velocity components in the horizontal and vertical directions for each pixel; The coherent motion tension derivation module is used to obtain the divergence at each pixel by utilizing the optical flow velocity components in the horizontal and vertical directions, and to obtain the coherent motion tension of each pixel based on the maximum eigenvalue, minimum eigenvalue, and divergence. The convergence potential energy calculation module is used to calibrate the target monitoring area and the core axis of danger corresponding to the polygonal mask in the two-dimensional pixel coordinate system of the reference image frame in the absence of dust obstruction. Based on the spatial Euclidean distance from each pixel within the target monitoring area to the core axis of the danger and the coherent motion tension of each pixel, the convergent potential energy formed by the superposition of the entire target monitoring area is obtained. The explosion-proof interlock command generation module is used to determine a dynamic potential energy activation threshold based on the total number of pixels in the target monitoring area; compare and verify the converged potential energy with the potential energy activation threshold; and generate an explosion-proof interlock control command when the converged potential energy is greater than the potential energy activation threshold.
2. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, The construction of the local structure tensor for each pixel based on the spatial gradient distribution in the neighborhood of each pixel in the current image frame includes: Extract the spatial gradient of each pixel in the current image frame in the horizontal and vertical directions. The local structure tensor is constructed by locally weighting the spatial gradients in the horizontal and vertical directions based on a preset Gaussian smoothing window.
3. The machine vision-based visual monitoring system for mining operations according to claim 2, characterized in that, The formula for calculating the local structure tensor is: ； In the formula, any pixel in the current image frame is taken as the target pixel. It is the local structure tensor of the target pixel in the current image frame; This represents the spatial gradient of the neighboring pixels of the target pixel in the current image frame in the horizontal direction. This represents the spatial gradient of the neighboring pixels of the target pixel in the current image frame in the vertical direction. These are the weighting coefficients for the Gaussian smoothing window; This represents a weighted integral summation operation performed within the neighborhood of the target pixel.
4. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, Using the horizontal and vertical optical flow velocity components, the divergence at each pixel is obtained, including: Calculate the partial derivative of the horizontal optical flow velocity component of the pixel with respect to the horizontal axis. Calculate the partial derivative of the vertical optical flow velocity component of a pixel with respect to the ordinate direction; Summing the two partial derivatives yields the divergence at the pixel.
5. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, Based on the maximum eigenvalue, minimum eigenvalue, and divergence, the coherent motion tension of each pixel is obtained, including: ； In the formula, The coherent motion tension of pixels in the current image frame; The maximum feature value of the pixel; The minimum feature value of a pixel; To prevent extremely small constants with a denominator of zero; It is a natural constant; This represents the absolute value operation; This represents the divergence at a pixel.
6. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, The process of calibrating the target monitoring area and the core axis of danger corresponding to the polygonal mask in the two-dimensional pixel coordinate system of the reference image frame under dust-free conditions includes: During the system edge computing deployment phase, the explosion-proof mining camera is controlled to acquire a reference image frame under dust-free conditions. In the two-dimensional pixel coordinate system of the reference image frame, based on the physical visual contour of the mining equipment on site, multiple pixels are marked as vertices to form a closed polygonal mask. The set of all pixels covered by this polygonal mask is defined as the target monitoring area. Two pixels are marked along the physical extension direction of the rotating component, and an analytical expression of a straight line is fitted based on the coordinates of these two pixels. Define it as the dangerous core axis. , , The linear analytical coefficients of the axis of the dangerous core are denoted as . The x-coordinate of the pixel; y is the ordinate of the pixel.
7. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, The method of obtaining the convergent potential energy formed by the superposition of the entire target monitoring area based on the spatial Euclidean distance from each pixel within the target monitoring area to the dangerous core axis and the coherent motion tension of each pixel includes: ； In the formula, To gather potential energy across the entire target monitoring area; Indicates the target monitoring area Iterate through and sum all pixels within the range; The coherent motion tension of the pixel; It is a natural constant; The spatial Euclidean distance from the pixel to the axis of the danger core; This represents the maximum diagonal length of the region.
8. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, Determining a dynamic potential energy activation threshold based on the total number of pixels within the target monitoring area includes: Count the total number of pixels contained within the target monitoring area; The dynamic potential activation threshold is obtained by multiplying the preset baseline potential energy density by the total number of pixels; the value of the baseline potential energy density ranges from 0.15 to 0.
35.
9. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, The method for obtaining the dense optical flow field of the current image frame includes: The Farneback dense optical flow algorithm based on polynomial expansion is used to analyze the relative displacement of each pixel in the neighborhood between the current image frame and the adjacent previous image frame, and to calculate the dense optical flow field of the current image frame. The dense optical flow field includes the optical flow velocity component in the horizontal direction and the optical flow velocity component in the vertical direction of each pixel.
10. The machine vision-based visual monitoring system for mining operations according to claim 1, characterized in that, When the converged potential energy is greater than the potential energy activation threshold, an explosion-proof interlock control command is generated, including: When the converged potential energy is greater than the potential energy activation threshold, a dangerous unauthorized intrusion is determined to have occurred, and the explosion-proof interlock control command is generated. The explosion-proof interlock control command is sent to the corresponding explosion-proof control equipment to cut off the main power supply circuit of the target mining equipment and achieve power outage and shutdown.