Cluster type whole mine AI video analysis and linkage method

Abstract

The invention discloses a cluster type whole mine AI video analysis and linkage method, and relates to the technical field of artificial intelligence and mine safety monitoring, and the method comprises the following steps: under a unified event time baseline, obtaining light source modulation information and a mine car moving trajectory, constructing a spatial-temporal feature dictionary of a shadow, and generating a shadow profile reference for dynamic comparison; based on shadow profile reference, a multi-view geometric trajectory is collected, a polarization image and a temperature image are fused, falling textures are stripped, and a shadow confidence spectrum is constructed. According to the method, a unified event time baseline and shadow profile reference is constructed, multi-view geometric, polarization and temperature images are fused, the dynamic shadow is identified, and the confidence is quantified; according to the method, stable features are extracted through anti-fact playback to generate misjudgment fingerprints, misjudgment is filtered through cluster consistency arbitration and dual field verification, and finally, shadow dynamics are actively intervened through a phase gating illumination structure, so that closed-loop regulation and control are realized, and the mine monitoring accuracy and safety are improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and mine safety monitoring technology, specifically to a cluster-based whole-mine AI video analysis and linkage method. Background Technology

[0002] "Cluster-based whole-mine AI video analysis and linkage" refers to the deployment of multiple video monitoring points and intelligent analysis terminals throughout the entire mine. Artificial intelligence algorithms are used to identify and comprehensively analyze information such as personnel behavior, equipment operating status, and environmental changes in the monitored footage in real time. Through clustered computing and collaborative mechanisms, the results from different monitoring points are interconnected and dynamically integrated to form a unified risk perception network covering the entire mine. Based on this, once an anomaly occurs in a certain area, the system can immediately trigger linkage control, such as alarms, starting or stopping ventilation equipment, dispatching personnel, or linking with other safety subsystems. This achieves cross-regional intelligent response from decentralized monitoring to overall coordination, significantly improving the real-time, comprehensive, and intelligent level of mine safety management.

[0003] The existing technology has the following shortcomings: In existing technologies, mine video monitoring largely relies on AI visual recognition algorithms to analyze images in real time to identify sudden disasters such as landslides. However, when mine trucks pass through the monitored area, their bodies cast large, rapidly moving shadows on the tunnel walls or ground under strong sunlight. These dynamic shadows are easily confused with the abnormal image features of ground subsidence in video footage. Because existing algorithms cannot effectively distinguish between the transient changes in shadows and the actual disaster signals of rock subsidence, the system often erroneously triggers large-scale evacuation commands. Under such misjudgment conditions, a large number of underground workers will rush into the limited escape routes in a very short time, causing short-term congestion and further triggering secondary safety accidents such as stampedes, thus turning the misjudgment itself into a serious mine safety risk.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a cluster-based, full-mine AI video analysis and linkage method to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a cluster-based whole-mine AI video analysis and linkage method, comprising the following steps: Under a unified event time baseline, light source modulation information and mine car running trajectory are acquired to construct a spatiotemporal feature dictionary of shadows and generate shadow profile references for dynamic comparison. Based on the shadow profile reference, multi-view geometric trajectories are acquired, polarization images and temperature images are fused, fall texture is stripped off, and shadow confidence spectrum is constructed. Based on the shadow confidence spectrum, a counterfactual replay sequence is generated, and image features that remain unchanged in multiple replays are extracted to form a feature fingerprint that triggers misjudgment. Based on feature fingerprints, consensus arbitration is performed in distributed nodes. Timestamp offset, viewpoint difference and light energy transition gradient are fused to calculate consensus score and generate dynamic adjustment threshold for evacuation response. Based on the dynamic adjustment threshold of the evacuation response threshold, a linkage pre-confirmation chain is constructed, a dual-phase verification window is introduced, voice prompts and geomagnetic passage counts are matched, and false judgment commands are eliminated. Under the condition that the chain is operating stably, a time-reversal phase-gated command is generated to drive the dynamic expansion of the extinguishing shadow of the polarization metasurface illumination structure, and the calibration matrix is ​​written back to the event time baseline to form a closed-loop control process.

[0007] Preferably, the steps for generating the shadow profile reference are as follows: Acquire light source modulation information and mine truck trajectory data under a unified event time baseline; The light intensity change, beam diffusion angle and illumination direction are collected based on the light source modulation information, and combined with the linear velocity, acceleration, vehicle posture and illumination projection area boundary of the mine car. Spatiotemporal path samples of shadows are constructed using combined data of light source and mine car trajectory, and the edge position and spatial morphology of shadows in consecutive image frames are recorded. A spatiotemporal feature dictionary of shadows is established based on path samples generated by multiple light sources and mining vehicles. The matching degree between shadow areas in the image and paths in the dictionary is compared in real-time image analysis, and shadow profile references are dynamically generated for subsequent anomaly identification and judgment.

[0008] Preferably, the shadow confidence spectrum construction steps are as follows: Under the constraint of shadow profile reference, multi-view images are acquired and geometric trajectory information is extracted. Spatial paths are calculated and candidate shadow regions are identified through edge detection and triangulation. Polarization and temperature images are acquired within the candidate shadow area, pixel-level alignment is performed, and the images are fused into three data streams: brightness, polarization, and temperature. A joint determination is performed based on the edge intensity, polarization direction consistency, and temperature gradient change rate of each image patch in the fused image stream, and a confidence spectrum image is generated by labeling the confidence level. The confidence spectrum image is output in time series for subsequent region confidence value extraction and recognition weight adjustment before image recognition.

[0009] Preferably, the steps for forming a feature fingerprint triggered by misjudgment are as follows: Based on the shadow confidence spectrum, the time window with drastic fluctuations in confidence values ​​is determined and the corresponding vehicle and illumination parameters are extracted; Multiple counterfactual replay image sequences were reconstructed based on the extracted behavioral parameters, and illumination and temperature normalization were performed. Stable structural regions are extracted from images in each playback sequence, and a feature table of stable image regions is established. Based on the feature table, a three-layer representation structure is constructed, consisting of a region structure index map, a region content feature table, and a contextual information set, to form a feature fingerprint. Feature fingerprints are embedded into the recognition and judgment process for spatial matching and comparison, which is used to identify high-risk areas for misjudgment and suppress the weight of misjudgment triggers.

[0010] Preferably, the steps for generating the dynamic adjustment threshold for the evacuation response are as follows: Using feature fingerprints as constraints, image data with the same image frame time as the main identification node is retrieved synchronously from multiple monitoring nodes, and image feature matching is performed to construct an image matching response matrix; By integrating the timestamp offset, viewpoint geometric difference, and light energy transition characteristics of each node, a consistency score is calculated, and the consistency score is used as a quantitative basis for the credibility of the identification. The threshold for evacuation response is automatically adjusted based on the consistency score, and the threshold boundary is dynamically corrected by combining the historical score distribution curve, so as to achieve continuous optimization and adjustment of the triggering conditions.

[0011] Preferably, the consistency score is calculated using time synchronization, spatial overlap and illumination response consistency as indicators, with weights set for each and weighted fusion is performed. When the score is lower than a preset lower limit, voice prompts are forcibly introduced and redundantly compared with the passage count data.

[0012] Preferably, the steps for constructing the linkage pre-confirmation chain are as follows: Set a verification time window and initiate a pre-confirmation process within the scoring threshold, calling the call center's voice prompt signal and the geomagnetic sensor's passage count information; Extract the main frequency features and keyword content of the audio within the verification time window, complete semantic matching, and simultaneously analyze the passage frequency and passage direction generated by geomagnetic induction to identify whether effective personnel response behavior has been formed. The voice prompt signal and the passage count signal are used as decision inputs and jointly judged with the current score. If both signals pass the verification and the score is not lower than the confidence threshold, the linkage response is allowed to be executed; otherwise, it is marked as an event to be confirmed.

[0013] Preferably, after confirming stable chain operation, the steps for generating time-reversal commands to drive the polarization metasurface illumination extinguishing shadow extension and writing the calibration results back to the event time baseline are as follows: Based on the confirmation chain feedback results, the shadow core region is extracted and a spatial light field modulation target map is generated to construct target data for guiding intervention waveform injection. Initiate a programmable polarization metasurface illumination array to inject a phase-conjugate micro-envelope waveform, which extinguishes the dynamic expansion of shadows through interference. Images were acquired after the intervention and the image feature vectors were reconstructed to generate an illumination intervention response matrix, which was then synchronously written back to a unified event time baseline. The region confidence mapping table is updated based on the response matrix, and the future identification threshold is dynamically adjusted.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows: This invention, by constructing a unified event time baseline and shadow profile reference, combined with multi-view geometric trajectory acquisition and polarization and temperature information fusion processing, achieves for the first time the structural identification and confidence quantification of dynamic shadows. Based on this, it utilizes counterfactual playback to extract stable image features and construct a misjudgment feature fingerprint, ensuring temporal consistency and image stability in the identification process. A cluster consensus arbitration mechanism calculates multi-source scores, and a dual field verification mechanism eliminates false triggering commands, achieving multi-dimensional confirmation of the identification results. Finally, time-reversal phase-gated control of the polarization metasurface illumination structure actively extinguishes the dynamic growth of shadows at the physical level, achieving a breakthrough from image recognition to light field intervention and completely eliminating misjudgment sources. The overall solution constructs a closed-loop control chain from image recognition and result verification to active intervention and calibration write-back, significantly improving the stability, accuracy, and intelligent response capabilities of mine safety monitoring systems, and significantly reducing the risk of ineffective evacuation, congestion, and secondary accidents caused by misjudgments. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0016] Figure 1 This is a flowchart of the cluster-based whole-mine AI video analysis and linkage method of the present invention. Detailed Implementation

[0017] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the description of this disclosure will be more complete and fully convey the concept of the exemplary embodiments to those skilled in the art.

[0018] This invention provides, for example Figure 1 The cluster-based, full-mine AI video analysis and linkage method shown includes the following steps: Under a unified event time baseline, prior information on light source modulation in the mine environment is obtained. Combined with mine car trajectory data, a spatiotemporal feature dictionary of shadows is established. The correspondence between the start boundary and the end boundary of the shadows is recorded in the dictionary, and a shadow profile reference for dynamic comparison is generated. In the process of AI video analysis in mines, in order to solve the problem of video misjudgment caused by dynamic interference of lighting and shadows, it is first necessary to construct a shadow profile reference for dynamic comparison under the premise of a unified event time baseline. This reference is based on the behavior of light sources and the trajectory of mine trucks, and integrates spatial structure and time series. The specific implementation process is as follows: A unified event time baseline was established, and prior information on light source modulation was collected. A high-precision time synchronization signal was generated by deploying a time reference device at the ground control center and synchronously transmitted to all video acquisition points in the roadways via fiber optic channels to ensure that the image frames acquired by each monitoring device maintained millisecond-level time accuracy. Subsequently, at each location with lighting devices, including underground overhead lights, handheld maintenance lights, mine car headlamps, rechargeable mobile lighting units, and personnel-worn area lighting equipment, the operating voltage, brightness curve, on / off cycle, beam diffusion angle, and illumination direction of the light source were recorded. Specifically, multi-channel optical receivers were placed near the equipment, and irradiance changes were collected at 20-millisecond sampling intervals. These changes were then timestamped and normalized with a reference brightness to obtain the brightness modulation curve of each light source per unit time. Furthermore, to recreate the natural background light disturbance caused by non-human-controlled light changes, a three-axis light intensity detection device was deployed in an area of ​​the mine without lighting equipment or vehicle traffic. This device continuously monitored the ambient light intensity and archived the data, serving as the baseline background for evaluating global brightness variations in subsequent modeling.

[0019] The process involves acquiring the mine car's trajectory data and combining it with light source modulation information for analysis. An inertial measurement unit (IMU), including a gyroscope and accelerometer, is installed on the mine car body, along with a real-time positioning system. This system receives signals emitted by positioning tags carried by the mine car via an ultra-wideband positioning base station, allowing for real-time calculation of the mine car's coordinates within the tunnel space. Within each time slice, the mine car's linear velocity, acceleration, direction of travel, body attitude, and steering angle are recorded. Combined with the illumination parameters of the overhead lighting fixtures, including the fixture's height from the ground, fixed angle, luminous cone coverage area, and the direction vector of the luminous surface center point, the precise illumination area boundaries of the headlight beam on the tunnel walls, ceiling, and ground are calculated. By superimposing the changes in the mine car's trajectory with the movement trend of the illuminated area, the change in shadow position caused by the mine car's movement can be calculated for each time period. Specifically, during sudden speed changes or turns, roll and yaw angle data should be additionally introduced to correct the illumination direction vector, eliminating beam offset errors caused by the vehicle's inertia. Ultimately, a physical correspondence between the light source and the illuminated surface is established under the guidance of the mine car's operation, providing a foundation for subsequent identification of the shadow space boundary.

[0020] Based on the combination of light source modulation curves and mine car trajectory data, a spatiotemporal feature dictionary of shadows is constructed. This dictionary is established along a unified time baseline, using each light source-vehicle combination as an index unit. The input conditions are set as the time, illumination angle, illuminated surface features, and mine car posture. The output content is the spatial expansion path of the shadow in the image sequence, recording the edge coordinates of its projection onto the ground, wall, and roof frame by frame. By performing spatiotemporal compression encoding on shadow samples generated by multiple mine cars under different operating speeds, turning radii, illumination intensities, and tunnel structure conditions, the shadow boundary changes in consecutive frames are modeled as a continuous and predictable path in three-dimensional space. This path includes not only the starting and ending boundary positions of the shadow but also the specific form of shadow changes in each frame between the start and end. In the specific encoding process, to improve recognition accuracy, each shadow path can be discretized into points with equal time intervals, and physical parameters such as the actual incident angle of illumination, the reflectivity of the tunnel surface material, and the surface tilt angle are introduced as supplementary parameters to characterize the regularity of shadow deformation under different physical environments. In addition, to enhance the adaptability of the dictionary, an incremental acquisition strategy is adopted, which automatically adds new samples to the feature dictionary during mine operation and reduces the weight of redundant data to maintain the accuracy and query efficiency of the dictionary.

[0021] Using the established shadow spatiotemporal feature dictionary, shadow profile references are generated for real-time dynamic comparison. In real-time image stream analysis, whenever a region with significant boundary drift, sudden brightness drop, or abrupt change in spatial texture appears in the monitored image, the time baseline and vehicle trajectory status corresponding to the current image frame are immediately retrieved, and the corresponding spatiotemporal path samples are searched in the shadow feature dictionary. By comparing the boundary position of the region to be identified in the real-time image with the typical shadow paths recorded in the dictionary one by one, the deviation value between the two in terms of spatial distribution and temporal progression trend is calculated. If the deviation value is lower than the set dynamic error tolerance threshold, the deformation in the current image is considered a normal shadow evolution process, and it is removed from the subsequent anomaly identification process; if the deviation exceeds the threshold, or the current shadow starting position is significantly inconsistent with the light source propagation direction, it is recorded as a potential abnormal image and enters the next stage of processing. The shadow profile reference in this step combines physical lighting behavior, changes in the spatial attitude of the mine car, and temporal image evolution during its construction. This enables predictable modeling of shadow behavior and significantly improves the accuracy and real-time performance of shadow misjudgment elimination in video monitoring without increasing computational resource consumption. It provides a reliable reference for the accurate identification of subsequent abnormal events.

[0022] Under the constraint of shadow profile reference, geometric trajectory information from multiple monitoring perspectives is collected, and polarization image data and temperature image data are fused. Layered stripping of fall texture information in the image that may be similar to rock collapse is performed to construct a shadow confidence spectrum with quantified confidence, which is used as a continuous input for subsequent anomaly identification and judgment. After establishing a shadow profile reference based on a unified event time baseline, to accurately distinguish shadows from anomalous event images during the image recognition stage, multi-view geometric analysis and multi-domain information fusion are introduced to construct a shadow confidence spectrum image with quantifiable confidence for subsequent continuous use in judgment. The specific operation includes the following steps: Based on the established shadow profile reference, image data from multiple fixed monitoring points where the spatial locations of the predicted shadow trajectories overlap are collected, and geometric trajectory information under the corresponding time windows is extracted. Within the tunnel structure, high-sensitivity cameras with fixed focal lengths, wide-angle lenses, and image synchronization clocks are symmetrically deployed along both sides and the top of the tunnel, based on the geographical distribution of lighting blind spots, high-frequency vehicle entry and exit points, and areas prone to misjudgment. This ensures at least three sets of acquisition devices with overlapping perspectives every two meters. To establish the geometric correspondence in three-dimensional space, each set of devices needs to be spatially calibrated using a fixed reference object before deployment. This reference object includes a standard plate with known side lengths and a high-contrast pattern. The intrinsic and extrinsic parameter matrices of each camera are obtained through direct shooting, rotational shooting, and multi-angle imaging. Subsequently, during the shadow prediction time period, image frames are synchronously extracted from multiple cameras. Extractable structural elements such as mine car outlines, light boundaries, ground feature lines, and support steel frames are subjected to edge detection in the image coordinate system. The projection path of the object edges in the three-dimensional physical coordinate system is calculated using a binocular triangulation algorithm. In these spatial paths, any part that overlaps with the starting boundary, extension path and ending position recorded in the shadow profile reference is marked as a candidate shadow region, which is used to identify and lock the processing range for subsequent spectral information.

[0023] Within the candidate shadow area, polarization image data and temperature image data are simultaneously acquired and then registered and fused. At each fixed video acquisition position, two types of special imaging equipment are deployed adjacent to the visible light imaging equipment: one type is a polarization imaging device with an omnidirectional linear polarization receiving array, which can obtain the main polarization direction and polarization degree value of the region at a specific time by placing polarizers at different angles in front of each pixel subarray; the other type is a high-sensitivity infrared thermal imaging device, which uses an uncooled thermopile infrared sensor to continuously acquire the radiation intensity changes of each image area in the eight to fourteen micrometer band and convert them into a temperature pseudo-color map in real time. After the two types of images are acquired, image coordinate correction is first performed to geometrically register the polarization image and the thermal imaging image along the spatial principal axis; then, through image resampling, the three types of images (visible light, polarization map, and thermal map) are aligned on the pixel-level spatial coordinates. After fusion, in each group of candidate areas, three sets of three-dimensional matrices are constructed, corresponding to the brightness change distribution, polarization direction abrupt change distribution, and temperature gradient change distribution, respectively, and arranged in time frame order to form a sensing data stream along the time axis.

[0024] Based on the fused image perception data stream, texture recognition, feature analysis, and layer-by-layer stripping are performed on image blocks falling into the shadow prediction region to extract credible shadow areas and exclude high-risk texture features that resemble rock falls but are not real events. In specific operations, firstly, edge gradient calculation is performed on each image block in the visible light image to extract boundaries with high contour intensity; then, the polarization direction consistency is checked on the polarization image at the corresponding location to determine if there are regions with abrupt changes in vibration direction or significant differences in scattering direction among multiple pixel sub-regions; further, the rate of change of radiation intensity at that location in the temperature image is combined to calculate whether each image block experiences a sudden temperature change exceeding a set threshold on the time axis. These three criteria are logically combined for judgment. If the image block boundary contour and shadow profile change are consistent, the polarization consistency is high, and the temperature curve is smooth, it is judged as normal shadow behavior and assigned a high confidence value; if the image block has irregular boundaries, disordered polarization direction, and a sudden drop in local temperature, it is marked as a potential source of misjudgment and assigned a low confidence value. This confidence value is marked in pixels within the image coordinates, and a single-frame shadow confidence spectrum image is generated in the form of a heatmap. To enhance spatial continuity and boundary smoothness, spatial Gaussian filtering and temporal median filtering are performed on the initial confidence spectrum to remove sudden noise or anomalous transition blocks.

[0025] The generated shadow confidence spectrum image is output in time series form as the input signal for subsequent image anomaly identification and participates in real-time linkage triggering decision weight adjustment. In actual use, before each frame of the main monitoring image enters the identification algorithm process, the corresponding shadow confidence spectrum image of the current frame is first called to perform position mapping and confidence value extraction on the region to be identified in the main image frame. If the confidence value of the corresponding position of the identification region in the confidence spectrum image is higher than the confidence threshold, the identification weight is automatically reduced to reduce the probability of entering the anomaly reporting path; if the region shows a decreasing confidence trend in the confidence spectrum and there are corresponding jump features in the heat map and polarization map, the weight is maintained or enhanced and recorded as an anomaly to be judged region. By introducing the confidence spectrum as a pre-intervention method, the image feature interference caused by the dynamic evolution of light and shadow can be effectively eliminated before the identification algorithm, reducing the false alarm rate caused by lighting and shadow from the source.

[0026] Based on the constructed shadow confidence spectrum, a counterfactual replay sequence is generated. The monitoring footage is replayed under simulated and actual mine truck speed and lighting conditions. Stable image features that remain unchanged in multiple replays are extracted to form the feature fingerprint of misjudgment trigger. Based on the constructed shadow confidence spectrum image sequence, to eliminate the impact of dynamic lighting or sporadic scene changes on anomaly identification, a counterfactual playback mechanism is proposed. By reproducing multiple monitoring images under near-realistic conditions, stable image features that recur and remain undisturbed are extracted, and based on these features, a unique and identifiable false alarm trigger feature fingerprint is generated. The specific steps include: Based on the constructed shadow confidence spectrum, time windows with a high risk of misjudgment in the target image sequence are identified, and the mine vehicle behavior parameters and illumination status information within the corresponding time periods are extracted. For frame sequences in the confidence spectrum with drastic fluctuations in confidence values, frequent abrupt changes in spatial region boundaries, and large deviations from the expected shadow path, according to the unified event time baseline index corresponding to each frame image, all parameter data at that time point are extracted, including the mine vehicle's running speed, direction of movement, attitude angle, vehicle coordinate position, and the on / off status of the vehicle's light source, beam projection angle, and illuminance intensity change curve. The data interval is extended by five seconds before and after the time point at 0.2-second intervals to form a complete ten-second playback window. This method ensures that each set of image frames is associated with accurate vehicle status and illumination behavior, providing a data basis for constructing the simulation replay.

[0027] Based on time windows and collected behavioral parameters, a series of representative counterfactual playback sequences are constructed, and the image evolution process consistent with the actual scene is reconstructed. First, key parameter points generated by the mining truck during actual operation are selected, including several typical combinations of states such as uniform straight-line travel, high-speed passage, turning acceleration, illumination surge, and light occlusion, as input boundary conditions for the replay simulation. Under each set of simulation conditions, according to the order of the original image frames collected by the actual monitoring equipment, the currently set illumination projection path and shadow evolution trajectory are superimposed using an image spatial reconstruction algorithm to reproduce the image change process generated by the combined action of light and the vehicle under the current parameter combination. In each playback, three types of multi-source image data—visible light image sequence, polarization image sequence, and temperature image sequence—are simultaneously restored, maintaining their one-to-one correspondence in spatial coordinates, timestamps, and viewing angles. To control the interference of external environmental variables on the recognition effect, all playback images are brightness-normalized based on the background brightness at the actual sampling time point during the reproduction process. Simultaneously, color normalization and edge smoothing are performed on the image frames to ensure that each simulation result can be used for subsequent comparisons.

[0028] After reconstructing multiple sets of counterfactual playback image sequences, image structural features were extracted frame by frame, and their stability across all playback rounds was statistically analyzed. Specifically, structural texture regions were identified frame by frame, including the side projection contours of mine cars, reflection areas of fixed objects on the mine wall, fixed crack lines on the ground, and shadow transition zones of support beams. The degree of change in each region was quantitatively scored based on three indicators: consistency of edge gradient direction, stability of brightness changes, and range of temperature field mean changes. A region was recorded as a "stable structural region" if it exhibited unchanged edges, approximate brightness curves, and no significant fluctuations in the temperature field under at least five different playback conditions. Simultaneously, in each playback round, key parameters such as spatial location, boundary shape, grayscale texture, and thermal characteristic curves of this region were encoded and archived to form a stable image region feature table.

[0029] Based on the extracted stable image region feature table, a corresponding misjudgment trigger feature fingerprint is constructed, and a spatial matching model is built and its expression method is encoded. Specifically, all regions that meet the stability conditions are spatially organized to determine their relative distribution pattern, region overlap, and adjacency relationship in the image coordinates, forming a two-dimensional spatial structure map. Then, the statistical features of each region in dimensions such as grayscale distribution, texture direction, and thermal imaging response are used to generate descriptive vectors, which are then incorporated into this structure map as supplementary data items. To enhance the scene adaptability of the feature fingerprint, it is also necessary to record behavioral context data such as the location and time of the first appearance of the image structure, the corresponding vehicle operating conditions, the current illumination disturbance type, and the confidence spectrum score at that time, as a mapping chain for identifying the causal relationship between the feature fingerprint and the misjudgment scenario. The final feature fingerprint has a three-layer expression structure: first, a region structure index map, which marks the spatial layout of all stable regions; second, a region content feature table, which encodes all image property indicators; and third, a context association information set, which records environmental parameters and trigger logic. This structure can be used as a reference template for subsequent recognition processes.

[0030] Write the generated feature fingerprint into the stability verification sub-process in the recognition and judgment process, which is used to identify possible misjudgment source areas in advance. In the actual image processing process, before each frame of input image enters the linkage judgment process, it is first spatially matched and compared with the constructed feature fingerprint structure, comparing its regional position, structural features and environmental context information; if a local area in the current image highly coincides with a known misjudgment feature fingerprint and its confidence spectrum score is relatively low, then this area is determined as a high-risk misjudgment area, and a judgment suppression weight is applied in the linkage decision-making stage to prevent incorrect responses triggered by such structures. Through the above mechanism, the present invention not only realizes the inversion simulation and feature attribution of typical misjudgment behaviors, but also precipitates them into high-risk reference information in the recognition database, forming a complete closed-loop process from image reconstruction, stability evaluation, feature extraction to scene fingerprint modeling.

[0031] Using the feature fingerprint as a constraint condition, perform cluster consistency arbitration in distributed nodes, fuse the timestamp offsets, perspective geometric differences and light energy transition gradients of different monitoring nodes, calculate the mutual consistency score between multi-source inputs, and generate a dynamic adjustment threshold for the evacuation response threshold according to the consistency score; After completing the extraction of the feature fingerprint of the stable image area in the counterfactual playback, in order to ensure that the recognition and judgment have cross-node credibility support, a cluster consistency arbitration method based on the feature fingerprint constraint condition is proposed, and on this basis, a dynamic adjustment threshold for the response threshold is generated for adapting to the linkage control precision management in a multi-node collaborative environment. The specific process includes the following steps: Based on the constructed image feature fingerprint structure, start the multi-monitoring node image calling and matching process according to the unified event time baseline. In the mine environment, fixed monitoring points at different positions receive the unified timing signal through the preset time synchronization mechanism, ensuring that the image acquisition time accuracy is controlled within milliseconds. Taking the abnormal trigger frame detected by the main recognition node as a reference, retrieve all other monitoring points whose field of view coverage overlaps with it, and retrieve the image data collected during the same period from these points. Such image data includes, but is not limited to, visible light images, polarization images and thermal imaging images, and each frame of image carries spatial position identification and timestamp data. After the image calling is completed, apply the constructed feature fingerprints to the corresponding areas in the node images one by one, and perform one-to-one matching and comparison of the image structure, texture edge, gray distribution and temperature response in units of image blocks. Each matching result will generate a corresponding matching index set, including structure coincidence degree, gray similarity, texture direction consistency and heat map response overlap degree, and fill these data into the fingerprint matching response matrix for the next step of consistency analysis. Through this method, cross-regional verification of the feature fingerprint in multi-node images can be realized, ensuring that the judgment result is not caused by accidental image disturbances under a certain fixed perspective.

[0032] Based on the image matching response results of each monitoring point, a multi-dimensional consistency score is calculated by combining timestamp offset, viewpoint geometric difference, and light energy transition characteristics to determine whether the current feature possesses cross-node consensus. In the temporal dimension, the timestamp data attached to each image frame is first extracted, and its reference time offset value relative to the main identification node is calculated to determine whether there is acquisition delay, synchronization loss, or other timing anomalies. An offset value less than five milliseconds is considered high temporal consistency; an offset value between five and twenty milliseconds requires timing compensation; and an offset value exceeding twenty milliseconds is marked as temporal mismatch. In the spatial dimension, based on the physical location, lens orientation, and relative height data recorded at the time of monitoring point installation, combined with known alleyway structure information, a spatial projection relationship of the monitoring images is established, and the degree of deformation of the same feature region in the image coordinate system under different viewpoints is calculated. Through affine transformation and image registration algorithms, the projected overlap area of ​​this region in the images from each viewpoint is obtained; the larger the overlap area, the higher the viewpoint consistency. In the energy dimension, the illuminance curve generated by the change of light source over time is extracted, and the slope of the brightness transition in the grayscale response of the matching region is superimposed. The location of the light energy transition point is identified by curve fitting, and it is determined whether the transition is caused by the actual change in lighting or by shadow occlusion or shading from a mining vehicle. Finally, the three indicators of temporal synchronization, spatial overlap, and illuminance response consistency are assigned weights, and the cluster consistency score of the feature fingerprint within the current recognition window is calculated. The score ranges from 0 to 1, with a higher value indicating a stronger credibility of the recognized event.

[0033] After obtaining the consistency score, a dynamic threshold adjustment mechanism for evacuation response is established based on the current score value. This mechanism automatically adjusts the minimum judgment value required to trigger an evacuation response based on the spatial support of the identified event, ensuring that the system can make differentiated responses to images with different levels of credibility. When the score value is higher than 0.85, the threshold can be automatically lowered, and the identified event enters the response execution stage without waiting for additional verification. When the score is between 0.60 and 0.85, the system keeps the threshold unchanged and prepares to introduce a redundant confirmation mechanism for further comparison and confirmation. When the score is lower than 0.60, the system raises the threshold and forcibly calls independent information sources such as voice announcements, personnel passage counts, and dynamic changes in video stream frames as additional comparison conditions to prevent low consistency events from misleading the triggering chain. To improve the response continuity of the dynamic threshold, a set of historical threshold regression curves is also set in this step. This curve is based on the statistical distribution of consistency scores and actual triggering results in the identified events of the previous hour, dynamically adjusting the score interval boundary to prevent frequent threshold fluctuations caused by frequent abnormal events in a short period of time. Ultimately, a dynamic response control logic tightly coupled with the recognition credibility is formed, which can automatically adjust the sensitivity of the response triggering mechanism according to the consistency of the data structure of the cluster nodes, and achieve bidirectional optimization of accurate response and false judgment suppression.

[0034] Based on the dynamic adjustment threshold of the evacuation response threshold, a linkage pre-confirmation chain is constructed. The consistency score is used as the input signal, and a millisecond-level dual-phase verification time window is introduced. Within this time window, the voice prompt signal of the call station and the passage counting information of the geomagnetic sensor are verified simultaneously to eliminate misjudged commands from a single source. After adjusting the cluster consistency score and dynamic response threshold, to avoid triggering mine safety linkage actions due to single-point misjudgment, a linkage pre-confirmation chain based on the score input is introduced. This chain utilizes a cross-domain dual-signal verification mechanism to perform physical-level redundant confirmation of the identification results. This confirmation chain operates within a millisecond timeframe, driven by both human voice commands and personnel access data, and specifically includes the following steps: After calculating the consistency score and generating the dynamic response threshold, the pre-confirmation process is initiated, and a verification time window is set. If the consistency score of the current image recognition event falls within the dynamic threshold range (e.g., between 0.65 and 0.85) after image structure analysis, multi-view comparison, and light energy transition judgment, the system enters the pre-confirmation process. To complete effective confirmation without delaying the linkage response, the system initiates a verification time window based on the current score completion time, with a duration not exceeding 500 milliseconds. During this period, the system does not immediately activate any physical linkage devices but waits for confirmation information to be received from two independent signal channels: a real-time voice prompt signal from the dispatch call console and personnel passage count information collected by the geomagnetic sensor. The length of this time window is determined by modeling the downhole broadcast response delay, sensor data feedback cycle, and network latency tolerance to ensure that the real-time and fault-tolerant requirements under actual production conditions are met.

[0035] Within the verification time window, both voice prompt signals and passage counting signals are accurately collected and their content verified, forming a dual verification logic for pre-confirmation. For voice verification, the dispatcher broadcasts the area where the identified event occurred via a broadcast terminal. Common prompts include "Please give way," "Anomaly has occurred," and "Numbered area alarm." This voice content is simultaneously transmitted to the underground loudspeaker and captured in real-time by sound acquisition devices deployed at roadway nodes. The sound acquisition devices digitize the received audio data, extracting its dominant frequency characteristics and keyword content, and perform semantic comparison and matching based on the event identification area's number information. If a key prompt related to the current identified event is clearly identified within the verification time window, and its content structure matches the system's preset template, the voice prompt is considered a valid confirmation signal. Simultaneously, within the roadway section involved in the identified event, geomagnetic sensors deployed at the main surface passageway also begin real-time collection of magnetic field disturbance data caused by passing personnel. Each time a metal object (such as a miner's lamp, tools, or safety helmet carried by personnel) passes through the geomagnetic induction coil, the device generates an electromagnetic change event, accompanied by a timestamp accurate to the millisecond level. The system compares the passage data within the time window with the historical average passage rate of the area. If it detects a jump in passage frequency exceeding three times the average baseline within 500 milliseconds, and the passage direction is consistent with the evacuation direction, it determines that personnel response has actually occurred, constituting a second confirmation signal.

[0036] After the two confirmation signals are collected and verified, they are used as decision inputs and jointly judged with the current consistency score to determine whether to allow the execution of the linkage command. If both the voice signal and the passage counting signal meet the four verification conditions of time window, semantic content, location number, and direction path, and the current event score is not lower than the score confidence threshold (e.g., not lower than 0.65), the current event is considered a valid linkage trigger event with image recognition basis and behavioral response support. The system will allow the execution of response commands, such as controlling local ventilation, turning off the power in a specific section, issuing an escape voice alarm, or dispatching personnel to evacuate, etc. If only one of the two confirmation signals is valid, or neither of them meets the verification conditions, the recognition event is marked as "pending confirmation" and will not enter the linkage execution process. The system will continue to track the evolution of subsequent image frames and score changes, and re-initiate the scoring and verification process in the next recognition cycle. This mechanism ensures that each linkage response execution undergoes a triple authentication process based on image analysis, consistency verification, and physical behavior signals, significantly reducing the risk of false triggering.

[0037] Under the condition that the linkage pre-confirmation chain runs continuously and stably, the time-reversal phase gating command is generated based on the feedback result of the confirmation chain. The programmable polarization metasurface illumination structure is controlled to inject the phase conjugate micro-envelope waveform into the shadow center region in the monitoring screen, extinguishing the dynamic expansion characteristics of the shadow, and synchronously writing back the calibration matrix after the illumination intervention to the unified event time baseline, forming a closed-loop reversible dynamic control process. After voice prompts and personnel access data have completed dual verification and the recognized event has been confirmed as valid, the active light field intervention phase begins. This step involves injecting a micro-envelope illumination waveform with phase conjugation characteristics to extinguish dynamic shadow areas in the image that pose a risk of misjudgment in real time, and then writing the intervention results into the time baseline to achieve closed-loop control. The specific steps are as follows: After completing the voice prompt verification and geomagnetic counting verification in the pre-confirmation chain and successfully determining the current identified event as a high-confidence event, the shadow core region of the identified event in the image is extracted. This region, identified by the system during the image processing stage, possesses complete edge information, dynamic evolution direction, and texture continuity. The system first reads the grayscale distribution curve, gradient direction map, texture break location, light reflection path, and historical change trajectory of this region during recognition, and combines this with prior information on light source modulation and mine car trajectory data to construct a "spatial light field modulation target map" that accurately describes the current shadow structure. In this target map, each pixel unit contains angular response, historical phase interference trajectory, and current wavefront perturbation trend, used to guide the next step of precise light field injection. This data structure obtains the optimal intervention waveform model suitable for the current spatiotemporal conditions by matching the current shadow with high-similarity segments in the historical shadow dictionary.

[0038] Based on the target image, the phase-driving unit in the corresponding region of the programmable polarization metasurface illumination array is activated. The corresponding control panel translates the intervention waveform parameters of the target region into a combination of phase delay, electro-polarized direction, and incident angle of multiple micro-illumination units, generating a complete intervention matrix signal. The polarization metasurface is composed of independently driveable electro-optic control units. Each unit changes the scattering direction and phase delay capability of its nanostructure to light waves by applying voltage, thereby achieving real-time control of the light waveform at the microscale. According to the modulation frequency, envelope width, interference angle, and phase inversion information of the desired injected waveform, the system drives several illumination units to synchronously emit micro-envelope waveforms with phase conjugation characteristics, injecting them into the shadow-forming core region along the optical reflection trajectory in the reverse path, and causing destructive interference with the original shadow wavefront, instantly canceling its intensity field. To ensure complete coverage of the intervention range, the system performs comparative analysis of image frames within 25 milliseconds before and after the illumination area, confirming at the image processing level whether the original gray-scale abnormal area has recovered to the average ambient brightness. If not, it automatically performs a phase fine-tuning and injects again.

[0039] After the lighting intervention is completed and the image changes meet the shadow extinguishing conditions, the system immediately acquires the latest image frame and enters the resampling and feature reconstruction process. This image frame, centered on the intervention area, undergoes pixel-by-pixel reanalysis of its grayscale distribution, texture arrangement direction, and boundary contour clarity, resulting in a new set of feature vectors. The system compares this set of vectors with the feature vectors of the corresponding area in the pre-intervention image, generating a difference map. This difference map is used to determine: 1) whether the intervention area completely extinguished the original light and shadow disturbance features; 2) whether new unexpected feature disturbances were introduced; and 3) whether the intervention effect had a spillover effect in neighboring areas. After the comparison, the intervention results are recorded in the form of a "lighting intervention response matrix," containing detailed information such as the start and end times of the intervention, intervention waveform parameters, image feature differences before and after the intervention, shadow stripping success rate, supplementary lighting offset angle, and successful inversion markers. This matrix is ​​used as a new calibration factor, bound to the unified time baseline of the current event, replacing the feature state before the intervention, and is used in subsequent image recognition processes.

[0040] By linking and comparing the response matrix after lighting intervention with the historical feature evolution model, the regional credibility mapping table of the image recognition system is updated, ultimately completing a closed-loop control mechanism of identification, verification, intervention, and feedback. The credibility mapping table is a dynamic map generated based on the intervention response results of historical shadow areas, used to reflect the potential misjudgment risk and processed status of each image area in future recognition cycles. This table appropriately raises the identification trigger threshold for successfully intervened areas to reduce the probability of repeated alarm triggers in the same area due to residual disturbances in a short period of time; at the same time, for areas where the intervention failed to completely eliminate misjudgment features, it is set to "enhanced monitoring state," incorporating higher-frequency image analysis in future monitoring cycles and prioritizing the triggering of the dual verification mechanism of voice and passage. Thus, image recognition is no longer a linear judgment process, but establishes a highly coupled closed-loop dynamic control mechanism from image input → shadow recognition → event verification → active intervention → image feedback → calibration write-back, truly realizing closed-loop engineering management of shadow misjudgment problems in complex mine environments.

[0041] This invention, by constructing a unified event time baseline and shadow profile reference, combined with multi-view geometric trajectory acquisition and polarization and temperature information fusion processing, achieves for the first time the structural identification and confidence quantification of dynamic shadows. Based on this, it utilizes counterfactual playback to extract stable image features and construct a misjudgment feature fingerprint, ensuring temporal consistency and image stability in the identification process. A cluster consensus arbitration mechanism calculates multi-source scores, and a dual field verification mechanism eliminates false triggering commands, achieving multi-dimensional confirmation of the identification results. Finally, time-reversal phase-gated control of the polarization metasurface illumination structure actively extinguishes the dynamic growth of shadows at the physical level, achieving a breakthrough from image recognition to light field intervention and completely eliminating misjudgment sources. The overall solution constructs a closed-loop control chain from image recognition and result verification to active intervention and calibration write-back, significantly improving the stability, accuracy, and intelligent response capabilities of mine safety monitoring systems, and significantly reducing the risk of ineffective evacuation, congestion, and secondary accidents caused by misjudgments.

[0042] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A cluster-based, full-mine AI video analysis and linkage method, characterized in that, Includes the following steps: Under a unified event time baseline, light source modulation information and mine car running trajectory are acquired to construct a spatiotemporal feature dictionary of shadows and generate shadow profile references for dynamic comparison. Based on the shadow profile reference, multi-view geometric trajectories are acquired, polarization images and temperature images are fused, fall texture is stripped off, and shadow confidence spectrum is constructed. Based on the shadow confidence spectrum, a counterfactual replay sequence is generated, and image features that remain unchanged in multiple replays are extracted to form a feature fingerprint that triggers misjudgment. Based on feature fingerprints, consensus arbitration is performed in distributed nodes. Timestamp offset, viewpoint difference and light energy transition gradient are fused to calculate consensus score and generate dynamic adjustment threshold for evacuation response. Based on the dynamic adjustment threshold of the evacuation response threshold, a linkage pre-confirmation chain is constructed, a dual-phase verification window is introduced, voice prompts and geomagnetic passage counts are matched, and false judgment commands are eliminated. Under stable operation of the confirmation chain, time-reversal phase-gated commands are generated to drive the dynamic expansion of the extinguishing shadow of the polarized metasurface illumination structure, and the calibration matrix is ​​written back to the event time baseline.

2. The cluster-based whole-mine AI video analysis and linkage method according to claim 1, characterized in that, The steps for generating the shadow profile reference are as follows: Acquire light source modulation information and mine truck trajectory data under a unified event time baseline; The light intensity change, beam diffusion angle and illumination direction are collected based on the light source modulation information, and combined with the linear velocity, acceleration, vehicle posture and illumination projection area boundary of the mine car. Spatiotemporal path samples of shadows are constructed using combined data of light source and mine car trajectory, and the edge position and spatial morphology of shadows in consecutive image frames are recorded. A spatiotemporal feature dictionary of shadows is established based on path samples generated by multiple light sources and mining vehicles. The matching degree between shadow areas in the image and paths in the dictionary is compared in real-time image analysis, and shadow profile references are dynamically generated for subsequent anomaly identification and judgment.

3. The cluster-based whole-mine AI video analysis and linkage method according to claim 2, characterized in that, The steps for constructing the shadow confidence spectrum are as follows: Under the constraint of shadow profile reference, multi-view images are acquired and geometric trajectory information is extracted. Spatial paths are calculated and candidate shadow regions are identified through edge detection and triangulation. Polarization and temperature images are acquired within the candidate shadow area, pixel-level alignment is performed, and the images are fused into three data streams: brightness, polarization, and temperature. A joint determination is performed based on the edge intensity, polarization direction consistency, and temperature gradient change rate of each image patch in the fused image stream, and a confidence spectrum image is generated by labeling the confidence level. The confidence spectrum image is output in time series for subsequent region confidence value extraction and recognition weight adjustment before image recognition.

4. The cluster-based whole-mine AI video analysis and linkage method according to claim 3, characterized in that, The steps for generating a feature fingerprint triggered by a misjudgment are as follows: Based on the shadow confidence spectrum, the time window with drastic fluctuations in confidence values ​​is determined and the corresponding vehicle and illumination parameters are extracted; Multiple counterfactual replay image sequences were reconstructed based on the extracted behavioral parameters, and illumination and temperature normalization were performed. Stable structural regions are extracted from images in each playback sequence, and a feature table of stable image regions is established. Based on the feature table, a three-layer representation structure is constructed, consisting of a region structure index map, a region content feature table, and a contextual information set, to form a feature fingerprint. Feature fingerprints are embedded into the recognition and judgment process for spatial matching and comparison, which is used to identify high-risk areas for misjudgment and suppress the weight of misjudgment triggers.

5. The cluster-based whole-mine AI video analysis and linkage method according to claim 4, characterized in that, The steps for generating the dynamic adjustment threshold for the evacuation response are as follows: Using feature fingerprints as constraints, image data with the same image frame time as the main identification node is retrieved synchronously from multiple monitoring nodes, and image feature matching is performed to construct an image matching response matrix; By integrating the timestamp offset, viewpoint geometric difference, and light energy transition characteristics of each node, a consistency score is calculated, and the consistency score is used as a quantitative basis for the credibility of the identification. The threshold for evacuation response is automatically adjusted based on the consistency score, and the threshold boundary is dynamically corrected by combining the historical score distribution curve.

6. The cluster-based whole-mine AI video analysis and linkage method according to claim 5, characterized in that, The consistency score is calculated based on time synchronization, spatial overlap and illumination response consistency as indicators, with weights assigned to each and weighted fusion. When the score is lower than the preset lower limit, voice prompts are forcibly introduced and redundantly compared with the passage count data.

7. The cluster-based whole-mine AI video analysis and linkage method according to claim 5, characterized in that, The steps for constructing the pre-confirmation chain are as follows: Set a verification time window and initiate a pre-confirmation process within the scoring threshold, calling the call center's voice prompt signal and the geomagnetic sensor's passage count information; Extract the main frequency features and keyword content of the audio within the verification time window, complete semantic matching, and simultaneously analyze the passage frequency and passage direction generated by geomagnetic induction to identify whether effective personnel response behavior has been formed. The voice prompt signal and the passage count signal are used as decision inputs and jointly judged with the current score. If both signals pass the verification and the score is not lower than the confidence threshold, the linkage response is allowed to be executed; otherwise, it is marked as an event to be confirmed.

8. The cluster-based whole-mine AI video analysis and linkage method according to claim 7, characterized in that, After confirming stable chain operation, the following steps are taken to generate time-reversal commands to drive the polarization metasurface illumination extinguishing shadow extension and write the calibration results back to the event time baseline: Based on the confirmation chain feedback results, the shadow core region is extracted and a spatial light field modulation target map is generated to construct target data for guiding intervention waveform injection. Initiate a programmable polarization metasurface illumination array to inject a phase-conjugate micro-envelope waveform, which extinguishes the dynamic expansion of shadows through interference. Images were acquired after the intervention and the image feature vectors were reconstructed to generate an illumination intervention response matrix, which was then synchronously written back to a unified event time baseline. The region confidence mapping table is updated based on the response matrix, and the future identification threshold is dynamically adjusted.

Images