A method and system for recognizing human-object interaction behavior in mines based on an improved PVIC model.

Abstract

This invention provides a method and system for mine behavior recognition based on human-object interaction. The method includes: Step S1, dataset construction, collecting underground monitoring videos, filtering and extracting frames, simulating complex environments through data augmentation, and labeling; Step S2, optimizing the PVIC model from the input, feature, and decoding dimensions to obtain the PVIC-FAGLI ​​model; Step S3, training and optimization, dividing the MineHOI dataset, performing end-to-end training using the Adam optimizer and cross-entropy loss, and adjusting parameters through ablation experiments; Step S4, preprocessing the video stream input algorithm model, detecting miner and tool interaction behaviors, outputting information, and triggering alarm recordings. This method improves the model's adaptability to complex underground environments by integrating prior knowledge and data augmentation. Combined with an end-to-end training and optimization mechanism, it enhances the recognition capability of human-computer interaction features, ultimately achieving accurate behavior perception and anomaly warning in dynamic scenarios, supporting intelligent mine safety management.

Description

Technical Field

[0001] This invention relates to the fields of mine safety monitoring and computer vision technology. Specifically, it relates to a method and system for identifying human-object interaction behavior in mines based on an improved PVIC model, which is particularly suitable for real-time detection and supervision of key safety operations such as "knocking on the walls and roof" underground. Background Technology

[0002] Mining is a high-risk operation. The underground environment is dark, with high dust concentrations, dense equipment, and complex geological conditions, leading to frequent accidents such as roof falls and side collapses, which seriously threaten the lives of miners. "Assessing the roof and sidewalls" is a core safety preparatory procedure before mining. Miners use tools such as crowbars and steel bars to tap the roof and sidewalls to determine the integrity of the rock mass and check for loose rocks. The standardization of its implementation directly determines the safety of subsequent operations.

[0003] Currently, mine supervision of such behaviors mainly relies on on-site inspections by safety officers or manual video monitoring. This approach suffers from problems such as low efficiency, strong subjectivity, susceptibility to fatigue and missed detections, and high labor costs, making it difficult to achieve reliable 24 / 7 monitoring from multiple locations. Existing computer vision-based behavior recognition technologies are mostly designed for general scenarios, recognizing simple actions such as raising a hand or falling, and lack the ability to model the interaction between people and specific tools and environments.

[0004] Existing behavior recognition technologies are mainly divided into three categories, each with significant shortcomings in mining environments: The first category is based on human skeletons or postures, focusing on the analysis of human movements. While performing well in general scenarios, it completely ignores object interaction information and cannot effectively identify specific behaviors that rely on tools, such as "knocking on the roof," leading to a higher false positive rate in the complex environment of mines. The second category is shallow interaction recognition methods based on object detection, which achieve preliminary interaction analysis by detecting the positions of people and objects. However, it lacks deep modeling of interaction relationships, making it difficult to distinguish semantically similar actions, and it has poor adaptability to low-light and high-noise videos. The third category is based on human-object interaction detection methods, which capture interaction semantics through <human body, verb, object> triples. This is currently a research hotspot, but mainstream algorithms are not optimized for mining environments: On the one hand, two-stage methods rely on the quality of the initial detector, consuming large computational resources and lacking real-time performance; on the other hand, while single-stage methods are more efficient, their computational complexity reaches O(N²) when processing long sequences, resulting in large memory consumption and limited feature extraction capabilities for dim or blurry images, making them prone to missed detections.

[0005] Furthermore, existing technologies generally face three major challenges: First, mine-specific datasets are scarce, and public datasets do not include behaviors such as "knocking on the roof and asking questions," resulting in weak model generalization ability; second, algorithms lack sufficient fusion of local details and global context, and attention mechanisms are prone to losing fine-grained features under noise interference; third, system deployment requires a balance between accuracy and real-time performance, and traditional methods struggle to meet the real-time analysis needs of underground operations with limited hardware resources. Therefore, there is an urgent need for a human-object interaction behavior recognition technology that adapts to the complex environment of mines and balances recognition accuracy and real-time performance to fill the gap in intelligent safety supervision of mines. Summary of the Invention

[0006] The purpose of this invention is to overcome the problems of poor adaptability to complex environments, insufficient real-time performance, and low accuracy in recognizing specific interactive behaviors in existing mine behavior recognition technologies. It provides a method and system for recognizing human-object interaction behaviors in mines based on an improved PVIC model, enabling accurate and real-time detection of key safety behaviors such as "knocking on the walls and roof," thereby improving the intelligence level of mine safety supervision and reducing the accident rate. Specific objectives include: First, improving the accuracy of behavioral feature extraction and recognition in complex mine environments such as dim lighting and dust through frequency domain enhancement and multi-module optimization; Second, improving the model's attention mechanism to reduce computational complexity and meet the real-time requirements of underground monitoring; Third, constructing a complete solution from data to application to promote the intelligence of mine safety supervision.

[0007] Firstly, embodiments of the present invention provide a method for recognizing human-object interaction behavior in mines based on an improved PVIC model. The core steps of this method are detailed below: Step S1, mine behavior dataset construction: Monitoring videos from the underground working face are collected, and after screening and frame extraction, data augmentation techniques such as contrast adjustment, Gaussian noise addition, blurring, and rotation scaling are used to simulate the complex underground environment, and annotations are constructed; Step S2, improved PVIC model construction: Based on the PVIC model, the PVIC-FAGLI ​​model is improved from the dimensions of input augmentation, feature extraction, and decoding efficiency, achieving multi-level algorithm improvement; Step S3, model training and optimization: The MineHOI dataset is divided into training, validation, and test sets proportionally. The PVIC-FAGLI ​​model is trained end-to-end using the Adam optimizer and cross-entropy loss function. Parameters are adjusted through ablation experiments to adapt to the mine scenario; Step S4, real-time behavior recognition and result output: After preprocessing the underground video stream, it is input into the trained model to detect the interaction behavior of miners and tools and output relevant information. When a target behavior or anomaly is detected, an alarm is triggered and video segments are recorded.

[0008] Secondly, embodiments of the present invention also provide a mine human-object interaction behavior recognition system for implementing the above-described method. This system adopts a layered, modular software architecture design to ensure high cohesion and low coupling, facilitating maintenance and expansion. The specific structure and functions of each layer are as follows:

[0009] 1. Application Layer (Interactive Presentation Layer): As the direct interface between the system and monitoring administrators, this layer provides a series of fully functional visual pages. These include user login and authentication pages, real-time video monitoring and behavior detection overlay display interfaces, alarm information management and handling interfaces, algorithm model parameter configuration and deployment task management interfaces, and video source device management interfaces. All interfaces adopt a responsive design, adapting to the large screens and various terminals in the mine monitoring center, providing an intuitive and clear operation and observation experience.

[0010] 2. Business Layer (Core Processing Layer): This layer carries all core business logic and data processing flow, and consists of the following three key service modules.

[0011] Streaming media service module: Responsible for video stream access and preprocessing. Supports multiple industry standard protocols such as RTSP, RTMP, and GB / T28181, utilizing a high-performance streaming media framework (such as ZLMediaKit) to achieve video stream reception, decoding, transcoding, encapsulation, and low-latency forwarding. It includes a built-in image preprocessing unit for downhole video, performing preliminary noise reduction and illumination enhancement on the input stream to provide higher-quality input for algorithm analysis.

[0012] Algorithm Analysis Module: As the core of the intelligence, it is responsible for loading and running the trained PVIC-FAGLI ​​model (usually converted to deployment formats such as ONNX). It utilizes inference acceleration frameworks (such as OpenVINO and TensorRT) to perform efficient inference on preprocessed video frames, completing the detection, classification, and confidence calculation of human-object interaction behaviors. This module directly connects to the alarm rule engine to determine in real time whether an alarm should be triggered.

[0013] Backend service module: Built on a mature web framework (such as Django), it provides unified business scheduling, user management, access control, device management, and data interface services. It adopts the MVC pattern, effectively separating business logic, data models, and frontend presentation. It is responsible for coordinating the workflows of the streaming media module and the algorithm module, managing task queues, and persisting analysis results to the database.

[0014] 3. Data Layer (Persistent Storage Layer): This layer is responsible for the reliable storage and efficient management of all system data. It uses a relational database (such as MySQL) to store structured data such as user information, device configurations, alarm records, deployment plans, and behavior statistics. Unstructured data, such as raw alarm video clips, model weight files, and system logs, is stored using a file system or object storage service. Through indexing, data caching, regular backups, and encryption of sensitive data, the layer comprehensively ensures data security and integrity, and supports rapid querying and statistical analysis based on multiple dimensions such as time, location, and behavior type.

[0015] The embodiments of the present invention bring the following beneficial effects:

[0016] This invention provides a method for recognizing human-object interaction behavior in mines based on an improved PVIC model. The core steps of this method are detailed below: Step S1: Constructing a mine behavior dataset. Monitoring videos from the underground working face are collected, and after screening and frame extraction, data augmentation techniques such as contrast adjustment, Gaussian noise addition, blurring, and rotation scaling are used to simulate the complex underground environment, and annotations are constructed. Step S2: Improving the PVIC model. Based on the PVIC model, the PVIC-FAGLI ​​model is improved from the dimensions of input augmentation, feature extraction, and decoding efficiency, achieving multi-level algorithm improvements. Step S3: Model training and optimization. The MineHOI dataset is divided into training, validation, and test sets proportionally. The PVIC-FAGLI ​​model is trained end-to-end using the Adam optimizer and cross-entropy loss function. Parameters are adjusted through ablation experiments to adapt to the mine scenario. Step S4: Real-time behavior recognition and result output. The preprocessed underground video stream is input into the trained model to detect the interaction behavior of miners and tools and output relevant information. When a target behavior or anomaly is detected, an alarm is triggered and video segments are recorded.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. Strong environmental adaptability: Through the frequency domain enhancement design of the gated frequency interaction unit (GFIU) and the dual-branch feature fusion strategy of the adaptive convolutional attention fusion module (ACAIM), the feature extraction problem in dim, high-noise, and multi-occluded environments of mines is effectively solved. Compared with the baseline PVIC model, the recognition accuracy (mAP) of this invention is improved by 1.7 percentage points, the false negative rate is reduced by 3.4 percentage points, and the false positive rate is reduced by 3.5 percentage points. This advantage is achieved through the collaborative design of the above two modules, which precisely corresponds to the technical features of claims 2 and 3.

[0019] 2. Excellent Real-Time Performance: The Grouped Linear Attention (GLIA) mechanism employed in this invention replaces the original full attention mechanism of the PVIC model by grouping query vectors into a preset number and applying a linear kernel function to calculate attention weights, thereby reducing the computational complexity from O(n²) to approximately O(n). This design significantly reduces the number of model parameters and computational load. The number of parameters is reduced by approximately 2.3M compared to the original PVIC model, and the computational load (GFLOPs) is reduced by 11.9%. Inference latency can be stably controlled within 200ms, supporting simultaneous processing of four 1080P video streams on a single server, fully adapting to the business needs of real-time mine monitoring, corresponding to the core technical features of claim 4.

[0020] 3. Highly Targeted by Scenario: The MineHOI dataset is specially constructed to focus on core safety behaviors in mines such as "knocking on the walls and roof". The model has been optimized for underground scenarios. Compared with the general HOI algorithm, the mAP in the mine scenario is improved by 7.7-9.7 percentage points. It can accurately capture the specific interaction relationship between miners and tools such as crowbars and steel bars.

[0021] 4. High practicality: The system adopts a modular architecture, supports multi-protocol video stream access, has a simple interface, and can be seamlessly integrated into the existing mine monitoring system, realizing the transformation from "post-event investigation" to "in-process intervention and pre-event early warning", reducing the cost of human supervision and improving the level of intelligent mine safety management.

[0022] Other features and advantages disclosed in this embodiment will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0023] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0024] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating a method for recognizing human-object interaction behavior in mines based on an improved PVIC model, provided for an embodiment of the present invention;

[0026] Figure 2A schematic diagram of the structure of a method for recognizing human-object interaction behavior in mines based on an improved PVIC model, provided in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of a mine human-object interaction behavior recognition system module based on an improved PVIC model, provided as an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0029] Mine construction is a high-risk, high-intensity working environment. "Knocking on the walls and roof" is a necessary safety procedure before mining, and its accurate identification is directly related to the safety of miners. Existing behavior recognition technologies have significant limitations: manual monitoring is inefficient and highly subjective; traditional algorithms ignore object interaction information, resulting in insufficient accuracy in recognizing tool-dependent behaviors such as "knocking on the walls and roof"; and general human-object interaction (HOI) detection models are prone to missed or false detections in dimly lit, high-noise mine environments, and have high computational complexity, making real-time deployment difficult. Therefore, this invention provides a mine behavior recognition method and system based on human-object interaction. Through innovative technologies such as data augmentation, gated frequency interaction units (GFIU), adaptive convolutional attention fusion modules (ACAIM), and grouped linear attention mechanisms (GLIA), feature extraction and interaction relationship modeling are optimized to achieve high-precision and high-efficiency mine behavior recognition. For ease of understanding, the method of this invention will first be described in detail.

[0030] Example 1

[0031] In this embodiment, Figure 1 This is a schematic diagram illustrating the research framework of the mine behavior recognition method provided in this embodiment of the invention. The core of this method includes three major stages: dataset construction, algorithm optimization, and system integration. The specific process is as follows:

[0032] Step S1: Construction of the Mine Behavior Dataset. In collaboration with three coal mining companies, monitoring videos were collected in underground working faces and transport tunnels using Hikvision DS-2CD3326DWD-I explosion-proof cameras (supporting 1080P resolution and low-light shooting). A total of 80 hours of video was collected. After manual screening to remove blurry and duplicate frames, 312 basic images were obtained. Frame extraction was performed using FFmpeg, and the following data augmentation techniques were used to expand the sample to 1560 images: contrast adjustment (brightness ±30%, contrast ±20%), Gaussian noise addition (signal-to-noise ratio 20-30dB), Gaussian blur (kernel size 3×3, 5×5), and rotation and scaling (rotation angle -15°~15°, scaling ratio 0.8~1.2) to simulate different lighting and dust concentration environments underground. The LabelImg tool was used for annotation, including human bounding boxes, tool bounding boxes (crowbar, steel chisel) and interactive behavior labels (knocking on the top and bottom, normal operation), generating XML format annotation files, constructing the MineHOI dataset, and dividing it into training set, validation set and test set in a 7:2:1 ratio.

[0033] Step S2: Improve the PVIC model construction. Based on the PVIC model, improve the PVIC-FAGLI ​​model for the mining scenario from the dimensions of input enhancement, feature extraction and decoding efficiency.

[0034] The input feature map is processed by a gated frequency interaction unit (GFIU), normalized, and then transformed to the frequency domain via a Fast Fourier Transform (FFT), reducing the channel dimension from C to C / 2+1. The amplitude branch uses two 1x1 convolutional layers, combined with ReLU and Sigmoid activation functions, to generate channel attention weights. Simultaneously, the phase branch generates a gated signal through a 1x1 convolution, which is then modulated element-wise with the amplitude weights. The modulated amplitude is added to the original phase, then transformed back to the spatial domain via an Inverse Fourier Transform (IFT), and residual connections are added to finally output the enhanced features.

[0035] In the Adaptive Convolutional Attention Fusion Module (ACAIM), a dual-branch parallel structure is employed. The local branch uses 1x1 convolutions, channel rearrangement, and two 3x3 depthwise separable convolutions to extract spatial detail features. The global branch generates query, key, and value vectors through three independent 3x3 depthwise separable convolutions and computes an attention weight map to achieve global context modeling. Finally, the outputs of the local branch and the global branch are added together and fused to generate the fused features.

[0036] The features are processed through a grouped linear attention mechanism module, where input features are mapped in parallel to eight subspaces. Each subspace has three independent sets of linear projection matrices (W^Q, W^K, W^V) to generate queries, keys, and values, respectively. Multi-head attention is divided into multiple groups, with each group sharing the projection weights for keys and values, thus reducing the number of parameters. By using linear kernel functions for approximation and adjusting the computation order, the computational complexity is reduced from O(N²) to O(N).

[0037] Step S3: Model training and optimization. The MineHOI dataset is divided into training, validation and test sets according to the proportion. The Adam optimizer and cross-entropy loss function are used to train the PVIC-FAGLI ​​model end-to-end. The parameters are adjusted through ablation experiments to adapt to the mining scenario.

[0038] The loss function used is the DETR series Hungarian matching loss:

[0039]

[0040] This represents the loss between the detected bounding box size and the ground truth bounding box size. For each matching pair, calculate the classification loss and the bounding box loss. This represents the GIOU loss between the detected bounding box and the ground truth bounding box. The formulas for calculating the loss and GIOU loss are shown below.

[0041]

[0042]

[0043] Where C represents the smallest rectangle covering both the predicted and ground truth bounding boxes. The model training environment used Ubuntu 20.04, an Intel i5-12400F CPU, an NVIDIA GeForce RTX 4060 GPU, 32GB of RAM, PyTorch 1.12 + CUDA 11.8 deep learning framework, and OpenVINO 2023.0 for accelerated inference. Training parameters included a batch size of 4, an Adam optimizer, an initial learning rate of 0.0001, a step size of 100, weight decay of 0.1, 300 training epochs, and cross-entropy loss (α=0.25, γ=2.0) to balance positive and negative samples and rare interaction categories. After training, the model was converted to ONNX format, accelerated for inference using OpenVINO, deployed to a mine monitoring server, and integrated with a streaming media service module to achieve real-time video stream parsing. Inference latency was controlled within 200ms, supporting simultaneous processing of four 1080P video streams on a single server.

[0044] The proposed method (PVIC-FAGLI) achieves an accuracy of 84.8%, a recall of 77.6%, and an mAP of 0.818 on MineHOI, representing an improvement of approximately 3.5% over the baseline PVIC model. Ablation experiments further validated the contributions of each module: GFIU alone improved accuracy to 82.0%, the ACAIM module improved it to 83.1%, the GLIA mechanism reduced the number of parameters to 49.7M, and GFLOPs were optimized to 85.35.

[0045] Step S4: Real-time Behavior Recognition and Result Output. The system pulls a video stream from a specified video source, converts the video stream format, decodes it, and encapsulates it into a format supported by the algorithm. It then calls the ONNX Runtime or OpenVINO inference engine to perform inference on the image and identify the target object. The system returns the detection results, including target category, confidence level, bounding box coordinates, etc. It determines whether alarm conditions are met; if target behavior or anomalies are detected, an alarm is triggered and a video clip is recorded.

[0046] Example 2

[0047] This embodiment provides a mine behavior recognition system based on human-object interaction. Figure 3 A schematic diagram of a mine behavior recognition system based on human-object interaction provided in an embodiment of the present invention includes:

[0048] The management module, a web management platform built on Django, is responsible for the core functions of user interaction, configuration management, and result display for the entire system. This module handles frontend requests through multiple view classes and persists device information, deployment strategies, and alarm records using a predefined database model. The frontend relies on JavaScript libraries to implement functions such as stream list management, polygon region drawing, low-latency video playback, and visual reports. In a specific business process, users add camera stream addresses through web_stream_add.html and configure parameters such as target recognition, algorithm model, detection area coordinates, and alarm intervals in control / add.html. The system serializes these configurations, stores them in the database, and triggers the background analyzer module to start the corresponding analysis task. When an alarm occurs, it reads the table records from the database and presents a list of alarm events with thumbnails and video playback.

[0049] The streaming media module is a high-performance streaming media service hub developed based on ZLMediaKit. It is responsible for video stream access, forwarding, recording, and protocol adaptation. Its underlying implementation relies on network, thread, and tool components provided by 3rdpart / ZLToolKit to support high-concurrency scenarios. The core capability of this module lies in its unified abstraction and processing of various streaming media protocols such as RTSP, RTMP, HLS, FLV, and WebRTC. It uses sub-libraries such as librtsp, librtmp, and libflv to demultiplex and encapsulate streams, and utilizes H264Encoder and AACEncoder in src / Codec to implement audio and video encoding and decoding. Simultaneously, server / main.cpp starts the main service process, listens for HTTP API queries to check stream status, and implements an event callback mechanism through WebApi.h and WebHook.h. The streaming media module also serves as the video source provider for the Analyzer and the receiver of processed video, and provides a WebSocket-FLV stream interface for the frontend wsPlayer.js.

[0050] The analyzer module is the analysis engine of the entire system, focusing on real-time inference of video frames and generation of anomalies. Internally, it includes AvPullStream, which pulls the raw video stream from the MediaServer and decodes it into BGR images; AlgorithmOnYolo or AlgorithmOvYolo, which encapsulates the ONNX Runtime / OpenVINO inference engine, loads pre-trained algorithm models, performs object detection on the images, and outputs bounding boxes and class confidence scores. When the detection result matches the configured object type and exceeds a threshold, an alarm logic is triggered, and the current frame and its preceding and following cached frames are encoded into MP4 video and JPEG cover images by the GenerateAlarmVideo module and stored in a specified directory for Admin display. Furthermore, if the user enables the "Inference Stream" option, AvPushStream will also re-encode video frames with overlaid detection boxes and push them back to the MediaServer via RTMP, enabling real-time visualization of the analysis results.

[0051] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

Claims

1. A method for recognizing human-object interaction behavior in mines based on an improved PVIC model, characterized in that, The steps include: Step S1: Mine behavior dataset construction. Collect monitoring videos of underground working faces, and after screening and frame extraction, use data augmentation techniques such as contrast adjustment, Gaussian noise addition, blurring and rotation scaling to simulate the complex underground environment and construct a mine human-object interaction dataset (MineHOI) labeled with <human body, verb, object> triple information. Step S2: Improve the PVIC model construction. Based on the PVIC model, improve the PVIC-FAGLI ​​model from the dimensions of input enhancement, feature extraction and decoding efficiency. The improvements include introducing a gated frequency interaction unit (GFIU), designing an adaptive convolutional attention fusion module (ACAIM) and adopting a grouped linear attention mechanism (GLIA). Step S3: Model training and optimization. The MineHOI dataset is divided into training, validation and test sets according to the proportion. The Adam optimizer and cross-entropy loss function are used to train the PVIC-FAGLI ​​model end-to-end. The parameters are adjusted through ablation experiments to adapt to the mining scenario. Step S4: Real-time behavior recognition and result output. After preprocessing the underground video stream, input it into the trained model to detect the interaction behavior between the miner and the tools and output relevant information. When the target behavior or abnormality is identified, trigger an alarm and record video clips.

2. The method according to claim 1, characterized in that, The gated frequency interaction unit (GFIU) is set at the front end of the model. It transforms the image features to the frequency domain through Fourier transform, enhances the amplitude information, and combines it with phase gating control to improve the feature discrimination of low-light images in the dim environment of the mine and optimize environmental adaptability.

3. The method according to claim 1, characterized in that, The Adaptive Convolutional Attention Fusion Module (ACAIM) adopts a dual-branch parallel structure, in which the convolutional branch captures fine-grained spatial features and the attention branch models global context information. The dual-branch features are fused through an adaptive fusion algorithm to solve the problem of feature detail loss in high-noise environments such as mines.

4. The method according to claim 1, characterized in that, The Grouped Linear Attention (GLIA) mechanism replaces the original full attention mechanism of the PVIC model. It groups the query vectors into a preset number of groups and applies a linear kernel function to each group of query vectors to calculate the attention weights, reducing the computational complexity from O(n²) to approximately O(n), thereby achieving lightweight model and real-time inference.

5. A mine human-object interaction behavior recognition system based on an improved PVIC model, characterized in that, A top-down, layered, modular architecture is adopted, including an application layer, a business layer, and a data layer. Each layer works together to implement the function of any one of the methods described in claims 1-4. The business layer is the core processing layer, which includes a streaming media service module, an algorithm analysis module, and a backend service module. The algorithm analysis module loads the ONNX format PVIC-FAGLI ​​model and accelerates inference through the OpenVINO framework.

6. The system according to claim 5, characterized in that, The streaming media service module supports RTSP and RTMP multi-protocol video stream access, and uses the ZLMediaKit framework to achieve encoding conversion, transpackaging and low-latency transmission. At the same time, it performs lighting and noise optimization preprocessing on the underground video. The data layer uses a MySQL database to store structured data and a local file system to store unstructured data, ensuring data security through backup, caching, and encryption technologies.

7. The system according to claim 5, characterized in that, The application layer includes login, real-time monitoring, alarm management, algorithm deployment and video source management interfaces, supporting video stream addition, parameter configuration, detection result viewing and historical record retrieval. The interface adopts a responsive design to adapt to the mine monitoring center scenario.

Images