A method and system for automatically monitoring personnel behavior analysis and intervention in a station building

Abstract

The application discloses a kind of automatic monitoring station personnel behavior analysis and intervention method and system in process, it is related to the technical field of pollution source automatic monitoring station supervision.The present application obtains real-time monitoring video stream and pollutant monitoring data and carries out pre-processing;Identify target personnel, monitoring equipment and segment sampling pipeline, generate personnel trajectory data, combine environmental target detection results and count the regional stay duration of each interest area;Through time shift network and spatiotemporal diagram convolution network dual path generates action candidate result, determines final action label according to preset station house scene performance benchmark and generates personnel action recognition time sequence;Through the rule engine of pre-established, execute multilevel risk determination, output standardized risk level and structured research and judgment result, trigger corresponding in-process early warning and generate structured law enforcement evidence package.The present application can improve the identification accuracy of station personnel behavior, realize the causal determination of operation behavior and monitoring data, support in-process supervision and law enforcement evidence.

Description

Technical Field

[0001] This invention relates to the field of automatic pollution source monitoring station technology, and in particular to a method and system for analyzing and intervening in the behavior of personnel at automatic monitoring stations. Background Technology

[0002] Automatic pollution source monitoring stations are the core infrastructure for environmental pollutant emission supervision, and the authenticity and accuracy of their monitoring data are crucial for environmental law enforcement and pollution control. Currently, station supervision mainly relies on regular manual inspections and post-event video recording review. This makes it difficult to identify in real time personnel actions that affect the authenticity of monitoring data, such as illegal sampling, equipment parameter tampering, and interference with sampling pipelines. This results in problems such as regulatory lag and low evidence collection efficiency, failing to meet the needs of in-process intervention in supervision.

[0003] Existing station personnel behavior analysis solutions mostly employ single-action recognition models, which struggle to simultaneously ensure accurate recognition of both continuous, global inspection actions and fine hand gestures. In complex station environments, this leads to false positives and false negatives, resulting in insufficient reliability. Furthermore, existing solutions often rely solely on video data for behavior assessment, failing to establish a spatiotemporal coupling between personnel actions and pollutant monitoring data. This makes it impossible to determine the causal relationship between personnel actions and abnormal monitoring data, distinguishing between normal maintenance operations and unauthorized interference, thus limiting the accuracy of risk assessment. In addition, existing systems often have fixed risk assessment rules that cannot be flexibly adjusted according to different station layouts and regulatory standards, resulting in poor scenario adaptability. Therefore, there is an urgent need for a high-precision, configurable, and automated method for analyzing and intervening in personnel behavior at monitoring stations, supporting the linkage between behavior and monitoring data. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes an automatic monitoring method and system for analyzing and intervening in the behavior of personnel at monitoring stations.

[0005] To achieve the above objectives, the present invention employs the following technical solution: An automated monitoring method for analyzing and intervening in the behavior of station personnel includes: S1. Obtain real-time monitoring video streams and pollutant monitoring data from automatic pollution source monitoring stations, and preprocess them to obtain standardized video frames and standardized pollutant monitoring data. S2. Identify target personnel and target monitoring equipment within standardized video frames, and perform instance segmentation on the sampling pipeline to output personnel detection boxes and environmental target detection results; assign personnel tracking tags associated with target personnel to the personnel detection boxes to obtain structured personnel detection boxes, and generate personnel trajectory data; preset regions of interest, and calculate the regional dwell time of each region of interest based on environmental target detection results and personnel trajectory data; S3. Based on standardized video frames, obtain candidate results for continuous operation actions and candidate results for hand interaction actions based on structured personnel detection boxes; determine the final action labels from the candidate results for continuous operation actions and candidate results for hand interaction actions according to the preset station scene performance benchmark, and generate a temporal sequence of personnel action recognition. S4. Based on the time sequence of personnel action recognition, personnel trajectory data, standardized pollutant monitoring data and regional stay duration, the system performs judgment through a preset rule engine and outputs standardized risk level and structured judgment results. S5. Trigger an in-process early warning that matches the standardized risk level based on the structured analysis results, and extract video clips of the violation period from the real-time monitoring video stream, and generate a structured law enforcement evidence package by associating the corresponding personnel trajectory data and standardized pollutant monitoring data.

[0006] As a preferred embodiment of the present invention, the preprocessing includes: The real-time monitoring video stream is sequentially denoised, frame-by-frame extracted, and frame-by-frame timestamp marked to obtain standardized video frames. The pollutant monitoring data are sequentially denoised, outliers removed, and timestamps calibrated to obtain standardized pollutant monitoring data. The standardized video frames and standardized pollutant monitoring data were aligned using a unified time reference.

[0007] As a preferred embodiment of the present invention, the preset interest region, based on the environmental target detection results and personnel trajectory data, calculates the dwell time in each interest region, including: The cabinet area, analysis area, and sampling area within the automatic pollution source monitoring station are predefined as regions of interest. By combining the location of monitoring equipment and sampling pipeline in the environmental target detection results, when the trajectory point in the personnel trajectory data falls into the location of the monitoring equipment or sampling pipeline in the corresponding area of ​​interest, the dwell time in each area of ​​interest is calculated. Simultaneously, trajectory points are continuously collected, and Gaussian kernel density estimation is used to generate and update the heat map of personnel staying in the station, combined with environmental target detection results.

[0008] As a preferred embodiment of the present invention, the step of obtaining candidate results for continuous operation-type actions based on standardized video frames and obtaining candidate results for hand interaction-type actions based on structured person detection boxes includes: Taking consecutive standardized video frames as input, global temporal features are extracted through a temporal shift network and coarse-grained action classification is performed. The output is a continuous operation action label and the corresponding continuous operation action label confidence, forming a continuous operation action candidate result. Human body regions are extracted using structured person detection boxes. High-resolution pose estimation networks are used to extract the coordinates and confidence scores of skeletal key points. A spatiotemporal skeleton map is constructed by building a temporal sequence of skeletal key points in consecutive frames. The spatiotemporal skeleton map is then input into a spatiotemporal graph convolutional network for action classification. The output includes hand interaction action labels and corresponding confidence scores for the hand interaction action labels, forming candidate results for hand interaction actions.

[0009] As a preferred embodiment of the present invention, the step of determining the final action label from the candidate results of continuous operation actions and the candidate results of hand interaction actions based on the preset station scene performance benchmark, and generating a personnel action recognition time sequence, includes: The preset station scenario performance benchmark is based on the performance comparison results of different action labels obtained by testing the temporal shift network and the spatiotemporal graph convolutional network respectively on normal inspection samples, equipment operation samples and illegal operation samples under the automatic pollution source monitoring station scenario; according to the performance comparison results, the continuous operation action set and the hand interaction action set are divided. The determination of the final action label includes: When a continuous operation action tag matches a set of continuous operation actions, but a hand interaction action tag does not match a set of hand interaction actions, the continuous operation action tag is used as the final action tag. When a hand interaction action tag matches a set of hand interaction actions, but a continuous operation action tag does not match a set of continuous operation actions, the hand interaction action tag is taken as the final action tag. When the continuous operation action label and the hand interaction action label are the same and both hit the corresponding set, the action label with the highest confidence value between the continuous operation action label and the hand interaction action label is taken as the final action label. When the continuous operation action tag and the hand interaction action tag are inconsistent and both hit the corresponding set, the action tag with the higher priority is selected as the final action tag according to the priority preset in the station scene performance benchmark. The final action label is bound to the corresponding confidence level, frame timestamp, and personnel tracking marker to obtain the personnel action recognition time sequence.

[0010] As a preferred embodiment of the present invention, the step of performing the judgment through a preset rule engine and outputting standardized risk levels and structured judgment results includes: The final action label in the time sequence of personnel action recognition is matched with a single violation judgment rule in the preset high-risk behavior database. If a match is found, a violation is judged, and the risk level corresponding to the single violation judgment rule is output. Extract the final action label sequence of continuous time sequence from the personnel action recognition time sequence, and match it with the time sequence combination judgment rule in the preset action sequence pattern library. If the match is successful, it is judged as a violation, and the risk level corresponding to the time sequence combination judgment rule is output. Based on the final action label in the personnel action recognition time sequence, the duration of the region of interest corresponding to the final action label, and the frame timestamp, the personnel operation event, the corresponding time window, and the region of interest are determined. Standardized pollutant monitoring data corresponding to the region of interest within the time window are extracted, and the personnel operation event and the standardized pollutant monitoring data are spatiotemporally aligned. If the standardized pollutant monitoring data exceeds the preset normal operating condition range after the personnel operation event ends, and the timestamp of the first time the standardized pollutant monitoring data exceeds the preset normal operating condition range falls within the preset time coupling window after the personnel operation event ends, it is determined that the personnel operation event caused the standardized pollutant monitoring data to deviate from the normal operating condition, and the corresponding risk level is output. Based on the obtained risk levels, the highest risk level is taken as the standardized risk level; The output includes a structured analysis result containing the trigger rule ID, evidence frame timestamp, final action label sequence, trajectory points, and corresponding standardized pollutant monitoring data; the trigger rule ID is the unique identifier of the judgment rule, and the evidence frame timestamp is the frame timestamp of the frame corresponding to the violation.

[0011] As a preferred embodiment of the present invention, the preset rule engine is updated through a visual configuration interface. Specifically, this includes updating the single violation judgment rule in the preset high-risk behavior library, the time sequence combination judgment rule in the preset action sequence pattern library, the preset normal working condition range, and the preset time coupling window through the visual configuration interface, and the update takes effect immediately.

[0012] An automated monitoring system for station personnel behavior analysis and real-time intervention includes: The data acquisition and preprocessing module is used to acquire real-time monitoring video streams and pollutant monitoring data from automatic pollution source monitoring stations, and to preprocess them to obtain standardized video frames and standardized pollutant monitoring data. The visual intelligence computing module is used to identify target personnel and target monitoring equipment within standardized video frames, and to perform instance segmentation on the sampling pipeline, outputting personnel detection boxes and environmental target detection results; it assigns personnel tracking tags associated with target personnel to the personnel detection boxes to obtain structured personnel detection boxes, and generates personnel trajectory data; it presets regions of interest, and calculates the dwell time of each region of interest based on environmental target detection results and personnel trajectory data. The dual-path calculation module is used to obtain candidate results of continuous operation actions based on standardized video frames and candidate results of hand interaction actions based on structured personnel detection boxes; the final action label is determined from the candidate results of continuous operation actions and hand interaction actions according to the preset station scene performance benchmark, and a temporal sequence of personnel action recognition is generated. The multi-dimensional risk assessment module is used to perform judgments based on the time sequence of personnel action recognition, personnel trajectory data, standardized pollutant monitoring data, and regional stay duration, and outputs standardized risk levels and structured assessment results through a preset rule engine. The early warning and configuration output module is used to trigger real-time early warnings that match the standardized risk level based on the structured analysis results, and to extract video clips of the violation period from the real-time monitoring video stream, and generate a structured law enforcement evidence package by associating the corresponding personnel trajectory data and standardized pollutant monitoring data.

[0013] The beneficial effects of this invention are as follows: This invention achieves accurate identification, risk assessment, and real-time intervention of personnel behavior at automatic pollution source monitoring stations through multi-source data collaborative processing and intelligent behavior analysis, demonstrating significant technical advantages. By synchronously preprocessing real-time monitoring video streams and pollutant monitoring data, standardized video frames and standardized pollutant monitoring data are obtained and time-aligned, ensuring the consistency of the spatiotemporal benchmark between video behavior and monitoring data, thus laying a reliable data foundation for subsequent joint analysis.

[0014] By acquiring personnel detection bounding boxes and environmental target detection results through target detection and instance segmentation, and combining personnel tracking tags to generate personnel trajectory data and statistically analyze the dwell time in each region of interest, it is possible to accurately depict the activity status of personnel in key areas such as cabinet area, analysis area, and sampling area, realize refined perception of personnel behavior in high-risk areas, and effectively distinguish between routine inspections and abnormal lingering behavior.

[0015] It adopts a dual-path recognition approach, which combines continuous operation actions and hand interaction actions. It selects the best action label based on the preset performance benchmark of the station scene, taking into account the recognition accuracy of global continuous actions and fine hand operations. This reduces false detections and false negatives caused by single model recognition, improves the reliability of the time sequence of personnel action recognition, and adapts to complex operation scenarios in the station.

[0016] Based on a pre-set rule engine, multi-level judgments are conducted, integrating behavioral characteristics, trajectory information, duration of stay in the area, and standardized pollutant monitoring data for comprehensive analysis. This enables accurate determination of the correlation between personnel operations and abnormal monitoring data, outputting standardized risk levels and structured analysis results, achieving quantitative risk assessment. Simultaneously, based on the analysis results, corresponding real-time warnings are triggered, and violation video clips are extracted and linked with trajectories and monitoring data to form a structured enforcement evidence package. This allows for real-time handling of risky behaviors and complete evidence preservation, improving the timeliness and standardization of supervision, and providing strong support for the intelligent and standardized supervision of automatic pollution source monitoring stations. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart of the method in an embodiment of the present invention; Figure 2 This is a system structure diagram in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, this is an embodiment of the present invention, which provides an automatic monitoring station personnel behavior analysis and real-time intervention method, including: S1. Acquire real-time monitoring video streams and pollutant monitoring data from automatic pollution source monitoring stations, and preprocess them to obtain standardized video frames and standardized pollutant monitoring data.

[0020] In this embodiment, the preprocessing includes: sequentially denoising, frame extraction, and marking frame timestamps on the real-time monitoring video stream to obtain standardized video frames; sequentially denoising, outlier removal, and timestamp calibration on the pollutant monitoring data to obtain standardized pollutant monitoring data; and aligning the standardized video frames and standardized pollutant monitoring data according to a unified time reference.

[0021] Specifically, the real-time monitoring video streams of the monitoring cameras inside the automatic monitoring station of pollution sources are accessed through RTSP, RTMP or GB / T28181 transmission protocols. An independent transmission session is established for each real-time monitoring video stream and an automatic reconnection mechanism is configured. The global time reference is synchronized based on the network time protocol, and the absolute timestamp of each frame of video data is recorded as a unified basis for the timestamp marking of subsequent frames.

[0022] The real-time monitoring video stream is processed sequentially through denoising, frame extraction, and frame-by-frame timestamp marking to obtain standardized video frames. First, the compressed video frames are decoded, prioritizing hardware decoding and automatically downgrading to software decoding in case of decoding failure, converting the compressed frames into raw RGB images. Then, spatiotemporal joint denoising is performed on the raw images. In the spatial domain, bilateral filtering is used to suppress random noise while preserving edge features of the cabinet, sampling pipeline, and monitoring equipment. In the temporal domain, inter-frame difference is used to distinguish moving and stationary regions, and weighted averaging of multiple consecutive frames is used to further reduce noise interference in stationary regions, resulting in a denoised video image. Next, frames are extracted from the denoised video image according to a preset frame extraction step size to reduce computational load while ensuring the accuracy of subsequent action recognition. Finally, each extracted frame is marked with its corresponding timestamp, forming a standardized video frame.

[0023] Simultaneously, pollutant monitoring data from the corresponding automatic monitoring stations of pollution sources is acquired through a read-only data interface. This data includes real-time values ​​of conventional monitoring indicators such as COD, ammonia nitrogen, total phosphorus, total nitrogen, pH, and flow rate, along with corresponding collection timestamps. The pollutant monitoring data undergoes denoising, outlier removal, and timestamp calibration processes to obtain standardized pollutant monitoring data. A sliding window mean filter is used to smooth and denoise the original monitoring data, and the 3σ criterion is used to identify and remove abnormal fluctuations that deviate from the normal fluctuation range. The collection timestamps of the pollutant monitoring data are calibrated according to the aforementioned unified network time reference to correct time deviations caused by data transmission and acquisition, resulting in time-accurate standardized pollutant monitoring data.

[0024] Standardized video frames and standardized pollutant monitoring data are time-aligned using a unified time reference: The frame timestamp of the standardized video frame is used as the query reference, and matching is performed within the time sequence of the standardized pollutant monitoring data. If the difference between the frame timestamp and the collection timestamp of a pollutant monitoring data point is within a preset tolerance range, the monitoring data point is directly associated with the corresponding video frame. In this embodiment, the preset tolerance range is ±1 second. If no matching pollutant monitoring data point is found within the preset tolerance range, linear interpolation is performed using two adjacent valid pollutant monitoring data points to calculate the pollutant monitoring value corresponding to the frame timestamp and complete the association. This achieves a unified spatiotemporal reference for video frame data and pollutant monitoring data, providing a consistent time dimension basis for subsequent joint analysis of personnel behavior and pollutant monitoring data.

[0025] S2. Identify target personnel and target monitoring equipment within standardized video frames, and perform instance segmentation on the sampling pipeline to output personnel detection boxes and environmental target detection results; assign personnel tracking tags associated with target personnel to the personnel detection boxes to obtain structured personnel detection boxes, and generate personnel trajectory data; preset regions of interest, and calculate the regional dwell time of each region of interest based on environmental target detection results and personnel trajectory data.

[0026] In this embodiment, the cabinet area, analysis area, and sampling area within the automatic pollution source monitoring station are preset as regions of interest. Combining the locations of monitoring equipment and sampling pipelines in the environmental target detection results, when the trajectory points in the personnel trajectory data fall into the corresponding monitoring equipment location or sampling pipeline location within the region of interest, the dwell time in each region of interest is calculated. At the same time, trajectory points are continuously collected, and Gaussian kernel density estimation is used to generate and update the heat map of personnel dwell time within the station, based on the environmental target detection results.

[0027] Specifically, standardized video frames are input into a multi-task visual detection model built on the YOLO26 architecture. The multi-task visual detection model includes a personnel detection branch, a target monitoring device detection branch, and a sampling pipeline instance segmentation branch, and performs three types of visual recognition operations in parallel.

[0028] The multi-task visual detection model is pre-trained and optimized. During training, YOLO26 official pre-trained weights are loaded as initial weights, and fine-tuning is performed using an annotated dataset of an automatic pollution source monitoring station scenario. The training dataset contains station monitoring images under different lighting conditions, shooting angles, and personnel operating postures. The annotation content covers three types of annotation information: personnel bounding boxes, monitoring equipment bounding boxes, and sampling pipeline instance masks. During training, a multi-scale feature fusion strategy and a small target label enhancement strategy are used to improve the small target detection accuracy. A hybrid stochastic gradient descent optimizer is used to iteratively optimize the network parameters until the model loss converges, resulting in a multi-task visual detection model adapted to the station scenario.

[0029] The model inference outputs a rectangular personnel detection box corresponding to the target personnel, and also outputs environmental target detection results including the target monitoring equipment detection box and the sampling pipeline instance segmentation mask. The personnel detection box is used to characterize the position and outer dimensions of the target personnel in the standardized video frame, and the environmental target detection results are used to characterize the spatial position of the key monitoring equipment in the station building, as well as the outline and distribution range of the sampling pipeline.

[0030] The personnel detection boxes output frame by frame are input into a multi-target tracking module using the ByteTrack algorithm. Each personnel detection box in each frame is assigned a unique personnel tracking tag bound to the corresponding target personnel, resulting in a structured personnel detection box carrying personnel identification. The multi-target tracking module uses Kalman filtering to predict the position state of the target personnel in the next frame, calculates the intersection-union cost matrix of the detection box and the predicted trajectory box, and first completes the matching of high-confidence detection boxes with existing trajectories using the Hungarian algorithm, then performs secondary association matching on low-confidence detection boxes. This ensures the continuity of personnel tracking tags and reduces identity switching and trajectory breaks in scenarios such as cabinet obstruction and staggered personnel passage within the station building. Based on the structured personnel detection boxes of continuous frames, the position coordinates corresponding to the same personnel tracking tag are concatenated according to the frame timestamp order to generate complete personnel trajectory data. The personnel trajectory data includes the pixel coordinates of each trajectory point, the corresponding frame timestamp, and the associated personnel tracking tag.

[0031] In the pre-defined pixel coordinate system of the station monitoring screen, three functional areas—cabinet area, analysis area, and sampling area—are designated as regions of interest (ROIs). Each ROI is pre-stored in the system configuration as polygon vertex coordinates. Combining the locations of monitoring equipment and sampling pipelines from the environmental target detection results, high-risk operation points such as equipment bodies and pipeline joints within each ROI are spatially labeled. All trajectory points in the personnel trajectory data are traversed, and each trajectory point is checked to determine whether it falls within the monitoring equipment location range or sampling pipeline location range of the corresponding ROI. When a trajectory point hits the above range, the effective dwell time of the target personnel in the corresponding ROI is accumulated based on the frame timestamp, and the final regional dwell time of each ROI is calculated.

[0032] Simultaneously, trajectory point data of all target personnel are continuously collected. Combined with the spatial layout of equipment and pipelines within the station determined by environmental target detection results, the spatial distribution density of trajectory points is continuously calculated using the Gaussian kernel density estimation algorithm to generate a heat map of personnel residence covering the entire station. The density distribution of the heat map is updated periodically with the input of new trajectory data, presenting the hot spots of personnel activity within the station in a visual manner, providing support for subsequent risk assessment and quantitative analysis of operation and maintenance behavior.

[0033] S3. Based on standardized video frames, obtain candidate results for continuous operation actions and candidate results for hand interaction actions based on structured personnel detection boxes. Based on the preset station scene performance benchmark, determine the final action label from the candidate results for continuous operation actions and candidate results for hand interaction actions, and generate a temporal sequence for personnel action recognition.

[0034] In this embodiment, multiple consecutive standardized video frames are used as input. A temporal shift network is used to extract global temporal features and perform coarse-grained action classification, outputting continuous operation action labels and corresponding continuous operation action label confidence scores to form continuous operation action candidate results. Based on structured personnel detection boxes, human body regions are cropped, and a high-resolution pose estimation network is used to extract skeletal keypoint coordinates and confidence scores. A spatiotemporal skeleton map is constructed after building a temporal sequence of continuous frame skeletal keypoints. The spatiotemporal skeleton map is input into a spatiotemporal graph convolutional network for action classification, outputting hand interaction action labels and corresponding hand interaction action label confidence scores to form hand interaction action candidate results.

[0035] Specifically, this embodiment uses a dual-path parallel inference architecture consisting of a continuous operation action recognition path and a hand interaction action recognition path to identify the actions of people in the station building, generating candidate results for continuous operation actions and candidate results for hand interaction actions respectively. The two inference paths run synchronously and output independently.

[0036] For continuous action recognition, an image sequence composed of multiple consecutive standardized video frames is used as input, such as 16 consecutive standardized video frames extracted according to a preset sampling step size. This sequence is then input into a temporal shift network. The temporal shift network uses a lightweight residual network as its backbone. By performing segmented shifting operations on the feature channels along the time dimension, it effectively fuses inter-frame temporal information without introducing additional parameters or computational overhead, simultaneously extracting global spatial semantic features and continuous temporal dynamic features of the image. Finally, a fully connected classification layer completes coarse-grained action classification, outputting continuous action labels and confidence scores for each action category, forming candidate results for continuous actions. This approach is suitable for recognizing continuous actions involving complete limb movements, such as opening and closing cabinet doors or inserting and unplugging sampling pipes.

[0037] For hand interaction action recognition paths, based on structured person detection boxes, the human body region image of the target person is cropped from the corresponding standardized video frames, and the human body region image is input into a high-resolution pose estimation network. The high-resolution pose estimation network adopts a multi-resolution parallel branch architecture, maintaining high-resolution feature representation throughout the feature extraction process, accurately outputting the pixel coordinates and corresponding keypoint confidence scores of human skeletal key points. These skeletal key points cover the key joints of the human torso, limbs, and hands. Subsequently, according to the chronological order of frame timestamps, the coordinates of corresponding skeletal key points from multiple consecutive frames are temporally concatenated to construct a temporal sequence of skeletal key points from consecutive frames. Spatial graph edges are constructed based on the natural connectivity of the human skeleton, and temporal graph edges are constructed based on the correlation between corresponding key points in adjacent frames, jointly constructing a spatiotemporal skeleton graph. This spatiotemporal skeleton graph is input into a spatiotemporal graph convolutional network, and spatiotemporal dynamic features of skeletal movement are extracted through multi-layer spatiotemporal graph convolution operations to complete fine-grained hand interaction action classification, outputting hand interaction action labels and hand interaction action label confidence scores for each action category, forming candidate results for hand interaction actions. This approach is suitable for identifying fine motor hand actions such as replacing water sample containers or tightening pipe fittings.

[0038] In this embodiment, the preset station scene performance benchmark is the performance comparison results of different action labels obtained by testing the temporal shift network and the spatiotemporal graph convolutional network respectively based on normal inspection samples, equipment operation samples and illegal operation samples under the automatic pollution source monitoring station scene; according to the performance comparison results, the continuous operation action set and the hand interaction action set are divided.

[0039] Specifically, the preset station scenario performance benchmark is constructed through pre-testing. A test sample set for the automatic pollution source monitoring station scenario is pre-constructed, covering three categories: normal inspection samples, equipment operation samples, and violation operation samples. The sample set includes station monitoring video clips under different lighting conditions, different personnel operating postures, and different shooting angles, with each sample labeled with a corresponding action category ground truth value. Inference tests are performed on the test sample set using a temporal shift network and a spatiotemporal graph convolutional network, respectively. The accuracy, recall, and F1 score of the two networks for each action label are statistically analyzed to obtain a performance comparison result of the two networks under different action labels, which serves as the preset station scenario performance benchmark.

[0040] Based on the performance comparison results, the action sets are divided into continuous operation action sets and hand interaction action sets: For action tags with global continuous motion characteristics, such as opening and closing cabinet doors and walking, if the recognition performance of the temporal shift network is better, they are classified into the continuous operation action set; For action tags with prominent fine hand interaction characteristics, such as replacing water sample containers and turning pipe joints, if the recognition performance of the spatiotemporal graph convolutional network is better, they are classified into the hand interaction action set.

[0041] Based on the performance comparison results, priorities are assigned to each action label. Action labels with better network recognition performance have higher priority levels. Specifically, in this embodiment, priorities are assigned to action labels based on the performance comparison results of the F1 scores of the temporal shift network and the spatiotemporal graph convolutional network on each action label. Action labels corresponding to networks with better performance have higher priority, as shown in Table 1. Table 1 Action Tag Priority Table

[0042] The determination of the final action label includes: when a continuous operation action label matches a set of continuous operation actions and a hand interaction action label does not match a set of hand interaction actions, the continuous operation action label is taken as the final action label; when a hand interaction action label matches a set of hand interaction actions and a continuous operation action label does not match a set of continuous operation actions, the hand interaction action label is taken as the final action label; when a continuous operation action label and a hand interaction action label are the same and both match the corresponding set, the action label with the highest confidence value between the continuous operation action label and the hand interaction action label is taken as the final action label; when a continuous operation action label and a hand interaction action label are different but both match the corresponding set, the action label with the higher priority is selected as the final action label according to the priority preset by the station scene performance benchmark. Specifically, if the action labels are cabinet door opening / closing and monitoring equipment button touching, and both have the same priority level, then the confidence levels of the corresponding action labels are further compared, and the action label with the higher confidence level is selected as the final action label. If the action labels are normal inspection walking and sampling pipeline connector tightening, then the sampling pipeline connector tightening operation with the higher priority level is selected as the final action label. If the action labels are personnel walking inside the station and equipment surface wiping, then the personnel walking inside the station with the higher priority level is selected as the final action label.

[0043] The final action label is bound to the corresponding confidence level, frame timestamp, and personnel tracking marker to obtain the personnel action recognition time sequence.

[0044] Specifically, after the final action label is determined, the final action label is structurally bound with the corresponding confidence level, frame timestamp, and personnel tracking mark to form a single-frame action recognition result. According to the chronological order of the frame timestamps, all single-frame action recognition results corresponding to the same personnel tracking mark are sorted and spliced ​​to obtain a continuous and complete personnel action recognition time sequence.

[0045] S4. Based on the time sequence of personnel action recognition, personnel trajectory data, standardized pollutant monitoring data, and regional stay duration, the system performs judgment through a preset rule engine and outputs standardized risk levels and structured judgment results.

[0046] In this embodiment, the final action label in the personnel action recognition time sequence is matched with a single violation judgment rule in the preset high-risk behavior database. If a match is found, a violation is determined, and the risk level corresponding to the single violation judgment rule is output.

[0047] Specifically, the preset high-risk behavior database stores multiple single violation judgment rules, each corresponding to a preset final action tag and a matching risk level. The rule engine traverses the final action tags of each frame in the personnel action recognition time sequence and matches them one by one with the rules in the preset high-risk behavior database; if the final action tag of a certain frame matches any single violation judgment rule, it is determined that a violation has occurred at that moment, and the risk level corresponding to that single violation judgment rule is output.

[0048] In this embodiment, examples of the high-risk behavior database are shown in Table 2: Table 2. Example of a High-Risk Behavior Database

[0049] Extract the final action label sequence of continuous time sequence from the personnel action recognition time sequence, and match it with the time sequence combination judgment rule in the preset action sequence pattern library. If the match is successful, it is judged as a violation, and the risk level corresponding to the time sequence combination judgment rule is output.

[0050] Specifically, the preset action sequence pattern library stores multiple temporal combination judgment rules. Each temporal combination judgment rule corresponds to a set of consecutive final action label sequences with sequential logical relationships, as well as a matching risk level. The rule engine extracts consecutive temporal final action label sequences corresponding to the same person tracking mark from the personnel action recognition temporal sequence and performs pattern matching on the action sequences using a sliding time window. If any temporal combination judgment rule is matched within the preset time window, it is determined that there is a combined violation, and the risk level corresponding to that temporal combination judgment rule is output.

[0051] In this embodiment, examples of the action sequence pattern library are shown in Table 3: Table 3. Example table of action sequence pattern library

[0052] Specifically, the rule engine combines the final action label in the personnel action recognition time sequence, the duration of the region of interest corresponding to the final action label, and the frame timestamp to determine the personnel operation event, its corresponding time window, and the region of interest to which it belongs. When the final action label belongs to a preset operation category label and the duration of the corresponding region of interest reaches a preset duration threshold, it is determined to be a complete personnel operation event. The start and end frame timestamps of the event are recorded to form a time window, and the region of interest to which the event belongs is marked. Subsequently, the standardized pollutant monitoring data corresponding to the region of interest within the time window is extracted, and the personnel operation event and the standardized pollutant monitoring data are spatiotemporally aligned based on a unified time reference. If the standardized pollutant monitoring data exceeds the preset normal operating condition range after the personnel operation event ends, and the timestamp of the first time the standardized pollutant monitoring data exceeds the preset normal operating condition range falls within the preset time coupling window after the personnel operation event ends, it is determined that the personnel operation event is the cause of the standardized pollutant monitoring data deviating from the normal operating condition, and the corresponding risk level is output according to the degree of data deviation. The greater the degree of data deviation, the higher the corresponding risk level.

[0053] Specifically, the preset duration thresholds in this embodiment include: if the region of interest is the sampling area, the preset duration threshold is 5 seconds; if the region of interest is the analysis area, the preset duration threshold is 3 seconds; and if the region of interest is the rack area, the preset duration threshold is 8 seconds. In this embodiment, taking pollutant concentration as an example, the preset normal operating condition range is shown in Table 4: Table 4 Normal Operating Condition Range

[0054] In this embodiment, the preset time coupling window is 300 seconds; a data deviation degree and risk level mapping table is used to output the corresponding risk level; in this embodiment, taking COD concentration as an example, the deviation rate (actual value / normal upper limit) is used as the data deviation degree, as shown in Table 5: Table 5 Mapping Table of Deviation Degree and Risk Level

[0055] After completing the above three-layer judgment, the rule engine summarizes the risk levels output by all triggered judgments and takes the highest risk level as the final standardized risk level. Simultaneously, the rule engine generates and outputs a structured analysis result, which includes the trigger rule ID, evidence frame timestamp, final action tag sequence, trajectory points, and corresponding standardized pollutant monitoring data. The trigger rule ID is a unique identifier for the matched judgment rule, used to trace the basis of the judgment; the evidence frame timestamp is the frame timestamp of the video frame corresponding to the time of the violation, used for location and evidence collection; the final action tag sequence and trajectory points are used to reconstruct the entire process of the violation, and the corresponding standardized pollutant monitoring data are used to corroborate the data anomaly, forming a complete and traceable analysis conclusion.

[0056] In this embodiment, the single violation judgment rule in the preset high-risk behavior library, the time combination judgment rule in the preset action sequence pattern library, the preset normal working condition range, and the preset time coupling window are updated through a visual configuration interface, and the update takes effect immediately.

[0057] S5. Trigger an in-process early warning that matches the standardized risk level based on the structured analysis results, and extract video clips of the violation period from the real-time monitoring video stream, and generate a structured law enforcement evidence package by associating the corresponding personnel trajectory data and standardized pollutant monitoring data.

[0058] Specifically, pre-configured early warning and response strategies correspond one-to-one with each standardized risk level: For low-risk events, alarm logs are recorded and pop-up notifications are displayed only on the local monitoring platform at the station, without triggering on-site alarms; for medium-risk events, low-frequency warning signals are triggered by the on-site audio-visual alerts at the station, and alarm notifications are pushed to the terminals of maintenance personnel within the station; for high-risk events, high-frequency audio-visual alarms are triggered at the station, and alarm information is simultaneously pushed to the environmental supervision platform in the jurisdiction; for extremely high-risk events, in addition to on-site audio-visual alarms and platform notifications, SMS messages and pop-up reminders are simultaneously sent to the mobile terminals of law enforcement personnel, with a total early warning push delay of no more than 10 seconds, enabling immediate intervention during the occurrence of violations.

[0059] Based on the timestamps of the evidence frames in the structured analysis results, a preset 10-second time period is traced back and a preset 5-second time period is extended forward to determine the complete evidence collection time window. Continuous video frames within this time window are extracted from the cached real-time monitoring video stream and synthesized into video segments of the violation period with original frame timestamp watermarks. The video encoding format is consistent with the original monitoring stream to ensure the originality and time traceability of the evidence.

[0060] In the structured enforcement evidence package generation stage, a unique evidence package number is used as the primary key to structurally bind and encapsulate the captured video clips from the violation period, the full-time personnel trajectory data associated with the corresponding personnel tracking tags, standardized pollutant monitoring data aligned with the evidence collection time window, trigger rule IDs, standardized risk levels, chronological descriptions of the violation, and the basis for the judgment conclusions, generating a standardized structured enforcement evidence package. This structured enforcement evidence package is simultaneously stored on the local evidence collection server at the station and on the upper-level cloud monitoring platform, supporting multi-dimensional retrieval and export by station identifier, time range, risk level, and personnel tracking tags, providing a complete and traceable closed-loop evidence chain for environmental enforcement.

[0061] like Figure 2 As shown, another embodiment of the present invention provides an automatic monitoring system for station personnel behavior analysis and real-time intervention, comprising: The data acquisition and preprocessing module is used to acquire real-time monitoring video streams and pollutant monitoring data from automatic pollution source monitoring stations, and to preprocess them to obtain standardized video frames and standardized pollutant monitoring data. The visual intelligence computing module is used to identify target personnel and target monitoring equipment within standardized video frames, and to perform instance segmentation on the sampling pipeline, outputting personnel detection boxes and environmental target detection results; it assigns personnel tracking tags associated with target personnel to the personnel detection boxes to obtain structured personnel detection boxes, and generates personnel trajectory data; it presets regions of interest, and calculates the dwell time of each region of interest based on environmental target detection results and personnel trajectory data. The dual-path calculation module is used to obtain candidate results of continuous operation actions based on standardized video frames and candidate results of hand interaction actions based on structured personnel detection boxes; the final action label is determined from the candidate results of continuous operation actions and hand interaction actions according to the preset station scene performance benchmark, and a temporal sequence of personnel action recognition is generated. The multi-dimensional risk assessment module is used to perform judgments based on the time sequence of personnel action recognition, personnel trajectory data, standardized pollutant monitoring data, and regional stay duration, and outputs standardized risk levels and structured assessment results through a preset rule engine. The early warning and configuration output module is used to trigger real-time early warnings that match the standardized risk level based on the structured analysis results, and to extract video clips of the violation period from the real-time monitoring video stream, and generate a structured law enforcement evidence package by associating the corresponding personnel trajectory data and standardized pollutant monitoring data.

[0062] In summary, this invention effectively eliminates temporal discrepancies between multi-source data by standardizing and preprocessing real-time monitoring video streams and pollutant monitoring data, and aligning them according to a unified time benchmark. This provides an accurate and reliable data foundation for subsequent joint analysis of behavior and data. A multi-task visual detection model is used to simultaneously achieve target detection and instance segmentation. Combined with a multi-target tracking mechanism based on the ByteTrack algorithm, the continuity of personnel trajectories is effectively maintained even in station equipment occlusion scenarios. The dwell time in each region of interest is statistically analyzed based on environmental target detection results, and a Gaussian kernel density estimation is used to generate a heatmap of personnel dwell time within the station, enabling refined perception and quantitative analysis of personnel activities. A dual-path action recognition architecture, combining a temporal shift network and a spatiotemporal graph convolutional network, is constructed. Based on a preset station scene performance benchmark, the final action label is output optimally, and a temporal sequence of personnel action recognition is generated. This approach balances the recognition accuracy of continuous operation actions and hand interaction actions, effectively overcoming the limitations of a single model and significantly improving the accuracy and robustness of action recognition in station scenes. By employing a pre-defined rules engine, a three-tiered progressive risk assessment is executed, establishing a causal relationship between personnel operation events and anomalies in standardized pollutant monitoring data. Combined with the degree of data deviation, a standardized risk level is output, significantly improving the accuracy and reliability of risk assessment. Based on the risk level, tiered real-time warnings are triggered, upgrading the regulatory model from post-event retrospection to in-process intervention. Video clips from the period of violation are automatically extracted and linked with multi-source data to generate a structured enforcement evidence package, forming a complete and traceable chain of enforcement evidence. The system supports updating judgment rules and threshold parameters through a visual configuration interface, with immediate effect. It can flexibly adapt to the differentiated regulatory needs of different monitoring stations, possessing excellent scenario adaptability and functional scalability.

[0063] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any other combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0064] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0065] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for automatic monitoring station personnel behavior analysis and in-process intervention, characterized in that, include: S1. Obtain real-time monitoring video streams and pollutant monitoring data from automatic pollution source monitoring stations, and preprocess them to obtain standardized video frames and standardized pollutant monitoring data. S2. Identify target personnel and target monitoring equipment within standardized video frames, perform instance segmentation on the sampling pipeline, and output personnel detection boxes and environmental target detection results; assign personnel tracking tags associated with target personnel to the personnel detection boxes to obtain structured personnel detection boxes, and generate personnel trajectory data; Pre-defined areas of interest are used, and the dwell time in each area of ​​interest is calculated based on environmental target detection results and personnel trajectory data. S3. Based on standardized video frames, candidate results for continuous operation actions are obtained, and based on structured personnel detection boxes, candidate results for hand interaction actions are obtained. Based on the preset station scene performance benchmark, the final action label is determined from the candidate results of continuous operation actions and hand interaction actions, and a time sequence of personnel action recognition is generated. S4. Based on the time sequence of personnel action recognition, personnel trajectory data, standardized pollutant monitoring data and regional stay duration, the system performs judgment through a preset rule engine and outputs standardized risk level and structured judgment results. S5. Trigger an in-process early warning that matches the standardized risk level based on the structured analysis results, and extract video clips of the violation period from the real-time monitoring video stream, and generate a structured law enforcement evidence package by associating the corresponding personnel trajectory data and standardized pollutant monitoring data.

2. The method for automatic monitoring station personnel behavior analysis and real-time intervention according to claim 1, characterized in that, The preprocessing includes: The real-time monitoring video stream is sequentially denoised, frame-by-frame extracted, and frame-by-frame timestamp marked to obtain standardized video frames. The pollutant monitoring data are sequentially denoised, outliers removed, and timestamps calibrated to obtain standardized pollutant monitoring data. The standardized video frames and standardized pollutant monitoring data were aligned using a unified time reference.

3. The method for automatic monitoring station personnel behavior analysis and in-process intervention according to claim 1, characterized in that, The preset regions of interest are determined based on environmental target detection results and personnel trajectory data, including the duration of dwell time in each region. The cabinet area, analysis area, and sampling area within the automatic pollution source monitoring station are predefined as regions of interest. By combining the location of monitoring equipment and sampling pipeline in the environmental target detection results, when the trajectory point in the personnel trajectory data falls into the location of the monitoring equipment or sampling pipeline in the corresponding area of ​​interest, the dwell time in each area of ​​interest is calculated. Simultaneously, trajectory points are continuously collected, and Gaussian kernel density estimation is used to generate and update the heat map of personnel staying in the station, combined with environmental target detection results.

4. The method for automatic monitoring station personnel behavior analysis and real-time intervention according to claim 1, characterized in that, The process of obtaining candidate results for continuous operation-type actions based on standardized video frames and candidate results for hand interaction-type actions based on structured person detection boxes includes: Taking consecutive standardized video frames as input, global temporal features are extracted through a temporal shift network and coarse-grained action classification is performed. The output is a continuous operation action label and the corresponding continuous operation action label confidence, forming a continuous operation action candidate result. Human body regions are extracted using structured person detection boxes. High-resolution pose estimation networks are used to extract the coordinates and confidence scores of skeletal key points. A spatiotemporal skeleton map is constructed by building a temporal sequence of skeletal key points in consecutive frames. The spatiotemporal skeleton map is then input into a spatiotemporal graph convolutional network for action classification. The output includes hand interaction action labels and corresponding confidence scores for the hand interaction action labels, forming candidate results for hand interaction actions.

5. The method for automatic monitoring station personnel behavior analysis and in-process intervention according to claim 4, characterized in that, The process involves determining the final action label from candidate results of continuous operation actions and candidate results of hand interaction actions based on a preset station scene performance benchmark, and generating a time sequence for personnel action recognition, including: The preset station scenario performance benchmark is based on the performance comparison results of different action labels obtained by testing the temporal shift network and the spatiotemporal graph convolutional network respectively on normal inspection samples, equipment operation samples and illegal operation samples under the automatic pollution source monitoring station scenario; according to the performance comparison results, the continuous operation action set and the hand interaction action set are divided. The determination of the final action label includes: When a continuous operation action tag matches a set of continuous operation actions, but a hand interaction action tag does not match a set of hand interaction actions, the continuous operation action tag is used as the final action tag. When a hand interaction action tag matches a set of hand interaction actions, but a continuous operation action tag does not match a set of continuous operation actions, the hand interaction action tag is taken as the final action tag. When the continuous operation action label and the hand interaction action label are the same and both hit the corresponding set, the action label with the highest confidence value between the continuous operation action label and the hand interaction action label is taken as the final action label. When the continuous operation action tag and the hand interaction action tag are inconsistent and both hit the corresponding set, the action tag with the higher priority is selected as the final action tag according to the priority preset in the station scene performance benchmark. The final action label is bound to the corresponding confidence level, frame timestamp, and personnel tracking marker to obtain the personnel action recognition time sequence.

6. The method for automatic monitoring station personnel behavior analysis and in-process intervention according to claim 1, characterized in that, The process of performing judgments through a preset rule engine and outputting standardized risk levels and structured assessment results includes: The final action label in the time sequence of personnel action recognition is matched with a single violation judgment rule in the preset high-risk behavior database. If a match is found, a violation is judged, and the risk level corresponding to the single violation judgment rule is output. Extract the final action label sequence of continuous time sequence from the personnel action recognition time sequence, and match it with the time sequence combination judgment rule in the preset action sequence pattern library. If the match is successful, it is judged as a violation, and the risk level corresponding to the time sequence combination judgment rule is output. Based on the final action label in the personnel action recognition time sequence, the duration of the region of interest corresponding to the final action label, and the frame timestamp, the personnel operation event, the corresponding time window, and the region of interest are determined. Standardized pollutant monitoring data corresponding to the region of interest within the time window are extracted, and the personnel operation event and the standardized pollutant monitoring data are spatiotemporally aligned. If the standardized pollutant monitoring data exceeds the preset normal operating condition range after the personnel operation event ends, and the timestamp of the first time the standardized pollutant monitoring data exceeds the preset normal operating condition range falls within the preset time coupling window after the personnel operation event ends, it is determined that the personnel operation event caused the standardized pollutant monitoring data to deviate from the normal operating condition, and the corresponding risk level is output. Based on the obtained risk levels, the highest risk level is taken as the standardized risk level; The output includes a structured analysis result containing the trigger rule ID, evidence frame timestamp, final action label sequence, trajectory points, and corresponding standardized pollutant monitoring data; the trigger rule ID is the unique identifier of the judgment rule, and the evidence frame timestamp is the frame timestamp of the frame corresponding to the violation.

7. The method for automatic monitoring station personnel behavior analysis and real-time intervention according to claim 6, characterized in that, The preset rule engine can be updated through a visual configuration interface. Specifically, this includes updating the single violation judgment rules in the preset high-risk behavior library, the time sequence combination judgment rules in the preset action sequence pattern library, the preset normal working condition range, and the preset time coupling window through the visual configuration interface, and the updates take effect immediately.

8. An automatic monitoring station personnel behavior analysis and real-time intervention system, applied to the automatic monitoring station personnel behavior analysis and real-time intervention method described in any one of claims 1 to 7, characterized in that, include: The data acquisition and preprocessing module is used to acquire real-time monitoring video streams and pollutant monitoring data from automatic pollution source monitoring stations, and to preprocess them to obtain standardized video frames and standardized pollutant monitoring data. The visual intelligence computing module is used to identify target personnel and target monitoring equipment within standardized video frames, and to perform instance segmentation on the sampling pipeline, outputting personnel detection boxes and environmental target detection results; it assigns personnel tracking tags associated with target personnel to the personnel detection boxes to obtain structured personnel detection boxes, and generates personnel trajectory data; Pre-defined areas of interest are used, and the dwell time in each area of ​​interest is calculated based on environmental target detection results and personnel trajectory data. The dual-path calculation module is used to obtain candidate results for continuous operation actions based on standardized video frames and candidate results for hand interaction actions based on structured personnel detection boxes. Based on the preset station scene performance benchmark, the final action label is determined from the candidate results of continuous operation actions and hand interaction actions, and a time sequence of personnel action recognition is generated. The multi-dimensional risk assessment module is used to perform judgments based on the time sequence of personnel action recognition, personnel trajectory data, standardized pollutant monitoring data, and regional stay duration, and outputs standardized risk levels and structured assessment results through a preset rule engine. The early warning and configuration output module is used to trigger real-time early warnings that match the standardized risk level based on the structured analysis results, and to extract video clips of the violation period from the real-time monitoring video stream, and generate a structured law enforcement evidence package by associating the corresponding personnel trajectory data and standardized pollutant monitoring data.

Images