Intelligent classification and identification system and method for multi-category power operation violation behaviors

Abstract

The application belongs to the technical field of illegal behavior classification and recognition, and discloses an intelligent classification and recognition system for multi-category illegal behavior of electric power operation, wherein through deep learning technology, in combination with three major scene businesses of safety briefing (substitute), climbing operation and fire operation and general business of the above-mentioned scenes, real-time analysis is carried out on the electric power facility monitoring video, and illegal behavior is automatically recognized, so as to improve the safety management level of the electric power facility. In the electric power video monitoring anti-illegal behavior, the method has a remarkable effect. First, through real-time analysis of the electric power facility monitoring video, the method can timely find illegal behavior and issue an alarm so as to immediately take corresponding measures and avoid potential safety accidents. Secondly, the method has high accuracy and can reduce the omissions and misjudgments of human monitoring and improve the safety management level of the electric power facility.

Description

Technical Field

[0001] This invention belongs to the field of violation classification and identification technology, and in particular relates to an intelligent classification and identification system and method for multiple categories of power operation violations. Background Technology

[0002] With the deepening of the construction of a new power system based on new energy sources, the scale of engineering construction for power grid companies is constantly expanding, production and maintenance tasks are becoming increasingly heavy, the safety situation is becoming more complex, and the pressure on personal safety management is continuously increasing. Statistics show that more than 80% of personal injury accidents in my country's power industry are directly caused by violations of regulations. This painful lesson has prompted power grid companies to prioritize anti-violation work. Although power grid companies have gradually formed a relatively complete working system through years of anti-violation practice, the sheer volume of daily operations means that relying solely on manual supervision is still insufficient to meet actual needs.

[0003] Currently, intelligent technologies have begun to be introduced into the field of safety management and control at power operation sites. For example, the "AI Supervisor for Power Operations" system developed by China Southern Power Grid deeply integrates digital technologies such as natural language understanding and large-scale models to build an AI assistant system for production command. Based on safety regulations and a knowledge base of typical violations, this system can intelligently identify more than 40 types of typical violation clues, such as errors in key content of work orders, omissions in safety measures, abnormal qualifications of operators, and incomplete risk assessments of operations. By automatically setting trigger conditions through real-time monitoring of work order status, it significantly improves the efficiency and accuracy of supervision. In addition, the system also has data processing and analysis capabilities, and can automatically complete in-depth mining of overall operation data and individual violation evidence chains, supporting multi-dimensional data analysis throughout the entire operation process.

[0004] However, the existing technology still has the following technical problems and defects:

[0005] While existing power operation violation identification systems have achieved a shift from manual to intelligent supervision, their core architecture is largely a simple combination of functional modules, lacking deep information interaction and collaborative mechanisms between these modules. Specifically, violation classification, feature extraction, and temporal recognition functions often operate independently. Classification results cannot effectively guide feature extraction, temporal recognition information is difficult to use to optimize classification decisions, and the introduction of attention mechanisms is often a general design lacking dynamic adaptation to specific violation scenarios. This "separate-doing" modular architecture makes it difficult for the system to form an end-to-end data processing and feature enhancement closed loop. When facing multiple complex challenges in power operation scenarios, such as noise interference, imbalanced samples, and subtle and temporally related behaviors, the recognition accuracy and generalization ability are significantly limited. In particular, for easily confused violations that rely on temporal context (such as "normal passage" and "illegal entry"), existing systems often struggle to make accurate judgments due to the lack of a deep fusion mechanism between temporal information and static classification. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides an intelligent classification and identification model for multiple categories of violations in electrical work.

[0007] This invention is implemented as follows: an intelligent classification and identification model for multiple categories of electrical work violations includes:

[0008] The module includes a preprocessing module, a violation classification module, a feature extraction module, an information recognition module, and an attention module.

[0009] The preprocessing module, connected to the violation classification module, is used to eliminate data inconsistencies by filling in missing data, smoothing or deleting outlier data, and to achieve data cleaning; it also uses technical means to process limited samples, supplement training samples, and improve the model's feature extraction performance.

[0010] The violation classification module, connected to the preprocessing module, feature extraction module, and information recognition module, is used to classify violations in power operations.

[0011] The feature extraction module, connected to the violation classification module and the information recognition module, is used to extract violation data;

[0012] The information recognition module, connected to the violation classification module, feature extraction module, and attention module, is used to identify power operation behavior information based on the time-shifting module;

[0013] The attention module, connected to the information recognition module, is used to introduce an attention mechanism into the neural network. During the training phase, it makes the network pay more attention to the characteristics of personnel violations, assigns more weight to important information, and reduces attention to relatively unimportant information, thereby obtaining a more accurate representation of the characteristics of personnel violations.

[0014] Furthermore, the violation classification module:

[0015] Violations at electrical work sites include: not wearing a safety helmet, improperly wearing a safety belt, not using a safety rope, crossing a hard fence, crossing or crawling over a warning line, smoking on site, placing oxygen cylinders upside down, placing acetylene cylinders upside down, placing oxygen and acetylene cylinders close together, not wearing insulated gloves when testing for electricity, improper clothing (short sleeves), no red vest on site, upside-down cable reels, messy power cables, not closing the doors of distribution boxes or switch boxes, not providing rain protection for equipment requiring rain protection, and improperly covering holes.

[0016] Furthermore, the feature extraction module:

[0017] (1) Dynamic behavioral characteristics of video content;

[0018] (2) Static behavior characteristics of image type.

[0019] Furthermore, the video-related dynamic behavior characteristics are as follows:

[0020] The human behavior information contained in video data includes two aspects: spatial correlation within a single frame and temporal correlation between frames, as well as the temporal sequence of the behavior process.

[0021] Furthermore, the static behavioral characteristics of the image class are as follows:

[0022] Using image samples as the main body, the violations are visually represented on the images; the static behavioral features of the images mainly include shape, color, texture, and spatial features; shape features include aspect ratio, invariant moments, etc., and are represented by region and boundary contour features.

[0023] Furthermore, the information recognition module:

[0024] First, the original video is segmented into M segments. Each frame from each segment is extracted as input. Data augmentation is used to improve the model's generalization ability. Each frame is sequentially fed into a ResNet-101 convolutional neural network to extract corresponding features. A time-shifting module (TSM) is embedded in each residual block of the ResNet-101, and the TSM is placed after the first convolutional layer in the residual module. Finally, an attention layer is added to the ResNet-101 to fuse and classify the extracted features, thus completing the task of identifying people's violations in the video.

[0025] Another objective of this invention is to provide a method for constructing an intelligent classification and identification model for multiple categories of violations in electrical work, including:

[0026] Step 1: The preprocessing module eliminates data inconsistencies by filling in missing data, smoothing or deleting outlier data, thus achieving data cleaning; and the limited samples are processed by technical means to supplement training samples and improve the model's feature extraction performance.

[0027] Step 2: Classify electrical operation violations using the violation classification module;

[0028] Step 3: Extract violation data using the feature extraction module;

[0029] Step 4: The information recognition module identifies the power operation behavior information based on the time-shifting module;

[0030] Step 5: Introduce an attention mechanism into the neural network through an attention module. During the training phase, make the network pay more attention to the characteristics of personnel violations, assign more weight to important information, and reduce attention to relatively unimportant information, so as to obtain a more accurate representation of the characteristics of personnel violations.

[0031] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the intelligent classification and identification model construction method for multi-category power operation violations.

[0032] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the intelligent classification and identification model construction method for the multi-category of power operation violations.

[0033] Another objective of this invention is to provide an information data processing terminal, which is used to implement an intelligent classification and identification model for the aforementioned multi-category violations of power operation regulations.

[0034] Based on the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solution to be protected by this invention from the following aspects:

[0035] First, addressing the technical problems existing in the prior art and the difficulty in solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0036] Comprehensive monitoring of power facilities and personnel operations, by real-time registration and fusion of images from different perspectives to form panoramic video images, provides more comprehensive and intuitive information for the safety management of power operations and power facilities.

[0037] This algorithm employs image registration to precisely align image data from multiple cameras or sensors, ensuring their spatial correspondence. Combined with transformation techniques, these registered images are then fused to generate a panoramic video image. Panoramic video provides a comprehensive view of power facilities, helping monitoring personnel fully understand the operational status of the facilities and their surrounding environment, promptly detect violations (such as illegal construction or unauthorized contact with live equipment), understand the real-time condition of the power facilities, and make timely decisions and responses.

[0038] By using deep learning technology and combining it with three major business scenarios—safety briefing (on behalf of others), working at height, and hot work—as well as general business processes for the above scenarios, the monitoring videos of power facilities are analyzed in real time to automatically identify violations, thereby improving the safety management level of power facilities.

[0039] The method for identifying illegal video information based on a convolutional neural network model first preprocesses the power facility monitoring video, such as cropping and grayscale conversion, to adapt it to the input requirements of the convolutional neural network. Then, a multi-layer convolutional neural network model is used to extract features and classify the video frames to identify illegal behaviors in the video, such as illegal construction and unauthorized contact with live equipment.

[0040] This method demonstrates significant effectiveness in preventing violations through power video surveillance. Firstly, by analyzing power facility monitoring videos in real time, it can promptly detect violations and issue alerts, enabling immediate action to prevent potential safety incidents. Secondly, its high accuracy reduces human oversights and misjudgments, improving the safety management of power facilities. Research on short-range IoT communication technology based on multi-protocol fusion aims to address the problems of poor interoperability and low resource utilization caused by single communication protocols in complex environments. By integrating multiple short-range communication protocols (such as ZigBee, Bluetooth, Wi-Fi, and LoRa), a highly efficient, flexible, and adaptable communication model is constructed.

[0041] This research on short-range IoT communication technology based on multi-protocol fusion first analyzes the technical characteristics, applicable scenarios, and performance advantages and disadvantages of different short-range communication protocols. A multi-protocol fusion architecture framework is designed to achieve seamless switching and collaborative operation between protocols. The data interaction mechanism between multiple protocols is studied, and protocol conversion and data fusion algorithms are developed to ensure interoperability between devices using different protocols. The resource scheduling strategy in the communication model is optimized to reduce energy consumption and improve data transmission efficiency, while also enhancing the model's anti-interference capability and stability. This research provides an efficient and flexible solution for short-range IoT communication.

[0042] Secondly, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0043] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0044] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0045] (3) Whether the technical solution of the present invention solves the technical problem that people have long wanted to solve but have never been able to solve successfully:

[0046] (4) Does the technical solution of the present invention overcome technical bias? Attached Figure Description

[0047] Figure 1 This is a block diagram of the intelligent classification and recognition model for multiple categories of violations of power operation regulations provided in this embodiment of the invention.

[0048] Figure 2 This is a flowchart of the intelligent classification and identification model construction method for multiple categories of power operation violations provided in this embodiment of the invention.

[0049] Figure 3 The embodiments of the present invention provide (a) a dynamic continuous video frame and (b) a static single-frame motion image.

[0050] Figure 4 This is a flowchart of the power operation site behavior recognition method based on a time-shifting module provided in this embodiment of the invention.

[0051] Figure 5 This is a diagram of the SeNet network structure provided in an embodiment of the present invention.

[0052] Figure 6 This is a structural diagram of a complex attention module provided in an embodiment of the present invention.

[0053] Figure 1 The module consists of: 1. Preprocessing module; 2. Violation classification module; 3. Feature extraction module; 4. Information recognition module; 5. Attention module. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0055] like Figure 1 As shown, the intelligent classification and identification model for multi-category electrical work violations provided in this embodiment of the invention includes:

[0056] Preprocessing module 1, violation classification module 2, feature extraction module 3, information recognition module 4, attention module 5;

[0057] Preprocessing module 1, connected to violation classification module 2, is used to eliminate data inconsistencies by filling in missing data, smoothing or deleting outlier data, and to achieve data cleaning; it also uses technical means to process limited samples, supplement training samples, and improve the model's feature extraction performance.

[0058] The violation classification module 2, connected to the preprocessing module 1, feature extraction module 3, and information recognition module 4, is used to classify violations in power operations.

[0059] Feature extraction module 3 is connected to violation classification module 2 and information recognition module 4, and is used to extract violation data;

[0060] Information recognition module 4, connected to violation behavior classification module 2, feature extraction module 3, and attention module 5, is used to identify power operation behavior information based on the time-shifting module;

[0061] Attention module 5, connected to information recognition module 4, is used to introduce an attention mechanism into the neural network. During the training phase, it makes the network pay more attention to the characteristics of personnel violations, assigns more weight to important information, and reduces attention to relatively unimportant information, thereby obtaining a more accurate representation of the characteristics of personnel violations.

[0062] This invention provides an intelligent classification and recognition model for multiple categories of violations in power operations. Its core lies in constructing an end-to-end closed-loop system for data processing and feature enhancement. This system is not a simple aggregation of preprocessing, classification, feature extraction, recognition, and attention mechanisms, but rather solves comprehensive technical challenges such as scarce violation samples, significant environmental interference, and subtle and diverse behavioral features in power operation scenarios through deep coupling and functional collaboration between modules. Its overall workflow and internal collaboration mechanism are as follows:

[0063] 1. Integrated preprocessing for data governance and sample augmentation (Module 1 → Module 2)

[0064] Upon input of raw power operation monitoring video streams or image data, preprocessing module 1 is initiated first. This module does not perform data cleaning in isolation; its primary function is to lay a "clean and rich" data foundation for all subsequent high-level analyses. On one hand, it eliminates data inconsistencies caused by acquisition equipment failures, transmission packet loss, or ambient light interference by filling in missing data, smoothing or removing outliers, preventing noise from being incorrectly amplified as "violation features" in subsequent deep networks. On the other hand, addressing the scarcity of samples for certain high-risk violations in power operations (such as specific operational errors), this module processes the limited samples using technical means (such as generative adversarial networks or data augmentation algorithms) to supplement training samples in a "data-driven" manner. This operation is directly tied to the performance of feature extraction module 3, aiming to force the model to learn more robust feature representations in the early stages of training, improving the model's generalization ability under long-tailed distributions. The processed, cleaned, and augmented data is then fed into violation classification module 2.

[0065] 2. Classification-guided multidimensional feature extraction (Module 2) Module 3)

[0066] The violation classification module 2, acting as the system's "task hub," receives preprocessed data and sends classification task guidance to the feature extraction module 3. The feature extraction module 3 does not perform general feature extraction aimlessly; instead, under the "supervision" of the classification task, it selectively extracts low- and mid-level features strongly correlated with violations in power operations, such as personnel outlines, tool positions, equipment status indicators, and operational trajectories. This collaborative mechanism of "classification-guided extraction" ensures the directionality of feature extraction and avoids wasting computational resources on background information unrelated to violations.

[0067] 3. Fusion of temporal behavior recognition and spatial features (Module 4) Module 3, Module 2)

[0068] Violations in electrical work often manifest as continuous time-series actions. To address this, the information recognition module 4 introduces an innovative "time-shift module." This module receives spatial features from the feature extraction module 3 and specifically performs time-series modeling of electrical work behavior information. Crucially, the interaction between the information recognition module 4 and the time-series features is bidirectional: it not only passively receives features, but the identified temporal context information (such as the sequence and duration of actions) is fed back to the violation classification module 2, assisting it in performing more accurate coarse-grained classification. Simultaneously, the results of time-series recognition also provide important weight allocation criteria for the subsequent attention mechanism.

[0069] 4. Dynamic feature calibration of attention mechanisms (Module 5) Module 4 (Closed-loop feedback)

[0070] Attention module 5 is crucial for enhancing the model's discriminative power, and its direct connection with information recognition module 4 forms the core of the system's "feature calibration." This module is not simply a generic attention layer introduced into the network, but rather customized for the power operation scenario: it dynamically introduces an attention mechanism into the neural network based on the temporal behavior segments identified by information recognition module 4. During the training phase, this mechanism drives the network to pay more attention to spatiotemporal regions containing personnel violations, assigning higher weights to feature channels or spatial locations containing key violation information, while suppressing interference from irrelevant information such as changes in background lighting and slight shaking. This collaborative strategy of "temporal domain recognition guiding spatial domain attention allocation" enables the model to obtain more accurate and robust representations of personnel violation features, ultimately returning the highly discriminative feature vectors to the classification module for final judgment.

[0071] The five modules of this invention form an organic whole that supports and interacts with each other through a complete technical chain of "data governance—task guidance—spatiotemporal extraction—temporal modeling—feature calibration." It is this end-to-end collaborative mechanism that enables the model to systematically solve a series of related technical problems, from data quality and sample imbalance to the capture of subtle behaviors. This results in the efficient and accurate identification of multiple categories of violations in power operations, achieving unexpected technical effects that cannot be achieved by a single module or simple combination.

[0072] The violation classification module provided in this embodiment of the invention:

[0073] Violations at electrical work sites include: not wearing a safety helmet, improperly wearing a safety belt, not using a safety rope, crossing a hard fence, crossing or crawling over a warning line, smoking on site, placing oxygen cylinders upside down, placing acetylene cylinders upside down, placing oxygen and acetylene cylinders close together, not wearing insulated gloves when testing for electricity, improper clothing (short sleeves), no red vest on site, upside-down cable reels, messy power cables, not closing the doors of distribution boxes or switch boxes, not providing rain protection for equipment requiring rain protection, and improperly covering holes.

[0074] The feature extraction module provided in this embodiment of the invention:

[0075] (1) Dynamic behavioral characteristics of video content;

[0076] (2) Static behavior characteristics of image type.

[0077] The video-related dynamic behavior features provided in this embodiment of the invention are as follows:

[0078] The human behavior information contained in video data includes two aspects: spatial correlation within a single frame and temporal correlation between frames, as well as the temporal sequence of the behavior process.

[0079] The static behavior characteristics of image classes provided in this embodiment of the invention are as follows:

[0080] Using image samples as the main body, the violations are visually represented on the images; the static behavioral features of the images mainly include shape, color, texture, and spatial features; shape features include aspect ratio, invariant moments, etc., and are represented by region and boundary contour features.

[0081] The information recognition module provided in this embodiment of the invention:

[0082] First, the original video is segmented into M segments. Each frame from each segment is extracted as input. Data augmentation is used to improve the model's generalization ability. Each frame is sequentially fed into a ResNet-101 convolutional neural network to extract corresponding features. A time-shifting module (TSM) is embedded in each residual block of the ResNet-101, and the TSM is placed after the first convolutional layer in the residual module. Finally, an attention layer is added to the ResNet-101 to fuse and classify the extracted features, thus completing the task of identifying people's violations in the video.

[0083] like Figure 2 As shown in the figure, the intelligent classification and identification model construction method for multi-category electrical operation violations provided by this invention includes:

[0084] S101 uses a preprocessing module to eliminate data inconsistencies by filling in missing data, smoothing or deleting outlier data, etc., to achieve data cleaning; it also uses technical means to process limited samples, supplement training samples, and improve the model's feature extraction performance.

[0085] S102, classify violations of electrical work regulations through the violation classification module;

[0086] S103, extract violation data through the feature extraction module;

[0087] S104, the information recognition module identifies power operation behavior information based on the time-shifting module;

[0088] S105 introduces an attention mechanism into the neural network through an attention module, making the network pay more attention to the characteristics of personnel violations during the training phase, assigning more weight to important information, while reducing attention to relatively unimportant information, thus obtaining a more accurate representation of the characteristics of personnel violations.

[0089] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention will be further described in detail below with reference to the accompanying drawings and flowcharts. It should be particularly noted that the various steps in this method are not isolated operations, but are deeply coupled in terms of timing logic and data flow. Through a collaborative mechanism where preceding steps lay the foundation for subsequent steps and subsequent steps in turn optimize preceding tasks, the technical challenge of identifying violations in power operations is systematically solved.

[0090] This invention provides a method for constructing an intelligent classification and identification model for multiple categories of electrical work violations. Its core lies in building a closed-loop processing chain of "data augmentation - task guidance - temporal modeling - feature calibration." Through the organic coordination of steps S101 to S105, this method achieves end-to-end optimization from raw data to high-precision violation identification. Its overall collaborative working mechanism is as follows:

[0091] 1. Underlying support for data quality and sample distribution (S101 → all subsequent steps)

[0092] Step S101 is not simply data cleaning, but rather the foundation for a high signal-to-noise ratio and balanced data distribution for the entire model.

[0093] Cooperative Point A (Noise Interference Countermeasures): Power work sites are subject to numerous noise sources, including variations in lighting and electromagnetic interference. In S101, smoothing or removing outliers effectively cuts off the path of noise propagation, preventing it from being misidentified as a violation feature in S103 feature extraction and S104 time-series identification. This improves the input signal-to-noise ratio for all subsequent steps.

[0094] Collaboration Point B (Solving Imbalanced Samples): To address the issue of insufficient samples of dangerous violations, S101 supplements the training samples through technical means. This operation directly benefits the training effect of the S105 attention module—only with sufficient samples can the attention mechanism learn the truly important "minority" violation features, rather than overfitting due to data scarcity.

[0095] 2. Targeted feature extraction guided by classification tasks (bidirectional coupling of S102 and S103)

[0096] Steps S102 (classification) and S103 (extraction) are not simply called sequentially, but rather form a two-way interaction between the task and the data.

[0097] Collaborative Mechanism: S102 first defines the category boundaries of violations (such as not wearing a safety helmet, improper operation, etc.). This classification task acts as a "higher-level instruction," driving S103 to selectively focus on features strongly correlated with these categories (such as head region, hand trajectories) when extracting data. Simultaneously, the quality of the underlying features extracted by S103 directly affects the adjustment of the classification boundaries in S102. This collaborative approach of "classification-guided extraction, extraction-optimized classification" ensures that feature extraction is no longer a blind global computation, but rather a precise computation with a strong task orientation.

[0098] 3. Spatiotemporal feature fusion for temporal recognition and static classification (S104) S102, S103)

[0099] The time-shifting module introduced in step S104 is a key bridge connecting spatial features and the time dimension, and it forms a spatiotemporal linkage with the preceding and following steps.

[0100] Collaborative Mechanism: S104 receives spatial features from S103 and uses a time-shifting module to mine the temporal coherence of actions (e.g., "approaching the meter" + "reaching out" + "reading the meter" constitutes "unauthorized close-range work"). More importantly, the temporal information identified by S104 is not the end point; it is fed back into the classification module of S102 to help the classification module distinguish easily confused violations that are similar in static images but have different temporal contexts (e.g., "normal passing by" versus "unauthorized entry"). This mechanism of temporal information feedback to optimize static classification significantly improves classification accuracy.

[0101] 4. Dynamic feature calibration closed loop of attention mechanism (S105→S104 & global feedback)

[0102] Step S105 is the accuracy guarantee valve for the entire collaborative link, and the attention mechanism it introduces has the core functions of dynamic perception and weight redistribution.

[0103] Collaborative Mechanism (Core Innovation): S105 does not operate independently; it is deeply integrated with the temporal recognition results of S104. Specifically, the key behavioral segments identified by S104 (such as the brief moment of a violation) guide the attention mechanism in S105 to dynamically assign higher weights to features from these segments during the neural network training phase. This means that the focus of the attention module is "awakened" in real time by the preceding temporal recognition module. These "calibrated" features then back-optimize the classification decision of S102, forming an internal closed loop of "identifying key temporal segments → amplifying key features → optimizing classification results."

[0104] These five steps, through forward data flow and backward gradient feedback, form a dynamically optimized whole during model training. This interconnected and mutually beneficial collaborative mechanism enables the model to systematically address multiple complex challenges in power operation scenarios, such as noise interference, sample scarcity, and subtle and time-dependent behaviors, achieving unexpected technical results that could not be achieved by any single module or simple step combination.

[0105] Those skilled in the art should understand that the above division of steps is only for clearly describing the core collaborative logic. In actual model training and inference, the modules corresponding to these steps are always in a state of close information interaction and parameter sharing, together constituting a complete intelligent recognition solution. 1. Principles of the Research Content of this Invention

[0106] Based on the analysis of current power operation construction and monitoring, this paper studies and analyzes the technical theories of image digitization technology, image distribution algorithms, image fusion algorithms, image recognition algorithms, and big data analysis. Building upon previous research, the following research approach is proposed:

[0107] First, we investigated the current status and development trend of key technologies for power operation and construction at home and abroad, and analyzed the feasibility of image digitization technology, image distribution algorithm, image fusion algorithm, image recognition algorithm, big data analysis and other technologies in the development and application of the intelligent panoramic integrated monitoring terminal for anti-violation.

[0108] Secondly, research was conducted on panoramic image fusion algorithms based on real-time image registration algorithms combined with transformation techniques, methods for identifying illegal video information based on convolutional neural network models, and short-range IoT communication technologies based on improving spread spectrum factors combined with frequency hopping techniques, laying a theoretical foundation for the development of intelligent early warning receivers, 360° panoramic monitoring terminals, and intelligent panoramic monitoring and control systems.

[0109] Finally, prototype scenarios for an intelligent early warning receiver, a 360° panoramic monitoring terminal, and an intelligent panoramic monitoring and control system were designed. By combining image digitization technology, image distribution algorithms, image fusion algorithms, image recognition algorithms, big data analysis, and other related technologies, the intelligent early warning receiver, 360° panoramic monitoring terminal, and intelligent panoramic monitoring and control system were constructed and validated, further verifying the feasibility of key technology applications.

[0110] 2.1. Research on Panoramic Image Fusion Algorithm Based on Real-time Image Registration Algorithm Combined with Transformation Technique

[0111] 2.1.1. Panoramic Image Fusion Preprocessing

[0112] Before image stitching and fusion, preprocessing of the acquired images is necessary to ensure the efficiency and accuracy of subsequent registration and fusion. Common image preprocessing methods include image correction and image enhancement.

[0113] Image denoising: The original image may contain various kinds of noise interference, which can be regarded as real information.

[0114] The deviation between the signal and the ideal signal can impair human visual perception and affect our understanding and analysis of image information. Noise is typically an unpredictable random signal, so image smoothing techniques, also known as filtering techniques, are generally used to effectively eliminate noise interference. Traditional filtering algorithms are divided into linear filtering and nonlinear filtering. Mean filtering is a common type of linear filtering, while median filtering is a type of nonlinear filtering.

[0115] Mean filtering: Mean filtering replaces the individual pixel values ​​of the original image with their average value. Consider a neighborhood s of a point (m, n) in image f(x, y). If s contains M pixel values, then the average value of these M pixels replaces the gray level of point (m, n). A 3×3 square neighborhood is typically used, with point (m, n) as its center. Then:

[0116]

[0117] Median filtering: Mean filtering can effectively suppress noise, but it can also blur the image. Therefore, in fields with high precision requirements, nonlinear filtering is often used to effectively filter out noise while preserving image details. Median filtering is a commonly used nonlinear filtering method.

[0118] Median filtering is a nonlinear signal processing technique based on order statistics theory. Its core idea is to first select a square neighborhood centered on a given pixel, then sort the data within that neighborhood, and finally take the median value of the sorted data as the new grayscale value of the center pixel. In this way, if there is noise such as a bright or dark spot, it will appear at either end of the sorted area, thus effectively removing noise. Moving this neighborhood window across the image then smooths the image.

[0119] Image registration: Image registration is the most important step in image stitching. It is defined as the process of optimally matching two or more images obtained from different sensors, at different times, or from different angles.

[0120] Image region fusion: After completing feature point extraction, feature point matching, feature point selection, and transformation matrix calculation, image registration is finished. The next important step in image stitching is image region fusion.

[0121] During image acquisition, variations in light intensity and other factors can lead to gaps during image stitching, necessitating smoothing of the fused image. Image smoothing refers to image processing methods used to highlight wide areas, low-frequency components, and the core of an image, or to suppress noise and high-frequency interference, resulting in gradual brightness changes, reduced abrupt gradients, and improved image quality. In image fusion, smoothing refers to resolving stitching boundaries and color / brightness differences arising after registration, ultimately stitching multiple images into a seamless panoramic image. Smoothing in image fusion must consider two crucial factors: 1. Maintaining smooth image boundaries to effectively eliminate gaps; 2. Ensuring that effective image information is not damaged or lost during fusion. Simply overlaying images will result in noticeable stitching gaps, potentially causing distortion or blurring. Therefore, selecting an appropriate smoothing method for seamless stitching is critical to image fusion. First, the two images to be fused need to be converted to grayscale. Then, a linear energy normalization method is used to balance the grayscale of the two images. By selecting appropriate parameters, the brightness of the two images can be adjusted to be approximately the same. Next, based on the affine transformation matrix obtained during registration, the overlapping area of ​​the two images can be determined. After adjusting the grayscale values ​​of the images to be fused, a new image to be fused can be obtained when the brightness of the two images is consistent.

[0122] Taking offline images as an example, due to issues such as shooting angle and lighting, there are relatively obvious contrasts in brightness and darkness in the images to be fused. However, these variations in brightness and darkness are displayed relatively smoothly in the final image, without obvious stitching artifacts. This demonstrates that the image region fusion method using gradient weights employed in this study can very effectively and smoothly fuse images with different angles and brightness and darkness contrasts, making it applicable to the panoramic video surveillance system of this invention.

[0123] 2.1.2. Implementation of Video Stitching

[0124] Video is composed of a series of still images arranged in a certain temporal and spatial order. These images are called video frames, and keyframes are certain video frames that represent the main content of the video and describe the main events. The sequence of keyframes can comprehensively reflect the main content of the video. By extracting and processing keyframes, the amount of video data can be effectively reduced, saving processing time. Therefore, how to select keyframes that meet the requirements is an important research direction for UAV video stitching.

[0125] Early keyframe extraction was primarily done manually. In recent years, however, an increasing number of keyframe extraction algorithms have been proposed, most of which are based on changes in low-level visual information such as color and texture, or methods like distance clustering. Current keyframe extraction methods can be broadly categorized into five types: methods based on shot boundaries, methods based on image content, clustering-based methods, motion analysis-based methods, and methods based on video stream compression.

[0126] 2.2. Research on Violation Video Information Recognition Method Based on Convolutional Neural Network Model

[0127] 2.2.1. Pre-processing of violations

[0128] The construction of a dataset of traffic violation behavior is crucial for research on violation recognition methods. Datasets form the basis for business analysis and model training, providing an important standard for comparing the performance of different recognition methods, and are a key part of business data analysis research and behavior recognition algorithm development. The updating and development of human behavior datasets serve as a trendsetter in the field of computer vision behavior recognition research. Various publicly available human behavior datasets differ in terms of shooting angle, behavior scene, behavior category, and data size, with some exhibiting significant variations. Based on the characteristics of the datasets, common publicly available datasets in the field of human behavior recognition can be divided into three categories: general datasets, real-world scene datasets, and multi-view datasets.

[0129] 2.2.1.1 Classification of Electricity Violations

[0130] Violations at electrical work sites include: not wearing a safety helmet, improperly wearing a safety belt, not using a safety rope, crossing rigid fences, crossing or crawling over warning lines, smoking on site, storing oxygen cylinders upside down, storing acetylene cylinders upside down, placing oxygen and acetylene cylinders close together, not wearing insulated gloves when testing for electricity, improper attire (short sleeves), no red vests on site, cable reels upside down, power cables haphazardly arranged, distribution boxes or switch boxes not closed, equipment requiring rain protection without rain protection measures, and improperly covering holes. The categories of violations and the rules for judging violations are shown in the table below.

[0131]

[0132] 2.2.1.2. Characteristics of Violation Data

[0133] Workplace violations can be categorized into personnel violations and equipment violations based on the perpetrator. Personnel violations include violations of actions and violations of personnel attire, while equipment violations include violations related to equipment location and defects. Violation data can be categorized into two types based on format: static images and dynamic videos. Static images primarily contain static features, while dynamic videos contain both static and dynamic features.

[0134] Video-related dynamic behavior characteristics

[0135] Compared to two-dimensional images, video data has an additional time dimension. Therefore, the human behavior information contained in video data includes two aspects: the spatial correlation in a single frame image, the temporal sequence of the behavior process, and the temporal correlation between frames. Figure 3 (a) shows a dynamic continuous video frame. Based on the consecutive frames, it can be determined that the person's action is climbing a ladder. Figure 3 (b) shows a static single-frame motion image, which cannot correctly determine whether the person is climbing a ladder, standing still, or descending a ladder.

[0136] Human behavior contains rich semantic information about actions. By tracking the postures and evolution of various parts of the body, human movements can be fully described. Generally, action descriptions primarily employ three types of features: features based on human models, local body features, and global body features. Features are typically described through key points. Key points in video images are usually points that change dramatically in the spatiotemporal dimension, reflecting important feature information of the object during movement. For example, when crossing a fence at a construction site, the position of a person's legs will move significantly between frames, causing substantial changes in the surrounding pixel information. Meanwhile, other parts of the body change very little, and the image pixel information remains largely unchanged. In algorithm design, extracting the data information that undergoes significant changes during this process and determining its positional information is beneficial for recognizing fence-crossing actions.

[0137] Static Behavior Characteristics of Image-related Items

[0138] This type of data primarily uses image samples to visually represent violations, such as construction workers not wearing safety helmets or not wearing safety belts while working at heights. Static behavioral features in images mainly include shape, color, texture, and spatial features. Shape features include aspect ratio and invariant moments, represented by region and boundary contour features. Common methods for shape feature description (also called shape feature descriptors) include chain code and Fourier transform, describing the contours of objects in the image for target detection and retrieval. Color features are visual features that are invariant in direction and position, relatively less affected by changes in image pixels, thus exhibiting good stability and robustness. Texture features mainly describe slowly and periodically changing structures or regions, expressed by the grayscale distribution of the image and its surrounding spatial neighborhood. Statistical features are related to target size and position and are rotationally invariant, primarily used for object appearance characteristics, but lacking specific object information, they are difficult to obtain high-level semantic information. Spatial relationship features represent the three-dimensional position or direction of the image target, including spatial relationships such as connection, adjacency, and intersection. Spatial position information includes relative position information describing up / down and left / right relationships, and absolute position information describing distance.

[0139] 2.2.1.3. Data Preprocessing for Violations

[0140] Data preprocessing mainly has two directions: data cleaning and data augmentation. Data cleaning eliminates inconsistencies by filling in missing data, smoothing, or removing outliers. Data augmentation uses techniques to process limited samples, supplementing training samples and improving model feature extraction performance. When samples are plentiful, data augmentation is generally unnecessary. However, in real-world applications, obtaining sufficient samples is costly, and the collection and acquisition of most samples involves considerable workload and difficulty. Therefore, directly obtaining sufficient samples from real-world applications is challenging. New training samples can be generated through data augmentation techniques such as geometric transformations and pixel adjustments.

[0141] 2.2.2. Violation Information Identification

[0142] This paper analyzes the algorithm and practical application scenarios for identifying violations at power sites, focusing primarily on feature extraction networks and attention networks. Regarding the feature extraction network, considering the algorithm's ability to model spatial and temporal information, a Tempora Shift Module (TSM) is embedded into the 2D convolutional network ResNet101, resulting in the TSM-ResNet101 2D convolutional network with spatial and temporal information modeling capabilities. This network serves as the feature extraction network for the algorithm in this invention. Regarding the attention network, most existing models assign the same weight to all input frames of a video, making it difficult to filter important information. Therefore, the model is susceptible to interference from irrelevant information. To address this issue, this invention introduces an attention mechanism into the TSM network, enabling the network to simultaneously focus on two parts in both temporal and spatial dimensions: frames important for action recognition and classification, and the spatiotemporal region. This method effectively identifies various violations such as not wearing a safety helmet, not wearing work clothes correctly, climbing over safety fences, climbing ladders, and falling.

[0143] 2.2.2.1. Power Operation Behavior Recognition Method Based on Time-Shift Module

[0144] The main process of the power operation site behavior recognition method based on time-shift module proposed in this invention is as follows: Figure 4As shown, the original video is first segmented into M segments. Each frame from each segment is extracted as input. Data augmentation is used to improve the model's generalization ability. Each frame is sequentially fed into a ResNet-101 convolutional neural network to extract corresponding features. A time-shifting module (TSM) is embedded in each residual block of the ResNet-101, placed after the first convolutional layer in the residual module. Finally, an attention layer is added to the ResNet-101 to fuse and classify the extracted features, thus completing the task of identifying people's violations in the video.

[0145] 2.2.2.2. Attention Module Construction

[0146] In each frame of a violation image, the various elements are often not of equal importance. Taking an image of a person falling to the ground as an example, for the model's final judgment, background information and the person's walking information before falling are secondary information, while the process from the moment of falling to complete collapse is the key information the model focuses on. However, most existing methods do not assess the importance of this information, making them prone to misjudgment due to interference from secondary information. By introducing an attention mechanism into the neural network, similar to how the human brain acquires information, the network is made to focus more on the characteristics of the violation during the training phase. More weight is allocated to important information, while less attention is paid to relatively unimportant information, thus obtaining a more accurate representation of the characteristics of the violation.

[0147] Attention mechanisms have been widely applied to tasks such as object detection, image recognition, and image segmentation. Their main function is to assign different weights to different feature maps, allowing the model to focus on effective feature maps and extract more crucial information. Unlike previous networks that fused feature channels spatially, SeNet (Squeeze-and-Excitation Networks) models the dependencies between feature channels, obtaining a feature channel importance score. Based on this score, it increases the weight of important feature channels while suppressing unimportant features, thereby improving network performance. The SeNet network structure is as follows: Figure 5 As shown, the SeNet module mainly consists of three parts: a global average pooling layer

[90] (GAP), a fully connected layer (FC), and a sigmoid function. The role of the fully connected layer is to capture cross-channel interaction information. The weight parameters output by the SeNet attention mechanism are channel weight parameters. After processing by the SeNet attention mechanism, the feature matrix obtains a set of weight parameters representing the channel weight values ​​of the feature matrix. The specific attention mechanism uses matrix dot product operation to process the channel weight parameters and the input feature matrix, thereby weakening the unimportant channel information of the input feature matrix and enhancing the important channel information of the input feature matrix.

[0148] Complex attention modules often achieve better performance, but they also increase the computational complexity of the model. To strike a balance between model performance and computational complexity, ECANet9 improves and optimizes SeNet by enhancing cross-channel interaction and dimensionality reduction. With fewer parameters, it effectively improves the network model's ability to extract important features, as shown in the following structure: Figure 6 As shown, unlike other channel attention mechanisms that acquire global information, ECANet primarily employs local cross-channel interaction, reducing computational complexity while maintaining model performance. Considering that fully connected layers represent global interactions between channels, but global interactions are meaningless for convolutional neural networks, ECANet removes fully connected layers from the original SeNet module. However, local interactions implemented through a one-dimensional convolutional structure are more efficient than global interactions between channels.

[0149] 2.3. Research on Short-Range IoT Communication Technology Based on Multi-Protocol Fusion

[0150] 2.3.1. Current Status of Wireless Communication Protocols

[0151] The detailed research on the current status of wireless communication protocols covers the technical characteristics, application scenarios, and development trends of mainstream protocols, aiming to comprehensively analyze the current status and future development direction of wireless communication technology.

[0152] The research focuses on the core technical characteristics of mainstream protocols such as Wi-Fi, Bluetooth, ZigBee, 5G, and LoRa, including key indicators such as transmission rate, coverage, power consumption, anti-interference capability, and security. For example, Wi-Fi, with its high bandwidth and wide coverage, has become the preferred choice for home and office networks, but its power consumption is relatively high; Bluetooth, with its low power consumption and short-range transmission characteristics, is widely used in smart devices and the Internet of Things (IoT); ZigBee focuses on low power consumption, low data rate, and self-organizing networking capabilities, making it suitable for industrial automation and smart homes; 5G, as a new generation of mobile communication technology, is driving the development of smart cities and connected vehicles with its ultra-high speed, low latency, and massive connectivity; and LoRa, with its long-range and low-power characteristics, occupies an important position in wide-area IoT.

[0153] This study provides an in-depth analysis of the current application status of various protocols in fields such as smart homes, industrial IoT, smart cities, connected vehicles, and healthcare, and assesses their advantages and limitations in practical applications. For example, Wi-Fi performs excellently in home entertainment and office scenarios, but is susceptible to interference in industrial environments; ZigBee performs outstandingly in smart homes, but its transmission rate is relatively low; 5G shows great potential in connected vehicles and smart cities, but its deployment cost is high.

[0154] 2.3.2. Research on the Convergence of Wireless Communication Protocols

[0155] Research on wireless communication protocol convergence aims to address the limitations of single protocols in terms of coverage, transmission rate, power consumption, and compatibility by integrating the advantages of multiple communication protocols to meet the needs of complex application scenarios. Research focuses on designing multi-protocol collaborative mechanisms to achieve seamless switching and data sharing between different protocols through a unified communication framework. For example, in smart homes, combining the high bandwidth of Wi-Fi with the low power consumption of ZigBee can improve overall network performance. Simultaneously, the research also addresses resource allocation and optimization issues in protocol convergence, reducing interference between protocols and improving spectrum utilization through dynamic spectrum management and intelligent scheduling algorithms. Furthermore, security is a key challenge in protocol convergence; research requires designing cross-protocol security mechanisms to ensure the privacy and integrity of data transmission. At the technical implementation level, the research explores the co-optimization of hardware and software, developing chips and modules that support multi-protocol communication to reduce device costs and power consumption. Through protocol convergence, the research aims to build efficient, flexible, and scalable communication systems, providing technical support for applications in fields such as the Internet of Things, smart cities, and the Industrial Internet, and promoting the innovation and development of wireless communication technology.

[0156] 1. Protocol Feature Analysis and Requirements Definition

[0157] In the protocol characteristic analysis and requirements definition phase, a comprehensive analysis of the technical characteristics of mainstream wireless communication protocols (such as Wi-Fi, Bluetooth, ZigBee, 5G, LoRa, etc.) is first conducted, including key indicators such as transmission rate, coverage, power consumption, anti-interference capability, and security. Through comparative analysis, the advantages and disadvantages of each protocol and their applicable scenarios are clarified. For example, Wi-Fi is suitable for high bandwidth requirements but has higher power consumption, while ZigBee performs excellently in low-power scenarios but has a lower transmission rate. Based on the analysis results, and combined with specific application scenarios (such as smart homes, industrial IoT, smart cities, etc.), specific requirements for protocol integration are defined. For example, in smart homes, it is necessary to simultaneously meet the requirements of high bandwidth (Wi-Fi) and low power consumption (ZigBee); in industrial IoT, it is necessary to balance high reliability (5G) and long-distance transmission (LoRa). Clarifying these requirements provides direction for the subsequent design of a multi-protocol collaborative framework.

[0158] 2. Multi-protocol collaborative framework design

[0159] Multi-protocol collaborative framework design is a core aspect of protocol fusion research, aiming to achieve seamless switching and data sharing between different protocols. First, a unified communication framework is designed to support the parallel operation and dynamic switching of multiple protocols. For example, in smart homes, devices can automatically switch to Wi-Fi (high bandwidth) or ZigBee (low power) based on task requirements. Second, dynamic spectrum management and intelligent scheduling algorithms are researched to optimize resource allocation, reduce interference between protocols, and improve spectrum utilization. For instance, intelligent scheduling algorithms dynamically allocate spectrum resources in environments where Wi-Fi and Bluetooth coexist, avoiding conflicts. Furthermore, hardware modules and software interfaces supporting multi-protocol communication are developed to ensure system efficiency and compatibility. Through the design of the multi-protocol collaborative framework, flexible and efficient communication solutions are provided for complex application scenarios.

[0160] 3. Development of cross-protocol security mechanisms

[0161] Developing cross-protocol security mechanisms is a crucial step in protocol convergence research, aiming to address security challenges in multi-protocol environments. First, it involves researching encryption algorithms and authentication technologies to ensure the privacy and integrity of data transmission. For example, designing a unified encryption key management mechanism to support secure communication between different protocols. Second, it requires developing data integrity verification technologies to prevent data tampering or loss during transmission. For instance, in Wi-Fi and ZigBee convergence scenarios, designing cross-protocol data verification mechanisms ensures data integrity during transmission between different protocols. Furthermore, it necessitates constructing a multi-layered security protection system, including security mechanisms at the network, transport, and application layers, to prevent data leakage and network attacks. Through the development of cross-protocol security mechanisms, reliable security guarantees are provided for multi-protocol convergence systems.

[0162] 4. Simulation and Experimental Verification

[0163] Simulation and experimental verification are crucial steps in protocol fusion research, aiming to test the performance and stability of multi-protocol collaborative frameworks. First, a simulation environment is built to model multi-protocol fusion scenarios, such as smart homes, industrial IoT, and smart cities. Simulation tests evaluate the real-time performance of protocol switching, the efficiency of resource allocation, and the effectiveness of security mechanisms. For example, in a smart home scenario, the seamless switching performance between Wi-Fi and ZigBee is tested, recording switching latency and data transmission success rate. Second, experimental verification tests the system's performance in real-world operating environments, recording key performance indicators (such as latency, throughput, and power consumption). For example, in an industrial IoT scenario, the collaborative performance of 5G and LoRa is tested, evaluating the system's reliability and stability. Simulation and experimental verification provide data support for system optimization.

[0164] 2.3.3. Research on Short-Range IoT Communication Technology with Multi-Protocol Fusion

[0165] This research on multi-protocol fusion short-range IoT communication technology aims to build an efficient, flexible, and low-power communication system by integrating the advantages of short-range communication protocols such as Wi-Fi, Bluetooth, and ZigBee to meet the needs of complex IoT application scenarios. The research focuses on designing multi-protocol collaborative mechanisms to achieve seamless switching and data sharing between different protocols through a unified communication framework. For example, in smart homes, combining the high bandwidth of Wi-Fi with the low power consumption of ZigBee can improve overall network performance. Simultaneously, dynamic spectrum management and intelligent scheduling algorithms are studied to optimize resource allocation, reduce interference between protocols, and improve spectrum utilization. In terms of security, cross-protocol security mechanisms are designed to ensure the privacy and integrity of data transmission, such as developing unified encryption key management technologies and data integrity verification mechanisms. Furthermore, hardware modules and software interfaces supporting multi-protocol communication are researched to reduce device costs and power consumption, and improve system compatibility and scalability. Through simulation and experimental verification, the performance of the multi-protocol fusion system in different scenarios is tested, such as the real-time performance of protocol switching, the efficiency of resource allocation, and the effectiveness of security mechanisms. Key performance indicators (such as latency, throughput, and power consumption) are recorded to provide data support for system optimization. Ultimately, through practical application verification, we will promote the implementation and popularization of multi-protocol integration technology, and provide innovative solutions for the development of short-range IoT communication technology.

[0166] 1. Protocol Feature Analysis and Requirements Definition

[0167] In the protocol characteristic analysis and requirements definition phase, a comprehensive analysis of the technical characteristics of short-range communication protocols such as Wi-Fi, Bluetooth, and ZigBee is conducted, covering key indicators such as transmission rate, coverage, power consumption, anti-interference capability, and security. Through comparative analysis, the advantages and disadvantages of each protocol and their applicable scenarios are clarified. For example, Wi-Fi is suitable for high bandwidth requirements but has higher power consumption; Bluetooth performs excellently in short-range transmission but has limited coverage; and ZigBee excels in low power consumption and self-organizing network capabilities but has a lower transmission rate. Based on the analysis results, and combined with specific application scenarios (such as smart homes and industrial IoT), specific requirements for protocol integration are defined. For example, in smart homes, the requirements of high bandwidth (Wi-Fi) and low power consumption (ZigBee) need to be met simultaneously; in industrial IoT, the requirements of high reliability (Bluetooth) and long-range transmission (ZigBee) need to be balanced. Clarifying these requirements provides direction for the subsequent design of a multi-protocol collaborative framework.

[0168] 2. Multi-protocol collaborative framework design

[0169] The core of this research is the design of a multi-protocol collaborative framework, aiming to achieve seamless switching and data sharing between different protocols, providing flexible and efficient communication solutions for complex application scenarios. A unified communication framework is designed to support the parallel operation and dynamic switching of multiple protocols. For example, in smart homes, devices can automatically switch to Wi-Fi (high bandwidth) or ZigBee (low power) based on task requirements to meet communication needs in different scenarios. Dynamic spectrum management and intelligent scheduling algorithms are researched to optimize resource allocation, reduce interference between protocols, and improve spectrum utilization. For example, in environments where Wi-Fi and Bluetooth coexist, intelligent scheduling algorithms dynamically allocate spectrum resources to avoid frequency band conflicts and ensure communication stability and efficiency. Hardware modules and software interfaces supporting multi-protocol communication are developed to ensure system efficiency and compatibility. Hardware modules integrate multiple communication chips to support rapid switching between protocols; software interfaces implement data format conversion and transmission control, improving system flexibility and scalability. Through the design of the multi-protocol collaborative framework, an efficient, flexible, and highly compatible communication system is constructed, providing reliable technical support for complex scenarios such as smart homes and the Industrial Internet of Things.

[0170] 3. Resource allocation and optimization

[0171] Resource allocation and optimization are key aspects of this research, aiming to improve the efficiency and performance of multi-protocol fusion systems and ensure their stable operation in complex application scenarios. The research focuses on dynamic spectrum management technology, which dynamically allocates spectrum resources by monitoring spectrum usage in real time, reducing interference between protocols; designing intelligent scheduling algorithms to optimize protocol switching and data transmission strategies based on task requirements and network status; and researching energy management technology to optimize device power consumption and extend battery life. Through resource allocation and optimization, the overall performance of multi-protocol fusion systems is improved, providing efficient and reliable communication support for scenarios such as smart homes and the Industrial Internet of Things (IIoT).

[0172] 4. Development of cross-protocol security mechanisms

[0173] This research focuses on encryption algorithms and authentication technologies, designing a unified encryption key management mechanism to support secure communication between different protocols. For example, in Wi-Fi and ZigBee convergence scenarios, Advanced Encryption Standard (AES) and Elliptic Curve Cryptography (ECC) are employed to ensure data confidentiality during transmission. Data integrity verification technologies are developed to prevent data tampering or loss during transmission. For instance, in cross-protocol communication, hash algorithms and digital signature technologies are introduced to ensure the integrity and authenticity of data transmitted between different protocols. A multi-layered security protection system is constructed, covering security mechanisms at the network, transport, and application layers. At the network layer, firewalls and Intrusion Detection Systems (IDS) are used to prevent external attacks; at the transport layer, Secure Sockets Layer (SSL) and Transport Layer Security (TLS) are used to encrypt communication; at the application layer, access control mechanisms and permission management systems are designed to prevent unauthorized access. Furthermore, cross-protocol security policy synchronization technologies are researched to ensure that security mechanisms between different protocols can work together, avoiding security vulnerabilities. Through the development of cross-protocol security mechanisms, comprehensive and reliable security guarantees are provided for multi-protocol convergence systems, meeting the high security requirements of complex application scenarios.

[0174] 5. Simulation and Experimental Verification

[0175] A simulation environment was built to model multi-protocol fusion scenarios. Simulation tests were used to evaluate the real-time performance of protocol switching, the efficiency of resource allocation, and the effectiveness of security mechanisms. Experiments were conducted to verify the system's performance in real-world operating environments, recording key performance indicators (such as latency, throughput, and power consumption). For example, in a power grid violation prevention scenario, the collaborative performance of Bluetooth and ZigBee was tested to evaluate the system's stability and efficiency under high reliability and long-distance transmission requirements. Furthermore, by combining simulation and experimental data, performance differences in different scenarios were analyzed, potential problems were identified, and optimization solutions were proposed. For example, for high-latency or low-throughput scenarios, resource allocation algorithms were adjusted or protocol switching strategies were optimized. Through simulation and experimental verification, the performance of the multi-protocol collaborative framework was comprehensively evaluated, providing reliable data support and technical assurance for system optimization and practical applications.

[0176] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0177] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An intelligent classification and identification system for multiple categories of violations in electrical work, characterized in that, include: The data preprocessing module is used to perform data cleaning and sample enhancement processing on the raw power operation data; The violation classification module, connected to the data preprocessing module, is used to classify violations of power operation based on the processed data. The feature extraction module is bidirectionally connected to the violation classification module and is used to extract violation features under the guidance of the classification task of the violation classification module. The time sequence recognition module is bidirectionally connected to the feature extraction module and connected to the violation classification module. The time sequence recognition module includes a time shift module, which is used to identify power operation behavior based on time sequence information. The attention allocation module, connected to the temporal recognition module, is used to dynamically allocate attention weights in the neural network training based on the temporal information identified by the temporal recognition module to highlight the characteristics of the violation, and to feed back the weighted features to the violation classification module.

2. The system according to claim 1, characterized in that, The data preprocessing module includes: The data cleaning unit is used to eliminate data inconsistencies by filling in missing data, smoothing outlier data, or deleting outlier data. The sample augmentation unit is used to process a limited number of samples to supplement the training samples using data augmentation techniques.

3. The system according to claim 1, characterized in that, The violation classification module is used to classify at least one of the following categories of violations at power operation sites: not wearing a safety helmet, improper wearing of a safety belt, not using a safety rope, crossing a hard fence, crossing or crawling over a warning line, smoking on site, oxygen cylinders upside down, acetylene cylinders upside down, oxygen and acetylene cylinders placed side by side, not wearing insulated gloves when testing for electricity, improper clothing, no red vest on site, cable reels upside down, power lines in disarray, distribution box or switch box doors not closed, equipment requiring rain protection without rain protection measures, and improper covering of holes.

4. The system according to claim 1, characterized in that, The violation behavior features extracted by the feature extraction module include dynamic behavior features of videos and static behavior features of images; the dynamic behavior features of videos include spatial correlation of a single frame image and temporal correlation between frames; the static behavior features of images include shape features, color features, texture features or spatial features.

5. The system according to claim 1, characterized in that, The time-series recognition module is specifically used to: segment the original video into M segments, extract each frame from each segment as input, improve the model's generalization ability by using data augmentation, sequentially feed each frame image into a ResNet-101 convolutional neural network to extract features, embed a time-shifting module in each residual block of the ResNet-101, the time-shifting module is placed after the first convolutional layer in the residual module, and add an attention layer to the ResNet-101 to fuse and classify the extracted features.

6. A method for intelligent classification and identification of multiple categories of electrical work violations using the system described in any one of claims 1 to 5, characterized in that, Includes the following steps: Step 1: The raw power operation data is cleaned and sample augmented using the data preprocessing module to obtain augmented data; Step 2: The violation classification module determines the classification task based on the augmented data and drives the feature extraction module to extract violation features related to the classification task. Step 3: The temporal recognition module performs temporal behavior recognition on the features extracted by the feature extraction module based on the time shift module to obtain the temporal recognition result; Step 4: The attention allocation module dynamically allocates attention weights based on the temporal recognition results to perform weighted calibration of the features; Step 5: Feed the weighted and calibrated features back to the violation classification module to optimize the classification results.

7. The method according to claim 6, characterized in that, The data cleaning in step 1 includes filling in missing data, smoothing outlier data, or deleting outlier data; sample augmentation includes processing limited samples to supplement training samples through data augmentation techniques.

8. The method according to claim 6, characterized in that, In step 2, the violation classification module classifies at least one of the following categories of violations at the power operation site: not wearing a safety helmet, improper wearing of a safety belt, not using a safety rope, crossing a hard fence, crossing or crawling over a warning line, smoking on site, oxygen cylinders upside down, acetylene cylinders upside down, oxygen and acetylene cylinders placed side by side, not wearing insulated gloves when testing for electricity, improper clothing, no red vest on site, cable reels upside down, power lines in disarray, distribution box or switch box doors not closed, equipment requiring rain protection without rain protection measures, and improper covering of holes.

9. The method according to claim 6, characterized in that, The violation behavior features extracted by the feature extraction module in step 2 include video-based dynamic behavior features and image-based static behavior features; the video-based dynamic behavior features include the spatial correlation of a single frame image and the temporal correlation between frames; the image-based static behavior features include shape features, color features, texture features, or spatial features.

10. The method according to claim 6, characterized in that, The specific process of temporal behavior recognition in step 3 includes: dividing the original video into M segments, extracting each frame from each segment as input, using data augmentation to improve the model's generalization ability, feeding each frame image sequentially into a ResNet-101 convolutional neural network to extract features, embedding a time-shifting module in each residual block of the ResNet-101, placing the time-shifting module after the first convolutional layer in the residual module, adding an attention layer to the ResNet-101, and fusing and classifying the extracted features.

Images