Electric power operation risk behavior violation intelligent identification method based on machine vision

Abstract

The invention relates to a machine vision-based electric power operation risk behavior violation intelligent identification method, which comprises the following steps of: acquiring an image of a violation scene, and labeling the collected image; preprocessing and storing the image; based on the preprocessed image, carrying out classification training on the extracted feature information, and constructing a classifier model used for identifying violation behaviors; performing autonomous feature learning based on calibrated positive and negative samples, and training a classifier model; and collecting behavior pictures of the on-site overhead working personnel in real time, and performing violation information intelligent identification through the trained classifier model. According to the method, the supervision efficiency of electric power safety supervision personnel is improved, electric power operation site violation behaviors can be timely found, effectively handled and reasonably solved, a template matching method, a deep learning algorithm and a feature extraction and video image processing analysis identification technology are utilized, an intelligent violation identification function is realized, and the method is suitable for popularization and application. Intelligent identification, automatic research and judgment, instantaneous alarm and log recording are carried out on common high-altitude operation personnel behaviors (such as ladder climbing), electrified interval or open fire area calibration, scaffold erection, foundation pit soil piling height, fence limb distance measurement and other violation behaviors on an electric power operation site, and a breakthrough from remote inspection to intelligent supervision is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of frequency image recognition and operation site inspection for power safety monitoring business, and in particular to a method for intelligently identifying violations of risky behaviors in power operations based on machine vision. Background Art

[0002] Grid safety management is a huge, complex, highly theoretical and operational system engineering. With the development of smart power plants (substations), the optimization and streamlining of professionals, the steady expansion of business, and the continuous advancement of terminal integration, the scope of safety has expanded accordingly, new safety risks have continued to emerge, and the pressure on grid safety management has increased. Although the current safety supervision methods for on-site operations are relatively complete, they still need to be manually implemented by relevant safety management personnel. On-site safety control work only relies on document supervision, material feedback, on-site supervision, and spot checks to improve the level of safety management at the power operation site, which cannot guarantee 100% implementation; and smart power plant (substation) operations have many problems such as multiple sites, multiple equipment, and multiple complex operations. It is impossible to supervise the power operation site around the clock, throughout the process, and in all directions, and it also consumes a lot of manpower, material and financial resources.

[0003] At present, video surveillance systems are usually used to remotely collect, identify and analyze images and videos of power operation scenes, key work areas and entrances and exits of specific construction areas; however, considering that the behavior management of most on-site workers in power operations is subject to factors such as construction complexity, instantaneous dynamics, behavioral variability and unpredictability, such as: not wearing safety helmets, operating mechanical equipment in violation of regulations, workers mistakenly entering live intervals, inadequate safety measures for high-altitude operations, and ranging of live equipment in substation operations, it is not economical to deploy fixed video surveillance systems, and it is difficult to provide real-time monitoring, timely reminders and strong protection for the safety of workers.

[0004] Taking all these factors into consideration, the monitoring equipment at the power operation site usually adopts mobile monitoring equipment, such as a dome or a mobile pan-tilt work recorder, to achieve functions such as instant video capture and image capture. However, the video images and instant images captured by mobile monitoring equipment usually have scene changes, complex surrounding environments, signal jitter and light interference. Traditional data processing methods and video image recognition methods cannot achieve accurate and rapid identification and analysis.

[0005] Therefore, there is an urgent need for a machine vision technology-based intelligent recognition function for risky behaviors and violations in power operations, automatic analysis and judgment of violations, instant alarms, and logs, thus achieving a breakthrough from remote inspection to intelligent supervision and supporting the effective improvement of the inherent safety level of on-site operations. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned technologies, the present invention proposes an intelligent identification method for illegal behaviors in power operations based on machine vision. Based on 3D vision and spatial perception, edge computing and image processing, the deep learning intelligent identification algorithm is used to train the algorithm model to realize intelligent identification of illegal behaviors such as the height of foundation pit piles and the distance measurement of fence edges, the calibration of live interval areas, the erection of scaffolding, the tightening of wire ropes, and the behavior of high-altitude workers, etc., which are common in power operation sites, and realize automatic analysis and instant alarm.

[0007] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a method for intelligently identifying risky behaviors and violations of power operations based on machine vision, comprising the following steps:

[0008] Collect images of traffic violation scenes and annotate the collected images;

[0009] Preprocess and store the image;

[0010] Based on the preprocessed images, the extracted feature information is classified and trained to build a classifier model for identifying illegal behaviors;

[0011] Conduct autonomous feature learning based on calibrated positive and negative samples to train the classifier model;

[0012] Collect behavior pictures of high-altitude workers on site in real time, and use the trained classifier model to intelligently identify violation information.

[0013] The step of labeling the collected images comprises the following steps:

[0014] In the videos or images taken by cameras and drones, samples of illegal targets are screened and collected, and the areas and categories used to identify illegal behaviors are marked; the illegal behaviors include: workers mistakenly entering live intervals, inadequate safety measures for high-altitude operations, and measuring the distance of live equipment during substation operations.

[0015] The construction of the classifier model includes:

[0016] Convolutional neural network, extracts features from the preprocessed image and forms a feature map;

[0017] Region Proposal Network: Use the region proposal network to generate multiple candidate boxes of different scales and aspect ratios as candidate regions by sliding the window on the feature map.

[0018] RoIPooling layer: For each candidate region, the RoIPooling layer is used to convert the candidate regions of different scales into feature vectors of uniform size;

[0019] The fully connected layer performs detection box classification and regression on the feature vector of uniform size to identify the target category and location;

[0020] The post-processing layer removes overlapping detection frames through non-maximum suppression and adjusts the position and size of the detection frame according to the regression results to optimize the category and position of the target;

[0021] The output layer outputs the optimized target classification, location, and confidence as the target detection result.

[0022] The autonomous feature learning based on the calibrated positive and negative samples and the training of the classifier model include the following steps:

[0023] First, for positive samples and negative samples, the moving targets are detected using the inter-frame difference method or the background subtraction method; the positive samples are samples used to represent illegal behaviors, and the negative samples are samples used to represent no illegal behaviors;

[0024] Then, the sample image is matched with the template image using the template matching method to obtain the best matching position;

[0025] Finally, for the best matching position in the sample image, the key features are classified by the classifier, and the classifier model is formed through continuous iteration.

[0026] The method of using the template matching method to match the sample image with the template image to obtain the best matching position includes the following steps:

[0027] For the sample image and the template image, slide the template image on the input sample image to traverse all possible positions in the input image; each time the template image slides to a new position, calculate the matching degree between the template image and the corresponding area in the input sample image; the matching degree adopts any one of square difference matching, correlation matching, and coefficient matching;

[0028] The matching degree of each position is saved in the result matrix; each element of the result matrix corresponds to a position in the current image, and its value represents the matching degree between the position and the template image;

[0029] By traversing the result matrix, the position with the extreme value of the matching degree is found as the best matching position.

[0030] The key features include at least one of color, texture, shape, edge, corner point, behavior state, and motion trajectory, which are used to characterize the behavior of high-altitude workers, energized intervals, or open flame area calibration.

[0031] The violation information includes: failure to wear a safety belt, incorrect climbing posture; personnel or equipment approaching energized intervals; detecting fire occurrence and calibrating the location of the fire source.

[0032] The intelligent identification system for risky behaviors and violations in power operations based on machine vision includes:

[0033] An image acquisition module is used to collect images of traffic violation scenes and annotate the collected images;

[0034] A preprocessing module, used for preprocessing and storing images;

[0035] A model building module is used to perform classification training on the extracted feature information based on the preprocessed images, and to build a classifier model for identifying illegal behaviors;

[0036] Model training module, used to perform autonomous feature learning based on calibrated positive and negative samples and train the classifier model;

[0037] The intelligent recognition module is used to collect real-time behavior images of high-altitude workers on site and intelligently identify violation information through a trained classifier model.

[0038] A device for intelligently identifying risky behaviors and violations in power operations based on machine vision comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the method for intelligently identifying risky behaviors and violations in power operations based on machine vision when executing the computer program.

[0039] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method for intelligently identifying risky behaviors and violations in power operations based on machine vision is implemented.

[0040] The present invention has the following beneficial effects and advantages:

[0041] 1. The present invention further improves the inspection efficiency of power safety inspectors, and ensures that violations at power operation sites are discovered in a timely manner, handled effectively, and resolved reasonably. It utilizes template matching method, deep learning algorithm, feature extraction, and video image processing analysis and recognition technology to realize the intelligent identification function of violations. It can intelligently identify, automatically analyze, instantly warn, and record logs of common violations such as high-altitude worker behaviors (such as climbing ladders), calibration of live intervals or open flame areas, scaffolding erection, foundation pit soil pile height, and fence edge distance measurement at power operation sites, thus achieving a breakthrough from remote inspection to intelligent inspection.

[0042] 2. Compared with the prior art, the advantages or effects of the steps and processes are that the present invention realizes the automation of the entire chain from data collection to model training to final application deployment through integrated process design. Compared with traditional manual visual inspection or a single video monitoring system, the present invention uses automated collection tools (such as drones and surveillance cameras) and uses machine vision technology and deep learning algorithms to automatically complete a series of processes such as image collection, annotation, preprocessing, and feature extraction, which greatly reduces the need for manual intervention and improves detection efficiency and accuracy.

[0043] 3. The advantage of the entire technical solution is that it covers the intelligent identification needs of some common violations at power operation sites (such as foundation pit soil height and fence edge distance measurement, live interval area calibration, scaffolding erection, wire rope fastening line cards, and high-altitude work personnel behavior and other violations). By using deep learning intelligent recognition algorithms, the present invention can realize automatic analysis and judgment of various violations such as high-altitude work personnel behavior and live interval area calibration. Once the system detects a violation, it can immediately trigger an alarm mechanism and record detailed log information, which is helpful for subsequent analysis and accountability. In addition, the solution also supports the combination of remote inspection and intelligent supervision, making the supervision of on-site operations more efficient and intelligent.

[0044] 4. The advantage of the entire technical solution is that it provides a one-stop solution from data collection to model training to field application. By using advanced image processing technology and intelligent algorithms, the solution can monitor and intelligently judge various violations at the power operation site in real time, and can issue warnings immediately to help on-site managers respond quickly and avoid accidents. In addition, it also has the function of recording logs, which is convenient for subsequent tracing and analysis, and helps to continuously improve safety management measures. Therefore, the present invention replaces a large amount of manual inspection work with intelligent means, which not only improves work efficiency and reduces the possibility of human errors, but also greatly saves human resource costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0047] like Figure 1 As shown in the figure, the intelligent identification method of risky behavior violations in power operations based on machine vision mainly includes:

[0048] Collect relevant image and video data, and manually annotate the collected image and video data. Manage and maintain data through data preprocessing and other methods, divide the data set in the sample library into training set, verification set and test set, and then conduct a large number of model training and optimization; after the model training is completed, the model needs to be evaluated and tested to verify its performance, obtain violation warnings and feedback. The method for intelligent identification of violations of risky behaviors in power operations includes the following steps:

[0049] S1: Collect images or video data of various violation scenes and manually annotate the collected images and video data;

[0050] S2: Preprocess the collected data, including denoising, enhancement, segmentation and other steps, to improve the accuracy and efficiency of subsequent processing;

[0051] S3: Store the pre-processed data in a safe and reliable place to ensure the integrity and accessibility of the data;

[0052] S4: Classify and train the extracted feature information, and output a detection model that can identify various types of target violations (model construction);

[0053] S5: Autonomous feature learning based on calibrated positive and negative samples. For the task of detecting illegal targets at power operation sites, it is necessary to comprehensively consider factors such as the specific violation type, violation characteristics, external environmental conditions, and data set size, optimize the model feature parameters, and make corrections and verifications based on the detection results;

[0054] S6: After the model training is completed, the model is evaluated and tested to verify its performance;

[0055] S7: On-site testing verifies the traffic violation intelligent recognition model, intelligently identifies the traffic violation information, outputs the processed video frames, and obtains traffic violation warnings and feedback.

[0056] 1. Obtain the image to be detected (data acquisition, preprocessing)

[0057] Building a high-quality violation sample library is crucial for training accurate and reliable violation target detection algorithms. This sample library contains image or video data of various violation scenes and corresponding label information. It can screen and collect samples of violation targets from videos or images taken by surveillance cameras and drones. In addition, it can also be supplemented with public data sets on the Internet to ensure the diversity of materials.

[0058] In the process of video image processing, adjustments and optimizations are continuously made according to specific tasks and data sets. The performance and generalization ability of the algorithm are improved by adjusting the network structure, optimizing the loss function, and adding data enhancement. At the same time, due to the complexity and diversity of video data, it is also necessary to consider how to effectively deal with occlusion, motion blur, lighting changes, and other issues in the video to improve the accuracy and robustness of target detection.

[0059] First, the video frames are preprocessed, including but not limited to image scaling, color space conversion (such as conversion from RGB to grayscale or other color spaces), normalization, etc. The purpose of preprocessing is to reduce noise and redundant information in the image and improve the efficiency and accuracy of subsequent processing.

[0060] At the same time, the collected images and video data need to be manually labeled. Based on their professional knowledge and experience, the labelers classify the collected target images, accurately identify violations and mark the corresponding areas and categories, such as workers mistakenly entering live intervals, inadequate safety measures for height operations, and distance measurement of live equipment at substation sites.

[0061] Given the complex and ever-changing environment of power operation sites, the quality and quantity of data are crucial to the performance of the model. Therefore, appropriate data enhancement and preprocessing techniques need to be used in the data distillation process to improve the generalization ability of the model. More training samples can be generated by scaling, rotating, and cropping the original image; at the same time, the collected video data can also be preprocessed, including denoising, enhancement, segmentation, and other steps to improve the accuracy and efficiency of subsequent processing. These include:

[0062] (1) Data cleaning: Clean the labeled data to remove duplicate, ambiguous or poor quality data to ensure the purity and reliability of the data set.

[0063] (2) Data augmentation: In order to enhance the generalization ability of the model, the dataset can be augmented. For example, images can be rotated, scaled, cropped, or videos can be cropped and spliced ​​in time. These operations can generate more training samples.

[0064] 2. Data management and maintenance (violation sample database)

[0065] Using distributed storage or cloud storage to store labeled data sets not only ensures data integrity and accessibility, but also improves data security and scalability. With the changes in power operation sites and the emergence of new violations, new data is regularly collected, labeled, and preprocessed, and then added to the sample library to continuously update and improve the sample library.

[0066] Note: The dataset in the sample library is divided into training set, validation set and test set. The training set is used to train the model, the validation set is used to adjust the model parameters and prevent overfitting, and the test set is used to evaluate the performance of the model.

[0067] 3. Model Construction

[0068] The convolutional neural network (CNN) in the improved FasterR-CNN algorithm is used to extract features from the preprocessed video frames. This process extracts rich feature information from the image through multiple layers of convolution, pooling, and activation functions for subsequent object detection tasks.

[0069] At the same time, based on feature extraction, a region proposal network (RPN) is used to generate candidate regions. RPN generates multiple candidate boxes of different scales and aspect ratios for each position by sliding a small window on the feature map, and predicts whether each candidate box contains the object and the position and size of the object.

[0070] For the candidate regions generated by RPN, the RoIPooling layer is used to map them to the feature map and extract a fixed-size feature vector. The purpose of this step is to convert candidate regions of different scales into feature vectors of uniform size for subsequent classification and regression tasks.

[0071] The feature vector output by the RoIPooling layer is input into the fully connected layer for classification and regression. The classifier is used to determine whether each candidate region contains an object and the category of the object, while the regressor is used to correct the position and size of the candidate region to obtain a more accurate object detection frame.

[0072] The classification and regression results are post-processed, including steps such as non-maximum suppression (NMS) to remove redundant detection boxes and improve detection accuracy.

[0073] In this embodiment, the model building process is as follows:

[0074] 1) Preprocess video frames:

[0075] Read video frame: read each frame image from the video stream.

[0076] Image preprocessing: This includes image resizing, normalization (e.g., scaling pixel values ​​to [0, 1] or [-1, 1]), and possible augmentation (such as flipping, cropping, color transformation, etc.) to improve the generalization ability of the model.

[0077] 2) Feature extraction using CNN

[0078] Convolution layer: Perform convolution operations with the input image through multiple layers of convolution kernels to extract local features of the image.

[0079] Activation functions (such as ReLU): Add nonlinearity to help the network learn complex patterns.

[0080] Pooling layer: Reduces the dimension of the feature map, extracts important features and reduces the amount of computation.

[0081] Batch Normalization (optional): speeds up the training process and improves model stability.

[0082] 3) Region Proposal Network (RPN)

[0083] Sliding window: Slide a small window on the feature map to generate anchors, which are candidate boxes of predefined multiple scales and aspect ratios.

[0084] Classification and regression: For each anchor point, RPN predicts whether it contains an object (binary classification problem) and adjusts its position and size (bounding box regression).

[0085] Non-maximum suppression (NMS) preliminary screening: NMS may have been applied in the RPN stage to reduce the number of candidate regions.

[0086] 4)RoIPooling layer

[0087] Mapping: Map the candidate region output by RPN back to the feature map.

[0088] Pooling: Perform a pooling operation on each candidate region to extract a feature map of a fixed size (such as 7x7).

[0089] 5) Fully connected layer

[0090] Feature Vector: The output of the RoIPooling layer is flattened into a fixed-length feature vector.

[0091] Classification and regression: The feature vector is input into the fully connected layer for classification (identifying the target category) and regression (further refining the target location).

[0092] 6) Post-processing

[0093] Non-maximum suppression (NMS): NMS is applied to the final detection result to remove highly overlapping detection boxes and keep only the best one.

[0094] Bounding box correction: adjust the position and size of the detection box based on the regression results so that it surrounds the object more accurately.

[0095] 7) Integrate the results

[0096] Output results: Output the object detection results in the processed video frames, which may include object category, confidence, and precise location information.

[0097] This process achieves efficient target detection in video frames through efficient feature extraction, accurate candidate region generation, and precise classification and regression.

[0098] 4. Model Training and Optimization

[0099] Autonomous feature learning is performed based on calibrated positive and negative samples (traffic violation samples are used as positive samples for algorithm learning, and non-traffic violation samples are used as negative samples excluded by the algorithm). Since video image processing involves continuous video frames, it is necessary to use time series information to associate the detection results in different frames, which can be achieved through tracking algorithms or models such as recurrent neural networks (RNN) in deep learning.

[0100] For the detection task of illegal targets at power operation sites, comprehensive consideration is given to factors such as the specific violation type, violation characteristics, external environmental conditions, and data set size, the model feature parameters are optimized, and correction and verification are performed based on the detection results. At the same time, you can also try to combine multiple algorithms to improve the real-time and accuracy of recognition. For example, first use the inter-frame difference method or background subtraction method to detect the moving target, then use the template matching method to match the current frame with the template image, and then use the deep learning algorithm (the focus of the training process is as follows) to extract the key features in the image (such as the color, texture, shape, edge, corner point, behavior state, motion trajectory, etc. of the image), and finally use the feature extraction combined with the classifier to classify the features, and form a reliable analysis model through continuous iteration.

[0101] First, for positive samples and negative samples, the moving targets are detected using the inter-frame difference method or the background subtraction method; the positive samples are samples used to represent illegal behaviors, and the negative samples are samples used to represent no illegal behaviors;

[0102] Then, the template matching method is used to match the current image with the template image, as follows:

[0103] 1) Prepare images: First, two images need to be prepared: one is a larger input image (current image), and the other is a smaller template image. Both images should be able to be matched, such as grayscale images or images that have been properly preprocessed.

[0104] 2) Sliding template: Then, slide the template image on the input image to traverse all possible positions in the input image. Every time it slides to a new position, the matching degree between the template image and the corresponding area in the input image is calculated.

[0105] 3) Calculate the matching degree: The matching degree can be calculated by a variety of methods, such as square difference matching, correlation matching, coefficient matching, etc. Each method has its own specific calculation method and scope of application. The higher the matching degree value (or the lower, depending on the specific matching method), the higher the similarity between the current position and the template image.

[0106] 4) Record the matching results: Save the matching degree of each position in a result matrix. Each element of the result matrix corresponds to a position in the input image, and its value represents the matching degree between the position and the template image.

[0107] 5) Find the best matching position: Finally, by traversing the result matrix, find the position with the highest match (or the lowest match, depending on the matching method). This position is the area in the input image that is most similar to the template image.

[0108] Next, for the entire image (whether it is a positive sample containing a violation or a negative sample without a violation), a deep learning algorithm is used to extract the key features in the image; the extracted features are input into the classifier for classification training.

[0109] Matching value: indicates the matching degree of the best matching position. This value can be further processed or analyzed as needed.

[0110] Finally, the features (shape features, texture features, color features, spatial relationship features, deep learning features, etc.) are classified through the classifier, and a classifier model is formed through continuous iteration to map the input features to the corresponding category labels, that is, to determine whether the image contains any violations.

[0111] In image processing, the combination of feature extraction algorithm and classifier algorithm is a key step in image processing. The key features are extracted from the image through the feature extraction algorithm, and then the classifier algorithm is used to classify and identify these features, thereby realizing intelligent processing and understanding of the image.

[0112] The key feature extraction of an image refers to identifying and extracting from the image the most valuable information for subsequent classification, recognition or understanding tasks. These features include: common behaviors of workers working at heights (such as climbing ladders: facing the ladder, three-point contact principle, stable climbing, correct use of safety belts, etc.), calibration of live intervals or open flame areas (edge ​​features, color histograms, texture features, and shape features of the image, such as: minimum safe spacing / minimum effective insulation length of insulating tools, flame shape features. Common shape features include rectangularity, circularity, sphericity and eccentricity, flame color and brightness, etc.).

[0113] With the rise of deep learning, models such as convolutional neural networks (CNN) have been widely used in image feature extraction. Through multi-layer convolution and pooling operations, CNN can automatically extract hierarchical feature representations from raw images, and CNN features perform well in tasks such as image classification, object detection, and image segmentation.

[0114] A classifier is an algorithm or model that maps input data (such as extracted image features) to predefined category labels (a library of violation samples).

[0115] CNN (Convolutional Neural Networks) extracts image features through multi-layer convolution and pooling operations, and uses fully connected layers for final classification. It can automatically extract image features and has good modeling capabilities for the spatial hierarchical structure of images.

[0116] In addition, it is also necessary to consider how to effectively deal with occlusion, motion blur, lighting changes and other issues in the video. Dealing with occlusion, motion blur and lighting changes in the video requires the comprehensive use of multiple technologies and methods. In practical applications, appropriate processing strategies can be selected according to the characteristics and needs of specific problems, and adjusted and optimized based on actual conditions.

[0117] 1. Dealing with occlusion issues

[0118] (1) Deep learning and computer vision technology: Use deep learning models (such as convolutional neural networks (CNNs)) to identify and reconstruct occluded areas. This method can learn the characteristics of occluders and the contextual information of occluded areas by training the model.

[0119] (2) Image inpainting technology: Use image inpainting algorithms (such as those based on texture synthesis) to fill in occluded areas. These algorithms generate reasonable pixel values ​​by analyzing the texture and structure of surrounding pixels.

[0120] 2. Dealing with motion blur

[0121] (1) Increase frame rate: Reduce motion blur between each frame by increasing the camera’s frame rate. A high frame rate camera can capture fast-moving objects more accurately, thus reducing blur.

[0122] (2) Motion estimation and compensation: Motion estimation algorithms (such as optical flow) are used to estimate the motion trajectory of pixels or feature points in video frames, and motion compensation is used to reduce blur. This method can restore a clear image by aligning the blurred frame with the adjacent clear frame.

[0123] (3) Image deblurring algorithm: Image deblurring algorithms (such as deep learning deblurring algorithms, etc.) are applied to directly process blurry frames. These algorithms can restore clear images by analyzing the statistical characteristics of blurry images and prior knowledge.

[0124] 3. Dealing with lighting changes

[0125] (1) Multi-mode background model: Establish background models under multiple lighting conditions and automatically select the corresponding model according to the current lighting conditions. This method can adapt to video processing requirements under different lighting environments.

[0126] (2) Deep learning technology: Use deep learning models (such as autoencoders, generative adversarial networks (GANs), etc.) to learn and adapt to lighting changes. These models can be trained to learn image features under different lighting conditions and automatically adjust to new lighting environments during testing.

[0127] The following two aspects need to be paid attention to during training:

[0128] (1) Loss function design: Select an appropriate loss function according to the specific task, such as cross entropy loss, smooth L1 loss, etc. The loss function should be designed to reflect the difference between the model's prediction results and the actual labels.

[0129] (2) Model training optimization: During the training process, some optimization techniques can be used to improve the convergence speed and performance of the model, such as gradient descent algorithm, momentum method, Adam optimizer, and learning rate decay strategy. In addition, during the optimization process, it is also necessary to adopt appropriate optimization algorithms, parameter settings, and comprehensive use of multiple technologies to ensure that the model can converge to a better solution.

[0130] The use of feature extraction combined with classifiers in specific application scenarios such as the behavior of high-altitude workers (such as climbing ladders), live intervals or open flame area calibration can effectively improve the efficiency and accuracy of safety monitoring and management. The following is a detailed analysis of these application scenarios:

[0131] 1. Monitoring of the behavior of workers working at heights (such as climbing ladders)

[0132] (1) Feature extraction

[0133] Video features: Extract the motion trajectory, posture, speed and other features of high-altitude workers from the surveillance video. This can be achieved by using image processing techniques such as background subtraction, optical flow method, target detection and tracking algorithms in deep learning, etc.

[0134] Behavioral characteristics: According to the specific behaviors of workers working at heights (such as climbing ladders, hanging operations, installing equipment, etc.), relevant specific behavioral characteristics are extracted, such as arm swing frequency, body tilt angle, tool usage, etc.

[0135] (2) Classifier selection and training

[0136] Classifier selection: Based on the extracted features, select a suitable classifier (SVM, decision tree, random forest, neural network (especially convolutional neural network CNN and recurrent neural network RNN, which are particularly effective when processing video data)) for training.

[0137] Training process: Use labeled data sets (including normal and abnormal behaviors) to train the classifier. During the training process, the model parameters need to be continuously adjusted to optimize the classification performance.

[0138] (3) Application scenarios

[0139] Real-time monitoring: Deploy the trained classifier to the actual high-altitude operation scenario to monitor the behavior of operators in real time.

[0140] Abnormal detection: When abnormal behavior is detected (such as not wearing a safety belt, incorrect climbing posture, etc.), an alarm will be issued immediately to alert managers and operators.

[0141] Data analysis: Conduct statistical analysis on the collected data to evaluate the safety status of aerial work and provide a basis for improving safety measures.

[0142] 2. Live interval calibration

[0143] (1) Feature extraction

[0144] Image features: Extract the shape, color, texture and other features of the charged interval area from the monitoring image. These features help to distinguish the charged area from the non-charged area.

[0145] Distance and location features: Combined with GPS or indoor positioning technology, the distance and location relationship features between the live interval and surrounding objects (such as personnel and equipment) are extracted.

[0146] (2) Classifier selection and training

[0147] Classifier selection: According to the requirements of the live interval calibration, select a suitable classifier for training. Since this is usually a binary classification problem (live / non-live), a simple classifier (such as SVM, logistic regression) can be used to learn the model.

[0148] Training process: Use the labeled images and location data to train the classifier. During the training process, attention should be paid to the accuracy and real-time performance of the classifier.

[0149] (3) Application scenarios

[0150] Automatic calibration: In power inspection and maintenance scenarios, the system automatically calibrates live interval areas to remind operators to pay attention to safe distances.

[0151] Early warning system: When it is detected that a person or equipment is approaching a live interval, an early warning signal is issued to prevent electric shock accidents.

[0152] 3. Open flame area calibration

[0153] (1) Feature extraction

[0154] Flame features: Extract flame features such as color, shape, texture, and motion pattern from surveillance videos or images. These features help distinguish flames from other similar objects (such as red lights).

[0155] Temperature characteristics: Combined with infrared thermal imaging technology, the temperature characteristics of the open fire area are extracted. Flames usually have a high temperature, so the temperature characteristics are very important in open fire detection.

[0156] (2) Classifier selection and training

[0157] Classifier selection: Since open fire detection usually needs to be performed in complex environments, a robust classifier is required. Therefore, a deep learning model (such as convolutional neural network CNN) is used because it performs particularly well in this regard.

[0158] Training process: Use a dataset containing flame and non-flame images to train the classifier. During the training process, attention should be paid to the generalization ability and real-time performance of the classifier.

[0159] (3) Application scenarios

[0160] Fire warning: In factories, warehouses, construction sites and other flammable and explosive places, open flame areas are monitored and marked in real time, and fire warning signals are issued in time.

[0161] Emergency response: When a fire occurs, quickly locate the fire source and provide accurate information for emergency response and firefighting and rescue.

[0162] In summary, the combination of feature extraction and classifier has important application value in practical application scenarios such as high-altitude worker behavior monitoring, live interval calibration, and open fire area calibration. By continuously optimizing the feature extraction algorithm and classifier model, the accuracy and real-time performance of the monitoring system can be further improved, providing strong support for safe production and emergency management.

[0163] 5. Model Evaluation and Testing

[0164] After the model training is completed, the model needs to be evaluated and tested to verify its performance. The violation intelligent recognition model is verified through field testing, and violation information is intelligently identified. Violation information such as warnings for not wearing a safety helmet, warnings for workers mistakenly entering a live interval, and warnings for inadequate safety measures for high-altitude operations are intelligently identified. Evaluation indicators can include commonly used target detection indicators such as accuracy, recall rate, and F1 score. In addition, the performance of the model in different scenarios and conditions can be further analyzed to evaluate and verify its feasibility in practical applications.

[0165] VI. Early Warning and Feedback

[0166] Finally, the processed video frames are output. When the operator is identified to have violated the regulations, the detected target is marked and displayed on the video frame, and the detection results are saved as files or alarm prompts to promptly feedback the violation identification information to the inspector.

Claims

1. A method for intelligently identifying risky behaviors and violations of power operations based on machine vision, characterized in that: The following steps are involved: Collect images of traffic violation scenes and annotate the collected images; Preprocess and store the image; Based on the preprocessed images, the extracted feature information is classified and trained to build a classifier model for identifying illegal behaviors; Conduct autonomous feature learning based on calibrated positive and negative samples to train the classifier model; Collect behavior pictures of high-altitude workers on site in real time, and use the trained classifier model to intelligently identify violation information.

2. According to the machine vision-based intelligent identification method for risky behaviors in power operations according to claim 1, it is characterized in that: The step of labeling the collected images comprises the following steps: In the videos or images taken by cameras and drones, samples of illegal targets are screened and collected, and the areas and categories used to identify illegal behaviors are marked; the illegal behaviors include: workers mistakenly entering live intervals, inadequate safety measures for high-altitude operations, and measuring the distance of live equipment during substation operations.

3. The method for intelligently identifying risky behaviors and violations of power operations based on machine vision according to claim 1 is characterized in that: The construction of the classifier model includes: Convolutional neural network, extracts features from the preprocessed image and forms a feature map; Region Proposal Network: Use the region proposal network to generate multiple candidate boxes of different scales and aspect ratios as candidate regions by sliding the window on the feature map. RoIPooling layer: For each candidate region, the RoIPooling layer is used to convert the candidate regions of different scales into feature vectors of uniform size; The fully connected layer performs detection box classification and regression on the feature vector of uniform size to identify the target category and location; The post-processing layer removes overlapping detection frames through non-maximum suppression and adjusts the position and size of the detection frame according to the regression results to optimize the category and position of the target; The output layer outputs the optimized target classification, location, and confidence as the target detection result.

4. The method for intelligently identifying risky behaviors and violations of power operations based on machine vision according to claim 1 is characterized in that: The autonomous feature learning based on the calibrated positive and negative samples and the training of the classifier model include the following steps: First, for positive samples and negative samples, the moving targets are detected using the inter-frame difference method or the background subtraction method; the positive samples are samples used to represent illegal behaviors, and the negative samples are samples used to represent no illegal behaviors; Then, the sample image is matched with the template image using the template matching method to obtain the best matching position; Finally, for the best matching position in the sample image, the key features are classified by the classifier, and the classifier model is formed through continuous iteration.

5. The method for intelligently identifying risky behaviors and violations of power operations based on machine vision according to claim 4 is characterized in that: The method of using the template matching method to match the sample image with the template image to obtain the best matching position includes the following steps: For the sample image and the template image, slide the template image on the input sample image to traverse all possible positions in the input image; each time the template image slides to a new position, calculate the matching degree between the template image and the corresponding area in the input sample image; the matching degree adopts any one of square difference matching, correlation matching, and coefficient matching; The matching degree of each position is saved in the result matrix; each element of the result matrix corresponds to a position in the current image, and its value represents the matching degree between the position and the template image; By traversing the result matrix, the position with the extreme value of the matching degree is found as the best matching position.

6. The method for intelligently identifying risky behaviors and violations of power operations based on machine vision according to claim 4 is characterized in that: The key features include at least one of color, texture, shape, edge, corner point, behavior state, and motion trajectory, which are used to characterize the behavior of high-altitude workers, energized intervals, or open flame area calibration.

7. The method for intelligently identifying risky behaviors and violations of power operations based on machine vision according to claim 1 is characterized in that: The violation information includes: failure to wear a safety belt, incorrect climbing posture; personnel or equipment approaching energized intervals; detecting fire occurrence and calibrating the location of the fire source.

8. The intelligent identification system for risky behaviors and violations of power operations based on machine vision is characterized by: include: An image acquisition module is used to collect images of traffic violation scenes and annotate the collected images; A preprocessing module, used for preprocessing and storing images; A model building module is used to perform classification training on the extracted feature information based on the preprocessed images, and to build a classifier model for identifying illegal behaviors; Model training module, used to perform autonomous feature learning based on calibrated positive and negative samples and train the classifier model; The intelligent recognition module is used to collect real-time behavior images of high-altitude workers on site and intelligently identify violation information through a trained classifier model.

9. A machine vision-based intelligent identification device for risky behavior violations in power operations, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the method for intelligently identifying risky behaviors and violations in power operations based on machine vision as described in any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for intelligently identifying risky behaviors and violations in power operations based on machine vision as described in any one of claims 1 to 6 is implemented.

Images