Operating site violation behavior recognition system and method based on convolutional neural network

Abstract

The invention relates to the technical field of operation site safety supervision, in particular to an operation site violation behavior recognition system and method based on a convolutional neural network, and the method comprises the steps: collecting an electric power operation scene picture and three-dimensional point cloud data; integrating the electric power operation scene picture and the three-dimensional point cloud data to obtain integrated data; inputting the integrated data of the power operation scene into a power operation scene violation behavior recognition model for recognition to obtain a recognition result; and performing judgment based on the identification result, if the violation behavior exists, giving an alarm to the working site, sending alarm data to the management end, and if the violation behavior does not exist, continuing monitoring. According to the invention, by monitoring the operation site in real time, automatically identifying violation behaviors and timely giving an alarm, the probability of accidents is reduced, and the safety of the operation site is improved. And the monitoring efficiency is improved by automatically identifying the violation behaviors, so that the problem of poor accuracy of the existing violation behavior identification system is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of on-site safety supervision of operations, and particularly to a system and method for identifying illegal behaviors at an operation site based on a convolutional neural network. Background Art

[0002] During the operation process at the construction site of a transmission line project, various behavior-based and device-based violations are likely to occur, seriously endangering the safety of construction workers and power grid equipment. The traditional manual safety supervision method has problems such as a large workload and difficulty in timely and correctly detecting illegal behaviors.

[0003] For the existing system that combines image recognition technology with video surveillance, although it can automatically identify safety hazard targets, it does not have the ability to judge distances and cannot identify the true distances between people and objects, or between objects, resulting in a large number of false alarms and the drowning of truly meaningful alarms. Summary of the Invention

[0004] The purpose of the present invention is to provide a system and method for identifying illegal behaviors at an operation site based on a convolutional neural network, aiming to solve the problem of poor accuracy of the existing illegal behavior identification system.

[0005] To achieve the above purpose, in the first aspect, the present invention provides a method for identifying illegal behaviors at an operation site based on a convolutional neural network, including the following steps:

[0006] Establish an illegal behavior recognition model for the power operation scenario;

[0007] Collect pictures and three-dimensional point cloud data of the power operation scenario;

[0008] Integrate the pictures and three-dimensional point cloud data of the power operation scenario to obtain integrated data;

[0009] Input the integrated data of the power operation scenario into the illegal behavior recognition model for the power operation scenario for recognition to obtain a recognition result;

[0010] Based on the recognition result, make a judgment. If there is an illegal behavior, send an alarm to the operation site and send alarm data to the management end. If there is no illegal behavior, continue to monitor.

[0011] Among them, in "establishing an illegal behavior recognition model for the power operation scenario", the following steps are included:

[0012] Obtain a historical data set of the power operation scenario, where the historical data set includes various operation environments and types of illegal behaviors;

[0013] Label the types, positions, and three-dimensional point cloud data of illegal behaviors in each picture in the historical data set;

[0014] Partition the labeled historical dataset to obtain a training set and a test set;

[0015] Train a neural network model based on the training set to obtain a trained model;

[0016] Test the trained model with the test set to obtain an illegal behavior recognition model for the power operation scenario.

[0017] Among them, in "Collecting images and 3D point cloud data of the power operation scenario", the following steps are included:

[0018] Use on-site cameras to collect images of the power operation scenario in real time;

[0019] Use a 3D laser scanner to obtain 3D point cloud data of the operation site.

[0020] Among them, in "Integrating the images and 3D point cloud data of the power operation scenario to obtain integrated data", the following steps are included:

[0021] Preprocess the collected images and point cloud data;

[0022] Extract visual features of illegal behaviors from the images and spatial features from the 3D point clouds;

[0023] Integrate the visual features and spatial features to obtain integrated data.

[0024] Among them, in "Inputting the integrated data of the power operation scenario into the illegal behavior recognition model for the power operation scenario for recognition to obtain a recognition result", the following steps are included:

[0025] Input the integrated data into the illegal behavior recognition model for the power operation scenario to analyze the input data and recognize illegal behaviors;

[0026] The illegal behavior recognition model for the power operation scenario outputs the recognition result of the illegal behavior, including information such as the type and location of the violation.

[0027] Among them, in "Making a judgment based on the recognition result. If there is an illegal behavior, an alarm is sent to the operation site and alarm data is sent to the management terminal. If there is no illegal behavior, continuous monitoring is continued", the following steps are included:

[0028] Analyze the recognition result output by the model to determine whether there is an illegal behavior;

[0029] If an illegal behavior is detected, trigger the alarm mechanism and send an audible and visual alarm signal to the operation site;

[0030] Send the illegal behavior data and alarm information to the management terminal for the management personnel to take corresponding measures;

[0031] If no violation is detected, the system continues to monitor the operation site.

[0032] In a second aspect, a violation behavior recognition system for an operation site based on a convolutional neural network is used for the method for recognizing violation behaviors at an operation site based on a convolutional neural network according to the first aspect, and includes an acquisition module, a three-dimensional point cloud ranging module, a violation behavior recognition module for an electric power operation scenario, and an alarm module. The acquisition module and the three-dimensional point cloud ranging module are respectively connected to the violation behavior recognition module for the electric power operation scenario, and the alarm module is connected to the violation behavior recognition module for the electric power operation scenario.

[0033] The method for recognizing violation behaviors at an operation site based on a convolutional neural network according to the present invention includes the following steps: establishing a violation behavior recognition model for an electric power operation scenario; acquiring pictures and three-dimensional point cloud data of the electric power operation scenario; integrating the pictures and three-dimensional point cloud data of the electric power operation scenario to obtain integrated data; inputting the integrated data of the electric power operation scenario into the violation behavior recognition model for the electric power operation scenario for recognition to obtain a recognition result; making a judgment based on the recognition result. If there is a violation behavior, an alarm is sent to the operation site and alarm data is sent to the management end. If there is no violation behavior, monitoring continues. By monitoring the operation site in real time, automatically recognizing violation behaviors, and sending alarms in a timely manner, the present invention reduces the probability of accidents and improves the safety of the operation site. Automatically recognizing violation behaviors reduces the need for manual monitoring, improves monitoring efficiency, and reduces labor costs. Once a violation behavior is recognized, the system can respond quickly and send an alarm immediately, shortening the time from the occurrence of the violation to the alarm. Thus, the problem of poor accuracy of the existing violation behavior recognition system is solved. Brief Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 is a flowchart of the method for recognizing violation behaviors at an operation site based on a convolutional neural network provided by the present invention.

[0036] Figure 2 is a flowchart of establishing a violation behavior recognition model for an electric power operation scenario.

[0037] Figure 3 is a flowchart of acquiring pictures and three-dimensional point cloud data of the electric power operation scenario.

[0038] Figure 4It is a flowchart for integrating pictures and 3D point cloud data of a power operation scenario to obtain integrated data.

[0039] Figure 5 It is a flowchart for inputting the integrated data of a power operation scenario into a recognition model for identifying illegal behaviors in the power operation scenario to obtain a recognition result.

[0040] Figure 6 It is a flowchart for making a judgment based on the recognition result. If there is an illegal behavior, an alarm is sent to the operation site and alarm data is sent to the management terminal. If there is no illegal behavior, continuous monitoring is continued.

[0041] Figure 7 It is a schematic diagram of an illegal behavior recognition system for an operation site based on a convolutional neural network provided by the present invention.

[0042] In the figure: 1 - acquisition module, 2 - 3D point cloud ranging module, 3 - illegal behavior recognition module for power operation scenario, 4 - alarm module. Detailed implementation manners

[0043] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present invention and should not be construed as a limitation of the present invention.

[0044] Please refer to Figures 1 to 6 , in the first aspect, the present invention provides an illegal behavior recognition method for an operation site based on a convolutional neural network, including the following steps:

[0045] S1 Establish an illegal behavior recognition model for a power operation scenario;

[0046] S11 Obtain a historical data set of a power operation scenario, where the historical data set includes various operation environments and types of illegal behaviors;

[0047] Specifically, first, collect a large number of pictures of power operation scenarios, which should cover various operation environments and types of illegal behaviors to ensure the generalization ability of the model. The various operation environments include substations, transmission lines, distribution rooms, etc.; the types of illegal behaviors include pictures of not wearing safety helmets, illegal operations, entering live areas by mistake, etc.

[0048] S12 Label the types of illegal behaviors, positions, and 3D point cloud data in each picture in the historical data set;

[0049] Specifically, the historical dataset is finely annotated to clearly label the types, locations of violations in each picture, and relevant 3D point cloud data (such as distances and positional relationships between objects). For image annotation, tools like LabelImg or Labelme are used, and for 3D point cloud data annotation, the 3D point cloud annotation tool CloudCompare is used. First, use the image annotation tool to annotate the violation behaviors in each picture, including behavior types, locations, and bounding boxes. Second, use the 3D point cloud annotation tool to annotate the point cloud data corresponding to the image, and mark key information such as distances and positional relationships between objects.

[0050] S13 Based on the annotated historical dataset, it is divided into a training set and a test set;

[0051] Specifically, the annotated dataset is randomly divided into a training set and a test set, with a ratio of 80% for the training set and 20% for the test set. And data augmentation (such as rotation, flipping, cropping, etc.) is performed on the training set to improve the generalization ability of the model and ensure the consistency of the training set and the test set in terms of violation behavior types, working environments, and data distributions.

[0052] S14 Based on the training set, the neural network model is trained to obtain a trained model;

[0053] Specifically, use the training set to train the selected CNN model, and adjust the network weights through forward propagation and backpropagation algorithms to enable the model to accurately identify violation behaviors. During the training process, monitor indicators such as the loss function and accuracy, and perform hyperparameter tuning (learning rate, batch size, etc.).

[0054] S15 The trained model is tested through the test set to obtain a violation behavior recognition model for the power operation scenario.

[0055] Specifically, use the test set to test the trained model, and evaluate indicators such as its recognition accuracy, recall rate, and F1 score. Fine-tune the model according to the test results until the preset performance level is reached. The finally obtained model is the violation behavior recognition model for the power operation scenario.

[0056] S2 Collect pictures and 3D point cloud data of the power operation scenario;

[0057] S21 Use on-site cameras to collect images of the power operation scenario in real time;

[0058] Specifically, install high-definition cameras at the power operation site and use image acquisition software to collect images of the operation scenario in real time. Ensure that the cameras can cover all key operation areas, and the cameras have night vision capabilities and are designed to be waterproof and dustproof.

[0059] S22 Acquire the three-dimensional point cloud data of the operation site using a three-dimensional laser scanner.

[0060] Specifically, deploy a three-dimensional laser scanner at the power operation site, and acquire the three-dimensional point cloud data of the operation site through laser scanning. Use point cloud processing software to preprocess the scanned data, including steps such as denoising, filtering, registration, and downsampling, to improve the data quality and processing efficiency.

[0061] S3 Integrate the power operation scene pictures and the three-dimensional point cloud data to obtain integrated data;

[0062] S31 Preprocess the acquired images and point cloud data;

[0063] Specifically, denoise, enhance, and adjust the size of the acquired images to improve the image quality. Filter, denoise, and convert the format of the acquired point cloud data to reduce data redundancy and improve the processing efficiency.

[0064] S32 Extract the visual features of the illegal behavior from the images, and extract the spatial features from the three-dimensional point cloud;

[0065] Specifically, use a CNN to extract the visual features of the illegal behavior from the images, such as edges, textures, and shapes; use point cloud processing algorithms to extract spatial feature points from the three-dimensional point cloud, such as point cloud density, normal vector, and curvature.

[0066] S33 Integrate the visual features and spatial features to obtain integrated data for illegal behavior recognition.

[0067] Specifically, fuse the extracted visual features and spatial features based on a feature fusion algorithm to obtain the integrated data.

[0068] S4 Input the integrated data of the power operation scene into the illegal behavior recognition model of the power operation scene for recognition to obtain the recognition result;

[0069] S41 Input the integrated data into the illegal behavior recognition model of the power operation scene to analyze the input data and recognize the illegal behavior;

[0070] Specifically, input the integrated data into the trained illegal behavior recognition model of the power operation scene for recognition.

[0071] S42 The illegal behavior recognition model of the power operation scene outputs the recognition result of the illegal behavior, including information such as the type and location of the violation.

[0072] Specifically, the model calculates and outputs the recognition result through forward propagation, including information such as the type, location, and confidence of the illegal behavior. The recognition result can be used for subsequent judgment and alarm processing.

[0073] S5 makes a judgment based on the recognition result. If there is a violation, it issues an alarm to the operation site and sends alarm data to the management terminal. If there is no violation, it continues to monitor.

[0074] S51 Analyze the recognition result output by the model to determine whether there is a violation.

[0075] Specifically, according to the safety threshold, judge whether there is a violation based on information such as the type of violation behavior and confidence level in the recognition result output by the model.

[0076] S52 If a violation is detected, trigger the alarm mechanism and send an audible and visual alarm signal to the operation site.

[0077] Specifically, when a violation is detected, trigger the alarm mechanism and send an alarm signal to the operation site through the audible and visual alarm installed on-site. The alarm signal should be obvious and easy to detect so as to attract the attention of the operators in time.

[0078] S53 Send the violation behavior data and alarm information to the management terminal for the management personnel to take corresponding measures.

[0079] Specifically, send the recognized violation behavior data and alarm information to the management terminal through the network. The management terminal can monitor the safety status of the operation site in real time.

[0080] S54 If no violation is detected, the system continues to monitor the operation site.

[0081] Specifically, when no violation is detected, the system will continue to monitor the operation site. By continuously collecting and processing data, the recognition result is updated in real time to ensure that the safety status of the operation site is effectively monitored. At the same time, the system should have the functions of automatic restart and fault recovery to ensure long-term stable operation.

[0082] Please refer to Figure 7 In the second aspect, a violation behavior recognition system for an operation site based on a convolutional neural network is used for the violation behavior recognition method for an operation site based on a convolutional neural network described in the first aspect, and includes a collection module 1, a three-dimensional point cloud ranging module 2, a violation behavior recognition module 3 for an electric power operation scenario, and an alarm module 4. The collection module 1 and the three-dimensional point cloud ranging module 2 are respectively connected to the violation behavior recognition module 3 for an electric power operation scenario, and the alarm module 4 is connected to the violation behavior recognition module 3 for an electric power operation scenario.

[0083] In this embodiment, the acquisition module 1 is used to acquire pictures of the power operation scenario; the 3D point cloud ranging module 2 is used to acquire 3D point cloud data, and the illegal behavior recognition module 3 for the power operation scenario identifies illegal behaviors based on the 3D point cloud data and the scenario pictures. The alarm module 4 makes a judgment based on the recognition result. If there is an illegal behavior, it issues an alarm to the operation site and sends alarm data to the management end. If there is no illegal behavior, it continues to monitor.

[0084] Beneficial effects:

[0085] 1. The illegal behavior recognition system and method for the operation site based on the convolutional neural network provided by the present invention improve the accuracy and timeliness of illegal behavior recognition: By combining image recognition technology and 3D point cloud data, this technical solution can not only identify the types of illegal behaviors, but also accurately judge the specific location where the illegal behavior occurs and the real distance between objects, thus greatly reducing the situations of false alarms and missed alarms, and improving the accuracy and timeliness of recognition. This is of great significance for ensuring the safety of construction personnel and power grid equipment.

[0086] 2. The illegal behavior recognition system and method for the operation site based on the convolutional neural network provided by the present invention enhance the generalization ability of the model: When establishing the illegal behavior recognition model for the power operation scenario, by collecting historical data sets covering various operation environments and types of illegal behaviors, and performing fine annotation and division on the data sets, as well as adopting data augmentation technology, the generalization ability of the model is effectively improved. This enables the model to maintain a high recognition accuracy in different operation environments and conditions.

[0087] 3. The illegal behavior recognition system and method for the operation site based on the convolutional neural network provided by the present invention realize automated and intelligent monitoring: This technical solution realizes real-time monitoring of the operation site and automatic recognition of illegal behaviors by automatically acquiring and processing pictures and 3D point cloud data of the power operation scenario. This not only reduces the workload of manual safety supervision, but also improves the efficiency and accuracy of monitoring. At the same time, the system has functions of automatic restart and fault recovery to ensure long-term stable operation.

[0088] 4. The illegal behavior recognition system and method for the operation site based on the convolutional neural network provided by the present invention provide rich alarm information: When an illegal behavior is detected, the system can trigger an alarm mechanism, issue an audible and visual alarm signal to the operation site, and send the illegal behavior data and alarm information to the management end. This provides timely alarm information and illegal behavior data for the management personnel, facilitating them to take corresponding measures for intervention and handling.

[0089] V. The system and method for identifying illegal behaviors at the operation site based on convolutional neural network provided by the present invention promote the improvement of safety production management level: The application of this technical solution helps power enterprises strengthen safety production management and improve the safety awareness and self-awareness of construction workers in observing regulations. Through real-time monitoring and automatic identification of illegal behaviors, illegal behaviors can be discovered and corrected in a timely manner, thereby effectively preventing accidents and ensuring the life safety of construction workers and the stable operation of power grid equipment.

[0090] The above-disclosed is only the preferred embodiment of the system and method for identifying illegal behaviors at the operation site based on convolutional neural network of the present invention. Of course, the scope of rights of the present invention cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present invention still fall within the scope covered by the invention.

Claims

1. A method for identifying illegal behaviors at work sites based on convolutional neural networks, characterized by: The following steps are involved: Establish a model for identifying illegal behaviors in power operation scenarios; Collect pictures and 3D point cloud data of power operation scenes; Integrate the power operation scene pictures and three-dimensional point cloud data to obtain integrated data; The integrated data of the power operation scene is input into the power operation scene violation behavior recognition model for recognition, and the recognition result is obtained; A judgment is made based on the recognition result. If there is any violation, an alarm is issued to the work site and the alarm data is sent to the management end. If there is no violation, monitoring continues.

2. The method for identifying illegal behaviors at work sites based on convolutional neural networks according to claim 1, characterized in that: In "Establishing a model for identifying illegal behaviors in power operation scenarios", the following steps are included: Obtain historical data sets of power operation scenarios, including various operation environments and violation behavior types; Label the violation type, location, and 3D point cloud data in each image in the historical dataset; Divide the labeled historical data set into training set and test set; Training the neural network model based on the training set to obtain a training model; The training model is tested on the test set to obtain a model for identifying illegal behaviors in power operation scenarios.

3. The method for identifying illegal behaviors at work sites based on convolutional neural networks as claimed in claim 2, characterized in that: In "Collecting power operation scene pictures and 3D point cloud data", the following steps are included: Use on-site cameras to collect images of power operation scenes in real time; Use a 3D laser scanner to obtain 3D point cloud data of the work site.

4. The method for identifying illegal behaviors at work sites based on convolutional neural networks as claimed in claim 3, characterized in that: In "Integrating power operation scene images and three-dimensional point cloud data to obtain integrated data", The following steps are included: Preprocess the collected images and point cloud data; Extract visual features of traffic violations from images and spatial features from 3D point clouds; The visual features and spatial features are integrated to obtain integrated data.

5. The method for identifying illegal behaviors at work sites based on convolutional neural networks as claimed in claim 4, characterized in that: In "inputting the integrated data of the power operation scene into the power operation scene violation behavior recognition model for recognition, and obtaining the recognition result", the following steps are included: The integrated data is input into the illegal behavior identification model of the power operation scene to analyze the input data and identify illegal behaviors; The illegal behavior recognition model for power operation scenarios outputs the recognition results of illegal behaviors, including information such as the type and location of the violation.

6. The method for identifying illegal behaviors at work sites based on convolutional neural networks as claimed in claim 5, characterized in that: In "making a judgment based on the recognition result, if there is a violation, sending an alarm to the work site and sending alarm data to the management end, and if there is no violation, continuing monitoring", the following steps are included: Analyze the recognition results output by the model to determine whether there is any violation; If a violation is detected, the alarm mechanism is triggered and an audible and visual alarm signal is sent to the work site; Send violation behavior data and alarm information to the management end so that management personnel can take corresponding measures; If no violation is detected, the system continues to monitor the work site.

7. A system for identifying illegal behaviors at work sites based on convolutional neural networks, used in a method for identifying illegal behaviors at work sites based on convolutional neural networks as claimed in any one of claims 1 to 6, characterized in that: It includes an acquisition module, a three-dimensional point cloud ranging module, an electric power operation scene illegal behavior recognition module and an alarm module. The acquisition module and the three-dimensional point cloud ranging module are respectively connected to the electric power operation scene illegal behavior recognition module, and the alarm module is connected to the electric power operation scene illegal behavior recognition module.

Images