A power operation site safety management and control method and system based on a hybrid neural network

Abstract

The application relates to a power operation site safety management and control method and system based on a hybrid neural network, which comprises the following steps: deploying a plurality of groups of image acquisition devices at a power operation site to collect power operation site images; pre-processing the collected power operation site images; inputting the pre-processed operation site images into a pre-trained hybrid neural network, intelligently identifying typical violation information of the power operation site by using the hybrid neural network; issuing a warning to the power operation site personnel according to the identified typical violation information and saving the identified violation information; and optimizing the hybrid neural network according to the violation information result, so that the safety management and control level of the power operation site is improved.

Description

Technical Field

[0001] This application relates to the field of power operation safety management and control technology, and more specifically, to a method and system for power operation on-site safety management and control based on a hybrid neural network. Background Technology

[0002] Electrical work performed in substations is characterized by complex environments, high risks, and stringent safety requirements, necessitating strict safety management to ensure the safety of personnel and equipment. Currently, dedicated safety supervisors are stationed at electrical work sites, but manual safety management inevitably carries the risk of oversights. Therefore, utilizing various auxiliary methods for hazard identification plays a crucial role in improving the level of safety management in electrical work scenarios.

[0003] Based on image information, artificial intelligence technology is used to intelligently identify various violations, and research and applications have been carried out in areas such as intelligent substation inspection and online monitoring of distribution substations. Current intelligent management methods for power operation scenarios need improvement in their ability to identify violations in complex environments.

[0004] Deploying mobile power operation safety management devices in substations, which enable intelligent monitoring of violations based on image information, can effectively improve the safety management of power operation scenarios and has the potential for large-scale application.

[0005] Existing technologies, such as Chinese patent CN112183265A, disclose a video monitoring and alarm method and system for power construction based on image recognition. This method specifically includes: first, real-time acquisition of image information from the power construction site; annotation processing of image information including personnel information; and data processing of the annotated image information to obtain image data; then, using a trained neural network as a pre-trained model, and retraining the pre-trained model based on violation data to obtain an image recognition model; finally, inputting the image data into the trained image recognition model to determine whether personnel at the power construction site have violated regulations, and issuing an alarm signal when violations occur. By using image recognition technology to identify the behavior of personnel entering the construction site, determining whether their behavior is in violation, and issuing different alarm signals according to the level of violation, the method improves the security monitoring level of substations and reduces the occurrence of safety incidents.

[0006] The problems with the aforementioned existing technology are as follows:

[0007] 1. Model training relies on a large amount of illegal data: Retraining a pre-trained model with illegal data to obtain an image recognition model means that a sufficient number of illegal data samples are needed to ensure the model's accuracy and reliability. However, in practical applications, illegal behaviors may not be common, making it difficult to obtain a large amount of illegal data.

[0008] 2. High data processing complexity: Data processing of the labeled image information requires a certain amount of computing resources and time, which may affect the real-time performance of the alarm system. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes a method and system for on-site safety management of power operations based on a hybrid neural network.

[0010] The technical solution of this invention is as follows:

[0011] This invention proposes a method for on-site safety management of power operations based on a hybrid neural network, comprising the following steps:

[0012] Step S1: Deploy several sets of image acquisition devices at the power operation site to acquire images of the power operation site;

[0013] Step S2: Preprocess the collected images of the power operation site;

[0014] Step S3: Input the pre-processed work site image into the pre-trained hybrid neural network and use the hybrid neural network to intelligently identify typical violations at the power work site.

[0015] Step S4: Issue warnings to personnel at the power operation site based on the identified typical violations and save the identified violation information;

[0016] Step S5: The on-site management personnel of the power operation optimize the hybrid neural network based on the identified violation information.

[0017] In a preferred embodiment, the hybrid neural network is a combination of a residual neural network and a self-attention mechanism, and ResNet50 is used to replace the convolutional module in the hybrid neural network.

[0018] As a preferred embodiment, the typical violation information includes: entering the work site without wearing safety protective equipment, performing high-altitude operations without protection, and workers crossing safety fences or safety warning lines.

[0019] As a preferred embodiment, the calculation of the self-attention mechanism in the hybrid neural network includes the following steps:

[0020] The input features are projected using three 1×1 convolutions and then reshaped into N parts, resulting in a set of intermediate features containing 3×N feature maps. These intermediate features are then grouped into N sets, each containing three features. The three feature maps are used as the query, key, and value, respectively. A multi-head self-attention mechanism is employed, and the calculation formula is as follows:

[0021]

[0022] In the formula, || represents splicing the attention of N heads; (a,b) represents a local region within the z-range of pixel (i,j); Points within the region; l represents the number of heads. ; q, k, and v are the attention scores for query, key, and value, respectively; Attention is calculated on the query and key feature maps; the calculation formula is as follows:

[0023]

[0024] In the formula: d is The dimension;

[0025] The attention scores for query, key, and value are calculated using the following formulas:

[0026]

[0027] In the formula: , , These are the weight matrices for the three feature maps: query, key, and value. Let be the feature value of the input image at position (i,j).

[0028] In a preferred embodiment, the ResNet50 convolutional part of the hybrid neural network uses a light fully connected layer to generate feature maps; the generated features are then shifted and aggregated to extract local features.

[0029] In a preferred embodiment, the hybrid neural network combines the attention component with the convolutional component, as shown in the following formula:

[0030]

[0031] In the formula, This is a self-attention feature map; For convolutional feature maps; For custom weights.

[0032] In a preferred embodiment, during the training process of the hybrid neural network, due to the non-uniformity of the power operation site image samples, the SMOTE algorithm for comprehensively sampled artificially synthesized data is used to expand the training samples. The specific steps are as follows:

[0033] Select training sample O as the baseline value for a single augmentation, and calculate the Euclidean distance from training sample O to other training sample points P. The specific calculation formula is as follows:

[0034]

[0035] In the formula: This represents the pixel value of sample O at pixel (r,s). This represents the pixel value of sample P at pixel point (r,s); n is the sample expansion ratio;

[0036] Linear interpolation is performed on training samples O and P to generate a new sample X. This process is repeated multiple times to expand the training samples.

[0037] The specific interpolation formula is as follows:

[0038]

[0039] In the formula, θ is a random number between (0,1).

[0040] On the other hand, the present invention also provides a power operation site safety management and control system based on a hybrid neural network, comprising:

[0041] The image acquisition device deployment module deploys several sets of image acquisition devices at the power operation site to acquire images of the power operation site.

[0042] The image preprocessing module preprocesses the acquired images of the power operation site.

[0043] The violation information identification module inputs the pre-processed work site image into a pre-trained hybrid neural network, and uses the hybrid neural network to intelligently identify typical violations at the power work site.

[0044] The violation warning module issues warnings to personnel at the power operation site based on the identified typical violation information and saves the identified violation information;

[0045] The feedback optimization module allows on-site power operation managers to optimize the hybrid neural network based on the identified violation information.

[0046] In another aspect, the present invention also provides an electronic device having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements a power operation site safety management method based on a hybrid neural network as described in any embodiment of the present invention.

[0047] In another aspect, the present invention also provides a computer-readable medium for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement a power operation site safety management method based on a hybrid neural network as described in any embodiment of the present invention.

[0048] The present invention has the following beneficial effects:

[0049] 1. Replacing the convolutional modules in a hybrid neural network with ResNet50 addresses the degradation problem of deep neural networks, enabling the learning of richer feature representations and thus improving the overall network performance. This results in higher accuracy in tasks such as image recognition and object detection.

[0050] 2. In hybrid neural networks, the convolutional part uses a light fully connected layer to generate feature maps, which can significantly reduce the number of parameters. Due to the reduction in the number of parameters, the computational cost of the light fully connected layer is also reduced accordingly. It can maintain performance comparable to that of the traditional fully connected layer to a certain extent, improve the efficiency of image processing, and ensure the timeliness of alarm information.

[0051] 3. In hybrid neural networks, the outputs of the attention mechanism and the convolution part are combined to obtain richer feature representations, which include both global dependencies and local details, enabling the model to understand the input data more comprehensively. This enhances the model's accuracy and robustness, improves its generalization ability, and achieves better performance without requiring extensive retraining.

[0052] 4. The SMOTE algorithm, which uses integrated sampling of artificially synthesized data, is used to expand the training samples, effectively solving the problem of unevenness in the training samples of power operation site images and ensuring the accuracy of model training. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram of the method flow in Example 1;

[0055] Figure 2 This is a schematic diagram of a hybrid neural network structure;

[0056] Figure 3 This is a schematic diagram of the sample expansion process. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] It should be understood that the step numbers used in the text are for ease of description only and are not intended to limit the order in which the steps are performed.

[0059] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0060] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0061] The term “and / or” refers to any combination of one or more of the associated listed items, as well as all possible combinations, and includes these combinations.

[0062] Example 1:

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, specific embodiments of this application will be described below, with reference to the accompanying drawings. Figure 1 The technical solution of the present invention will be clearly and completely described.

[0064] To address the problems of existing technologies, this invention provides a method for on-site safety management and control of power operations based on hybrid neural networks, comprising the following steps:

[0065] Step S1: Deploy several sets of image acquisition devices at the power operation site to acquire images of the power operation site;

[0066] Step S2: Preprocess the collected images of the power operation site;

[0067] Step S201, Image reading and format conversion: Read the collected images of the power operation site and convert the image format into a format suitable for the input of the hybrid neural network;

[0068] Step S202, image size adjustment: The size of the acquired power operation site image is adjusted to meet the input requirements of the hybrid neural network. When adjusting the image size, it is usually necessary to consider maintaining the aspect ratio of the image to avoid image distortion.

[0069] Step S203, image normalization and enhancement operation: normalize the pixel values ​​of the image and adjust the range of pixel values ​​to [0, 1]; after normalization, perform data enhancement operation, which includes random rotation, flipping, translation, scaling, color transformation, etc.

[0070] Step S204, data format conversion and batch processing, converts the preprocessed image data into the tensor format required by the hybrid neural network; in order to improve the efficiency of training and inference, multiple images are usually processed in batches to form a batch tensor.

[0071] Step S3: Input the pre-processed work site image into the pre-trained hybrid neural network and use the hybrid neural network to intelligently identify typical violations at the power work site.

[0072] Step S301, the hybrid neural network is as follows Figure 2 As shown, combining the advantages of residual neural networks and self-attention mechanisms, the input features are first projected through three 1×1 convolutions, and then reshaped into N parts, resulting in a set of intermediate features containing 3×N feature maps.

[0073] Step S302: Next, perform self-attention and convolution calculations in parallel. For the self-attention part, the intermediate features are grouped into N sets, each set containing 3 features, each feature derived from a 1×1 convolution. The three corresponding feature maps serve as the query, key, and value, respectively, using a multi-head self-attention module. The calculation formula is:

[0074]

[0075] In the formula, || represents splicing the attention of N heads; (a,b) represents a local region within the z-range of pixel (i,j); Points within the region; l represents the number of heads. ; q, k, and v are the attention scores for query, key, and value, respectively; Attention is calculated on the query and key feature maps; the calculation formula is as follows:

[0076]

[0077] In the formula: d is The dimension;

[0078] The attention scores for query, key, and value are calculated using the following formulas:

[0079] .

[0080] In the formula: , , These are the weight matrices for the three feature maps: query, key, and value. The input image value at position (i,j).

[0081] Step S303: The convolutional module in the hybrid neural network adopts a ResNet50 structure; the convolutional part uses a light fully connected layer to generate feature maps; the generated features are shifted and aggregated to extract local features.

[0082] Step S304: Combine the attention part with the convolution part, as shown in the following formula:

[0083]

[0084] In the formula, This is a self-attention feature map; For convolutional feature maps; For custom weights.

[0085] Step S304, see Appendix Figure 3 Because the image samples from power operation sites are non-uniform, the hybrid neural network training process uses the SMOTE algorithm, which integrates artificially synthesized data from comprehensive sampling, to augment the training samples. Specifically, this includes the following steps:

[0086] The SMOTE algorithm is an algorithm for augmenting minority class samples in imbalanced samples, and it is an improvement on the random oversampling algorithm.

[0087] First, iterate through the minority class samples, selecting each minority class sample O as the baseline for a single augmentation, and calculate its Euclidean distance to other minority class sample points P; the specific calculation formula is as follows:

[0088]

[0089] In the formula: This represents the pixel value of sample O at pixel (r,s). This represents the pixel value of sample P at pixel point (r,s); n is the sample expansion ratio;

[0090] Based on the Euclidean distance, several nearest neighbors of sample point O are obtained. A nearest neighbor P is randomly selected from these neighbors and linearly interpolated with the reference sample O using the following formula to generate a new sample X. This process is repeated n times (where n is the sample expansion ratio). The specific interpolation formula is as follows:

[0091]

[0092] In the formula, θ is a random number between (0,1).

[0093] In this way, for each baseline sample xi, n new samples will be generated. After traversing all the minority class samples, the original number of samples will be expanded to n times the original number. Sample expansion is achieved through oversampling.

[0094] Step S305: Input the images of the work site into the trained hybrid neural network to intelligently identify typical violations at the power work site. The violations include: entering the work site without wearing safety protective equipment, performing high-altitude work without protection, and workers crossing safety fences or safety warning lines.

[0095] Step S4: Issue warnings to personnel at the power operation site based on the identified typical violations and save the identified violation information;

[0096] Step S401: Issue a warning to personnel at the power operation site based on the identified typical violation information;

[0097] Step S402: Save the images and violation information identified by the hybrid neural network as violations, so that the power operation site management personnel can trace the violations in the future.

[0098] Step S5: The on-site management personnel of the power operation optimize the hybrid neural network based on the identified violation information.

[0099] Based on the identified violation information, on-site management personnel in power operations provide feedback to the hybrid neural network on inaccurate violation behaviors output; the hybrid neural network then performs reinforcement learning based on the feedback to improve the accuracy of model recognition.

[0100] Example 2:

[0101] This embodiment provides a power operation site safety management and control system based on a hybrid neural network, including:

[0102] The image acquisition device deployment module deploys several sets of image acquisition devices at the power operation site to acquire images of the power operation site.

[0103] The image preprocessing module preprocesses the acquired images of the power operation site.

[0104] The violation information identification module inputs the pre-processed work site image into a pre-trained hybrid neural network, and uses the hybrid neural network to intelligently identify typical violations at the power work site.

[0105] The violation warning module issues warnings to personnel at the power operation site based on the identified typical violation information and saves the identified violation information;

[0106] The feedback optimization module allows on-site power operation managers to optimize the hybrid neural network based on the identified violation information.

[0107] Example 3:

[0108] This embodiment provides an electronic device that stores a computer program. When the computer program is executed by a processor, it implements a power operation site safety management method based on a hybrid neural network as described in any embodiment of the present invention.

[0109] Example 4:

[0110] This embodiment provides a computer-readable medium for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement a power operation site safety management method based on a hybrid neural network as described in any embodiment of the present invention.

[0111] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0112] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented using electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0113] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0114] In the several embodiments provided in this application, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for on-site safety management and control of power operations based on hybrid neural networks, characterized in that, Includes the following steps: Step S1: Deploy several sets of image acquisition devices at the power operation site to acquire images of the power operation site; Step S2: Preprocess the collected images of the power operation site; Step S3: Input the preprocessed work site image into the pre-trained hybrid neural network, and use the hybrid neural network to intelligently identify typical violations at the power work site; the hybrid neural network is a combination of a residual neural network and a self-attention mechanism, and ResNet50 is used to replace the convolutional modules in the hybrid neural network; the self-attention mechanism calculation in the hybrid neural network includes the following steps: The input features are projected using three 1×1 convolutions and then reshaped into N parts, resulting in a set of intermediate features containing 3×N feature maps. These intermediate features are then grouped into N sets, each containing three features. The three feature maps are used as the query, key, and value, respectively. A multi-head self-attention mechanism is employed, and the calculation formula is as follows: In the formula, || represents splicing the attention of N heads; (a,b) represents a local region within the z-range of pixel (i,j); Points within the region; l represents the number of heads. ; q, k, and v are the attention scores for query, key, and value, respectively; Attention is calculated on the query and key feature maps; the calculation formula is as follows: In the formula: d is The dimension; The attention scores for query, key, and value are calculated using the following formulas: In the formula: , , These are the weight matrices for the three feature maps: query, key, and value. Let (i,j) be the feature value of the input image at position (i,j). During the training process of the hybrid neural network, due to the non-uniformity of the power operation site image samples, the SMOTE algorithm, which integrates artificially synthesized data through comprehensive sampling, is used to expand the training samples. The specific steps are as follows: Select training sample O as the baseline value for a single augmentation, and calculate the Euclidean distance from training sample O to other training sample points P. The specific calculation formula is as follows: In the formula: This represents the pixel value of sample O at pixel (r,s). This represents the pixel value of sample P at pixel point (r,s); n is the sample expansion ratio; Linear interpolation is performed on training samples O and P to generate a new sample X. This process is repeated multiple times to expand the training samples. The specific interpolation formula is as follows: In the formula, θ is a random number between (0,1); Step S4: Issue warnings to personnel at the power operation site based on the identified typical violations and save the identified violation information; Step S5: The on-site management personnel of the power operation optimize the hybrid neural network based on the identified violation information.

2. The method for on-site safety management and control of power operations based on a hybrid neural network according to claim 1, characterized in that, The typical violations include: entering the work site without wearing safety protective equipment, performing high-altitude operations without protection, and workers crossing safety fences or safety warning lines.

3. The method for on-site safety management and control of power operations based on a hybrid neural network according to claim 1, characterized in that: In the hybrid neural network, the ResNet50 convolutional part uses a light fully connected layer to generate feature maps; the generated features are then shifted and aggregated to extract local features.

4. The method for on-site safety management and control of power operations based on a hybrid neural network according to claim 1, characterized in that: The hybrid neural network combines the attention component with the convolutional component, as shown in the following formula: In the formula, This is a self-attention feature map; For convolutional feature maps; For custom weights.

5. A power operation site safety management and control system based on a hybrid neural network, using the power operation site safety management and control method based on a hybrid neural network as described in any one of claims 1 to 4, characterized in that, include: The image acquisition device deployment module deploys several sets of image acquisition devices at the power operation site to acquire images of the power operation site. The image preprocessing module preprocesses the acquired images of the power operation site. The violation information identification module inputs the pre-processed work site image into a pre-trained hybrid neural network, and uses the hybrid neural network to intelligently identify typical violations at the power work site. The violation warning module issues warnings to personnel at the power operation site based on the identified typical violation information and saves the identified violation information; The feedback optimization module allows on-site power operation managers to optimize the hybrid neural network based on the identified violation information.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a power operation site safety management method based on a hybrid neural network as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a power operation site safety management method based on a hybrid neural network as described in any one of claims 1 to 4.

Images