Power operation site safety management and control method and system based on hybrid neural network
By combining residual neural network and self-attention mechanism in the power operation site safety control method at the power operation site, using ResNet50 and SMOTE algorithms, the identification of violations in the existing technology depends on a large amount of violation data and data processing complexity. Problem, improving identification accuracy and real-time.
Patent Information
- Application Number
- CN202411186011.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-27
AI Technical Summary
The prior art relies on a large number of violation data training models in the identification of violations at power operations sites, and the high complexity of data processing affects real-time performance, making it difficult to ensure accuracy and reliability in practical applications.
The power operation site safety control method based on hybrid neural network is adopted, combined with residual neural network and self-attention mechanism, and the convolution module is replaced by ResNet50, and the training samples are expanded through the SMOTE algorithm to reduce the amount of parameters and calculations, and improve the generalization of the model. ability and accuracy.
It improves the accuracy of violation recognition and the robustness of the model, ensures the timeliness and accuracy of alarm information, reduces the dependence on a large number of violation data, and solves the problem of data processing complexity.
Smart Images

Figure CN119049130A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of power operation safety management and control, and more specifically, to a power operation site safety management and control method and system based on a hybrid neural network. Background Art
[0002] The power operations carried out in substations are characterized by complex environments, high risks, and high safety requirements. Strict safety management and control must be carried out to ensure the safety of personnel and equipment. At present, the power operation site is equipped with special safety supervisors. Manual safety management and control inevitably have the risk of omissions. Hazard identification through various auxiliary means plays an important role in improving the safety management and control level of power operation scenes.
[0003] Based on image information, artificial intelligence technology is used to intelligently identify various violation information, and research and application have been carried out in the fields of substation intelligent inspection and distribution station online monitoring. The current intelligent control method of power operation scene has the effect of improving the recognition of power operation violations under complex backgrounds.
[0004] Deploying mobile power operation safety control devices in substations and realizing intelligent monitoring of violations based on image information can effectively improve the safety control effects of power operation scenarios and has the potential for large-scale promotion and application.
[0005] Existing technologies such as the Chinese patent with the patent number "CN112183265A" disclose a method and system for video monitoring and alarming of electric power construction based on image recognition. The method specifically includes: firstly, collecting image information of the electric power construction site in real time, annotating the image information including the person information in the image information, and performing data processing on the annotated image information to obtain image data; then using the trained neural network as a pre-training model, and then retraining the pre-training model according to the violation data to obtain an image recognition model; finally, inputting the image data into the trained image recognition model to determine whether the personnel at the electric power construction site have violated the regulations, and issuing an alarm prompt signal when a violation occurs. By using image recognition technology to identify the behavior of personnel entering the construction site, it is determined whether the behavior of personnel entering the construction site is in violation of the regulations, and different alarm signals can be issued according to the violation level to improve the security supervision level of the substation and reduce the occurrence of security incidents.
[0006] The problems existing in the above prior art are:
[0007] 1. Model training relies on a large amount of violation data: The image recognition model is obtained by retraining the pre-trained model with violation data, which means that there must be enough violation data samples to ensure the accuracy and reliability of the model. However, in actual applications, violations may not be common, and it is difficult to obtain a large amount of violation data.
[0008] 2. High data processing complexity: Data processing of the marked image information requires a certain amount of computing resources and time, which may affect the real-time performance of the alarm system. Summary of the invention
[0009] In order to solve the above technical problems, the present invention proposes a method and system for safety management and control of power operation sites based on hybrid neural network.
[0010] The technical solution of the present invention is as follows:
[0011] The present invention proposes a method for safety management and control of power operation sites based on a hybrid neural network, comprising the following steps:
[0012] Step S1, deploying several groups of image acquisition devices at the power operation site to collect images of the power operation site;
[0013] Step S2, preprocessing the collected power operation site images;
[0014] Step S3, inputting the preprocessed work site image into a pretrained hybrid neural network, and using the hybrid neural network to intelligently identify typical violation information at the power work site;
[0015] Step S4, issuing a warning to the power operation site personnel according to the identified typical violation information and saving the identified violation information;
[0016] Step S5: The power operation site manager optimizes the hybrid neural network based on the identified violation information.
[0017] As a preferred implementation, the hybrid neural network is a combination of a residual neural network and a self-attention mechanism, and ResNet50 is used to replace the convolution module in the hybrid neural network.
[0018] As a preferred implementation, the typical violation information includes: entering the work site without wearing safety protection tools, performing high-altitude operations without protection, and workers crossing safety fences or safety cordons.
[0019] As a preferred implementation, the calculation of the self-attention mechanism in the hybrid neural network includes the following steps:
[0020] The input features are projected through three 1×1 convolutions and then reshaped into N parts to obtain a set of intermediate features containing 3×N feature maps; the intermediate features are grouped into N groups, each containing 3 features, and the corresponding three feature maps are used as query, key and value respectively. The multi-head self-attention mechanism is used, and the calculation formula is as follows:
[0021]
[0022] In the formula, || represents the splicing of N attention heads; N z(i,j) Represents the local area of the z range near the pixel point (i, j); (a, b) is N z(i,j) points in the region; l is the number of heads, l∈N=1,2,3,…,N; q, k, v are the attention scores of query, key and value respectively; To calculate the attention of the query and key feature graphs; the calculation formula is as follows:
[0023]
[0024] Where: d is Dimensions;
[0025] The calculation formula for the attention scores of query, key and value is as follows:
[0026]
[0027] Where: W q ,W k ,W v They are the weight matrices of the three feature maps: query, key, and value; f i,j is the eigenvalue of the input image at position (i, j).
[0028] As a preferred implementation, the ResNet50 convolution part in the hybrid neural network uses a light fully connected layer to generate a feature map; the generated features are shifted and aggregated to extract local features.
[0029] As a preferred implementation, the hybrid neural network combines the attention part with the convolution part, and the specific combination formula is as follows:
[0030] F out =αF att +βF conv
[0031] In the formula, F att is the self-attention feature map; F conv is the convolution feature map; α and β are custom weights.
[0032] As a preferred implementation, during the hybrid neural network training process, since the power operation site image samples are non-uniform, the comprehensive sampling artificial synthetic data SMOTE algorithm is used to expand the training samples. The specific steps are as follows:
[0033] Select training sample O as the benchmark value for single expansion, and calculate the Euclidean distance from training sample O to other training sample points P. The specific calculation formula is as follows:
[0034]
[0035] In the formula: o r,s represents the pixel value of sample O at the pixel point (r, s), p r,s represents the pixel value of sample P at pixel point (r, s); n is the sample expansion ratio;
[0036] Perform linear interpolation on training samples O and P to generate a new sample X. Repeat this process multiple times to expand the training samples.
[0037] The specific interpolation formula is as follows:
[0038] x r,s =o r,s +(p r,s -o r,s )·θ
[0039] In the formula, θ is a random number between (0,1).
[0040] On the other hand, the present invention also provides a power operation site safety management and control system based on a hybrid neural network, comprising:
[0041] An image acquisition device deployment module deploys several groups of image acquisition devices at the power operation site to collect images of the power operation site;
[0042] An image acquisition preprocessing module is used to preprocess the acquired images of the power operation site;
[0043] The violation information recognition module inputs the pre-processed work site image into the pre-trained hybrid neural network and uses the hybrid neural network to intelligently recognize typical violation information at the power work site;
[0044] The violation warning module warns the personnel on the power operation site based on the identified typical violation information and saves the identified violation information;
[0045] In the feedback optimization module, the on-site management personnel of power operations optimize the hybrid neural network based on the results of the identified violation information.
[0046] On the other hand, the present invention further provides an electronic device having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for safety management and control of power operation sites based on a hybrid neural network as described in any embodiment of the present invention is implemented.
[0047] On the other hand, the present invention also provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement a method for safety management and control of power operation sites based on a hybrid neural network as described in any embodiment of the present invention.
[0048] The present invention has the following beneficial effects:
[0049] 1. Use ResNet50 to replace the convolution module in the hybrid neural network to solve the degradation problem of deep neural networks and learn richer feature representations, thereby improving the performance of the entire network. In tasks such as image recognition and object detection, it will bring higher accuracy.
[0050] 2. The convolution part of the hybrid neural network uses a light fully connected layer to generate feature maps, which can significantly reduce the number of parameters. Due to the reduction in the number of parameters, the computational complexity of the light fully connected layer is also reduced accordingly, and it can maintain performance comparable to that of the traditional fully connected layer to a certain extent, thereby improving image processing efficiency and ensuring the timeliness of warning information.
[0051] 3. In the hybrid neural network, the output of the attention mechanism and the convolution part are combined to obtain a richer feature representation, in which the feature representation contains both global dependencies and local detail information, so that the model can understand the input data more comprehensively; enhance the accuracy and robustness of the model; improve the generalization ability of the model, and achieve better performance without a lot of retraining.
[0052] 4. The comprehensive sampling artificial synthetic data SMOTE algorithm is used to expand the training samples, which effectively solves the problem of uneven training samples of power operation site images and ensures the accuracy of model training. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0054] Figure 1 This is a schematic diagram of the method flow of Example 1;
[0055] Figure 2 Schematic diagram of hybrid neural network structure;
[0056] Figure 3 Schematic diagram of the sample expansion process. DETAILED DESCRIPTION
[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.
[0059] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0060] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0061] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.
[0062] Embodiment 1:
[0063] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following will be combined with the specific embodiments of the present application and refer to the attached Figure 1 , clearly and completely describe the technical solution of the present invention.
[0064] In order to solve the problems of the prior art, the present invention provides a method for safety management and control of power operation sites based on a hybrid neural network, comprising the following steps:
[0065] Step S1, deploying several groups of image acquisition devices at the power operation site to collect images of the power operation site;
[0066] Step S2, preprocessing the collected power operation site images;
[0067] Step S201, image reading and format conversion, reading the collected power operation site image, and converting the image format into a format suitable for hybrid neural network input;
[0068] Step S202, image resizing, resizing the collected power operation site image to make it meet the input requirements of the hybrid neural network. When resizing the image, it is usually necessary to consider maintaining the aspect ratio of the image to avoid image deformation;
[0069] Step S203, image normalization and enhancement operation, normalizing the pixel values of the image, adjusting the range of the pixel values to between [0, 1]; performing data enhancement operation after normalization, data enhancement includes random rotation, flipping, translation, scaling, color conversion, etc.;
[0070] Step S204, data format conversion and batch processing, converting the preprocessed image data into the tensor format required by the hybrid neural network; in order to improve the efficiency of training and reasoning, multiple images are usually grouped into a batch for processing to form a batch tensor.
[0071] Step S3, inputting the preprocessed work site image into a pretrained hybrid neural network, and using the hybrid neural network to intelligently identify typical violation information at the power work site;
[0072] Step S301, the hybrid neural network is as follows: Figure 2 As shown in the figure; combining the advantages of residual neural network and self-attention mechanism, the input features are first projected through 3 1×1 convolutions and then reshaped into N parts to obtain a set of intermediate features containing 3×N feature maps.
[0073] Step S302, secondly, self-attention calculation and convolution calculation are carried out in parallel. For the self-attention part, the intermediate features are grouped into N groups, each group contains 3 features, and each feature comes from 1×1 convolution. The corresponding three feature maps are used as query, key and value respectively, and a multi-head self-attention module is used. The calculation formula is:
[0074]
[0075] In the formula, || represents the splicing of N attention heads; N z(i,j) Represents the local area of the z range near the pixel point (i, j); (a, b) is N z(i,j) points in the region; l is the number of heads, l∈N=1,2,3,…,N; q, k, v are the attention scores of query, key and value respectively; To calculate the attention of the query and key feature graphs; the calculation formula is as follows:
[0076]
[0077] Where: d is Dimensions;
[0078] The calculation formula for the attention scores of query, key and value is as follows:
[0079]
[0080] Where: W q ,W k ,W v They are the weight matrices of the three feature maps: query, key, and value; f i,j is the value of the input image at position (i, j).
[0081] Step S303, the convolution module in the hybrid neural network adopts the resnet50 structure; the convolution part uses a light fully connected layer to generate a feature map; the generated features are shifted and aggregated to extract local features.
[0082] Step S304, combining the attention part with the convolution part, the specific combination formula is as follows:
[0083] F out =αF att +βF conv
[0084] In the formula, F att is the self-attention feature map; F conv is the convolution feature map; α and β are custom weights.
[0085] Step S304, see attached Figure 3 Since the image samples of the power operation site are non-uniform, the hybrid neural network training process uses the comprehensive sampling artificial synthetic data SMOTE algorithm to expand the training samples, which specifically includes the following steps:
[0086] The SMOTE algorithm is an algorithm for expanding minority class samples in unbalanced samples and is an improvement on the random oversampling algorithm.
[0087] First, traverse the minority class samples, select each minority class sample O in turn as the benchmark value for a single expansion, and calculate its Euclidean distance to other sample points P in the minority samples of this class; the specific calculation formula is as follows:
[0088]
[0089] In the formula: o r,s represents the pixel value of sample O at the pixel point (r, s), p r,s represents the pixel value of sample P at pixel point (r, s); n is the sample expansion ratio;
[0090] According to the size of the Euclidean distance, several neighbors of the sample point O are obtained, and a neighbor P is randomly selected from them. It is linearly interpolated with the benchmark sample O according to the following formula to generate a new sample X. This is repeated n times (n is the sample expansion ratio). The specific interpolation formula is as follows:
[0091] x r,s =o r,s +(p r,s -o r,s )·θ
[0092] In the formula, θ is a random number between (0,1).
[0093] In this way, for each benchmark sample xi, n new samples will be generated. After traversing all the minority class samples, the number of original samples will be expanded to n times the original number, and sample expansion is achieved through oversampling.
[0094] Step S305, the work site image is input and combined with the trained hybrid neural network to intelligently identify typical violation information at the power work site, where the violation information includes: entering the work site without wearing safety protection tools, performing high-altitude operations without protection, and workers crossing safety fences or safety warning lines.
[0095] Step S4, issuing a warning to the power operation site personnel according to the identified typical violation information and saving the identified violation information;
[0096] Step S401, issuing a warning to the personnel at the power operation site according to the identified typical violation information;
[0097] Step S402, the pictures and violation information identified as violations by the hybrid neural network are saved to facilitate the subsequent tracing of violations by the power operation site management personnel.
[0098] Step S5: The power operation site manager optimizes the hybrid neural network based on the identified violation information.
[0099] The on-site managers of power operations provide feedback on the inaccurate violations output by the hybrid neural network based on the identified violation information results; the hybrid neural network performs reinforcement learning based on the feedback content to improve the accuracy of model recognition.
[0100] Embodiment 2:
[0101] This embodiment provides a power operation site safety management and control system based on a hybrid neural network, including:
[0102] An image acquisition device deployment module deploys several groups of image acquisition devices at the power operation site to collect images of the power operation site;
[0103] An image acquisition preprocessing module is used to preprocess the acquired images of the power operation site;
[0104] The violation information recognition module inputs the pre-processed work site image into the pre-trained hybrid neural network and uses the hybrid neural network to intelligently recognize typical violation information at the power work site;
[0105] The violation warning module warns the personnel on the power operation site based on the identified typical violation information and saves the identified violation information;
[0106] In the feedback optimization module, the on-site management personnel of power operations optimize the hybrid neural network based on the results of the identified violation information.
[0107] Embodiment three:
[0108] This embodiment provides an electronic device having a computer program stored thereon, wherein when the computer program is executed by a processor, a method for safety management and control of a power operation site based on a hybrid neural network as described in any embodiment of the present invention is implemented.
[0109] Embodiment 4:
[0110] This embodiment provides a computer-readable medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement a method for safety management and control of power operation sites based on a hybrid neural network as described in any embodiment of the present invention.
[0111] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.
[0112] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0114] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.
[0115] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for safety management and control of power operation sites based on hybrid neural network, characterized in that: The following steps are involved: Step S1, deploying several groups of image acquisition devices at the power operation site to collect images of the power operation site; Step S2, preprocessing the collected power operation site images; Step S3, inputting the preprocessed work site image into a pretrained hybrid neural network, and using the hybrid neural network to intelligently identify typical violation information at the power work site; Step S4, issuing a warning to the power operation site personnel according to the identified typical violation information and saving the identified violation information; Step S5: The power operation site manager optimizes the hybrid neural network based on the identified violation information.
2. A method for safety management and control of electric power operation site based on hybrid neural network according to claim 1, characterized in that: The hybrid neural network is a combination of a residual neural network and a self-attention mechanism, and ResNet50 is used to replace the convolution module in the hybrid neural network.
3. The method for safety management and control of electric power operation site based on hybrid neural network according to claim 1 is characterized in that: The typical violation information includes: entering the work site without wearing safety protection tools, performing high-altitude work without protection, and workers crossing safety fences or safety cordons.
4. The method for safety management and control of power operation site based on hybrid neural network according to claim 2 is characterized in that: The calculation of the self-attention mechanism in the hybrid neural network includes the following steps: The input features are projected through three 1×1 convolutions and then reshaped into N parts to obtain a set of intermediate features containing 3×N feature maps; the intermediate features are grouped into N groups, each containing 3 features, and the corresponding three feature maps are used as query, key and value respectively. The multi-head self-attention mechanism is used, and the calculation formula is as follows: In the formula, || represents the splicing of N attention heads; N z(i,j) Represents the local area of the z range near the pixel point (i, j); (a, b) is N z(i,j) points in the region; l is the number of heads, l∈N=1,2,3,…,N; q, k, v are the attention scores of query, key and value respectively; To calculate the attention of the query and key feature graphs; the calculation formula is as follows: Where: d is Dimensions; The calculation formula for the attention scores of query, key and value is as follows: Where: W q ,W k ,W v They are the weight matrices of the three feature maps: query, key, and value; f i,j is the eigenvalue of the input image at position (i, j).
5. The method for safety management and control of electric power operation site based on hybrid neural network according to claim 2 is characterized in that: The ResNet50 convolution part in the hybrid neural network uses a light fully connected layer to generate feature maps; the generated features are shifted and aggregated to extract local features.
6. The method for safety management and control of power operation site based on hybrid neural network according to claim 2 is characterized by: The hybrid neural network combines the attention part with the convolution part. The specific combination formula is as follows: F out =αF att +βF conv In the formula, F att is the self-attention feature map; F conv is the convolution feature map; α and β are custom weights.
7. The method for safety management and control of power operation site based on hybrid neural network according to claim 2, characterized in that: During the training process of the hybrid neural network, since the image samples of the power operation site are non-uniform, the comprehensive sampling artificial synthetic data SMOTE algorithm is used to expand the training samples. The specific steps are as follows: Select training sample O as the benchmark value for single expansion, and calculate the Euclidean distance from training sample O to other training sample points P. The specific calculation formula is as follows: In the formula: o r,s represents the pixel value of sample O at the pixel point (r, s), p r,s represents the pixel value of sample P at pixel point (r, s); n is the sample expansion ratio; Perform linear interpolation on training samples O and P to generate a new sample X. Repeat this process multiple times to expand the training samples. The specific interpolation formula is as follows: x r,s =o r,s +(p r,s -o r,s )·θ In the formula, θ is a random number between (0,1).
8. A power operation site safety management and control system based on hybrid neural network, characterized in that: include: An image acquisition device deployment module deploys several groups of image acquisition devices at the power operation site to collect images of the power operation site; An image acquisition preprocessing module is used to preprocess the acquired images of the power operation site; The violation information recognition module inputs the pre-processed work site image into the pre-trained hybrid neural network and uses the hybrid neural network to intelligently recognize typical violation information at the power work site; The violation warning module warns the personnel on the power operation site based on the identified typical violation information and saves the identified violation information; In the feedback optimization module, the on-site management personnel of power operations optimize the hybrid neural network based on the results of the identified violation information.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements a method for safety management and control of power operation sites based on a hybrid neural network as described in any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the method for safety management and control of power operation site based on hybrid neural network as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Operation site digital safety management and control method, device and equipment and storage medium
CN115471865A
Real-time early warning method for typical violation behaviors of electric power operation
CN115909212A
Real-time risk identification method for substation near-electricity operation site
CN117523437A
Multi-task multi-modal machine learning system
US20200089755A1
