A method, system and terminal for real-time safety monitoring of electric power operation targets

Abstract

The present invention provides a method, system and terminal for real-time safety monitoring of targets of power operations, wherein a multi-head target detection network model is read and loaded from a local file or an online file through an initialization module; an initial target detection is performed on a real-time video data stream based on the loaded multi-head target detection network model through a target detection module to obtain an initial detection target; a scale distribution of the initial detection target is calculated through a target scale analysis module, and if the scale distribution of the target is consistent, a corresponding single-head target detection network model is selected through a network model replacement module according to the scale distribution of the target to replace the multi-head target detection network model, and formal target detection is performed. The present invention can effectively reduce the calculation time required in the real-time target detection operation process, thereby improving the real-time target detection frame rate of the safety monitoring system and maintaining the detection accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of power operation safety, and in particular to a method, system and terminal for real-time safety monitoring of power operation targets. Background Art

[0002] Real-time safety monitoring systems are very important for power operations. Safety monitoring systems with high detection accuracy and strong real-time detection can effectively reduce unsafe behaviors and potential risks in power operation scenarios. Improving the accuracy or real-time performance of the safety monitoring system is conducive to improving the safety of power operation scenarios.

[0003] At present, the implementation of real-time safety monitoring systems in power operation scenarios is mainly based on the multi-head target detection method of deep learning. However, for real-time safety monitoring in power operation scenarios, this multi-head detection method is not necessary and will reduce the frame rate of real-time detection.

[0004] Multi-head detection in deep learning target detection is a multi-scale target detection method that can detect targets of multiple scales at the same time, but it also requires more calculations and longer detection time. The advantage of multi-head target detection is that it can detect multiple targets of different sizes at the same time, which improves the accuracy and robustness of detection. For example, the currently popular target detection method YOLO series is based on multi-scale multi-head target detection method. However, the multi-head detection method also has significant disadvantages. Because it requires a lot of computing resources and computing time, the speed and frame rate of real-time detection are limited. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method, system and terminal for real-time safety monitoring of targets in power operations, which can effectively reduce the calculation time required during the operation of real-time target detection, thereby improving the real-time target detection frame rate of the safety monitoring system and maintaining the detection accuracy.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A method for real-time safety monitoring of electric power operations, comprising the steps of:

[0008] S1. Read and load the multi-head target detection network model from a local file or an online file through the initialization module;

[0009] S2, performing initial target detection on the real-time video data stream based on the loaded multi-head target detection network model through the target detection module to obtain an initial detection target;

[0010] S3, calculating the scale distribution of the initial detection target through the target scale analysis module, if the scale distribution of the target is consistent, proceeding to step S4;

[0011] S4, selecting a corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target through a network model replacement module;

[0012] S5. Enter formal target detection.

[0013] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0014] A target real-time safety monitoring system for electric power operation, comprising:

[0015] Initialization module: read and load the multi-head target detection network model from local files or online files;

[0016] Target detection module: performs initial target detection on the real-time video data stream based on the loaded multi-head target detection network model to obtain the initial detection target;

[0017] Target scale analysis module: calculates the scale distribution of the initial detection target, and if the scale distribution of the target is consistent, calls the target detection module to perform target detection;

[0018] Network model replacement module: selects the corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target, and enters the formal target detection.

[0019] In order to solve the above technical problems, another technical solution adopted by the present invention is:

[0020] A terminal for real-time safety monitoring of targets for electric power operations, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for real-time safety monitoring of targets for electric power operations when executing the computer program.

[0021] The beneficial effects of the present invention are as follows: the present invention provides a method, system and terminal for real-time safety monitoring of targets in power operations, which completes safety monitoring of real-time targets in power operation scenarios through four modules: an initialization module, a target detection module, a target scale analysis module and a network model replacement module. The initialization module first loads a multi-head target detection network model so that the target detection model first uses the multi-head target detection network model to perform target detection on real-time video in the initial stage, and then uses the target scale analysis module to perform scale distribution analysis on the targets detected in the initial stage. Since the position of a camera for shooting real-time video is generally fixed, and no zoom or steering operation is performed during the operation of the safety monitoring system, the camera The category of target objects that need to be identified generally does not change during the operation of the monitoring system, so the safety monitoring system of the power operation scene needs to monitor a target of a fixed scale during operation. If the sizes of these fixed-scale targets are close, the scale distribution of each target analyzed by the target scale analysis module will be consistent, which means that the accuracy of the original multi-head target detection network model can be achieved by using a single-head target detection network model of the corresponding scale. Therefore, the network model replacement module can select the corresponding single-head target detection network model to replace the original multi-head target detection network model for subsequent formal target detection, thereby requiring less computing resources and time, and faster detection speed and detection frame rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a main flow chart of a method for real-time safety monitoring of electric power operations according to an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of module connections of a target real-time safety monitoring system for electric power operations according to an embodiment of the present invention;

[0024] Figure 3 for Figure 2 The YOLOv7 network architecture diagram used in index 3;

[0025] Figure 4 for Figure 3 Schematic diagram of the Conv2D_BN_SiLU module structure;

[0026] Figure 5 for Figure 3 Schematic diagram of the Multi_Concat_Block module structure;

[0027] Figure 6 for Figure 3 Schematic diagram of the Transition_Block module structure in;

[0028] Figure 7This is a large-scale single-head target detection model architecture diagram according to an embodiment of the present invention;

[0029] Figure 8 This is a diagram of the architecture of a mesoscale single-head target detection model according to an embodiment of the present invention;

[0030] Fig. 9 This is a diagram of the architecture of a small-scale single-head target detection model according to an embodiment of the present invention;

[0031] Fig.10 The present invention is a schematic diagram of the structure of a target real-time safety monitoring terminal for electric power operations according to an embodiment of the present invention.

[0032] Description of labels:

[0033] 1. A target real-time safety monitoring terminal for power operations; 2. Memory; 3. Processor. DETAILED DESCRIPTION

[0034] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0035] Please refer to Figure 1 , a method for real-time safety monitoring of electric power operations, comprising the steps of:

[0036] S1. Read and load the multi-head target detection network model from a local file or an online file through the initialization module;

[0037] S2, performing initial target detection on the real-time video data stream based on the loaded multi-head target detection network model through the target detection module to obtain an initial detection target;

[0038] S3, calculating the scale distribution of the initial detection target through the target scale analysis module, if the scale distribution of the target is consistent, proceeding to step S4;

[0039] S4, selecting a corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target through a network model replacement module;

[0040] S5. Enter formal target detection.

[0041] From the above description, it can be seen that the beneficial effect of the present invention is that the safety monitoring of real-time targets in the power operation scene is completed through four modules: initialization module, target detection module, target scale analysis module and network model replacement module. The initialization module first loads the multi-head target detection network model so that the target detection model first uses the multi-head target detection network model to detect the target in the real-time video in the initial stage, and then uses the target scale analysis module to perform scale distribution analysis on the target detected in the initial stage. Since the position of the camera shooting the real-time video is generally fixed, and no zoom or steering operation is performed during the operation of the safety monitoring system, the target object type that the camera needs to identify is The scale of the targets analyzed by the target scale analysis module will be consistent, which means that the single-head target detection network model of the corresponding scale can achieve the accuracy of the original multi-head target detection network model. Therefore, the network model replacement module can select the corresponding single-head target detection network model to replace the original multi-head target detection network model for subsequent formal target detection, thereby requiring less computing resources and time, and faster detection speed and detection frame rate.

[0042] Furthermore, the step S2 further includes:

[0043] After each initial target detection, the position frame coordinate information of the initial detected target is transmitted to the target scale analysis module.

[0044] From the above description, it can be seen that for the target detected by the target detection model from each real-time video data stream, its position frame coordinate information is also transmitted to the target scale analysis module in real time, so that the target scale analysis module can subsequently analyze the scale distribution according to the position coordinate information of each target.

[0045] Furthermore, the steps S2 and S3 further include the following steps:

[0046] S23, the target scale analysis module receives and saves the position frame coordinate information, and counts the initial detection targets. If the count does not reach the threshold, it is determined that the current operating state of the system is a preheating state, and returns to step S2 to continue using the multi-head target detection network model to perform the initial target detection on the next segment of the real-time video data stream;

[0047] Otherwise go to step S3.

[0048] From the above description, it can be seen that after each target detection by the target detection module, the target scale analysis module will judge the operating status of the system based on the number of targets received. When the number of targets does not reach the threshold, the system will return to the target detection module to continue to perform target detection based on the loaded multi-head target detection network model and the real-time video data stream until the number of targets received by the target scale analysis module reaches the threshold. The purpose of this is to collect as many target samples as possible before performing target scale analysis, so that the scale distribution of targets detected in a current shorter initial time period can be used to simulate the scale distribution of targets that will be detected in a longer time period in the future.

[0049] Furthermore, the step S3 is specifically as follows:

[0050] S31, the target scale analysis module switches the operating state of the system from the preheating state to the detection state, and classifies the scale of each target through a clustering algorithm according to the acquired position frame coordinate information of each of the initial detection targets, and calculates the proportional relationship between the length and width of each target and the length and width of the video;

[0051] S32, if the proportions are within the preset consistent range, it indicates that the scale distribution of the target is consistent, and the classification result is sent to the network model replacement module and the process goes to step S4;

[0052] Otherwise, it indicates that the target size in the current scene has multiple different scales compared to the video size, then the multi-head target detection network model continues to be used and the process goes to step S5.

[0053] Furthermore, the classification results are obtained according to preset size standards, including a large-size target range, a medium-size target range, and a small-size target range;

[0054] The single-head target detection network model includes a large-scale single-head target detection network model, a medium-scale single-head target detection network model or a small-scale single-head target detection network model.

[0055] Furthermore, the step S4 is specifically as follows:

[0056] The network model replacement module analyzes the classification result, and when the classification result belongs to the large-size target range, replaces the multi-head target detection network model with the large-scale single-head target detection network model, wherein the large-scale single-head target detection network model is obtained by deleting the part of the multi-head target detection network model responsible for medium-scale target detection and small-scale target detection and retaining the part for large-scale target detection;

[0057] When the classification result belongs to the medium-sized target range, the multi-head target detection network model is replaced by the medium-scale single-head target detection network model, wherein the medium-scale single-head target detection network model is obtained by deleting the parts of the multi-head target detection network model responsible for large-scale target detection and small-scale target detection and retaining the part for medium-scale target detection;

[0058] When the classification result belongs to the small-size target range, the small-scale single-head target detection network model is used to replace the multi-head target detection network model. The small-scale single-head target detection network model is obtained by deleting the part of the multi-head target detection network model responsible for large-scale target detection and medium-scale target detection and retaining the part for small-scale target detection.

[0059] From the above description, it can be seen that the main content of the target scale analysis module is the distribution of the scale range of each target obtained, that is, according to the position frame coordinate information of each target, clustering calculation is performed to obtain the proportional relationship between the length and width of the target and the length and width of the video. At the same time, the clustering algorithm is also used to classify the scale distribution of each target, which is divided into a large-size target range, a medium-size target range and a small-size target range; if the proportional relationships are not within the preset consistent range, it can be explained that the target size in the current scene has multiple different scales compared to the video size. There may be both large-scale targets and medium-scale and / or small-scale targets. The system cannot use a single-head target detection network model to detect all targets, so the system will continue to Continue to use the previously loaded multi-head target detection network model for target detection, end the warm-up state, switch to the detection state to start formal detection; if the proportion relationships are within the preset consistent range, it means that the targets in the current scene are all large-scale, medium-scale or small-scale targets, that is, there may be only one classification result, so you can directly select the corresponding large-scale single-head target detection network model, medium-scale single-head target detection network model or small-scale single-head target detection network model according to the classification result to replace the previously loaded multi-head target detection network model for target detection, end the warm-up state, switch to the detection state to start formal detection, and continue target detection through a loop in the subsequent detection state until the system ends or there is no real-time video data stream input.

[0060] Furthermore, the real-time video data stream is obtained by real-time shooting by a fixed camera.

[0061] As can be seen from the above description, the real-time video data stream is captured by a fixed camera to ensure the consistency of the video size.

[0062] Please refer to Figures 2 to 9 , a target real-time safety monitoring system for power operations, comprising:

[0063] Initialization module: read and load the multi-head target detection network model from local files or online files;

[0064] Target detection module: performs initial target detection on the real-time video data stream based on the loaded multi-head target detection network model to obtain the initial detection target;

[0065] Target scale analysis module: calculates the scale distribution of the initial detection target, and if the scale distribution of the target is consistent, calls the target detection module to perform target detection;

[0066] Network model replacement module: selects the corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target, and enters the formal target detection.

[0067] From the above description, it can be seen that the beneficial effects of the present invention are: based on the same technical concept, in conjunction with the above-mentioned real-time safety monitoring method for electric power operations, a real-time safety monitoring system for electric power operations is provided, which completes the safety monitoring of real-time targets in electric power operation scenarios through four modules: an initialization module, a target detection module, a target scale analysis module and a network model replacement module. The initialization module first loads a multi-head target detection network model so that the target detection model first uses the multi-head target detection network model to perform target detection on the real-time video in the initial stage, and then uses the target scale analysis module to perform scale distribution analysis on the targets detected in the initial stage. Since the position of the camera that shoots the real-time video is generally fixed, and during the operation of the safety monitoring system, No zoom or steering operation is performed during the operation, and the target object category that the camera needs to identify generally does not change during the operation of the monitoring system. Therefore, the safety monitoring system of the power operation scene needs to monitor a target of a fixed scale during operation. If the sizes of these fixed-scale targets are close, the scale distribution of each target analyzed by the target scale analysis module will be consistent, which means that the single-head target detection network model of the corresponding scale can achieve the accuracy of the original multi-head target detection network model. Therefore, the network model replacement module can select the corresponding single-head target detection network model to replace the original multi-head target detection network model for subsequent formal target detection, thereby requiring less computing resources and time, and faster detection speed and detection frame rate.

[0068] Please refer to Fig.10 A terminal for real-time safety monitoring of electric power operations comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for real-time safety monitoring of electric power operations when executing the computer program.

[0069] From the above description, it can be seen that the beneficial effects of the present invention are: based on the same technical concept, in conjunction with the above-mentioned real-time safety monitoring method for electric power operation targets, a real-time safety monitoring terminal for electric power operation targets is provided, and the safety monitoring of real-time targets in the electric power operation scenario is completed through four modules: initialization module, target detection module, target scale analysis module and network model replacement module. The initialization module first loads the multi-head target detection network model so that the target detection model first uses the multi-head target detection network model to perform target detection on the real-time video in the initial stage, and then uses the target scale analysis module to perform scale distribution analysis on the targets detected in the initial stage. Since the position of the camera for shooting real-time video is generally fixed, and during the operation of the safety monitoring system, the real-time target detection network model is used to detect the real-time video. No zoom or steering operation is performed during the operation, and the target object category that the camera needs to identify generally does not change during the operation of the monitoring system. Therefore, the safety monitoring system of the power operation scene needs to monitor a target of a fixed scale during operation. If the sizes of these fixed-scale targets are close, the scale distribution of each target analyzed by the target scale analysis module will be consistent, which means that the single-head target detection network model of the corresponding scale can achieve the accuracy of the original multi-head target detection network model. Therefore, the network model replacement module can select the corresponding single-head target detection network model to replace the original multi-head target detection network model for subsequent formal target detection, thereby requiring less computing resources and time, and faster detection speed and detection frame rate.

[0070] The present invention provides a method, system and terminal for real-time safety monitoring of electric power operations, which are applied to real-time safety monitoring in electric power operation scenarios. Figures 1 to 10 And specific embodiments are described.

[0071] Please refer to Figure 1 , Embodiment 1 of the present invention is:

[0072] A method for real-time safety monitoring of power operations, such as Figure 1 As shown, the steps include:

[0073] S0, the default system operating state is preheating state, ensuring smooth system initialization.

[0074] S1. Read and load the multi-head target detection network model from local files or online files through the initialization module.

[0075] S2. Perform initial target detection on the real-time video data stream through the target detection module based on the loaded multi-head target detection network model to obtain the initial detection target; after each initial target detection, the position frame coordinate information of each initial detection target is passed to the target scale analysis module, that is, the position frame coordinate information of the target detected by the target detection model from each segment of the real-time video data stream is also passed to the target scale analysis module in real time, so that the target scale analysis module can subsequently analyze the scale distribution according to the position coordinate information of each target; wherein, the real-time video data stream is obtained by real-time shooting by a fixed camera to ensure the consistency of the video size.

[0076] S23, the target scale analysis module receives and saves the position box coordinate information, and counts the initial detection targets. If the count does not reach the threshold, the current operating state of the system is judged to be the preheating state, and returns to step S2 to continue using the multi-head target detection network model to perform initial target detection on the next real-time video data stream; otherwise, enter step S3.

[0077] That is, after each target detection is completed by the target detection module, the target scale analysis module will judge the system operation status according to the number of targets received. When the number of targets does not reach the threshold, the system will return to the target detection module to continue to perform target detection according to the loaded multi-head target detection network model and the real-time video data stream until the number of targets received by the target scale analysis module reaches the threshold. The purpose of this is to collect as many target samples as possible before performing target scale analysis, so that the scale distribution of targets detected in a shorter initial time period can be used to simulate the scale distribution of targets that will be detected in a longer time period in the future.

[0078] S3, calculate the scale distribution of the initial detection target through the target scale analysis module. If the scale distribution of the target is consistent, proceed to step S4, which is specifically:

[0079] S31, the target scale analysis module switches the system operation state from the preheating state to the detection state, and classifies the scale of each target through a clustering algorithm according to the acquired position frame coordinate information of each initial detection target, and calculates the proportional relationship between the length and width of each target and the length and width of the video;

[0080] S32, if the proportions are within the preset consistent range, it indicates that the scale distribution of the target is consistent, and the classification result is sent to the network model replacement module and enters step S4;

[0081] Otherwise, it indicates that the target size in the current scene has multiple different scales compared to the video size, then the multi-head target detection network model continues to be used and the process goes to step S5.

[0082] Among them, the classification results are obtained according to preset size standards, including large-size target range, medium-size target range and small-size target range; the single-head target detection network model includes a large-scale single-head target detection network model, a medium-scale single-head target detection network model or a small-scale single-head target detection network model.

[0083] S4. The network model replacement module selects the corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target, specifically:

[0084] The network model replacement module analyzes the classification results. When the classification results belong to the large-size target range, the large-scale single-head target detection network model is used to replace the multi-head target detection network model. The large-scale single-head target detection network model is obtained by deleting the part responsible for medium-scale target detection and small-scale target detection in the multi-head target detection network model and retaining the part responsible for large-scale target detection.

[0085] When the classification result belongs to the medium-sized target range, the medium-scale single-head target detection network model is used to replace the multi-head target detection network model. The medium-scale single-head target detection network model is obtained by deleting the parts responsible for large-scale target detection and small-scale target detection in the multi-head target detection network model and retaining the part responsible for medium-scale target detection.

[0086] When the classification result belongs to the small-size target range, the small-scale single-head target detection network model is used to replace the multi-head target detection network model. The small-scale single-head target detection network model is obtained by deleting the parts responsible for large-scale target detection and medium-scale target detection in the multi-head target detection network model and retaining the part responsible for small-scale target detection.

[0087] S5. Enter formal target detection.

[0088] That is, in this embodiment, four modules are used: initialization module, target detection module, target scale analysis module and network model replacement module to complete the safety monitoring of real-time targets in the power operation scenario, and a logical judgment is made based on the detected targets whether there are unsafe operating behaviors and potential risks. In this embodiment, the multi-head target detection network model is first loaded by the initialization module, so that the target detection model first uses the multi-head target detection network model to detect the target of the real-time video in the initial stage, and then the target scale analysis module performs scale distribution analysis on the target detected in the initial stage. Since the position of the camera shooting the real-time video is generally fixed, and no zoom or steering operation is performed during the operation of the safety monitoring system, the target object category that the camera needs to identify will generally not change during the operation of the monitoring system, so that the safety monitoring system of the power operation scene needs to monitor a fixed-scale target during operation. If the sizes of these fixed-scale targets are close, the scale distribution of each target analyzed by the target scale analysis module will be consistent, which means that the accuracy of the original multi-head target detection network model can be achieved by using a single-head target detection network model of the corresponding scale. Therefore, the network model replacement module can select the corresponding single-head target detection network model to replace the original multi-head target detection network model for subsequent formal target detection, so that the safety monitoring of the power operation scene does not need to perform multi-scale target detection in all stages of operation, thereby requiring less computing resources and time, and faster detection speed and detection frame rate.

[0089] Among them, the main content of the target scale analysis module is the distribution of the scale range of each target obtained, that is, according to the position frame coordinate information of each target, clustering calculation is performed to obtain the proportional relationship between the length and width of the target and the length and width of the video. At the same time, the clustering algorithm is also used to classify the scale distribution of each target, which is divided into a large-size target range, a medium-size target range and a small-size target range; if the proportional relationships are not within the preset consistent range, it can be explained that the target size in the current scene has multiple different scales compared to the video size. There may be both large-scale targets and medium-scale and / or small-scale targets. The system cannot use a single-head target detection network model to detect all targets, so the system will continue to use Use the previously loaded multi-head target detection network model to perform target detection, end the preheating state, switch to the detection state to start formal detection; if the proportional relationships are within the preset consistent range, it means that the targets in the current scene are all large-scale, medium-scale or small-scale targets, that is, there may be only one classification result, so you can directly select the corresponding large-scale single-head target detection network model, medium-scale single-head target detection network model or small-scale single-head target detection network model according to the classification result to replace the previously loaded multi-head target detection network model for target detection, end the preheating state, switch to the detection state to start formal detection, and continue target detection through a loop in the subsequent detection state until the system ends or there is no real-time video data stream input.

[0090] Please refer to Figures 2 to 9 , Embodiment 2 of the present invention is:

[0091] A target real-time safety monitoring system for power operations, such as Figure 2 As shown, including:

[0092] Initialization module: read and load the multi-head target detection network model from local files or online files.

[0093] Target detection module: Based on the loaded multi-head target detection network model, initial target detection is performed on the real-time video data stream to obtain the initial detection target.

[0094] Target scale analysis module: Calculates the scale distribution of the initial detection target. If the scale distribution of the target is consistent, the target detection module is called to perform target detection.

[0095] Network model replacement module: According to the scale distribution of the target, the corresponding single-head target detection network model is selected to replace the multi-head target detection network model and enter the formal target detection.

[0096] That is, in this embodiment, based on the same technical concept, in conjunction with a real-time safety monitoring method for electric power operation targets in the above-mentioned embodiment 1, a real-time safety monitoring system for electric power operation targets is provided, which completes the safety monitoring of real-time targets in the electric power operation scenario through four modules: an initialization module, a target detection module, a target scale analysis module and a network model replacement module. The initialization module first loads a multi-head target detection network model so that the target detection model first uses the multi-head target detection network model to perform target detection on the real-time video in the initial stage, and then uses the target scale analysis module to perform scale distribution analysis on the targets detected in the initial stage. Since the position of the camera that shoots the real-time video is generally fixed and is not performed during the operation of the safety monitoring system, When zooming or turning, the target object category that the camera needs to identify generally does not change during the operation of the monitoring system, so that the safety monitoring system of the power operation scene needs to monitor a target of a fixed scale during operation. If the sizes of these fixed-scale targets are close, the scale distribution of each target analyzed by the target scale analysis module will be consistent, which means that the accuracy of the original multi-head target detection network model can be achieved by using a single-head target detection network model of the corresponding scale. Therefore, the network model replacement module can select the corresponding single-head target detection network model to replace the original multi-head target detection network model for subsequent formal target detection, thereby requiring less computing resources and time, and faster detection speed and detection frame rate.

[0097] In this embodiment, the initialization module includes a loading model submodule and a multi-head target detection network model, so it is possible to load the multi-head target detection network model in deep learning.

[0098] The model loading submodule can select different network model loading tools or libraries according to the different platforms and programming languages ​​used to load the deep network model. For example, in Python, the PyTorch library can be used to load the deep network model. In C++, the Opencv library can be used to load the deep network model. The multi-head target detection network model is a network model trained by deep learning back propagation technology. As a target detection technology, multi-head target detection is not limited to a specific multi-head target detection network model in this embodiment. As long as the target detection network architecture using the multi-head target detection method can be used.

[0099] In this embodiment, the currently used open source multi-head real-time target detection model architecture YOLOv7 is taken as an example. Figure 2The multi-head target detection network model used in the real-time safety monitoring system for power operation targets index 3 is used to improve the frame rate and rate of real-time target detection in the safety detection system for power operation scenarios. At the same time, the large-scale single-head target detection network model, medium-scale single-head target detection network model, and small-scale single-head target detection network model used in subsequent systems are also generated based on the YOLOv7 network architecture.

[0100] Figure 3 Shown Figure 2 The YOLOv7 network architecture used in Index 3 in the figure is mainly composed of three parts, from front to back: feature extraction part, feature enhancement part and output part. The feature extraction part is used to receive the image as input and output a vector, which expresses the relevant features of the image; the feature enhancement part receives the output of the feature extraction part as input, further extracts and enhances the feature vector of the image, so that the vector used to express the image features is easier to correctly identify in the subsequent stages; the output part receives the output of the feature extraction part and the feature enhancement part as input at the same time, and outputs the detection results based on multiple output heads. Each output head needs to perform multiple multi-scale feature fusions before outputting the detection results.

[0101] A method for real-time safety monitoring of electric power operation targets in this embodiment mainly modifies the "output part" of the multi-head target detection network model to reduce unnecessary multi-scale feature fusion to improve the detection speed.

[0102] in, Figure 3 The Conv2D_BN_SiLU module in the image is mainly used for image feature extraction. The structure is as follows Figure 4 As shown in the figure, the Conv2D_BN_SiLU module consists of three network layers and is calculated in sequence. The input data first passes through a 2D convolutional network layer (Conv2D), the output of which is passed to a batch normalization network layer (BN), and finally to the SiLU activation function layer (SiLU) to output the result.

[0103] in, Figure 3 The Multi_Concat_Block module in the example is composed of multiple convolutional layers stacked after calculation, the purpose is to extract image features, and its structure is as follows Figure 5 As shown, there are four paths. The four paths pass through different numbers of Conv2D_BN_SiLU modules from left to right to extract features of different scales. Finally, the four feature layers generated by the four paths are stacked (Concat) and output.

[0104] in, Figure 3The Transition_Block module in the image is mainly used to downsample the image. Its structure is as follows Figure 6 As shown, the left branch is a combination of a maximum pooling layer (MaxPool_2D) and a Conv2D_BN_SiLU module, and the right branch is two sequentially connected Conv2D_BN_SiLU modules. The results of the two branches are stacked and output.

[0105] In this embodiment, the target detection module includes four parts: a video stream processing submodule, a target detection submodule, a system operation status judgment submodule and an alarm system submodule. Figure 2 Index 4, index 5, index 6, and index 7 in .

[0106] Among them, the purpose of the video stream processing submodule is to obtain the streaming media information of the fixed camera arranged at the power operation site, and process the streaming media so that the real-time video data stream can meet the requirements of target detection. The specific processing is as follows: First, the video stream processing submodule needs to obtain the streaming media address of the fixed camera, and then extract the frames of the video streaming media according to the needs of use, and convert the video streaming media into pictures to obtain a single picture in a loop; secondly, after obtaining a picture, the picture resolution needs to be adjusted. Assuming that the network model used by the system requires the size of the input picture to be 640*640 (length*width), the resolution of the original picture needs to be adjusted to 640*640 pixels; finally, after the picture resolution is adjusted, the value range of each pixel point in the picture needs to be mapped from 0-255 to 0-1 as a whole, so as to facilitate the subsequent detection of targets.

[0107] The purpose of the target detection submodule is to output the target detection result based on the loaded network model (which can be a multi-head target detection network model or a single-head target detection network model) and the input image (the size is required to be 640*640 pixels). Specifically, the forward propagation method of the network model is used, and the processed image is used as the input parameter to perform forward propagation calculation to obtain the target detection result. The result of target detection may contain data of multiple targets. The data of each target contains three parts: one part of the data is the target category class, which is represented by the index number of the target category (such as 0, 1, 2...); one part of the data is the position of the target in the picture. In order to facilitate calculation and expression, the position of the target in the picture is replaced by the position of the smallest rectangular box that can contain the target. The position of the rectangular box is expressed by an array xywh, where x represents the horizontal coordinate of the center point or the upper left corner of the rectangular box, y represents the vertical coordinate of the center point or the upper left corner of the rectangular box, w represents the width of the rectangular box, and h represents the height of the rectangular box; the last part of the data is the confidence of the category to which the target in the rectangular box belongs, which is represented by a decimal between 0 and 1, 1 represents complete credibility, and 0 represents very low credibility. Each detected target is expressed by an array, which contains the target category, target confidence, and target rectangular box coordinates. The target detection submodule may generate multiple arrays representing target information when it runs once. These target information arrays will be selectively input to the target scale analysis module according to the system status.

[0108] The function of the system operation status judgment submodule is to obtain the current system operation status and execute different processes according to the obtained status, namely, the two operation states of the system: preheating state and detection state. When the system starts running, the default state is the preheating state; when the system state is the preheating state, the target detection module will continuously pass the obtained target information array to the target scale analysis module for further processing until the number of target information arrays received by the target scale analysis module reaches the threshold. At this time, the target detection module no longer needs to continue to send the detected target information array to the target scale analysis module, but passes it to the alarm subsystem. The alarm subsystem can make logical judgments based on the obtained target information to decide whether an alarm is needed.

[0109] The alarm system submodule is used to receive the target information array output by the target detection, and make logical judgments based on the target information to determine whether there are unsafe operating behaviors or potential risks in the power scenario. If so, an alarm is pushed. The form of the alarm can include various types, such as pop-up push alarms, SMS push alarms, phone push alarms, etc. The alarm system submodule will only run when the system is in the detection state. When the system is in the preheating state, the alarm system submodule will not run.

[0110] The target scale analysis module is used to receive the target information detected in the system preheating state and output the length and width size distribution of the target information. At the same time, this module is also responsible for switching the system state. It mainly includes: target information storage submodule, judgment submodule whether the number of saved targets meets the threshold, modification submodule of the system state, target scale analysis submodule and target scale distribution consistency judgment submodule, which are respectively Figure 2 Index 8, index 9, index 10, index 11, and index 12 in .

[0111] Among them, the target information saving submodule is used to save the target data information output by the target detection module. This submodule saves the detected target data information in the form of a list.

[0112] The submodule for judging whether the number of saved targets meets the threshold value is used to judge whether the number of saved target data has reached the threshold value. If the threshold value is reached, the system will switch from the preheating state to the detection state; if the number does not meet the threshold value, the system will continue to maintain the preheating state and return to the target detection module until the number of saved targets reaches the threshold value. For common power safety and supervision scenarios, the number threshold can be set to 100 or 1000 according to the frequency of target occurrence. 100 is suitable for situations where the target appears less frequently in the current scenario, and 1000 is suitable for situations where the target appears more frequently in the current scenario. The purpose of setting the threshold is to allow the targets detected by the system within a certain period of time to cover all possible targets in the scenario. Therefore, the threshold can also be adjusted according to actual usage.

[0113] The function of the Modify System Status submodule is to modify the system status so that the system status can be switched between "preheating" and "detection". This submodule is implemented through a system Boolean variable. When the system is started and in the preheating state, the value of the Boolean variable is true. After running for a period of time and the number of saved targets reaches the threshold, the submodule changes the value of the Boolean variable to false. At this time, the system running state changes to the detection state, thereby completing the switching of the system state.

[0114] The target scale analysis submodule is used to analyze the information of the saved targets and output the range of the length and width distribution information of the saved targets. The function of this submodule is mainly based on the clustering algorithm. It is necessary to extract the information of each target. Only the length and width information of the rectangular box of each target is retained, and other information is deleted; then, the clustering algorithm is used for the length and width information of these rectangular boxes. For example, the number of clusters is set to 3, and the calculation method of the monomer distance is set to the intersection over union (IOU). After the clustering algorithm is completed, 3 value pairs representing the length and width of the rectangular box will be generated, denoted as anchor_3, which contains 3 length and width value pairs [w1,h1,w2,h2,w3,h3]; then, the intersection over union of each target rectangular box and the generated three rectangular boxes is calculated respectively, and the maximum intersection over union value of the intersection over union of each target rectangular box and the three rectangular boxes is taken, and the average intersection over union value is calculated, denoted as average_IOU. The target size analysis submodule outputs the anchor and average_IOU data to the next step.

[0115] The target scale distribution consistency judgment submodule receives the output of the target size analysis submodule as input and judges whether the input meets the conditions for process control. This submodule performs logical judgment through the value anchor_3 of average_IOU. The judgment steps are as follows:

[0116] First, determine whether average_IOU is 1. If it is 1, it means that 100% of the length and width of the target are contained in the three rectangular boxes in anchor_3, and continue to judge; secondly, take all the w and h values ​​in [w1,h1,w2,h2,w3,h3] and compare them respectively to obtain the value range of w and the value range of h. If the value range of w and the value range of h completely belong to one of the large, medium and small target scale standard ranges, it means that the target size is all large-scale targets, or all medium-scale targets, or all small-scale targets, indicating that the scale distribution of the targets in the current scene is uniform, and there are only targets of one scale. The multi-head target detection network model can be replaced with a single-head target detection network model without causing a loss in detection accuracy, and the analysis results are passed to the network model replacement module as the basis for selecting the single-head target detection network model.

[0117] The numerical standards for large, medium and small scales are based on the statistics of objects in the large public image dataset COCO. For example, the standard range for large-scale objects is 140-238 in length and 264-542 in width; the standard range for medium-scale objects is 86-180 in length and 68-152 in width; the standard range for small-scale objects is 19-38 in length and 27-94 in width. The above numerical units are all pixel values, and are pixel values ​​under 640*640 images. When the image size is not 640*640, the image needs to be scaled for comparison.

[0118] When average_IOU is not 1, or the value range of the length and width of the rectangular box in anchor_3 does not completely belong to one of the large, medium, or small scale standards, it indicates that the scale distribution of the targets in the current scene is uneven, and there may be both large-scale targets and medium-scale targets or small-scale targets. If the multi-head target detection network model is replaced with a single-head target detection network model, the detection accuracy will be reduced. Therefore, the system will still use the original multi-head target detection network model and start the formal detection phase.

[0119] The network model replacement module receives the output of the target scale analysis module as input, selects the corresponding single-head target detection network model according to the input content, and replaces the original multi-head target detection network model. The three single-head target detection network models are used to detect large-scale targets, medium-scale targets, and small-scale targets, respectively. Among them, the large-scale single-head target detection network model is obtained by deleting the part responsible for "medium-scale target detection" in the output part of the multi-head target detection network model architecture, and deleting the part responsible for "small-scale target detection"; the medium-scale single-head target detection network model is obtained by deleting the part responsible for "large-scale target detection" in the output part of the multi-head target detection network model architecture, and deleting the part responsible for "small-scale target detection"; the small-scale single-head target detection network model is obtained by deleting the part responsible for "large-scale target detection" in the output part of the multi-head target detection network model architecture, and deleting the part responsible for "medium-scale target detection".

[0120] The positions of the three single-head target detection network models in the system are: Figure 2 The specific model architectures of the three single-head target detection network models are shown in Index 14, Index 16 and Index 17 in Figure 6 (Large-scale single-head target detection model), Figure 7 (Mesoscale Single-Head Object Detection Model) and Figure 8 (Small-scale single-head object detection model).

[0121] in, Figure 6 , Figure 7 and Figure 8The UpSampling2D module in the image is used to upsample the input content and expand the size of the input content. The RepConv module is a convolutional network layer that deforms the input features to facilitate subsequent conversion into the final output results.

[0122] Please refer to Fig.10 , Embodiment 2 of the present invention is:

[0123] A terminal 1 for real-time safety monitoring of a target of an electric power operation comprises a memory 2, a processor 3 and a computer program stored in the memory 2 and executable on the processor 3. When executing the computer program, the processor 3 implements the steps in a method for real-time safety monitoring of a target of an electric power operation in the first embodiment described above.

[0124] In summary, the present invention provides a method, system and terminal for real-time safety monitoring of targets in power operations. In the system preheating stage, a multi-head target detection model is used to identify the scale of the target to be detected in the current scene, and the scales of the multiple targets detected by preheating are clustered and analyzed to determine whether the scales of the targets to be detected in the current scene are consistent; after the system preheating stage is over, if the scales of the targets to be detected in the current scene are consistent, the target scale is analyzed to be large, medium or small, and the original multi-head target detection network model is replaced with a single-head target detection model of the corresponding scale, and the system is switched to the "detection" stage to start formal detection. There is no need to adjust the hardware equipment. It only needs to count and analyze the scales of the targets to be detected and select the corresponding single-head target detection network model to effectively improve the frame rate of the system's real-time target detection, and maintain the detection accuracy without decreasing, thereby improving the real-time detection frame rate of target detection, which is conducive to reducing unsafe operations and potential risks in power operation scenes.

[0125] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for real-time safety monitoring of electric power operations, characterized in that: Includes steps: S1. Read and load the multi-head target detection network model from a local file or an online file through the initialization module; S2, performing initial target detection on the real-time video data stream based on the loaded multi-head target detection network model through the target detection module to obtain an initial detection target; S3, calculating the scale distribution of the initial detection target through the target scale analysis module, if the scale distribution of the target is consistent, proceeding to step S4; S4, selecting a corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target through a network model replacement module; S5, enter the formal target detection; The step S2 further comprises: After each initial target detection, the position frame coordinate information of the initial detected target is transmitted to the target scale analysis module; The steps S2 and S3 also include the following steps: S23, the target scale analysis module receives and saves the position frame coordinate information, and counts the initial detection targets. If the count does not reach the threshold, it is determined that the current operating state of the system is a preheating state, and returns to step S2 to continue using the multi-head target detection network model to perform the initial target detection on the next segment of the real-time video data stream; Otherwise, proceed to step S3; The step S3 is specifically as follows: S31, the target scale analysis module switches the operating state of the system from the preheating state to the detection state, and classifies the scale of each target through a clustering algorithm according to the acquired position frame coordinate information of each of the initial detection targets, and calculates the proportional relationship between the length and width of each target and the length and width of the video; S32, if the proportions are within the preset consistent range, it indicates that the scale distribution of the target is consistent, and the classification result is sent to the network model replacement module and the process goes to step S4; Otherwise, it indicates that the target size in the current scene has multiple different scales compared to the video size, then the multi-head target detection network model continues to be used and the process goes to step S5.

2. A method for real-time safety monitoring of electric power operations according to claim 1, characterized in that: The classification results are obtained according to preset size standards, including a large-size target range, a medium-size target range, and a small-size target range; The single-head target detection network model includes a large-scale single-head target detection network model, a medium-scale single-head target detection network model or a small-scale single-head target detection network model.

3. A method for real-time safety monitoring of electric power operations according to claim 2, characterized in that: The step S4 is specifically as follows: The network model replacement module analyzes the classification result, and when the classification result belongs to the large-size target range, replaces the multi-head target detection network model with the large-scale single-head target detection network model, wherein the large-scale single-head target detection network model is obtained by deleting the part of the multi-head target detection network model responsible for medium-scale target detection and small-scale target detection and retaining the part for large-scale target detection; When the classification result belongs to the medium-sized target range, the multi-head target detection network model is replaced by the medium-scale single-head target detection network model, wherein the medium-scale single-head target detection network model is obtained by deleting the parts of the multi-head target detection network model responsible for large-scale target detection and small-scale target detection and retaining the part for medium-scale target detection; When the classification result belongs to the small-size target range, the small-scale single-head target detection network model is used to replace the multi-head target detection network model. The small-scale single-head target detection network model is obtained by deleting the part of the multi-head target detection network model responsible for large-scale target detection and medium-scale target detection and retaining the part for small-scale target detection.

4. A method for real-time safety monitoring of electric power operations according to claim 1, characterized in that: The real-time video data stream is obtained by real-time shooting by a fixed camera.

5. A target real-time safety monitoring system for power operations, characterized in that: include: Initialization module: read and load the multi-head target detection network model from local files or online files; Target detection module: performs initial target detection on the real-time video data stream based on the loaded multi-head target detection network model to obtain the initial detection target. After each initial target detection, the position frame coordinate information of each initial detection target is transmitted to the target scale analysis module; Target scale analysis module: receiving and saving the position frame coordinate information, and counting the initial detection targets. If the count does not reach the threshold, the current running state of the system is judged to be the preheating state, and at the same time, the target detection module is called back to use the multi-head target detection network model to perform the initial target detection on the next segment of the real-time video data stream. Otherwise, the scale distribution of the initial detection target is calculated. If the scale distribution of the target is consistent, the target detection module is called to perform target detection. Network model replacement module: selects the corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target, and enters the formal target detection; The target scale analysis module calculates the scale distribution of the initial detection target. If the scale distribution of the target is consistent, the target detection module is called to perform target detection, specifically: Switching the operating state of the system from the preheating state to the detection state, and classifying the scale of each target through a clustering algorithm according to the acquired position frame coordinate information of each of the initial detection targets, and calculating the proportional relationship between the length and width of each target and the length and width of the video; If the proportions are within the preset consistent range, it indicates that the scale distribution of the target is consistent, and the classification result is sent to the network model replacement module and the network model replacement module is called to select the corresponding single-head target detection network model to replace the multi-head target detection network model according to the scale distribution of the target; Otherwise, it indicates that the target size in the current scene has multiple different scales compared to the video size, then the multi-head target detection network model continues to be used, and the network model replacement module is called to enter the formal target detection.

6. A target real-time safety monitoring terminal for electric power operation, characterized in that: It comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a method for real-time safety monitoring of electric power operations as described in any one of claims 1 to 4 when executing the computer program.

Images