Electric power high-altitude operation image processing method and system

Abstract

The invention provides an electric power high-altitude operation image processing method and system, and relates to the technical field of image processing, and the method comprises the steps: obtaining a plurality of operation image sets including an operator and wearing equipment, and constructing a plurality of operation image sequences; obtaining a plurality of wearing action safety levels; obtaining a plurality of wearing safety level sets, and respectively carrying out wearing safety stability analysis calculation to obtain a plurality of recognized wearing safety levels; and according to the plurality of identification wearing safety levels and the plurality of wearing action safety levels, calculating to obtain a real-time wearing equipment safety level, and taking the real-time wearing equipment safety level as a high-altitude operation monitoring image processing result. The technical problem that real-time safety feedback cannot be provided due to the fact that multi-angle and multi-moment continuous analysis is difficult to achieve in the prior art can be solved, and the technical effects of improving the safety monitoring level of electric power high-altitude operation and reducing the safety accident risk caused by improper wearing of equipment are achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and system for high-altitude power operations. Background Art

[0002] In the field of safety monitoring of power aerial work, existing technologies mainly rely on manual inspections and some traditional target detection algorithms, such as the YOLO series, Faster R-CNN, etc. These technologies have achieved remarkable results in many fields. However, for the detection of safety wear for power aerial work, existing algorithms have some limitations.

[0003] First, the results of manual judgment are easily affected by subjective factors and lack consistency, which may lead to safety issues being ignored. Second, existing target detection algorithms find it difficult to dynamically identify the wearing status of equipment, especially when the worn equipment undergoes unstable changes at consecutive moments. It is difficult to achieve continuous analysis of multiple angles and multiple moments, resulting in the inability to provide real-time safety feedback. Summary of the invention

[0004] The purpose of this application is to provide an image processing method and system for high-altitude power operations, so as to solve the technical problem in the prior art that it is difficult to achieve continuous analysis of multiple angles and multiple moments, resulting in the inability to provide real-time safety feedback.

[0005] In view of the above problems, the present application provides an image processing method and system for high-altitude power operations.

[0006] In a first aspect, the present application provides an image processing method for high-altitude electric power operations, which is implemented by an image processing system for high-altitude electric power operations, wherein the image processing method for high-altitude electric power operations comprises: in high-altitude electric power operations, performing image acquisition of operators at multiple angles through image acquisition equipment, obtaining multiple operation image sets including operators and wearable equipment, and constructing multiple operation image sequences; performing safety analysis of the operation actions of the wearable equipment in high-altitude operations on the multiple operation image sequences respectively, and obtaining multiple wearing action safety levels; performing separate wearing safety analysis of the wearable equipment on each operation image in the multiple operation image sets respectively, and obtaining multiple wearing safety level sets, and performing wearing safety stability analysis and calculation respectively, and obtaining multiple identification wearing safety levels; obtaining a real-time wearable equipment safety level based on the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude operation monitoring image processing result.

[0007] Furthermore, the safety analysis of the working actions of the worn equipment in the high-altitude work is performed on the multiple working image sequences respectively to obtain multiple wearing action safety levels, including: pre-training a wearing equipment action safety analyzer for identifying the safety level of wearing actions based on the slowfast network; extracting working images from the multiple working image sequences respectively according to a first extraction step length, and performing Haar wavelet transform down-sampling processing to obtain multiple fast working image sets; extracting working images from the multiple working image sequences respectively according to a second extraction step length to obtain multiple slow working image sets, and the second extraction step length is greater than the first extraction step length; and correspondingly combining the multiple fast working image sets and the multiple slow working image sets respectively, and inputting them into the wearing equipment action safety analyzer to obtain multiple wearing action safety levels.

[0008] Furthermore, in the high-altitude electric power operation, the images of the workers are captured at multiple angles by image acquisition equipment to obtain multiple work image sets including the workers and the equipment they wear, and multiple work image sequences are constructed, including: in the high-altitude electric power operation, the images of the workers are captured at multiple angles by image acquisition equipment, and the work images within the past P moments are counted to obtain multiple work image sets, where P is an integer greater than 1; the multiple work image sets are arranged according to the timestamps of the P moments to obtain multiple work image sequences.

[0009] Furthermore, based on the slowfast network, a wearable equipment motion safety analyzer for identifying the safety level of wearing actions is pre-trained, including: based on the image acquisition data of high-altitude operations of power grids in historical time, according to the first extraction step size, Haar wavelet transform downsampling and the second extraction step size, acquisition and processing are performed to obtain multiple sample fast operation image sets and multiple sample slow operation image sets, and according to the standard degree of the wearable equipment operation action, the sample wearing action safety level set is labeled to obtain; based on the slowfast network, the network structure of the wearable equipment motion safety analyzer is constructed; using the multiple sample fast operation image sets, the multiple sample slow operation image sets and the sample wearing action safety level set, the wearable equipment motion safety analyzer is trained until convergence to complete the training.

[0010] Furthermore, the wearing safety of wearable equipment is analyzed separately for each operation image in the multiple operation image sets to obtain multiple wearing safety level sets, including: collecting a sample operation picture set based on image acquisition data of high-altitude operations of power grids in historical time, and annotating to obtain a sample wearing safety level set according to the standard degree of wearing of wearable equipment in each sample operation image; using the sample operation picture set and the sample wearing safety level set, using a convolutional neural network, training a wearing safety analyzer; inputting each operation image in the multiple operation image sets into the wearing safety analyzer, and outputting to obtain multiple wearing safety level sets.

[0011] Furthermore, wearing safety stability analysis and calculation are performed respectively to obtain multiple identification wearing safety levels, including: respectively calculating the means of the multiple wearing safety level sets to obtain multiple average wearing safety levels; randomly selecting multiple first wearing safety levels from the first wearing safety level set within the multiple wearing safety level sets, respectively calculating the deviation amplitudes of the multiple first wearing safety levels and the first average safety wearing level and calculating the mean to obtain the first wearing safety level instability coefficient; using 1 minus the difference of the first wearing safety level instability coefficient, multiplying by the first average wearing safety level, to perform a correction calculation of the wearing safety level to obtain a first identification wearing safety level; continuing to perform correction calculations on the other multiple average wearing safety levels to obtain multiple first identification wearing safety levels.

[0012] Furthermore, the real-time wearable equipment safety level is calculated based on the multiple identification wear safety levels and the multiple wearing action safety levels, including: calculating the ratio of each identification wear safety level to the average of the multiple identification wear safety levels as multiple wear safety coefficients; using the multiple wear safety coefficients to respectively perform correction calculations on the products of the multiple wearing action safety levels to obtain multiple corrected wearing action safety levels; calculating the average of the multiple corrected wearing action safety levels to obtain the real-time wearable equipment safety level as the high-altitude work monitoring image processing result.

[0013] In a second aspect, the present application also provides an image processing system for high-altitude electric power operations, which is used to execute the image processing method for high-altitude electric power operations as described in the first aspect, wherein the image processing system for high-altitude electric power operations includes: an image acquisition module, which is used to perform image acquisition at multiple angles on the operating personnel through image acquisition equipment during high-altitude electric power operations, obtain multiple operating image sets including the operating personnel and the worn equipment, and construct multiple operating image sequences; an action safety analysis module, which is used to perform safety analysis on the operating actions of the worn equipment in high-altitude operations on the multiple operating image sequences, and obtain multiple wearing action safety levels; a separate wearing safety analysis module, which is used to perform separate wearing safety analysis on the wearing equipment for each operating image in the multiple operating image sets, obtain multiple wearing safety level sets, and perform wearing safety stability analysis calculations respectively, to obtain multiple identification wearing safety levels; a safety level acquisition module, which is used to calculate and obtain a real-time wearing equipment safety level based on the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude operation monitoring image processing result.

[0014] One or more technical solutions provided in this application have at least the following technical effects or advantages: in the high-altitude operation of electric power, the image acquisition device is used to acquire images of the operator at multiple angles, and multiple sets of operation images including the operator and the equipment worn are obtained, and multiple sequences of operation images are constructed; the safety analysis of the operation actions of the equipment worn in the high-altitude operation is performed on the multiple operation image sequences respectively, and multiple safety levels of wearing actions are obtained; the wearing safety of the equipment worn is analyzed separately for each operation image in the multiple operation image sets respectively, and multiple sets of wearing safety levels are obtained, and the wearing safety stability analysis and calculation are performed respectively to obtain multiple identification wearing safety levels; the real-time wearing equipment safety level is obtained according to the multiple identification wearing safety levels and the multiple wearing action safety level calculations as the image processing result of the high-altitude operation monitoring. Through the multi-angle image acquisition device and image processing technology, the wearing actions and equipment wearing conditions of the operator are analyzed in real time, and dynamic safety level evaluation and early warning prompts are provided, thereby greatly improving the safety monitoring level of the high-altitude operation of electric power and reducing the risk of safety accidents caused by improper equipment wearing.

[0015] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented according to the contents of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are specifically cited below. It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easy to understand through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of the image processing method for high-altitude power operations in this application.

[0018] Figure 2 This is a schematic diagram of the structure of the image processing system for high-altitude power operations in this application.

[0019] Explanation of the reference numerals: image acquisition module 11 , action safety analysis module 12 , wearing safety separate analysis module 13 , safety level acquisition module 14 . DETAILED DESCRIPTION

[0020] This application solves the technical problem in the prior art that it is difficult to provide real-time safety feedback due to the difficulty in achieving continuous analysis of multiple angles and multiple moments by providing an image processing method and system for high-altitude power operations. Through multi-angle image acquisition equipment and image processing technology, real-time analysis of the wearing actions and equipment wearing conditions of operators is performed to provide dynamic safety level evaluation and early warning prompts, thereby greatly improving the safety monitoring level of high-altitude power operations and reducing the risk of safety accidents caused by improper equipment wearing.

[0021] Below, the technical solutions in the present application will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments of the present application. It should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application. It should also be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, rather than all of them.

[0022] For example, please refer to the attached Figure 1 The present application provides an image processing method for high-altitude electric power operations, wherein the image processing method for high-altitude electric power operations is applied to an image processing system for high-altitude electric power operations, and the image processing method for high-altitude electric power operations specifically includes the following steps.

[0023] Step 1: During the high-altitude power operation, the image acquisition equipment is used to capture images of the workers from multiple angles, obtain multiple work image sets including the workers and the equipment they wear, and construct multiple work image sequences.

[0024] Specifically, during the high-altitude operation of electric power, real-time image acquisition of the workers and their equipment (such as safety belts and fall protection devices) is performed through image acquisition equipment installed at the operation site (such as high-definition cameras or smart helmet cameras). In order to ensure comprehensive monitoring of the safety status of the workers from different perspectives, the acquisition equipment is set at different fixed positions, which can capture images of the workers from the front, back, left and right, and up and down angles, and multiple sets of operation images containing the workers and their equipment can be obtained. The image acquisition equipment will continue to operate during the high-altitude operation and capture images at a certain time interval within P moments. Specifically, P represents the number of time points. Assuming that an image is collected once per second, an image set of 60 moments can be obtained within one minute. By analogy, during the entire duration of the high-altitude operation, image data sets corresponding to multiple angles and containing different moments can be obtained as multiple sets of operation images.

[0025] Furthermore, in order to effectively analyze multiple work image sets, multiple work image sets are constructed into multiple work image sequences, that is, the images in the work image sets are arranged in chronological order to form a complete work image sequence. For example, in a certain high-altitude work experiment, assuming that the acquisition device acquires images every 5 seconds and acquires a total of 300 images at a time, the obtained work image set will contain image data at 300 time points, which will be arranged in chronological order to generate the corresponding work image sequence. The work image sequence will be used for subsequent safety analysis, focusing on analyzing whether the workers are wearing safety belts and fall protection devices correctly, and judging the safety status of the workers by comparing and analyzing images at different time points.

[0026] Step 2: Perform safety analysis on the multiple operation image sequences of the operation actions of wearing equipment during high-altitude operations to obtain multiple wearing action safety levels.

[0027] Specifically, after obtaining multiple work image sequences, each work image sequence will be analyzed for the safety of the work actions of wearing equipment during high-altitude work. The purpose is to evaluate whether the actions of workers wearing equipment during high-altitude work meet safety standards. This process is analyzed through the slowfast network, which is a deep learning network that combines fast and slow paths. It can efficiently process motion features of different time scales. The fast path can capture subtle and rapidly changing motion features, while the slow path is responsible for analyzing the overall motion trend over a long period of time and identifying whether the work actions of wearing equipment meet the standards. For example, whether the operation of workers wearing safety belts meets the safety operation specifications and whether the actions are quick and in place. If the worker's actions at a certain moment are not standardized, such as the safety belt is not worn correctly, they will be scored according to the standard degree of the action. The higher the score, the safer it is. Specifically, a wearing action safety level will be assigned to the wearing action at each time point, with a score range from 1 to 10, where 10 means full compliance with safety standards and 1 means there are serious safety hazards. For example, in a test, if the analysis process identifies that the operator is following the standard actions, the final score may be 9. If it is found in a certain operation that the operator did not use the fall protection device correctly in multiple operations, the slowfast network will give a lower score, such as between 3-5, and use this as the wearing action safety level, which can effectively identify the safety hazards of the operators and provide reference data for subsequent high-altitude operation safety monitoring.

[0028] Step three: performing a separate analysis of the wearing safety of the wearable equipment for each operation image in the plurality of operation image sets to obtain a plurality of wearing safety level sets, and performing wearing safety stability analysis and calculation to obtain a plurality of identified wearing safety levels.

[0029] Specifically, in the image processing of high-altitude power operations, in order to accurately evaluate the safety of the equipment worn by the workers, a separate analysis of the wearing safety is performed on each frame of each operation image set. The separate analysis of wearing safety involves detecting the equipment status of each image. The purpose of this process is to determine whether the safety equipment worn by the workers meets the standards at each moment and angle, and to assign a wearing safety level to each image based on the analysis results. These levels will be aggregated into a wearing safety level set as the basis for the overall safety evaluation of the operation image set. For example, in an operation image set consisting of 100 frames of images, the safety level of 90 frames is between 9 and 10, while the levels of the remaining 10 frames are between 5 and 6.

[0030] Next, a wearing safety stability analysis is performed on each wearing safety level set. The purpose is to detect whether the safety level remains stable in the time series, so as to determine whether the safety of the equipment worn by the operators is stable, thereby obtaining multiple identification wearing safety levels corresponding to multiple angles, and comprehensively evaluating the status of the operators' wearing equipment, ensuring that safety hazards are identified and warned in a timely manner, and providing important safety guarantees for high-altitude power operations.

[0031] Step 4: Calculate the real-time wearing equipment safety level according to the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude work monitoring image processing result.

[0032] Specifically, multiple identification wearing safety levels are based on the state of the wearing equipment in each frame of the image, indicating static safety. For example, it analyzes whether the operator's safety belt is correctly fastened, whether the fall protection device is in a compliant position, etc., and gives a score of 1 to 10 (10 means the safest). The wearing action safety level reflects the dynamic action safety, which is mainly based on the state of the operator's wearing equipment when performing the action, such as whether the equipment is kept in normal use during climbing or moving. The score is also between 1 and 10. The fusion of these two types of safety levels can obtain a real-time wearing equipment safety level. As the image processing result of high-altitude work monitoring, it can subsequently issue a prompt warning when the real-time wearing equipment safety level does not meet the requirements, providing real-time safety level monitoring for high-altitude workers, helping to timely identify and prevent possible safety hazards.

[0033] Furthermore, step one of the present application includes: in high-altitude power operations, using image acquisition equipment to acquire images of operators at multiple angles, and counting the work images within the past P moments to obtain multiple work image sets, where P is an integer greater than 1; arranging the multiple work image sets according to the timestamps of the P moments to obtain multiple work image sequences.

[0034] Specifically, in power aerial work, image acquisition equipment (such as high-definition cameras or smart helmet cameras) are installed at different locations on the work site, which can monitor workers in real time from multiple angles. It can not only capture the movements of workers, but also accurately record the status of the safety equipment they wear (such as safety belts, fall protection devices, etc.). In the actual operation process, the images of workers are continuously collected, and the image data of the past P moments are counted, where P is an integer greater than 1. The image set collected at each moment will contain images of the workers from different angles, such as the front, side, and back. Images from one angle will form an operation image set. Assuming that an image is collected once per second, P moments correspond to the image data of the past P seconds. For example, assuming P=60, the operation image data within the past 60 seconds will be counted.

[0035] Next, for multiple work image sets, they are arranged according to the timestamps of P moments (i.e., the recording time of each moment), and multiple work image sequences are generated in chronological order. Each work image sequence contains images of the workers and their equipment captured from one angle, and these images are arranged in chronological order, which is convenient for analyzing the status changes of the equipment worn during the work process. For example, it is analyzed whether the workers' safety belts are worn correctly at every moment, and whether the anti-fall device is always in an effective state. If it is found at a certain moment that the safety belt is not correctly fastened or the anti-fall device is not enabled, this problem will be recorded and the current safety risk will be calculated to ensure that the workers can adjust their equipment in time to avoid safety hazards.

[0036] Furthermore, step two of the present application includes: based on the slowfast network, pre-training a wearable equipment motion safety analyzer for identifying the safety level of wearing actions; extracting operation images for the multiple operation image sequences according to a first extraction step size, and performing Haar wavelet transform down-sampling processing to obtain multiple fast operation image sets; extracting operation images for the multiple operation image sequences according to a second extraction step size to obtain multiple slow operation image sets, and the second extraction step size is greater than the first extraction step size; respectively combining the multiple fast operation image sets and the multiple slow operation image sets, and inputting them into the wearable equipment motion safety analyzer to obtain multiple wearing action safety levels.

[0037] Furthermore, the present application also includes the following steps: based on the image acquisition data of high-altitude operations of power grids in historical time, according to the first extraction step size, Haar wavelet transform downsampling and the second extraction step size, the acquisition and processing are performed to obtain multiple sample fast operation image sets and multiple sample slow operation image sets, and according to the standard degree of the wearable equipment operation action, the sample wearing action safety level set is labeled to obtain; based on the slowfast network, the network structure of the wearable equipment action safety analyzer is constructed; using the multiple sample fast operation image sets, multiple sample slow operation image sets and the sample wearing action safety level set, the wearable equipment action safety analyzer is trained until convergence to complete the training.

[0038] Specifically, the slowfast network is an action recognition model that can capture the characteristics of fast-paced and slow-paced actions at the same time. It is suitable for analyzing the detailed actions and overall behaviors of wearable equipment in high-altitude operations. The wearable equipment action safety analyzer trained in this way has the ability to process fast and slow-paced actions at the same time. It can detect whether the subtle actions of the operator are standardized at the micro level, and judge whether the overall working posture and equipment operation meet the safety requirements at the macro level. Further, images are extracted from multiple working image sequences, and different extraction steps are used for sampling processing. The first extraction step is relatively small, which is used to capture the details of the rapidly changing actions during the operation. For example, it can be set to collect one frame of image every 0.5 seconds, so as to obtain high-frequency images of fast actions such as the operator's hand operating the safety belt or adjusting the anti-fall device. These high-frequency images will be down-sampled by Haar wavelet transform, mainly to reduce data redundancy while retaining key details in the image. The multiple image sets extracted from multiple working image sequences according to the first extraction step are recorded as multiple fast working image sets, which are used to detect small changes in actions.

[0039] At the same time, sampling is performed at larger intervals according to the second extraction step size to extract the long-term trend of the overall movement of the operator. For example, it can be set to collect one frame of image every 4 seconds. These images are used to monitor the posture changes, equipment usage status, etc. of the operators over a period of time. The multiple image sets extracted from the multiple operation image sequences according to the second extraction step size are recorded as multiple slow operation image sets, which can help identify larger action patterns, such as whether the operators always maintain the correct safety belt status when working at heights or whether they operate according to standard procedures.

[0040] Next, multiple fast operation image sets and multiple slow operation image sets are combined accordingly, that is, the fast operation image set and the slow operation image set collected from one angle are combined and input into the wearable equipment action safety analyzer at the same time. Since the slowfast network has the ability to process fast and slow rhythmic actions at the same time, the wearable equipment action safety analyzer can detect whether the subtle movements of the operators are standardized at the micro level, and judge whether the overall operation posture and equipment operation meet the safety requirements at the macro level, thereby outputting multiple wearing action safety levels. For example, a certain electric power high-altitude operation was monitored for 10 minutes, and a frame of fast operation image was extracted every 0.5 seconds, and a frame of slow operation image was extracted every 5 seconds. A total of 1200 frames of fast operation images and 120 frames of slow operation images were extracted. Through the slowfast network analysis, the wearing action fully meets the safety standards, and the corresponding wearing action safety level is output as level 10. The wearable equipment action safety analyzer can timely identify the safety hazards in the operation and generate the corresponding wearing action safety level, thereby providing support for the monitoring of electric power high-altitude operations, ensuring the safety of high-altitude operations, and reducing the risk of accidents.

[0041] Among them, based on the slowfast network, the specific steps of pre-training a wearable equipment action safety analyzer for identifying the safety level of wearing actions are as follows: first, collect image acquisition data of high-altitude operations of power grids in the past period of time, that is, historical time, and sample them separately according to the first extraction step size and the second extraction step size using Haar wavelet transform downsampling, so that multiple sample fast operation image sets and multiple sample slow operation image sets with corresponding relationships can be obtained. At the same time, professional and technical personnel in this field combine the standard degree of wearable equipment operation actions corresponding to the image acquisition data of high-altitude operations of power grids in the historical time period to label the wearing action safety level and obtain a sample wearing action safety level set. The safety level labeling is performed by professional and technical personnel in this field according to the degree of compliance of the operator's actions in the image acquisition data with the safety standards. For example, if the use of the safety belt meets the standards and the operation is standardized, it can be labeled as a high safety level (such as 9); if the safety belt is not used in a standardized manner or the fall protection device is not used correctly, it may be labeled as a low level (such as 1).

[0042] Haar wavelet transform downsampling is to perform Haar wavelet transform first and then sample according to the first decimation step and the second decimation step. Haar wavelet transform is a commonly used discrete wavelet transform. Haar wavelet transform can be used to reduce the spatial resolution of feature maps while retaining information integrity to the maximum extent. Its principle is to decompose the signal into components of different frequencies and can effectively perform data compression and feature extraction. The core of Haar wavelet transform is to decompose the signal into two parts: approximation and detail. This process is achieved through a series of low-pass and high-pass filters. In Haar wavelet transform downsampling, simple average filtering (low-pass) and difference filtering (high-pass) are used. The decomposition formula used is as follows: , ;in It is The approximation coefficient of the layer, It is The approximation coefficient of the layer, It is The detail factor of the layer, It is in Index into the array of approximation coefficients for the layer.

[0043] The reconstruction formula is as follows: , . This means that the length is The signal can be divided into The two parts can be interpreted as low-pass and high-pass filters, respectively. When the Haar wavelet transform is applied to a two-dimensional signal such as a grayscale image, four components will be generated, each with half the spatial resolution of the original signal, and the number of channels of the feature map has increased four times. In other words, the Haar wavelet transform can encode part of the information in the spatial dimension into the channel dimension without losing any information. During the sampling process, the image is downsampled by the Haar wavelet transform. Through this transformation method, the image can reduce redundant data while retaining key action information, thereby improving computational efficiency. Next, the fast operation image set and the slow operation image set are processed separately, and the safety level of the wearing action in each image sequence is marked.

[0044] Next, the slowfast network is used to build the network structure of the wearable equipment motion safety analyzer. The slowfast network is a deep learning network designed for motion recognition. It improves the accuracy of recognition by processing fast-paced and slow-paced motion features at the same time. The fast path is used to identify the detailed changes in the motion during the operation, such as the hand operation completed by the operator in a few milliseconds, and the slow path is responsible for analyzing the overall motion trend over a long period of time, such as the adjustment of the operator's body posture. Exemplarily, the slow path processes video frames at a lower sampling rate and uses a 3D convolution layer to simultaneously extract features in the time and space dimensions. In high-altitude operation scenarios, this convolution layer can detect macroscopic actions such as the operator's wearing of the safety belt and overall posture changes. In order to further improve the training effect of the network, residual connections are usually added to the convolution layer of the slow path, which helps to capture subtle changes over a long period of time and prevent information loss. The main purpose of the fast path is to capture detailed actions. It uses a higher frame rate and lightweight convolution operations to focus on fast-changing details. The fast path uses smaller convolution kernels and fewer channels to extract detailed features more quickly. The fast path contains a pooling layer to aggregate local detailed features, while reducing the size of the feature map and reducing computational complexity. In the slowfast network structure, the slow path and the fast path will perform feature fusion regularly. The multi-scale fusion layer can effectively integrate the features of the fast and slow paths, retaining the overall trend information of the action and capturing subtle changes in the action. By fusing the feature maps of the fast and slow paths in multiple network layers, information of different time scales can be combined. At each fusion, the feature map of the slow path will be added to the fast path to enhance the fast path's perception of macro information. The features after multi-scale fusion will be input into the final classification layer to output the safety level of the wearable action. The classification layer uses the Softmax activation function to output the probability distribution of different safety levels, and the safety level with the highest probability will be used as the output.

[0045] After the network structure of the wearable equipment action safety analyzer is constructed, multiple sample fast work image sets, multiple sample slow work image sets and corresponding sample wearing action safety level sets are input into the slowfast network for training. The training process repeatedly adjusts the network parameters until the network can accurately predict the safety level of the wearing action. As the training progresses, the network will gradually converge, which means that the safety of the working action can be accurately judged by the input fast and slow rhythm image sequences. For example, 500 hours of power grid high-altitude operation image data were collected in the past year. After processing, 20,000 fast operation image sets and 20,000 slow operation image sets were obtained. By annotating these image sets and inputting them into the slowfast network for training, the network's recognition accuracy was finally increased to more than 90%, which can effectively predict the safety level of wearing actions. After the training, the wearable equipment action safety analyzer can evaluate the safety of each action in high-altitude operations in real time, thereby providing more reliable safety guarantees for on-site operations.

[0046] Furthermore, step three of the present application includes: collecting a set of sample operation pictures based on the image acquisition data of high-altitude operations of power grids in historical time, and annotating to obtain a set of sample wearing safety levels based on the standard degree of wearing of the wearable equipment in each sample operation image; using the sample operation picture set and the sample wearing safety level set, using a convolutional neural network, training a wearing safety analyzer; inputting each operation image in the multiple operation image sets into the wearing safety analyzer, and outputting multiple wearing safety level sets.

[0047] Specifically, the step of performing a separate analysis of the wearing safety of the wearable equipment on each operation image in the plurality of operation image sets, and obtaining a plurality of wearing safety level sets includes: in the monitoring of the high-altitude operation of the electric power, based on the image acquisition data in the historical time, constructing a safety analysis model of the wearable equipment, firstly, collecting sample images of multiple historical time periods in the high-altitude operation of the power grid, these image sets are called sample operation image sets, each sample image records the wearing equipment of the operator, including the wearing status of key safety equipment (such as safety belts, anti-fall devices, etc.), in order to establish an effective analysis model, the professional and technical personnel in this field analyze the status of the wearable equipment for each image, and mark the corresponding safety level according to the standard degree of the wearable equipment, so as to form a sample wearing safety level set. For example, if the safety belt in the image is worn in accordance with the specification and the anti-fall device is correctly fixed, a higher safety level (such as 9 to 10) is marked for the image; if the equipment in some images is not worn in accordance with the specification, such as the safety belt is not fastened or the anti-fall device is not in the right position, a lower safety level (such as 1) will be marked, and these marked level sets constitute the sample wearing safety level set as the label in the training data set.

[0048] Next, a wearable safety analyzer is constructed using a convolutional neural network, and is trained using a set of input sample operation graphs and a set of sample wearable safety levels. A convolutional neural network is a deep learning algorithm that is particularly suitable for image recognition tasks. Its core is to extract key features from images through multi-layer convolution and pooling operations, thereby realizing the recognition of the status of wearable equipment. The convolutional neural network includes convolutional layer, pooling layer, activation function layer, fully connected layer and output layer. For example, the typical structure of a wearable safety analyzer based on a convolutional neural network is as follows: input layer: a color image of size 32x32x3; convolutional layer 1: 32 3x3 filters, step size 1, ReLU activation function; pooling layer 1: 2x2 maximum pooling, step size 2; convolutional layer 2: 64 3x3 filters, step size 1, ReLU activation function; pooling layer 2: 2x2 maximum pooling, step size 2; flattening: flattening the feature map into a one-dimensional vector; fully connected layer 1: 128 neurons, ReLU activation function; fully connected layer 2: 64 neurons, ReLU activation function; output layer: 10 neurons, Softmax activation function, for 10 classification tasks.

[0049] During the training process, each sample image in the sample operation image set is input into the convolutional neural network. The network will extract the safety equipment features in the image layer by layer, such as the closure of the seat belt buckle, the position of the fall protection device, etc. Then, the network output is compared with the corresponding sample wearing safety level in the sample wearing safety level set, and the model is optimized by adjusting the network parameters so that it can gradually learn to identify the characteristics of the equipment safety status. When the training reaches a certain accuracy, that is, the network's predicted output can accurately correspond to the actual marked safety level, the training process is completed. After the training of the wearing safety analyzer is completed, each operation image in the multiple operation image sets in the actual operation can be input into the wearing safety analyzer to obtain multiple wearing safety level sets, which can realize real-time monitoring and risk identification of the equipment wearing safety of power high-altitude workers, and effectively ensure the safety of the operation.

[0050] Furthermore, the present application also includes the following steps: respectively calculating the means of the multiple wearing safety level sets to obtain multiple average wearing safety levels; randomly selecting multiple first wearing safety levels from the first wearing safety level set within the multiple wearing safety level sets, respectively calculating the deviation amplitudes of the multiple first wearing safety levels and the first average safety wearing level and calculating the mean to obtain the first wearing safety level instability coefficient; using 1 minus the difference of the first wearing safety level instability coefficient, multiplying it by the first average wearing safety level, to perform a correction calculation of the wearing safety level to obtain a first identified wearing safety level; continuing to perform correction calculations on the other multiple average wearing safety levels to obtain multiple first identified wearing safety levels.

[0051] Specifically, after obtaining multiple sets of wearing safety levels, it is necessary to perform wearing safety stability analysis and calculation respectively to obtain multiple identification wearing safety levels. The specific steps are as follows: For each wearing safety level set, calculate the mean to obtain multiple average wearing safety levels. The mean represents the overall safety level of the wearable equipment in the entire operating image set. For example, if a wearing safety level set contains 10 scores (for example: 9, 8, 9, 7, 8, 9, 10, 8, 7, 9), then its average wearing safety level is 8.4, which reflects the overall situation of wearing safety in the operating image set. Next, randomly select several first wearing safety levels in the first wearing safety level set, calculate the difference between several first wearing safety levels and the first average safety wearing level, and then calculate the ratio of the difference to the first average safety wearing level as the deviation amplitude. After obtaining several deviation amplitudes, perform mean calculation to evaluate the instability of safety and generate the instability coefficient of the first wearing safety level. The larger the deviation amplitude is, the greater the difference in the safety level of the wearable equipment in different operation images is, and the greater the instability of the wearable safety level is. The wearable safety level may be high at the current moment and low at the next moment. Among them, the first wearable safety level set refers to any wearable safety level set in multiple wearable safety level sets, multiple first wearable safety levels refer to elements in the first wearable safety level set, and the first average safety wearable level refers to the average wearable safety level corresponding to the first wearable safety level set in multiple average wearable safety levels.

[0052] The larger the instability coefficient of the first wearing safety level, the greater the fluctuation of the safety level in the set, that is, the safety of the wearable equipment is relatively unstable. After obtaining the instability coefficient of the first wearing safety level, a correction calculation of the wearing safety level will be performed to further reflect the actual wearing safety status. The correction calculation formula is: subtract the instability coefficient of the first wearing safety level from 1, and then multiply it by the average wearing safety level to obtain the first identification wearing safety level. The corrected identification wearing safety level can more accurately reflect the overall safety level of the wearable equipment. Using the same method, traverse the multiple wearing safety level sets to obtain multiple identification wearing safety levels.

[0053] Furthermore, step four of the present application includes: calculating the ratio of each identified wearing safety level to the average of the multiple identified wearing safety levels as multiple wearing safety factors; using the multiple wearing safety factors to respectively perform correction calculations on the products of the multiple wearing action safety levels to obtain multiple corrected wearing action safety levels; calculating the average of the multiple corrected wearing action safety levels to obtain the real-time wearing equipment safety level as the high-altitude work monitoring image processing result.

[0054] Specifically, first calculate the mean of multiple identification wearing safety levels, and then calculate the ratio of each identification wearing safety level to the mean to obtain multiple wearing safety factors. The wearing safety factor reflects the relative relationship between the wearing safety status at a certain moment and the overall safety level. Therefore, the relative safety of the identification wearing safety level below the mean is slightly lower, while the safety of the identification wearing safety level above the mean is relatively higher. Next, use the wearing safety factor to correct the safety level of each wearing action. The correction calculation method is to multiply each wearing safety factor by the corresponding wearing action safety level to obtain multiple corrected wearing action safety levels. This method can dynamically adjust the wearing action safety level, reflecting the adjustment of the safety status at that moment relative to the overall safety level. Example: If a wearing action safety level is 8.5, and the corresponding wearing safety factor is 0.875, then the corrected wearing action safety level is 8.5×0.87=7.44. Finally, the average of multiple corrected wearing action safety levels is calculated to obtain the real-time wearing equipment safety level. The real-time wearing equipment safety level is the final safety score, which is used for overall safety monitoring of the work site, reflecting the overall safety status of the wearing equipment during the current operation, and can be fed back to the monitoring system in real time so that safety managers can grasp the safety status of the workers in a timely manner. Through this correction method based on identification and action level, the safety of the equipment of the workers in different states can be more accurately reflected, providing a more valuable real-time safety score for reference for high-altitude work safety monitoring, so as to facilitate timely protective measures.

[0055] In summary, the image processing method for high-altitude electric power operations provided by the present application has the following technical effects: in high-altitude electric power operations, the image acquisition device is used to acquire images of the operators at multiple angles, and multiple sets of operation images including the operators and the equipment being worn are obtained, and multiple sequences of operation images are constructed; the safety analysis of the operation actions of the equipment being worn in high-altitude operations is performed on the multiple operation image sequences respectively, and multiple safety levels of wearing actions are obtained; the wearing safety of the equipment being worn is separately analyzed for each operation image in the multiple operation image sets respectively, and multiple sets of wearing safety levels are obtained, and the wearing safety stability analysis and calculation are performed respectively, and multiple identification wearing safety levels are obtained; the real-time wearing equipment safety level is obtained according to the multiple identification wearing safety levels and the multiple wearing action safety level calculations, as the image processing result for monitoring high-altitude operations. Through multi-angle image acquisition equipment and image processing technology, the wearing actions and equipment wearing conditions of the operators are analyzed in real time, and dynamic safety level evaluation and early warning prompts are provided, thereby greatly improving the safety monitoring level of high-altitude electric power operations and reducing the risk of safety accidents caused by improper equipment wearing.

[0056] Embodiment 2: Based on the image processing method for high-altitude power operation in the above embodiment, and with the same inventive concept, this application also provides an image processing system for high-altitude power operation, as shown in the attached Figure 2 The electric power aerial work image processing system includes: an image acquisition module 11, which is used to acquire images of operators at multiple angles through image acquisition equipment during electric power aerial work, obtain multiple work image sets including operators and wearable equipment, and construct multiple work image sequences.

[0057] The action safety analysis module 12 is used to perform safety analysis of the action of wearing equipment during high-altitude operations on the multiple operation image sequences respectively, and obtain multiple wearing action safety levels.

[0058] The wearing safety separate analysis module 13 is used to perform a separate wearing safety analysis of the wearable equipment on each operation image in the multiple operation image sets to obtain multiple wearing safety level sets, and perform wearing safety stability analysis calculations to obtain multiple identified wearing safety levels.

[0059] The safety level acquisition module 14 is used to calculate the real-time safety level of the worn equipment according to the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude work monitoring image processing result.

[0060] Furthermore, the image acquisition module 11 in the electric power aerial work image processing system is also used for: in electric power aerial work, using image acquisition equipment to acquire images of workers at multiple angles, and counting the work images within the past P moments to obtain multiple work image sets, where P is an integer greater than 1; arranging the multiple work image sets according to the timestamps of the P moments to obtain multiple work image sequences.

[0061] Furthermore, the action safety analysis module 12 in the electric power aerial work image processing system is also used to: pre-train a wearable equipment action safety analyzer for identifying the safety level of wearing actions based on a slowfast network; extract work images for the multiple work image sequences according to a first extraction step size, and perform Haar wavelet transform down-sampling processing to obtain multiple fast work image sets; extract work images for the multiple work image sequences according to a second extraction step size to obtain multiple slow work image sets, and the second extraction step size is greater than the first extraction step size; respectively combine the multiple fast work image sets and the multiple slow work image sets, and input them into the wearable equipment action safety analyzer to obtain multiple wearing action safety levels.

[0062] Furthermore, the action safety analysis module 12 in the electric power aerial work image processing system is also used for: based on the image acquisition data of the power grid aerial work in the historical time, according to the first extraction step size, Haar wavelet transform downsampling and the second extraction step size, the acquisition and processing obtains multiple sample fast operation image sets and multiple sample slow operation image sets, and according to the standard degree of the wearable equipment operation action, annotates and obtains the sample wearing action safety level set; based on the slowfast network, constructs the network structure of the wearable equipment action safety analyzer; uses the multiple sample fast operation image sets, multiple sample slow operation image sets and sample wearing action safety level sets to train the wearable equipment action safety analyzer until convergence, and completes the training.

[0063] Furthermore, the separate wearing safety analysis module 13 in the electric power aerial work image processing system is also used to: collect a set of sample operation pictures based on the image acquisition data of aerial work in the power grid in historical time, and mark and obtain a set of sample wearing safety levels based on the standard degree of wearing of the worn equipment in each sample operation image; use the sample operation picture set and the sample wearing safety level set to train a wearing safety analyzer using a convolutional neural network; input each operation image in the multiple operation image sets into the wearing safety analyzer, and output multiple wearing safety level sets.

[0064] Furthermore, the separate wearing safety analysis module 13 in the power aerial work image processing system is also used to: calculate the means of the multiple wearing safety level sets respectively to obtain multiple average wearing safety levels; randomly select multiple first wearing safety levels from the first wearing safety level set within the multiple wearing safety level sets, calculate the deviation amplitudes of the multiple first wearing safety levels and the first average safety wearing level respectively, and calculate the mean to obtain the first wearing safety level instability coefficient; use 1 minus the difference of the first wearing safety level instability coefficient, multiply by the first average wearing safety level, perform a correction calculation of the wearing safety level, and obtain a first identified wearing safety level; continue to perform correction calculations on the other multiple average wearing safety levels to obtain multiple first identified wearing safety levels.

[0065] Furthermore, the safety level acquisition module 14 in the power aerial work image processing system is also used to: calculate the ratio of each identified wearing safety level to the average of the multiple identified wearing safety levels as multiple wearing safety coefficients; use the multiple wearing safety coefficients to respectively perform correction calculations on the products of the multiple wearing action safety levels to obtain multiple corrected wearing action safety levels; calculate the average of the multiple corrected wearing action safety levels to obtain the real-time wearing equipment safety level as the aerial work monitoring image processing result.

[0066] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. Figure 1 The image processing method for high-altitude electric power operations and the specific examples in the first embodiment are also applicable to the image processing system for high-altitude electric power operations in the present embodiment. Through the above detailed description of the image processing method for high-altitude electric power operations, those skilled in the art can clearly understand the image processing system for high-altitude electric power operations in the present embodiment, so for the sake of brevity of the specification, it will not be described in detail here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0067] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0068] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technology, the present application is also intended to include these modifications and variations.

Claims

1. A method for processing images of electric power aerial work, characterized in that: The electric power aerial work image processing method comprises: In the high-altitude power operation, the image acquisition equipment is used to acquire images of the operators from multiple angles, obtain multiple sets of operation images including the operators and the equipment they wear, and construct multiple operation image sequences; Performing safety analysis on the multiple operation image sequences of the operation actions of wearing equipment during high-altitude operations to obtain multiple wearing action safety levels; Performing a separate wearing safety analysis of the wearable equipment on each of the plurality of operation image sets to obtain a plurality of wearing safety level sets, and performing a wearing safety stability analysis calculation to obtain a plurality of identified wearing safety levels; Calculate the real-time wearing equipment safety level according to the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude work monitoring image processing result; The safety analysis of the operation actions of wearing equipment in high-altitude operations is performed on the multiple operation image sequences respectively to obtain multiple wearing action safety levels, including: Based on the slowfast network, a wearable device action safety analyzer is pre-trained to identify the safety level of wearing actions; Extracting operation images from the plurality of operation image sequences according to a first extraction step length, and performing Haar wavelet transform down-sampling processing to obtain a plurality of fast operation image sets; Extracting job images from the plurality of job image sequences respectively according to a second extraction step length to obtain a plurality of slow job image sets, wherein the second extraction step length is greater than the first extraction step length; The multiple fast operation image sets and the multiple slow operation image sets are respectively combined correspondingly and input into the wearing equipment action safety analyzer to obtain multiple wearing action safety levels.

2. The method for processing images of power aerial work according to claim 1, characterized in that: In the high-altitude operation of electric power, the image acquisition equipment is used to collect images of the operators from multiple angles, obtain multiple operation image sets including the operators and the equipment they wear, and construct multiple operation image sequences, including: In the electric power aerial work, the image acquisition equipment is used to collect images of the workers from multiple angles, and the work images in the past P moments are counted to obtain multiple work image sets, where P is an integer greater than 1; The multiple job image sets are arranged according to the timestamps of the P moments to obtain multiple job image sequences.

3. The method for processing images of power aerial work according to claim 1, characterized in that: Based on the slowfast network, a wearable device motion safety analyzer is pre-trained to identify the safety level of wearable motions, including: Based on the image acquisition data of the high-altitude operation of the power grid in the historical time, according to the first extraction step size, Haar wavelet transform down sampling and the second extraction step size, the acquisition and processing are performed to obtain a plurality of sample fast operation image sets and a plurality of sample slow operation image sets, and according to the standard degree of the operation action of the wearing equipment, the sample wearing action safety level set is annotated to obtain; Based on the slowfast network, a network structure of the wearable equipment action safety analyzer is constructed; The plurality of sample fast operation image sets, the plurality of sample slow operation image sets and the sample wearing action safety level sets are used to train the wearing equipment action safety analyzer until convergence, thereby completing the training.

4. The method for processing images of power aerial work according to claim 1, characterized in that: Performing a separate analysis of the wearing safety of the wearable equipment on each operation image in the plurality of operation image sets, to obtain a plurality of wearing safety level sets, including: Based on the image acquisition data of high-altitude operations of power grids in historical time, a set of sample operation images is collected, and according to the standard degree of wearing equipment in each sample operation image, a set of sample wearing safety levels is obtained by annotation; Using the sample operation graph set and the sample wearing safety level set, and using a convolutional neural network, training a wearing safety analyzer; Each operation image in the plurality of operation image sets is input into the wearing safety analyzer, and a plurality of wearing safety level sets are obtained as output.

5. The method for processing images of power aerial work according to claim 1, characterized in that: Perform wear safety stability analysis and calculations respectively to obtain multiple identification wear safety levels, including: Calculating the averages of the plurality of wearing safety level sets respectively to obtain a plurality of average wearing safety levels; Randomly extracting a plurality of first wearing safety levels from a first wearing safety level set within the plurality of wearing safety level sets, respectively calculating deviation amplitudes of the plurality of first wearing safety levels and a first average safety wearing level and calculating an average, to obtain an instability coefficient of the first wearing safety level; The difference between 1 and the instability coefficient of the first wearing safety level is multiplied by the first average wearing safety level to perform correction calculation of the wearing safety level to obtain a first identified wearing safety level; Continue to perform correction calculation on other multiple average wearing safety levels to obtain multiple first identified wearing safety levels.

6. The method for processing images of power aerial work according to claim 1, characterized in that: The real-time wearable equipment safety level is calculated according to the multiple identification wear safety levels and the multiple wear action safety levels, including: Calculating a ratio of each identified wearing safety level to an average of the plurality of identified wearing safety levels as a plurality of wearing safety coefficients; Using the multiple wearing safety factors, respectively multiplying and correcting the products of the multiple wearing action safety levels to obtain multiple corrected wearing action safety levels; The average of the multiple corrected wearing action safety levels is calculated to obtain the real-time wearing equipment safety level as the high-altitude work monitoring image processing result.

7. The image processing system for electric power aerial work is characterized by: The steps for implementing the method for processing images of power aerial work according to any one of claims 1 to 6, the power aerial work image processing system comprising: The image acquisition module is used to acquire images of operators from multiple angles through image acquisition equipment during high-altitude power operations, obtain multiple sets of operation images including operators and equipment worn, and construct multiple operation image sequences; An action safety analysis module, used to respectively analyze the safety of the action of wearing equipment in the aerial work on the plurality of operation image sequences, and obtain a plurality of wearing action safety levels; A separate wearing safety analysis module, for performing separate wearing safety analysis of the wearable equipment on each operation image in the plurality of operation image sets, obtaining a plurality of wearing safety level sets, and performing wearing safety stability analysis calculations on each of the plurality of operation image sets, obtaining a plurality of identified wearing safety levels; The safety level acquisition module is used to calculate the real-time safety level of the worn equipment according to the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude work monitoring image processing result.

Images