Power high-altitude operation image processing method and system

Abstract

The present application provides an image processing method and system for high-altitude power operation, which relates to the technical field of image processing and includes: obtaining a plurality of operation image sets including operators and wearing equipment, and constructing a plurality of operation image sequences; obtaining a plurality of safety levels of wearing actions; obtaining a plurality of wearing safety level sets, and respectively performing wearing safety stability analysis calculations to obtain a plurality of recognized wearing safety levels; calculating a real-time wearing equipment safety level based on the plurality of recognized wearing safety levels and the plurality of wearing action safety levels as the image processing result of high-altitude operation monitoring. Through the present application, the technical problem in the prior art that it is difficult to achieve continuous analysis of multiple angles and multiple moments, resulting in the inability to provide real-time safety feedback, can be solved, and the technical effects of improving the safety monitoring level of high-altitude power operation and reducing the risk of safety accidents caused by improper wearing of equipment can be achieved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to an image processing method and system for high-altitude power operation. Background Art

[0002] In the field of safety monitoring for high-altitude power operation, the existing technology mainly relies on manual inspections and some traditional object detection algorithms, such as the YOLO series, Faster R-CNN, etc. These technologies have achieved remarkable results in multiple fields. However, for the detection of safety wear in high-altitude power operation, the existing algorithms have some limitations.

[0003] Firstly, the results of manual judgment are easily affected by subjective factors and lack consistency, which may lead to the neglect of safety issues. Secondly, the existing object detection algorithms are difficult to dynamically identify the wearing status of equipment. Especially when the wearing equipment changes unstably at continuous moments, it is difficult to achieve continuous analysis of multiple angles and multiple moments, resulting in the inability to provide real-time safety feedback. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method and system for high-altitude power operation, so as to solve the technical problem in the existing technology that due to the difficulty in achieving continuous analysis of multiple angles and multiple moments, real-time safety feedback cannot be provided.

[0005] In view of the above problems, this application provides an image processing method and system for high-altitude power operation.

[0006] In the first aspect, this application provides an image processing method for high-altitude power operation. The image processing method for high-altitude power operation is implemented through an image processing system for high-altitude power operation. Among them, the image processing method for high-altitude power operation includes: during high-altitude power operation, using an image acquisition device to collect images of the operator from multiple angles, obtaining multiple sets of operation images including the operator and the wearing equipment, and constructing multiple operation image sequences; respectively performing safety analysis on the wearing equipment operation actions in the high-altitude operation for the multiple operation image sequences to obtain multiple wearing action safety levels; respectively performing individual safety analysis on the wearing equipment for each operation image in the multiple sets of operation images to obtain multiple sets of wearing safety levels, and respectively performing wearing safety stability analysis calculations to obtain multiple recognized wearing safety levels; calculating the real-time wearing equipment safety level based on the multiple recognized wearing safety levels and the multiple wearing action safety levels as the image processing result of high-altitude operation monitoring.

[0007] Further, perform safety analysis on the operation actions of the wearing equipment during high-altitude operations for the multiple operation image sequences respectively, and obtain multiple wearing action safety levels, including: based on the SlowFast network, pre-train a wearing equipment action safety analyzer for identifying the wearing action safety level; extract operation images from the multiple operation image sequences respectively according to the first extraction step length, and perform downsampling processing by Haar wavelet transform to obtain multiple fast operation image sets; extract operation images from the multiple operation image sequences respectively according to the second extraction step length, and obtain multiple slow operation image sets, where the second extraction step length is greater than the first extraction step length; respectively combine the multiple fast operation image sets and the multiple slow operation image sets in a corresponding manner, and input them into the wearing equipment action safety analyzer to obtain multiple wearing action safety levels.

[0008] Further, during high-altitude power operations, perform image acquisition on the operator from multiple angles through an image acquisition device to obtain multiple operation image sets including the operator and the wearing equipment, and construct multiple operation image sequences, including: during high-altitude power operations, perform image acquisition on the operator from multiple angles through an image acquisition device, and count the operation images within the past P time instants to obtain multiple operation image sets, where P is an integer greater than 1; arrange the multiple operation image sets according to the timestamps of the P time instants to obtain multiple operation image sequences.

[0009] Further, based on the SlowFast network, pre-train a wearing equipment action safety analyzer for identifying the wearing action safety level, including: according to the image acquisition data of high-altitude grid operations within the historical time, collect and process to obtain multiple sample fast operation image sets and multiple sample slow operation image sets according to the first extraction step length, Haar wavelet transform downsampling, and the second extraction step length, and label and obtain a sample wearing action safety level set according to the standard degree of the wearing equipment operation actions; based on the SlowFast network, construct the network structure of the wearing equipment action safety analyzer; use the multiple sample fast operation image sets, the multiple sample slow operation image sets, and the sample wearing action safety level set to train the wearing equipment action safety analyzer until convergence, and complete the training.

[0010] Further, separately analyze the wearing safety of each operation image in the multiple operation image sets to obtain multiple sets of wearing safety levels, including: collecting a set of sample operation images according to the image acquisition data of high-altitude power grid operations within a historical time, and marking to obtain a set of sample wearing safety levels according to the standard degree of wearing equipment in each sample operation image; using the set of sample operation images and the set of sample wearing safety levels, training a wearing safety analyzer with a convolutional neural network; inputting each operation image in the multiple operation image sets into the wearing safety analyzer, and outputting to obtain multiple sets of wearing safety levels.

[0011] Further, perform wearing safety stability analysis and calculation separately to obtain multiple recognized wearing safety levels, including: calculating the mean values of the multiple sets of wearing safety levels respectively to obtain multiple average wearing safety levels; randomly extracting multiple first wearing safety levels from the first set of wearing safety levels in the multiple sets of wearing safety levels, calculating the deviation amplitudes between the multiple first wearing safety levels and the first average wearing safety level respectively and calculating the mean value to obtain the instability coefficient of the first wearing safety level; using the difference obtained by subtracting the instability coefficient of the first wearing safety level from 1, multiplying by the first average wearing safety level, and performing correction calculation of the wearing safety level to obtain the first recognized wearing safety level; continuing to perform correction calculation on the other multiple average wearing safety levels to obtain multiple first recognized wearing safety levels.

[0012] Further, calculate the real-time wearing equipment safety level according to the multiple recognized wearing safety levels and the multiple wearing action safety levels, including: calculating the ratio of each recognized wearing safety level to the mean value of the multiple recognized wearing safety levels as multiple wearing safety factors; using the multiple wearing safety factors to perform correction calculation on the product of the multiple wearing action safety levels respectively to obtain multiple corrected wearing action safety levels; calculating the mean value of the multiple corrected wearing action safety levels to obtain the real-time wearing equipment safety level, which is used as the processing result of high-altitude operation monitoring image processing.

[0013] Second aspect, the present application also provides a power high-altitude operation image processing system for performing the power high-altitude operation image processing method as described in the first aspect. Wherein, the power high-altitude operation image processing system includes: an image acquisition module, configured to acquire images of the operator from multiple angles through an image acquisition device during the power high-altitude operation, obtain multiple sets of operation images including the operator and the wearable equipment, and construct multiple operation image sequences; an action safety analysis module, configured to separately perform safety analysis on the operation actions of the wearable equipment during the high-altitude operation for the multiple operation image sequences, and obtain multiple wearable action safety levels; a wearable safety individual analysis module, configured to separately perform individual safety analysis on the wearing of the wearable equipment for each operation image in the multiple sets of operation images, obtain multiple sets of wearable safety levels, and separately perform wearing safety stability analysis calculations to obtain multiple recognized wearable safety levels; a safety level acquisition module, configured to calculate and obtain the real-time wearable equipment safety level based on the multiple recognized wearable safety levels and the multiple wearable action safety levels, as the high-altitude operation monitoring image processing result.

[0014] One or more technical solutions provided in the present application have at least the following technical effects or advantages: During the power high-altitude operation, images of the operator are acquired from multiple angles through an image acquisition device, multiple sets of operation images including the operator and the wearable equipment are obtained, and multiple operation image sequences are constructed; separately perform safety analysis on the operation actions of the wearable equipment during the high-altitude operation for the multiple operation image sequences, and obtain multiple wearable action safety levels; separately perform individual safety analysis on the wearing of the wearable equipment for each operation image in the multiple sets of operation images, obtain multiple sets of wearable safety levels, and separately perform wearing safety stability analysis calculations to obtain multiple recognized wearable safety levels; calculate and obtain the real-time wearable equipment safety level based on the multiple recognized wearable safety levels and the multiple wearable action safety levels, as the high-altitude operation monitoring image processing result. Through the multi-angle image acquisition device and image processing technology, real-time analysis is performed on the wearing actions of the operator and the wearing status of the equipment, providing dynamic safety level evaluation and warning prompts, thereby greatly improving the safety monitoring level of the power high-altitude operation and reducing the risk of safety accidents caused by improper wearing of the equipment.

[0015] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the description. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] To more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are merely exemplary, and for those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative efforts.

[0017] Figure 1 It is a schematic flowchart of the image processing method for high-altitude power operation in the present application.

[0018] Figure 2 It is a schematic structural diagram of the image processing system for high-altitude power operation in the present application.

[0019] Explanation of reference numerals: Image acquisition module 11, action safety analysis module 12, wearable safety individual analysis module 13, safety level acquisition module 14. Specific embodiments

[0020] By providing the image processing method and system for high-altitude power operation, the present application solves the technical problem in the prior art that it is difficult to continuously analyze multiple angles and multiple moments, resulting in the inability to provide real-time safety feedback. Through the multi-angle image acquisition device and image processing technology, the wearing actions and equipment wearing conditions of the operators are analyzed in real time, providing dynamic safety level evaluation and warning prompts, thereby greatly improving the safety monitoring level of high-altitude power operation and reducing the risk of safety accidents caused by improper equipment wearing.

[0021] Next, the technical solutions in the present application will be clearly and completely described with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the example embodiments described here. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. Additionally, it should be noted that for the convenience of description, only the parts related to the present application are shown in the drawings, rather than all of them.

[0022] Embodiment 1. Please refer to the attached Figure 1 , the present application provides an image processing method for high-altitude power operation. Among them, the image processing method for high-altitude power operation is applied to an image processing system for high-altitude power operation, and the image processing method for high-altitude power operation specifically includes the following steps.

[0023] Step 1: In high-altitude power operation, use an image acquisition device to collect images of the operator from multiple angles, obtain multiple sets of operation images including the operator and the wearable equipment, and construct multiple operation image sequences.

[0024] Specifically, during high-altitude electrical work, real-time image acquisition is carried out on the workers and the equipment they wear (such as safety belts and anti-fall devices) through image acquisition devices installed at the work site (such as high-definition cameras or intelligent helmet camera devices). To ensure comprehensive monitoring of the safety status of workers from different perspectives, the acquisition devices are set at different fixed positions, capable of capturing images of workers from the front, back, left, right, and up and down angles, and multiple sets of work images containing workers and their worn equipment can be obtained. The image acquisition devices will operate continuously during high-altitude work and capture images at certain time intervals within P moments. Specifically, P represents the number of time points. Assuming that an image is acquired every second, 60 sets of images at different moments can be obtained within one minute, and so on. During the entire duration of high-altitude work, multiple sets of image data corresponding to different angles and containing different moments can be obtained, which are used as multiple sets of work images.

[0025] Furthermore, to effectively analyze multiple sets of work images, multiple sets of work images are constructed into multiple work image sequences, that is, the images in the work image sets are arranged in chronological order to form complete work image sequences. For example, in an experiment of a certain high-altitude work, assume that the acquisition device acquires images every 5 seconds and a total of 300 images at different moments are acquired. The obtained work image set will contain image data at 300 time points, and arranging them in the order of acquisition time generates the corresponding work image sequence. The work image sequence will be used for subsequent safety analysis, focusing on analyzing whether the workers are correctly wearing safety belts and anti-fall devices, and judging the safety status of the workers by comparing and analyzing images at different time points.

[0026] Step 2: Respectively perform safety analysis on the wearing equipment operation actions during high-altitude work for the multiple work image sequences to obtain multiple wearing action safety levels.

[0027] Specifically, after obtaining multiple sequences of operation images, safety analysis of the operation actions of the wearing equipment during high-altitude operations will be performed on each sequence of operation images. The purpose is to evaluate whether the actions of the operator wearing the equipment during high-altitude operations comply with safety standards. This process is analyzed through the SlowFast network, which is a deep learning network that combines fast and slow paths and can efficiently process action features at different time scales. The fast path can capture subtle and rapidly changing action features, while the slow path is responsible for analyzing the overall action trend over a long time range to identify whether the operation actions of wearing the equipment comply with the standards. For example, for the operation of the operator wearing a safety belt, whether it complies with the safety operation specifications and whether the action is rapid and in place. If at a certain moment the operator's action is not standardized, such as the safety belt is not worn correctly, a score will be given according to the standard degree of the action. The higher the score, the safer it indicates. Specifically, a wearing action safety level will be assigned to the wearing action at each time point, with a scoring range from 1 to 10, where 10 means fully compliant with safety standards and 1 means there are serious safety hazards. For example, in a test, if it is identified during the analysis that the operator's actions are standardized, the final score may be 9. If it is found in a certain operation that the operator fails to correctly use the fall arrest device during multiple operations, the SlowFast network will give a lower score, such as between 3 and 5, and this will be used as the wearing action safety level. Thus, it can effectively identify the safety hazards of the operator and provide reference data for subsequent high-altitude operation safety monitoring.

[0028] Step 3: Separately perform individual analysis on the wearing safety of the wearing equipment for each operation image in the multiple operation image sets, obtain multiple sets of wearing safety levels, and separately perform wearing safety stability analysis and calculation to obtain multiple identified wearing safety levels.

[0029] Specifically, in the image processing of high-altitude electrical operations, in order to accurately evaluate the safety of the operator wearing the equipment, individual analysis on the wearing safety is performed on each frame of image in each operation image set. The individual analysis on the wearing safety involves detecting the equipment status for each image. The purpose of this process is to determine whether the safety equipment worn by the operator complies with the standards at each moment and angle, and based on the analysis results, assign a wearing safety level to each image. These levels will be aggregated into a set of wearing safety levels as the basis for the overall safety evaluation of the operation image set. For example, in an operation image set consisting of 100 frames of images, the safety levels of 90 frames of images are between 9 and 10, while the levels of the remaining 10 frames of images are between 5 and 6.

[0030] Next, perform a wear safety stability analysis on each set of wear safety levels. The purpose is to detect whether the safety levels remain stable over time, thereby determining whether the safety of the equipment worn by the operator is stable, and obtaining multiple recognized wear safety levels corresponding to multiple angles to comprehensively evaluate the status of the equipment worn by the operator, ensuring that potential safety hazards are identified and warned in a timely manner, providing an important safety guarantee for high-altitude electric power operations.

[0031] Step 4: Calculate the real-time wear equipment safety level based on the multiple recognized wear safety levels and the multiple wear action safety levels, which is used as the image processing result of high-altitude operation monitoring.

[0032] Specifically, the multiple recognized wear safety levels are based on the state of the wear equipment in each frame of the image, representing static safety. For example, analyze whether the operator's safety belt is properly fastened, whether the fall protection device is in a compliant position, etc., and give a score from 1 to 10 (10 represents the safest). The wear action safety level reflects the dynamic action safety, mainly based on the state of the wear equipment when the operator performs an action. For example, whether the equipment is maintained in a normal use state during climbing or moving, and the score is also between 1 and 10. By integrating these two types of safety levels, a real-time wear equipment safety level can be obtained, which is used as the image processing result of high-altitude operation monitoring. A prompt warning can be given when the real-time wear equipment safety level does not meet the requirements, providing real-time safety level monitoring for high-altitude operation personnel and helping to identify and prevent potential safety hazards in a timely manner.

[0033] Furthermore, step 1 of this application includes: In high-altitude electric power operations, use an image acquisition device to collect images of the operator from multiple angles, and count the operation images within the past P moments to obtain multiple sets of operation images, where P is an integer greater than 1; According to the timestamps of the P moments, arrange the multiple sets of operation images to obtain multiple operation image sequences.

[0034] Specifically, in high-altitude electric power operations, an image acquisition device (such as a high-definition camera or an intelligent helmet camera device) is installed at different positions at the operation site, which can monitor the operator from multiple angles in real time, not only capturing the operator's actions but also accurately recording the status of the safety equipment (such as safety belts, fall protection devices, etc.) worn by the operator. During the actual operation process, continuously collect images of the operator, and count the image data of the past P moments, where P is an integer greater than 1. Each set of images collected at each moment will contain images of the operator from different angles, such as the front, side, and back. The images from one angle will form a set of operation images. Assuming that images are collected once per second, the P moments correspond to the image data of the past P seconds. For example, assuming P = 60, then the operation image data within the past 60 seconds will be counted.

[0035] Next, for multiple sets of operation images, arrange them according to the timestamps at P moments (i.e., the recording time of each moment), and generate multiple sequences of operation images in chronological order. Each sequence of operation images contains images of the operator and their wearing equipment captured from one angle, and these images are arranged in chronological order, facilitating the analysis of the state changes of the wearing equipment during the operation. For example, analyze whether the safety belt of the operator is correctly worn at each moment, and whether the fall prevention device is always in an effective state. If it is found at a certain moment that the safety belt is not properly fastened or the fall prevention device is not enabled, record this problem and calculate the current safety risk to ensure that the operator can adjust the equipment in time to avoid potential safety hazards.

[0036] Furthermore, step two of the present application includes: based on the slowfast network, pre-train a safety analyzer for wearing equipment actions to identify the safety level of wearing actions; extract operation images from the multiple sequences of operation images respectively according to the first extraction step size, and perform downsampling processing under Haar wavelet transform to obtain multiple sets of fast operation images; extract operation images from the multiple sequences of operation images respectively according to the second extraction step size to obtain multiple sets of slow operation images, where the second extraction step size is greater than the first extraction step size; respectively combine the multiple sets of fast operation images and multiple sets of slow operation images correspondingly, and input them into the safety analyzer for wearing equipment actions to obtain multiple safety levels of wearing actions.

[0037] Furthermore, the present application also includes the following steps: according to the image acquisition data of high-altitude power grid operations within the historical time, collect and process to obtain multiple sets of sample fast operation images and multiple sets of sample slow operation images according to the first extraction step size, Haar wavelet transform downsampling, and the second extraction step size, and label and obtain a set of sample safety levels of wearing actions according to the standard degree of wearing equipment operation actions; based on the slowfast network, construct the network structure of the safety analyzer for wearing equipment actions; use the multiple sets of sample fast operation images, multiple sets of sample slow operation images, and the set of sample safety levels of wearing actions to train the safety analyzer for wearing equipment actions until convergence to complete the training.

[0038] Specifically, the SlowFast network is an action recognition model that can capture both fast-paced and slow-paced action features simultaneously. It is suitable for analyzing the detailed actions and overall behaviors of workers wearing equipment during high-altitude operations. The wearable equipment action safety analyzer trained thereby has the ability to process fast and slow-paced actions simultaneously, and can detect at the micro level whether the subtle actions of the workers are standardized, and at the macro level determine whether the overall working postures and equipment operations meet safety requirements. Further, images are extracted from multiple operation image sequences and sampled using different extraction step sizes. The first extraction step size is relatively small and is used to capture the rapidly changing action details during the operation. For example, it can be set to capture one frame of image every 0.5 seconds, so as to obtain high-frequency images of the rapid actions of the worker's hand operating the safety belt or adjusting the fall arrest device. These high-frequency images will be downsampled through Haar wavelet transform, mainly to reduce data redundancy while retaining the key details in the images. The multiple image sets extracted from multiple operation image sequences according to the first extraction step size are denoted as multiple fast operation image sets and are used to detect the minute changes in actions.

[0039] At the same time, sampling is performed at larger intervals according to the second extraction step size to extract the long-term trends of the overall actions of the workers. For example, it can be set to capture one frame of image every 4 seconds. These images are used to monitor the posture changes and equipment usage status of the workers over a period of time. The multiple image sets extracted from the multiple operation image sequences according to the second extraction step size are denoted as multiple slow operation image sets, which can help identify larger action patterns, such as whether the worker always maintains the correct safety belt state or operates according to the standard process during high-altitude operations.

[0040] Next, the multiple fast operation image sets and multiple slow operation image sets are combined correspondingly, that is, the fast operation image set and the slow operation image set collected from the same angle are combined and input into the wearable equipment action safety analyzer simultaneously. Since the SlowFast network has the ability to process fast and slow-paced actions simultaneously, the wearable equipment action safety analyzer can detect at the micro level whether the subtle actions of the workers are standardized, and at the macro level determine whether the overall working postures and equipment operations meet safety requirements, and thus can output multiple wearable action safety levels. For example, during a 10-minute monitoring of a certain high-altitude power operation, one frame of fast operation image is extracted every 0.5 seconds and one frame of slow operation image is extracted every 5 seconds. A total of 1200 frames of fast operation images and 120 frames of slow operation images are extracted. Through the analysis of the SlowFast network, the wearable actions fully meet the safety standards, and the corresponding wearable action safety level of 10 is output. Through the wearable equipment action safety analyzer, potential safety hazards during the operation can be identified in a timely manner, and the corresponding wearable action safety levels can be generated, thereby providing support for the monitoring of high-altitude power operations, ensuring the safety of high-altitude operations, and reducing accident risks.

[0041] Among them, based on the slowfast network, the specific steps of pre-training a wearable equipment action safety analyzer for identifying the safety level of wearing actions are as follows: first, collect image acquisition data of high-altitude operations of power grids in the past period of time, that is, historical time, and sample them separately according to the first extraction step size and the second extraction step size using Haar wavelet transform downsampling, so that multiple sample fast operation image sets and multiple sample slow operation image sets with corresponding relationships can be obtained. At the same time, professional and technical personnel in this field combine the standard degree of wearable equipment operation actions corresponding to the image acquisition data of high-altitude operations of power grids in the historical time period to label the wearing action safety level and obtain a sample wearing action safety level set. The safety level labeling is performed by professional and technical personnel in this field according to the degree of compliance of the operator's actions in the image acquisition data with the safety standards. For example, if the use of the safety belt meets the standards and the operation is standardized, it can be labeled as a high safety level (such as 9); if the safety belt is not used in a standardized manner or the fall protection device is not used correctly, it may be labeled as a low level (such as 1).

[0042] Haar wavelet transform downsampling is to perform Haar wavelet transform first and then sample according to the first decimation step and the second decimation step. Haar wavelet transform is a commonly used discrete wavelet transform. Haar wavelet transform can be used to reduce the spatial resolution of feature maps while retaining information integrity to the maximum extent. Its principle is to decompose the signal into components of different frequencies and can effectively perform data compression and feature extraction. The core of Haar wavelet transform is to decompose the signal into two parts: approximation and detail. This process is achieved through a series of low-pass and high-pass filters. In Haar wavelet transform downsampling, simple average filtering (low-pass) and difference filtering (high-pass) are used. The decomposition formula used is as follows: , ;in It is The approximation coefficient of the layer, It is The approximation coefficient of the layer, It is The detail factor of the layer, It is in Index into the array of approximation coefficients for the layer.

[0043] The reconstruction formula is as follows: , . This means that the length is The signal can be divided into The two parts can be respectively interpreted as a low-pass filter and a high-pass filter. When the Haar wavelet transform is applied to two-dimensional signals such as grayscale images, four components will be generated, each component having half the spatial resolution of the original signal, while the number of channels of the feature map increases fourfold. In other words, the Haar wavelet transform can encode some information in the spatial dimension into the channel dimension without losing any information. During the sampling process, the image is downsampled by the Haar wavelet transform. Through this transformation method, the image can reduce redundant data while retaining key action information, improving the computational efficiency. Next, the fast job image set and the slow job image set are processed separately, and the safety level of the wearing actions in each image sequence is labeled.

[0044] Next, the slowfast network is used to construct the network structure of the wearable equipment action safety analyzer. The slowfast network is a deep learning network designed specifically for action recognition, which improves the recognition accuracy by processing fast-paced and slow-paced action features simultaneously. The fast path is used to identify the detailed changes in actions during the operation, such as the hand operations completed by the operator within a few milliseconds. The slow path is responsible for analyzing the overall action trend over a long period, such as the adjustment of the operator's body posture. Exemplarily, the slow path processes video frames at a lower sampling rate and uses 3D convolutional layers to extract features in both the temporal and spatial dimensions simultaneously. In the high-altitude operation scenario, such convolutional layers can detect macroscopic actions such as the operator's seat belt wearing situation and overall posture changes. To further improve the training effect of the network, residual connections are usually added to the convolutional layers of the slow path, which helps to capture subtle changes over a long period and prevent information loss. The main purpose of the fast path is to capture detailed actions, using a higher frame rate and lightweight convolutional operations to focus on the rapidly changing details. The fast path uses smaller convolutional kernels and fewer channels so that it can extract detailed features more quickly. The fast path contains pooling layers to aggregate local detailed features and at the same time reduce the size of the feature map, reducing the computational complexity. In the slowfast network structure, the slow path and the fast path will perform feature fusion regularly. The multi-scale fusion layer can effectively integrate the features of the fast and slow paths, retaining both the overall trend information of the action and capturing the subtle changes in the action. By fusing the feature maps of the fast and slow paths in multiple network layers, information at different time scales can be combined. In each fusion, the feature map of the slow path will be added to the fast path to enhance the fast path's perception of macroscopic information. After multi-scale fusion, the features will be input into the final classification layer for outputting the safety level of the wearing action. The classification layer uses the Softmax activation function to output the probability distribution of different safety levels, and the safety level with the highest probability will be used as the output.

[0045] After the network structure of the wearable equipment operation safety analyzer is constructed, multiple sets of sample fast-operation images, multiple sets of sample slow-operation images, and the corresponding set of sample wearable action safety levels are input into the SlowFast network for training. During the training process, the network parameters are repeatedly adjusted until the network can accurately predict the safety level of the wearable action. As the training progresses, the network will gradually converge, meaning that it can accurately judge the safety of the operation action through the input fast and slow rhythm image sequences. For example, 500 hours of power grid high-altitude operation image data in the past year are collected. After processing, 20,000 sets of fast-operation images and 20,000 sets of slow-operation images are obtained. Through the annotation of these image sets and input into the SlowFast network for training, the recognition accuracy of the network is finally increased to more than 90%, which can effectively predict the safety level of the wearable action. After the training is completed, the wearable equipment operation safety analyzer can evaluate the safety of each action in high-altitude operations in real time, thus providing more reliable safety protection for on-site operations.

[0046] Further, step three of the present application includes: according to the image acquisition data of the power grid high-altitude operation within the historical time, collecting a set of sample operation images, and annotating to obtain a set of sample wearable safety levels according to the standard degree of the wearable equipment worn in each sample operation image; using the set of sample operation images and the set of sample wearable safety levels, and using a convolutional neural network to train the wearable safety analyzer; inputting each operation image in the multiple sets of operation images into the wearable safety analyzer, and outputting to obtain multiple sets of wearable safety levels.

[0047] Specifically, the step of separately analyzing the wearing safety of the wearable equipment for each operation image in the multiple sets of operation images to obtain multiple sets of wearable safety levels includes: in the monitoring of power high-altitude operations, based on the image acquisition data within the historical time, constructing a safety analysis model for the wearable equipment. First, collect sample images of multiple historical time periods in the power grid high-altitude operation. These image sets are called the set of sample operation images. Each sample image records the situation of the operator wearing the equipment, including the wearing status of key safety equipment (such as safety belts, anti-fall devices, etc.). In order to establish an effective analysis model, professional technical personnel in the field analyze the status of the wearable equipment for each image, and annotate the corresponding safety level according to the standard degree of the wearable equipment to form a set of sample wearable safety levels. For example, if the safety belt in the image is worn in a standard manner and the anti-fall device is correctly fixed, a higher safety level (such as level 9 to 10) is annotated for this image; if the equipment wearing in some images does not meet the standard, such as the safety belt is not fastened tightly or the position of the anti-fall device is inappropriate, a lower safety level (such as 1) will be annotated. These annotated level sets constitute the set of sample wearable safety levels and serve as the labels in the training dataset.

[0048] Next, a wearable safety analyzer is constructed using a convolutional neural network and trained with the input sample job graph set and sample wearable safety level set. A convolutional neural network is a deep learning algorithm particularly suitable for image recognition tasks. Its core is to extract key features in images through multiple layers of convolution and pooling operations, thereby achieving the recognition of the status of wearable equipment. A convolutional neural network includes a convolutional layer, a pooling layer, an activation function layer, a fully connected layer, and an output layer. For example, a typical structure of a wearable safety analyzer based on a convolutional neural network is as follows: Input layer: a color image with a size of 32x32x3; Convolutional layer 1: 32 3x3 filters, a stride of 1, and a ReLU activation function; Pooling layer 1: 2x2 max pooling, a stride of 2; Convolutional layer 2: 64 3x3 filters, a stride of 1, and a ReLU activation function; Pooling layer 2: 2x2 max pooling, a stride of 2; Flattening: flatten the feature map into a one-dimensional vector; Fully connected layer 1: 128 neurons, a ReLU activation function; Fully connected layer 2: 64 neurons, a ReLU activation function; Output layer: 10 neurons, a Softmax activation function, for a 10-classification task.

[0049] During the training process, each sample image in the sample job graph set is input into the convolutional neural network. The network will extract the safety equipment features in the image layer by layer, such as the closing condition of the seat belt buckle, the position of the fall arrest device, etc. Then, the output of the network is compared with the corresponding sample wearable safety level in the sample wearable safety level set, and the model is optimized by adjusting the network parameters, enabling it to gradually learn the features for recognizing the safety status of the equipment. When the training reaches a certain accuracy, that is, the predicted output of the network can accurately correspond to the actual labeled safety level, the training process is completed. After the wearable safety analyzer is trained, each job image in multiple job image sets during actual operations can be input into the wearable safety analyzer to obtain multiple wearable safety level sets, which can achieve real-time monitoring and risk identification of the equipment wearing safety of high-altitude power operation personnel and effectively ensure operation safety.

[0050] Furthermore, this application also includes the following steps: calculate the mean values of the multiple wearable safety level sets respectively to obtain multiple average wearable safety levels; randomly select multiple first wearable safety levels from the first wearable safety level set within the multiple wearable safety level sets, calculate the deviation amplitudes between the multiple first wearable safety levels and the first average safety wearable level respectively and calculate the mean value to obtain the instability coefficient of the first wearable safety level; use the difference obtained by subtracting the instability coefficient of the first wearable safety level from 1, multiply it by the first average wearable safety level, perform a correction calculation on the wearable safety level to obtain the first recognized wearable safety level; continue to perform correction calculations on the other multiple average wearable safety levels to obtain multiple first recognized wearable safety levels.

[0051] Specifically, after obtaining multiple sets of wearable safety levels, it is necessary to perform wearable safety stability analysis and calculation respectively to obtain multiple recognized wearable safety levels. The specific steps are as follows: For each set of wearable safety levels, calculate the mean value to obtain multiple average wearable safety levels. The mean value represents the overall safety level of the wearable equipment within the entire set of operation images. For example, if a set of wearable safety levels contains 10 scores (e.g., 9, 8, 9, 7, 8, 9, 10, 8, 7, 9), then its average wearable safety level is 8.4, and this value reflects the overall situation of wearable safety within the set of operation images. Next, randomly select several first wearable safety levels within the first set of wearable safety levels, calculate the difference between the several first wearable safety levels and the first average safe wearable level, and then calculate the ratio of the difference to the first average safe wearable level as the deviation amplitude. After obtaining several deviation amplitudes, perform a mean value calculation to evaluate the instability of safety and generate the first wearable safety level instability coefficient. The larger the deviation amplitude, the greater the difference in the safety levels of the wearable equipment in different operation images, and the greater the degree of instability of the wearable safety level. It is possible that the wearable safety level is high at the current moment and low at the next moment. Among them, the first set of wearable safety levels generally refers to any one of the multiple sets of wearable safety levels, the multiple first wearable safety levels refer to the elements within the first set of wearable safety levels, and the first average safe wearable level refers to the average wearable safety level corresponding to the first set of wearable safety levels among the multiple average wearable safety levels.

[0052] The larger the first wearable safety level instability coefficient, the greater the fluctuation of the safety levels within the set, that is, the safety of the wearable equipment is relatively unstable. After obtaining the first wearable safety level instability coefficient, a wearable safety level correction calculation will be performed to further reflect the actual wearable safety situation. The formula for the correction calculation is: Subtract the first wearable safety level instability coefficient from 1, and then multiply by the average wearable safety level to obtain the first recognized wearable safety level. The corrected recognized wearable safety level can more accurately reflect the overall safety level of the wearable equipment. Using the same method, traverse the multiple sets of wearable safety levels to obtain multiple recognized wearable safety levels.

[0053] Furthermore, step four of this application includes: calculating the ratio of each recognized wearable safety level to the mean value of the multiple recognized wearable safety levels as multiple wearable safety coefficients; using the multiple wearable safety coefficients to perform product correction calculations on the multiple wearable action safety levels respectively to obtain multiple corrected wearable action safety levels; calculating the mean value of the multiple corrected wearable action safety levels to obtain the real-time wearable equipment safety level as the result of the high-altitude operation monitoring image processing.

[0054] Specifically, first calculate the mean of multiple recognized wearing safety levels. Then, for each recognized wearing safety level, calculate the ratio of it to the mean respectively to obtain multiple wearing safety factors. The wearing safety factor reflects the relative relationship between the wearing safety condition at a certain moment and the overall safety level. Therefore, the relative safety of the recognized wearing safety levels below the mean is slightly lower, while the safety of the recognized wearing safety levels above the mean is relatively higher. Next, use the wearing safety factors to correct each wearing action safety level. The correction calculation method is to multiply each wearing safety factor by the corresponding wearing action safety level to obtain multiple corrected wearing action safety levels. This method can dynamically adjust the wearing action safety level and reflect the adjustment of the safety state at that moment relative to the overall safety level. Example: If a wearing action safety level is 8.5 and the corresponding wearing safety factor is 0.875, then the corrected wearing action safety level is 8.5×0.87 = 7.44. Finally, calculate the average of the multiple corrected wearing action safety levels to obtain the real-time wearing equipment safety level. The real-time wearing equipment safety level is the final safety score, which is used for the overall safety monitoring of the operation site, reflects the overall safety condition of the wearing equipment during the current operation, and can be fed back to the monitoring system in real time so that safety management personnel can timely grasp the safety situation of the operators. Through this correction method based on recognition and action levels, it can more accurately reflect the equipment safety of the operators in different states, provide a more valuable real-time safety score for high-altitude operation safety monitoring, and facilitate timely taking of protective measures.

[0055] In summary, the power high-altitude operation image processing method provided by this application has the following technical effects: In power high-altitude operations, multiple-angle image acquisition of the operators is performed through an image acquisition device to obtain multiple sets of operation images including the operators and wearing equipment, and multiple operation image sequences are constructed; the safety of the wearing equipment operation actions in high-altitude operations is analyzed for the multiple operation image sequences respectively to obtain multiple wearing action safety levels; the wearing safety of the wearing equipment is analyzed separately for each operation image in the multiple sets of operation images to obtain multiple sets of wearing safety levels, and the wearing safety stability analysis and calculation are performed respectively to obtain multiple recognized wearing safety levels; the real-time wearing equipment safety level is calculated based on the multiple recognized wearing safety levels and the multiple wearing action safety levels and used as the image processing result of high-altitude operation monitoring. Through the multi-angle image acquisition device and image processing technology, the wearing actions of the operators and the equipment wearing conditions are analyzed in real time, and dynamic safety level evaluation and warning prompts are provided, thereby greatly improving the safety monitoring level of power high-altitude operations and reducing the risk of safety accidents caused by improper equipment wearing.

[0056] Embodiment 2. Based on the power high-altitude operation image processing method in the foregoing embodiment and with the same inventive concept, the present application further provides a power high-altitude operation image processing system. Please refer to the attached Figure 2 , the power high-altitude operation image processing system includes: an image acquisition module 11, configured to collect images of the operator from multiple angles through an image acquisition device during power high-altitude operations, obtain multiple sets of operation images including the operator and the wearable equipment, and construct multiple operation image sequences.

[0057] An action safety analysis module 12, configured to separately analyze the safety of the wearable equipment operation actions in the multiple operation image sequences during high-altitude operations, and obtain multiple wearable action safety levels.

[0058] A wearable safety individual analysis module 13, configured to separately analyze the wearable safety of the wearable equipment for each operation image in the multiple sets of operation images, obtain multiple sets of wearable safety levels, and separately perform wearable safety stability analysis calculations to obtain multiple recognized wearable safety levels.

[0059] A safety level acquisition module 14, configured to calculate and obtain a real-time wearable equipment safety level based on the multiple recognized wearable safety levels and the multiple wearable action safety levels, as the high-altitude operation monitoring image processing result.

[0060] Further, the image acquisition module 11 in the power high-altitude operation image processing system is further configured to: during power high-altitude operations, collect images of the operator from multiple angles through an image acquisition device, and count the operation images within the past P time instants to obtain multiple sets of operation images, where P is an integer greater than 1; arrange the multiple sets of operation images according to the timestamps of the P time instants to obtain multiple operation image sequences.

[0061] Further, the action safety analysis module 12 in the power high-altitude operation image processing system is further configured to: based on the slowfast network, pre-train a wearable equipment action safety analyzer for identifying the wearable action safety level; separately extract operation images from the multiple operation image sequences according to a first extraction step size, and perform Haar wavelet transform downsampling processing to obtain multiple sets of fast operation images; separately extract operation images from the multiple operation image sequences according to a second extraction step size, where the second extraction step size is greater than the first extraction step size, to obtain multiple sets of slow operation images; respectively combine the multiple sets of fast operation images and multiple sets of slow operation images in a corresponding manner, and input them into the wearable equipment action safety analyzer to obtain multiple wearable action safety levels.

[0062] Further, the action safety analysis module 12 in the power high-altitude operation image processing system is further configured to: according to the image acquisition data of power grid high-altitude operations within a historical time, collect and process to obtain multiple sample fast-operation image sets and multiple sample slow-operation image sets according to the first extraction step, Haar wavelet transform downsampling, and the second extraction step, and label and obtain a sample wearing action safety level set according to the standard degree of the wearing equipment operation actions; based on the slowfast network, construct the network structure of the wearing equipment action safety analyzer; use the multiple sample fast-operation image sets, multiple sample slow-operation image sets, and the sample wearing action safety level set to train the wearing equipment action safety analyzer until convergence, and complete the training.

[0063] Further, the wearing safety independent analysis module 13 in the power high-altitude operation image processing system is further configured to: according to the image acquisition data of power grid high-altitude operations within a historical time, collect a sample operation image set, and label and obtain a sample wearing safety level set according to the standard degree of wearing equipment in each sample operation image; use the sample operation image set and the sample wearing safety level set to train the wearing safety analyzer by using a convolutional neural network; input each operation image in the multiple operation image sets into the wearing safety analyzer, and output to obtain multiple wearing safety level sets.

[0064] Further, the wearing safety independent analysis module 13 in the power high-altitude operation image processing system is further configured to: respectively calculate the means of the multiple wearing safety level sets to obtain multiple average wearing safety levels; randomly extract multiple first wearing safety levels from the first wearing safety level set in the multiple wearing safety level sets, respectively calculate the deviation amplitudes between the multiple first wearing safety levels and the first average safety wearing level and calculate the mean value to obtain the first wearing safety level instability coefficient; use the difference obtained by subtracting the first wearing safety level instability coefficient from 1, multiply it by the first average wearing safety level, perform a correction calculation on the wearing safety level, and obtain the first recognized wearing safety level; continue to perform correction calculations on the other multiple average wearing safety levels to obtain multiple first recognized wearing safety levels.

[0065] Further, the safety level acquisition module 14 in the power high-altitude operation image processing system is further configured to: calculate the ratios of each recognized wearing safety level to the mean value of the multiple recognized wearing safety levels as multiple wearing safety factors; use the multiple wearing safety factors to respectively perform correction calculations on the products of the multiple wearing action safety levels to obtain multiple corrected wearing action safety levels; calculate the mean value of the multiple corrected wearing action safety levels to obtain the real-time wearing equipment safety level, which is used as the high-altitude operation monitoring image processing result.

[0066] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The Figure 1 The power high-altitude operation image processing method and specific examples in the first embodiment are equally applicable to the power high-altitude operation image processing system in this embodiment. Through the detailed description of the power high-altitude operation image processing method above, those skilled in the art can clearly understand the power high-altitude operation image processing system in this embodiment. Therefore, for the sake of brevity of the specification, it will not be elaborated here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple. For the relevant parts, refer to the description in the method section.

[0067] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but rather will be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0068] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalent technologies, the present application also intends to include these changes and modifications.

Claims

1. A method for processing images of electric power aerial work, characterized in that: The electric power aerial work image processing method comprises: In the high-altitude power operation, the image acquisition equipment is used to acquire images of the operators from multiple angles, obtain multiple sets of operation images including the operators and the equipment they wear, and construct multiple operation image sequences; Performing safety analysis on the multiple operation image sequences of the operation actions of wearing equipment during high-altitude operations to obtain multiple wearing action safety levels; Performing a separate wearing safety analysis of the wearable equipment on each of the plurality of operation image sets to obtain a plurality of wearing safety level sets, and performing a wearing safety stability analysis calculation to obtain a plurality of identified wearing safety levels; Calculate the real-time wearing equipment safety level according to the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude work monitoring image processing result; The safety analysis of the operation actions of wearing equipment in high-altitude operations is performed on the multiple operation image sequences respectively to obtain multiple wearing action safety levels, including: Based on the slowfast network, a wearable device action safety analyzer is pre-trained to identify the safety level of wearing actions; Extracting operation images from the plurality of operation image sequences according to a first extraction step length, and performing Haar wavelet transform down-sampling processing to obtain a plurality of fast operation image sets; Extracting job images from the plurality of job image sequences respectively according to a second extraction step length to obtain a plurality of slow job image sets, wherein the second extraction step length is greater than the first extraction step length; The plurality of fast operation image sets and the plurality of slow operation image sets are respectively combined correspondingly, and input into the wearing equipment action safety analyzer to obtain a plurality of wearing action safety levels; Perform wear safety stability analysis and calculations respectively to obtain multiple identification wear safety levels, including: Calculating the averages of the plurality of wearing safety level sets respectively to obtain a plurality of average wearing safety levels; Randomly extracting a plurality of first wearing safety levels from a first wearing safety level set within the plurality of wearing safety level sets, respectively calculating deviation amplitudes of the plurality of first wearing safety levels and a first average safety wearing level and calculating an average, to obtain an instability coefficient of the first wearing safety level; The difference between 1 and the instability coefficient of the first wearing safety level is multiplied by the first average wearing safety level to perform correction calculation of the wearing safety level to obtain a first identified wearing safety level; Continue to perform correction calculation on other multiple average wearing safety levels to obtain multiple first identified wearing safety levels; The real-time wearable equipment safety level is calculated according to the multiple identification wear safety levels and the multiple wear action safety levels, including: Calculating a ratio of each identified wearing safety level to an average of the plurality of identified wearing safety levels as a plurality of wearing safety coefficients; Using the multiple wearing safety factors, respectively multiplying and correcting the products of the multiple wearing action safety levels to obtain multiple corrected wearing action safety levels; The average of the multiple corrected wearing action safety levels is calculated to obtain the real-time wearing equipment safety level as the high-altitude work monitoring image processing result.

2. The method for processing images of power aerial work according to claim 1, characterized in that: In the high-altitude operation of electric power, the image acquisition equipment is used to collect images of the operators from multiple angles, obtain multiple operation image sets including the operators and the equipment they wear, and construct multiple operation image sequences, including: In the electric power aerial work, the image acquisition equipment is used to collect images of the workers from multiple angles, and the work images in the past P moments are counted to obtain multiple work image sets, where P is an integer greater than 1; The multiple job image sets are arranged according to the timestamps of the P moments to obtain multiple job image sequences.

3. The method for processing images of power aerial work according to claim 1, characterized in that: Based on the slowfast network, a wearable device motion safety analyzer is pre-trained to identify the safety level of wearable motions, including: Based on the image acquisition data of the high-altitude operation of the power grid in the historical time, according to the first extraction step size, Haar wavelet transform down sampling and the second extraction step size, the acquisition and processing are performed to obtain a plurality of sample fast operation image sets and a plurality of sample slow operation image sets, and according to the standard degree of the operation action of the wearing equipment, the sample wearing action safety level set is annotated to obtain; Based on the slowfast network, a network structure of the wearable equipment action safety analyzer is constructed; The plurality of sample fast operation image sets, the plurality of sample slow operation image sets and the sample wearing action safety level sets are used to train the wearing equipment action safety analyzer until convergence, thereby completing the training.

4. The method for processing images of power aerial work according to claim 1, characterized in that: Performing a separate analysis of the wearing safety of the wearable equipment on each operation image in the plurality of operation image sets, to obtain a plurality of wearing safety level sets, including: Based on the image acquisition data of high-altitude operations of power grids in historical time, a set of sample operation images is collected, and according to the standard degree of wearing equipment in each sample operation image, a set of sample wearing safety levels is obtained by annotation; Using the sample operation graph set and the sample wearing safety level set, and using a convolutional neural network, training a wearing safety analyzer; Each operation image in the plurality of operation image sets is input into the wearing safety analyzer, and a plurality of wearing safety level sets are obtained as output.

5. The image processing system for electric power aerial work is characterized by: The steps for implementing the method for processing images of power aerial work according to any one of claims 1 to 4, the power aerial work image processing system comprising: The image acquisition module is used to acquire images of operators from multiple angles through image acquisition equipment during high-altitude power operations, obtain multiple sets of operation images including operators and equipment worn, and construct multiple operation image sequences; An action safety analysis module, used to respectively analyze the safety of the action of wearing equipment in the aerial work on the plurality of operation image sequences, and obtain a plurality of wearing action safety levels; A separate wearing safety analysis module, for performing separate wearing safety analysis of the wearable equipment on each operation image in the plurality of operation image sets, obtaining a plurality of wearing safety level sets, and performing wearing safety stability analysis calculations on each of the plurality of operation image sets, obtaining a plurality of identified wearing safety levels; The safety level acquisition module is used to calculate the real-time safety level of the worn equipment according to the multiple identification wearing safety levels and the multiple wearing action safety levels as the high-altitude work monitoring image processing result.

Images