A method and system for intelligent monitoring and alarm of power operation deployment status based on image recognition and multi-feature fusion

Abstract

This invention discloses an intelligent monitoring and alarm method and system for the status of power operation deployment ball surveillance cameras based on image recognition and multi-feature fusion. It performs initial alarm judgment based on a power operation deployment ball surveillance status classification model, and secondary alarm judgment based on the proportion of ground area in the real-time image, the proportion of the lower body of personnel at the highest point of the image to the overall height of the image, and corresponding thresholds. This method features short anomaly response time and high anomaly recognition accuracy, effectively avoiding safety accidents caused by blind spots in monitoring. Furthermore, this method eliminates the need for continuous intervention from remote supervisors, reducing labor costs. It provides a distortion-free, clearly defined image foundation for intelligent analysis of operational behavior, comprehensively enhancing the effectiveness of power safety supervision.

Description

Technical Field

[0001] This invention belongs to the field of power operation alarm, and in particular relates to a method and system for intelligent monitoring and alarm of power operation deployment ball status based on image recognition and multi-feature fusion. Background Technology

[0002] Currently, safety supervision at power operation sites mainly relies on manual inspections and fixed monitoring equipment. Manual inspections suffer from low efficiency, high cost, and limited coverage, and are easily affected by personnel fatigue or negligence. While fixed monitoring equipment can provide continuous monitoring, its fixed installation location makes it difficult to adapt to complex and changing working environments. In recent years, the application of mobile surveillance cameras has improved monitoring flexibility, but in actual operation, improper installation (such as placing the camera directly on the ground without a tripod) often results in the camera's viewing angle being too low, only capturing the feet of personnel or a small part of the ground, failing to cover the entire operation.

[0003] In existing technologies, some solutions involve intervention through remote manual supervision or periodic spot checks of surveillance footage, but these suffer from slow response times and high labor costs. Furthermore, traditional image processing techniques rely solely on single features (such as ground area ratio) for analysis, making them prone to misjudgments in complex ground textures (such as grass or gravel) or densely populated scenes. They also fail to consider other features, such as people in the image, resulting in a high false alarm rate.

[0004] Therefore, there is an urgent need for a method and system for intelligent monitoring and alarm of the status of power operation deployment ball that integrates multi-dimensional features and adapts to complex scenarios, so as to improve the effectiveness of power operation safety supervision. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention proposes an intelligent monitoring and alarm method and system for the status of power operation control ball based on image recognition and multi-feature fusion. This system enables accurate judgment and real-time alarm of the installation status of the control ball, reduces manual intervention, and improves the accuracy of edge image recognition.

[0006] The technical solution of the present invention is as follows:

[0007] A method for intelligent monitoring and alarming of the status of power operation control spheres based on image recognition and multi-feature fusion, wherein the status of the control spheres includes a standard status (installed according to preset specifications) and a non-standard status (installed according to non-preset specifications), including:

[0008] Image samples were collected from surveillance cameras at different power work sites under standard and non-standard installation conditions. Key features of the ground area and lower body features of personnel in the image samples were marked to form multi-feature fusion data of the image samples.

[0009] A power operation deployment ball status classification model is constructed, using the multi-feature fusion data as input data and the deployment ball status corresponding to the image sample as output data, and the power operation deployment ball status classification model is trained.

[0010] The initial alarm judgment is made based on the real-time image output of the power operation control ball status according to the power operation control ball status classification model. If the output control ball status is a standard status, no alarm is issued; if the output control ball status is a non-standard status, a secondary alarm judgment is made based on the proportion of the ground area in the real-time image, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

[0011] Furthermore, the different power operation sites include at least different lighting conditions, different ground materials, and different personnel activity states.

[0012] Furthermore, the key features of the ground area include at least texture features, color distribution, and edge shape; the lower body features include at least the coordinates of key points consisting of the hip joint, knee joint, ankle joint, trousers, and shoes.

[0013] Furthermore, the texture features are quantized using a gray-level co-occurrence matrix to determine the ground granularity; the color distribution is obtained by extracting the color difference between the ground and the work equipment in the HSV color space; and the edge shape is determined by using an improved Canny algorithm with adaptive multi-scale bilateral filtering preprocessing, combined with the edge-preserving properties of bilateral filtering, to suppress noise while retaining edge details and detect the boundary line between the ground and the work area.

[0014] Furthermore, the lower body features are identified using the OpenPose algorithm, including the coordinates of key points such as the hip joint, knee joint, ankle joint, pants, and shoes.

[0015] Furthermore, the power operation deployment ball status classification model uses ResNet-50 as the backbone network and combines it with transfer learning methods to optimize model training efficiency.

[0016] Furthermore, the specific method for making secondary alarm judgments based on the proportion of the ground area in the real-time image, the proportion of the lower body of the person at the highest point of the image to the overall height of the image, and the corresponding thresholds is as follows:

[0017] Set dynamic dual thresholds, which include a ground area ratio threshold that considers the influence of the scene and a ratio threshold that considers the influence of the person's posture that considers the ratio of the person's lower body at the highest point of the image to the overall height of the image.

[0018] The system compares the proportion of the ground area in the real-time image with the ground area proportion threshold, and compares the proportion of the lower body of a person in the real-time image at the highest point of the image to the overall height of the image with the lower body proportion threshold. If the ground area proportion is greater than the ground area proportion threshold, or the lower body proportion of a person in the real-time image at the highest point of the image to the overall height of the image is greater than the lower body proportion threshold, an alarm is triggered.

[0019] Furthermore, the alarm includes transmitting the secondary alarm judgment process as the alarm cause, combined with rectification suggestions, to the operation and maintenance personnel using a multi-channel alarm method.

[0020] Furthermore, the method also includes performing three alarm judgments based on the proportion of the ground area in the image uploaded by the maintenance personnel after alarm rectification, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

[0021] A smart monitoring and alarm system for the status of power operation deployment ball based on image recognition and multi-feature fusion, wherein the status of the deployment ball includes a standard status installed according to a preset specification and a non-standard status that is different from the preset specification, and includes a multi-feature fusion data formation module, a model building and training module and an alarm judgment module;

[0022] The multi-feature fusion data forming module is used to acquire image samples collected by the deployment ball in standard and non-standard installation states at different power operation sites, and to mark the key features of the ground area and the lower body features of the personnel in the image samples to form multi-feature fusion data of the image samples.

[0023] The model building and training module is used to build a power operation deployment ball status classification model. It uses the multi-feature fusion data as input data and the deployment ball status corresponding to the image sample as output data to train the power operation deployment ball status classification model.

[0024] The alarm judgment module is used to make an initial alarm judgment based on the real-time image output of the control ball corresponding to the power operation control ball status classification model. If the output control ball status is a standard status, no alarm is triggered; if the output control ball status is a non-standard status, a secondary alarm judgment is made based on the proportion of the ground area in the real-time image, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] This invention proposes an intelligent monitoring and alarm method and system for the status of power operation deployment spheres based on image recognition and multi-feature fusion. It uses a power operation deployment sphere status classification model for initial alarm judgment, and then performs secondary alarm judgment based on the proportion of ground area in the real-time image, the proportion of the lower body of personnel at the highest point of the image to the overall height of the image, and corresponding thresholds. This method features short anomaly response time and high anomaly recognition accuracy, effectively avoiding safety accidents caused by monitoring blind spots. Furthermore, this method eliminates the need for continuous intervention from remote supervisors, reducing labor costs. It provides a distortion-free, clearly defined image foundation for intelligent analysis of operational behavior, comprehensively enhancing the effectiveness of power safety supervision.

[0027] The method of this invention integrates key features of the ground area with the lower body features of personnel to form multi-feature fusion data and trains a power operation deployment ball status classification model, which significantly reduces the false alarm rate in complex scenarios and effectively overcomes the failure problems of traditional single feature analysis such as grass texture interference and rain reflection.

[0028] The present invention employs a dynamic dual-threshold mechanism for secondary alarm judgment. The dynamic dual thresholds include a ground area ratio threshold that considers the influence of the scene and a ratio threshold that considers the influence of the person's posture, which is the ratio of the person's lower body at the highest point of the screen to the overall height of the screen. This enables the secondary judgment to adapt to environmental changes, reduce the false alarm rate, and avoid misjudgments caused by people temporarily crouching or inclement weather.

[0029] The edge computing optimization scheme of this invention also adopts an alarm closed-loop management mechanism. The secondary alarm judgment process is used as the alarm cause and combined with the rectification suggestions to transmit to the operation and maintenance personnel through a multi-channel alarm method. Then, the proportion of the ground area in the image uploaded by the operation and maintenance personnel after the alarm rectification, the proportion of the lower body of the person in the highest point of the image to the overall height of the image, and the corresponding threshold are used for a third alarm judgment. This improves the rectification qualification rate and forms a complete supervision closed loop of "monitoring-feedback-verification".

[0030] The method of this invention supports integration with the power operation safety supervision platform to realize alarm data visualization, historical record tracing and big data analysis. Attached Figure Description

[0031] Figure 1 This is an example image of the control ball in a standard state according to the present invention (the ground occupies a small proportion, and the lower body is in the lower part of the field of view of the image);

[0032] Figure 2 This is an example image of the control ball in a non-standard state (the ground occupies a large proportion, and the lower body is in the upper part of the field of view).

[0033] Figure 3 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0034] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0035] Example 1:

[0036] This invention discloses an intelligent monitoring and alarm method for the status of power operation control cameras based on image recognition and multi-feature fusion. The status of the control cameras includes the standard status of installation according to preset specifications (e.g., ...). Figure 1 (as shown) and non-standard installation states that differ from the preset specifications (such as...) Figure 2 As shown), Figure 3 As shown, the method includes the following steps:

[0037] S1. Acquire image samples from the surveillance cameras at different power operation sites under standard and non-standard installation conditions, mark the key features of the ground area and the lower body features of personnel in the image samples, and form multi-feature fusion data of the image samples.

[0038] S2. Construct a power operation deployment ball status classification model, using multi-feature fusion data as input data and the deployment ball status corresponding to the image sample as output data, and train the power operation deployment ball status classification model.

[0039] S3. Based on the power operation deployment ball status classification model, the deployment ball status output from the real-time image collected by the deployment ball is used for the initial alarm judgment. If the output deployment ball status is a standard status, no alarm is triggered. If the output deployment ball status is a non-standard status, a secondary alarm judgment is made based on the proportion of the ground area in the real-time image, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

[0040] Furthermore, preset specifications can be set according to the target, for example, the preset specification is to install the control ball on a 1.3 to 1.5-meter tripod.

[0041] Furthermore, an edge computing module can be built into the control sphere to deploy a lightweight model accelerated by TensorRT, supporting real-time processing of video streams at 30 frames per second with a detection latency of ≤150ms.

[0042] Furthermore, the following example illustrates the sample data collection and model training process. 10,000 samples were collected from 10 typical power operation sites (such as high-voltage tower maintenance and substation maintenance), covering different climatic conditions throughout the four seasons. Labeling standards: Pixel-level labeling was performed using Label Studio, with a ground area labeling accuracy error ≤ 5 pixels and a personnel key point labeling error ≤ 2 pixels. Model training: The PyTorch framework was used, with ResNet-50 as the backbone network. The training period was 100 epochs, with an initial learning rate of 0.001, decreasing by 10% every 20 epochs.

[0043] In one embodiment, different power work sites include at least different lighting conditions, different ground materials, and different personnel activity states.

[0044] Furthermore, power operation sites include substations, transmission line maintenance points, and other power operation sites; lighting conditions include strong light, cloudy days, and infrared supplemental lighting at night; ground materials include cement ground, grass, and mud ground; and personnel activity states include standing, squatting, and walking.

[0045] In one embodiment, the key features of the ground area include at least texture features, color distribution, and edge shape; the lower body features include at least the coordinates of key points consisting of the hip joint, knee joint, ankle joint, and trousers, shoes, etc.

[0046] In one embodiment, texture features are quantized using a gray-level co-occurrence matrix (GLCM) to determine ground granularity; color distribution is achieved by extracting the color difference between the ground and the work equipment in the HSV color space; and edge shape is determined using an improved Canny algorithm to address the issue that traditional Canny algorithms, particularly in ground scenes, cannot adapt to multi-scale variations in ground granularity through Gaussian filtering, thus easily blurring true edges. Adaptive multi-scale bilateral filtering preprocessing is employed, combined with the edge-preserving properties of bilateral filtering, to suppress noise while retaining edge details, thereby detecting the boundary line between the ground and the work area.

[0047] Filter parameters are dynamically adjusted based on local variance σ. 2 (x,y) determines the spatial domain standard deviation σ of the bilateral filter. d (x,y) and the standard deviation of the range σ r (x,y):

[0048]

[0049] In the formula, K1 and K2 are scaling coefficients, with values ​​ranging from [0.5 to 2.0], and σ d0 ,σ r0 The base standard deviation is used, with a default value of 1.0. This formula makes the filtering scale adaptively vary with ground grain size: coarse-grained regions (high σ)2 Large-scale filtering is used to suppress noise, especially in fine-textured areas (low σ). 2 Preserve edge details.

[0050] The formula for calculating the filtering result F(x,y) of pixel (x,y) in a grayscale image G(x,y) is as follows:

[0051]

[0052] Where p and q are the coordinate parameters of the pixels in the neighborhood, which are limited to integers and satisfy the following:

[0053] p∈[xk,x+k],q∈[yk,y+k]

[0054] In the formula ( (This is for rounding up), meaning the range of p is k pixels to the left and right of x, and the range of q is k pixels above and below y.

[0055] Where N(x,y) is the (2k+1)×(2k+1) neighborhood centered at (x,y).

[0056] Spatial domain weights Reflects the impact of spatial distance on filtering;

[0057] Range weight This reflects the impact of pixel value similarity on filtering.

[0058] Compared to traditional Gaussian filtering, this filter, while suppressing noise, preserves edges through range-domain weighting (where pixel values ​​differ significantly). r When the value approaches 0, the filtering effect is weak.

[0059] In one embodiment, the lower body features are identified using the OpenPose algorithm, including the coordinates of key points such as the hip, knee, and ankle joints, as well as trousers and shoes. A lightweight OpenPose model is used, retaining only the lower body key point detection branch to reduce computational resource consumption.

[0060] In one embodiment, the power operation deployment ball status classification model uses ResNet-50 as the backbone network and combines it with transfer learning methods to optimize model training efficiency.

[0061] In one embodiment, the specific method for secondary alarm judgment based on the proportion of the ground area in the real-time image, the proportion of the lower body of the person at the highest point of the image to the overall height of the image, and the corresponding threshold is as follows:

[0062] Set dynamic dual thresholds, which include a ground area ratio threshold that considers the scene's influence and a ratio threshold that considers the person's posture that considers the person's lower body at the highest point of the image to the overall height of the image.

[0063] The system compares the proportion of the ground area in the real-time image with a ground area proportion threshold, and compares the proportion of the lower body of a person in the highest point of the image to the overall height of the image with a threshold. If the ground area proportion is greater than the ground area proportion threshold, or the proportion of the lower body of a person in the highest point of the image to the overall height of the image is greater than the threshold, an alarm is triggered.

[0064] Generally, the threshold for the proportion of ground area can be set using the following scheme:

[0065] When installed correctly, the ground area should be small, generally less than or equal to 32%, because the camera looks down at the work area, and the ground area is small.

[0066] When installed improperly, the ground area occupies ≥64% of the frame because the camera is close to the ground, and the ground occupies most of the frame.

[0067] Therefore, the alarm threshold is set to 50%, and dynamic adjustment is allowed according to the scene. For example, the threshold is increased to 55% when there is strong reflection in the rain.

[0068] The following scheme can be used to set the threshold for the proportion of the lower body of a person in the highest point of the image to the overall height of the image:

[0069] When installed correctly, the lower half of the person's body should account for ≤14% of the total height of the image from the highest point of the frame. This is because the camera is positioned high, so the entire human body is captured in the frame.

[0070] When installed improperly, the lower body of the person is at the highest point of the image, accounting for ≥47% of the total height of the image. This is because the camera only captures the legs.

[0071] Therefore, the alarm threshold is set at 30%, and the threshold is temporarily relaxed to 35% when the person is crouching, taking into account their posture.

[0072] In one embodiment, the alarm includes transmitting the secondary alarm judgment process as the alarm cause, combined with rectification suggestions, to the operation and maintenance personnel using a multi-channel alarm method.

[0073] Furthermore, the alarm reason could be something like "Ground coverage exceeds limit: 65%" or "Personnel lower body height is abnormal: 42%".

[0074] Furthermore, rectification suggestions can be pre-set based on the alarm cause, such as "Please use a tripod to raise the equipment to 1.5 meters".

[0075] Furthermore, multiple alarm channels, including SMS, app pop-ups, and voice broadcasts, ensure that maintenance personnel receive alerts in a timely manner.

[0076] In one embodiment, the method further includes performing three alarm judgments based on the proportion of the ground area in the image uploaded by the maintenance personnel after alarm rectification, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

[0077] Example 2:

[0078] The present invention provides an intelligent monitoring and alarm system for the status of power operation deployment ball based on image recognition and multi-feature fusion. The deployment ball status includes a standard status installed according to a preset specification and a non-standard status that differs from the preset specification. The system includes a multi-feature fusion data formation module, a model building and training module, and an alarm judgment module.

[0079] The multi-feature fusion data generation module is used to acquire image samples collected by the deployment ball in different power operation sites under standard and non-standard installation conditions, mark the key features of the ground area and the lower body features of the personnel in the image samples, and combine the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image to form multi-feature fusion data of the image samples.

[0080] The model building and training module is used to build a classification model of the status of the power operation deployment ball. It uses multi-feature fusion data as input data and the deployment ball status corresponding to the image sample as output data to train the power operation deployment ball status classification model.

[0081] The alarm judgment module is used to make an initial alarm judgment based on the real-time image output of the control ball corresponding to the power operation control ball status classification model. If the output control ball status is a standard status, no alarm is issued; if the output control ball status is a non-standard status, a secondary alarm judgment is made based on the proportion of the ground area in the real-time image, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

[0082] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent monitoring and alarming of the status of power operation control spheres based on image recognition and multi-feature fusion, wherein the status of the control spheres includes a standard status (installed according to preset specifications) and a non-standard status (installed according to non-preset specifications), characterized in that, include: Image samples were collected from surveillance cameras at different power work sites under standard and non-standard installation conditions. Key features of the ground area and lower body features of personnel in the image samples were marked to form multi-feature fusion data of the image samples. A power operation deployment ball status classification model is constructed, using the multi-feature fusion data as input data and the deployment ball status corresponding to the image sample as output data, and the power operation deployment ball status classification model is trained. The initial alarm judgment is made based on the real-time image output of the power operation control ball status according to the power operation control ball status classification model. If the output control ball status is a standard status, no alarm is issued; if the output control ball status is a non-standard status, a secondary alarm judgment is made based on the proportion of the ground area in the real-time image, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

2. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 1, characterized in that, The different power operation sites include at least different lighting conditions, different ground materials, and different personnel activity states.

3. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 1, characterized in that, The key features of the ground area include at least texture features, color distribution, and edge shape; the lower body features include at least the coordinates of key points consisting of the hip joint, knee joint, ankle joint, trousers, and shoes.

4. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 3, characterized in that, The texture features are quantified by gray-level co-occurrence matrix to determine ground granularity; the color distribution is obtained by extracting the color difference between the ground and the working equipment in the HSV color space; the edge shape is determined by an improved Canny algorithm, using adaptive multi-scale bilateral filtering preprocessing, combined with the edge-preserving properties of bilateral filtering to suppress noise while retaining edge details, and detecting the boundary line between the ground and the working area.

5. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 3, characterized in that, The lower body features were identified using the OpenPose algorithm, which included the coordinates of key points such as the hip joint, knee joint, ankle joint, pants, and shoes.

6. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 1, characterized in that, The power operation deployment status classification model uses ResNet-50 as the backbone network and combines it with transfer learning to optimize model training efficiency.

7. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 1, characterized in that, The specific method for secondary alarm judgment based on the proportion of ground area in the real-time image, the proportion of the lower body of the person at the highest point of the image to the overall height of the image, and the corresponding threshold is as follows: Set dynamic dual thresholds, which include a ground area ratio threshold that considers the influence of the scene and a ratio threshold that considers the influence of the person's posture that considers the ratio of the person's lower body at the highest point of the image to the overall height of the image. The system compares the proportion of the ground area in the real-time image with the ground area proportion threshold, and compares the proportion of the lower body of a person in the real-time image at the highest point of the image to the overall height of the image with the lower body proportion threshold. If the ground area proportion is greater than the ground area proportion threshold, or the lower body proportion of a person in the real-time image at the highest point of the image to the overall height of the image is greater than the lower body proportion threshold, an alarm is triggered.

8. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 7, characterized in that, The alarm includes transmitting the secondary alarm judgment process as the alarm cause, combined with rectification suggestions, to the operation and maintenance personnel using a multi-channel alarm method.

9. The intelligent monitoring and alarm method for power operation deployment ball status based on image recognition and multi-feature fusion according to claim 8, characterized in that, The method also includes making three alarm judgments based on the proportion of the ground area in the image uploaded by the maintenance personnel after the alarm rectification, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

10. A smart monitoring and alarm system for the status of power operation control spheres based on image recognition and multi-feature fusion, wherein the status of the control spheres includes a standard installation status (installed according to preset specifications) and a non-standard installation status (distinct from the preset specifications), characterized in that... It includes a multi-feature fusion data formation module, a model building and training module, and an alarm judgment module; The multi-feature fusion data forming module is used to acquire image samples collected by the deployment ball in standard and non-standard installation states at different power operation sites, and to mark the key features of the ground area and the lower body features of the personnel in the image samples to form multi-feature fusion data of the image samples. The model building and training module is used to build a power operation deployment ball status classification model. It uses the multi-feature fusion data as input data and the deployment ball status corresponding to the image sample as output data to train the power operation deployment ball status classification model. The alarm judgment module is used to make an initial alarm judgment based on the real-time image output of the control ball corresponding to the power operation control ball status classification model. If the output control ball status is a standard status, no alarm is triggered; if the output control ball status is a non-standard status, a secondary alarm judgment is made based on the proportion of the ground area in the real-time image, the proportion of the lower body of the personnel at the highest point of the image to the overall height of the image, and the corresponding threshold.

Images