A method and related device for monitoring and early warning of the stress state of a telegraph pole

Abstract

The application discloses a power pole stress state monitoring and early warning method and related equipment, first through a shooting device to collect normal and inclined power pole images of different angles, mark the stress value to form a data set, and then preprocess and divide into a training set, a validation set and a test set. Then design a power pole stress state analysis model, including batch normalization fusion, local feature fusion and other modules, respectively realizing feature extraction, fusion and analysis. The training set is used to train the model, and the validation set and the test set are used to verify the effect and performance. In actual application, real-time collected images are input into the model, and the preset threshold is combined for early warning. Compared with the traditional method, the scheme reduces the dependence on sensors and reduces the cost; through image preprocessing, the environmental adaptability is enhanced; the modules are cooperated to improve the detection performance, and complex features are better captured; the model design optimizes the perception ability, reduces the calculation complexity, and realizes efficient and accurate judgment.

Description

Technical Field

[0001] This application relates to the field of utility pole condition monitoring, and more specifically, to a method and related equipment for monitoring and early warning of the stress state of utility poles. Background Technology

[0002] Power poles play a crucial role in power transmission systems, supporting power lines and maintaining their stable operation. Their stress state directly impacts the safety and stability of the transmission corridor. Under the influence of complex power line operating environments and natural disasters, the risk of abnormal stress or damage to power poles increases, especially during severe weather and prolonged periods of excessive load, making them prone to tilting and breakage. Therefore, timely monitoring of abnormal stress on power poles and subsequent maintenance are extremely important for ensuring power safety.

[0003] Existing technologies for monitoring the stress state of utility poles mainly include traditional physical inspection methods and image processing-based visual inspection methods. Traditional physical inspection methods utilize devices such as strain gauges and tilt sensors, which can provide some real-time data, but suffer from high installation and maintenance costs, limited inspection locations, and an inability to comprehensively monitor all parts of the utility pole. Image processing-based visual inspection methods use image acquisition equipment to photograph the utility pole and analyze image features to determine its stress state. This can cover various angles of the pole and reduce installation and maintenance complexity, but it has limitations in image quality, feature extraction, and accurate identification of stress states.

[0004] Traditional monitoring methods still have many shortcomings: First, they are highly dependent on sensors, relying on physical sensors to collect data, which are complex to install, costly, and prone to data loss due to improper sensor placement or malfunction, affecting the accuracy and reliability of detection. Second, they have poor environmental adaptability, as environmental conditions (such as wind speed, temperature, humidity, etc.) can interfere with sensor data, affecting the accuracy of detection results. Third, traditional image detection methods have deficiencies, relying on simple convolution and feature extraction, making it difficult to capture subtle changes in force, lacking multi-scale feature fusion and enhancement, sensitive to environmental changes, easily overlooking image details, and lacking feature enhancement mechanisms, resulting in limited detection accuracy and failing to meet the needs of power systems for real-time and intelligent monitoring.

[0005] Based on this, this application provides a monitoring and early warning scheme for the stress state of utility poles, which avoids the defects of existing monitoring methods and achieves more efficient monitoring of the stress state of utility poles. Summary of the Invention

[0006] This application provides a method and related equipment for monitoring and early warning of the stress state of utility poles, which shows significant beneficial effects in monitoring and early warning of the stress state of utility poles, effectively overcomes the defects of the prior art, and greatly improves the efficiency of monitoring the stress state of utility poles.

[0007] A method for monitoring and early warning of the stress state of utility poles includes:

[0008] Acquire images of normal utility poles and tilted utility poles from different angles captured by a camera, and label the force values ​​of the utility poles in each image to form a utility pole image dataset;

[0009] The utility pole image dataset is subjected to histogram equalization, noise removal, affine transformation, rotation transformation, and noise addition operations, and is divided into training set, validation set, and test set according to the proportions.

[0010] Design a stress state analysis model for utility poles, and train the model using the training set. Validate the training effect and model performance using the validation set and test set, respectively. The stress state analysis model for utility poles includes:

[0011] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0012] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0013] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0014] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0015] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0016] The real-time acquired images of the utility pole to be monitored are input into the trained and validated force state analysis model of the utility pole to obtain the current force state detection result output by the force state analysis model of the utility pole. The model is then combined with the preset threshold to perform early warning detection and output an abnormal early warning signal.

[0017] Optionally, the stress state analysis model of the utility pole is composed of two batch normalization fusion modules, two local feature fusion modules, a self-feedback feature fusion module, an attention-enhanced convolution module, an upsampling module, a stitching module, and a feature map stress analysis module.

[0018] The first batch normalization fusion module performs two convolution operations on the input utility pole image, adds them after batch normalization, performs average pooling and activates them with the sigmoid function, and generates the first fused feature map by dot product with the original input utility pole image.

[0019] The second batch normalization fusion module performs two convolution operations on the first fused feature map, adds them after batch normalization, performs average pooling and activates them with the sigmoid function, and multiplies them with the first fused feature map to generate the second fused feature map.

[0020] The first local feature fusion module performs double convolution operations on the second fused feature map to obtain the first convolution result. After ReLU activation, average pooling and sigmoid activation, the first convolution result is multiplied by the first convolution result to output the first enhanced feature map.

[0021] The self-feedback feature fusion module performs three self-addition operations on the first enhanced feature map, and generates three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the first enhanced feature map and added to generate the self-feedback feature map.

[0022] The attention-enhancing convolutional module enhances the channel and spatial attention of the self-feedback feature map and generates a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0023] The second local feature fusion module performs double convolution operations on the reduced feature map to obtain the second convolution result. After ReLU activation, average pooling and sigmoid activation, the second convolution result is multiplied by the second convolution result to output the second enhanced feature map.

[0024] The upsampling module performs bilinear interpolation upsampling on the second enhanced feature map to generate an upsampled feature map;

[0025] The stitching module stitches the upsampled feature map and the first fused feature map along the channel dimension and adjusts the number of channels to generate a stitched feature map;

[0026] The feature map stress analysis module analyzes the deformation gradient, texture change and edge offset in the stitched feature map, and outputs the stress state detection result.

[0027] Optionally, the processing procedure of the batch normalization fusion module includes:

[0028] The input feature maps are subjected to convolution operations with kernel sizes of 1 and 3 respectively to generate the first convolution feature map and the second convolution feature map;

[0029] The first convolutional feature map and the second convolutional feature map are batch normalized respectively, and then the sum is used to generate the first pooled feature map by average pooling.

[0030] The pooled feature map is activated by sigmoid to generate attention weights;

[0031] The attention weights are multiplied element-wise with the original input feature map to output a fused feature map.

[0032] Optionally, the processing procedure of the local feature fusion module includes:

[0033] A third convolutional feature map is generated by performing a convolution operation with a kernel size of 1 on the input feature map;

[0034] The third convolutional feature map is subjected to a convolution operation with a kernel size of 3 to generate a fourth convolutional feature map, which is then activated by ReLU to generate an activation feature map.

[0035] The activated feature map is subjected to global average pooling, and then activated by sigmoid to generate the first channel weights;

[0036] The fourth convolutional feature map is multiplied element-wise based on the weights of the first channel to output an enhanced feature map.

[0037] Optionally, the processing procedure of the self-feedback feature fusion module includes:

[0038] Perform three increment operations on the input feature map to generate an incremented feature map;

[0039] The self-added feature map is subjected to average pooling and then size compression through a fully connected layer to generate a compressed feature map;

[0040] After ReLU activation, the compressed feature map is expanded by a fully connected layer to generate an expanded feature map.

[0041] The extended feature map is divided into three sub-feature maps along the channel dimension. Each sub-feature map is activated by sigmoid and then multiplied element-wise with the original input feature map. The sub-feature maps are then added together to generate a self-feedback feature map.

[0042] Optionally, the processing procedure of the attention-enhanced convolutional module includes:

[0043] The input feature map is subjected to global average pooling, and then the weights of the second channel are generated by passing through a fully connected layer and sigmoid activation.

[0044] Channel compression is performed on the input feature map, and spatial weights are generated through convolution and sigmoid activation;

[0045] The second channel weight is multiplied element-wise with the spatial weight to generate a double-enhanced feature map;

[0046] The dual-enhanced feature map is subjected to convolutional dimensionality reduction to output a dimensionality-reduced feature map.

[0047] A power pole stress state monitoring and early warning device, comprising:

[0048] The data acquisition unit is used to acquire images of normal utility poles and tilted utility poles from different angles captured by the camera, and to label the force values ​​of the utility poles in each image to form a utility pole image dataset.

[0049] The data classification unit is used to perform histogram equalization, noise removal, affine transformation, rotation transformation and noise addition operations on the utility pole image dataset, and divide it into training set, validation set and test set according to the proportion.

[0050] A model training unit is used to design a stress state analysis model for utility poles, train the model using the training set, and verify the training effect and model performance using the validation set and the test set, respectively. The stress state analysis model for utility poles includes:

[0051] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0052] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0053] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0054] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0055] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0056] The state detection unit is used to input the real-time acquired images of the utility pole to be monitored into the trained and verified force state analysis model of the utility pole, obtain the current force state detection result output by the force state analysis model of the utility pole, and perform early warning detection and output an abnormal early warning signal in combination with the preset threshold.

[0057] A device for monitoring and early warning of the stress state of a utility pole, comprising a memory and a processor;

[0058] The memory is used to store programs;

[0059] The processor is used to execute the program to implement the various steps of the power pole stress state monitoring and early warning method as described in any of the above claims.

[0060] A readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the various steps of the power pole stress state monitoring and early warning method as described in any of the preceding claims.

[0061] A computer program product includes a computer program, characterized in that the computer program, when run by a processor, executes the various steps of the utility pole stress state monitoring and early warning method as described in any of the preceding claims.

[0062] As can be seen from the above technical solutions, the utility pole stress state monitoring and early warning method and related equipment provided in this application construct a utility pole image dataset, perform data preprocessing, design and train a utility pole stress state analysis model network, and finally apply it to real-time monitoring and early warning of abnormalities in the stress state of utility poles.

[0063] First, this application significantly reduces reliance on sensors. It utilizes image analysis technology based on a stress state analysis model for utility poles, using captured image data to monitor the stress state of the poles. This eliminates excessive dependence on physical sensors, reduces detection costs, and makes the detection scheme more flexible and economical.

[0064] Secondly, it significantly improves environmental adaptability. Traditional force detection methods are highly susceptible to interference from environmental factors, which in turn affects the accuracy of the detection results. This application employs image preprocessing and data augmentation techniques, such as histogram equalization, affine transformation, and rotation transformation, to effectively enhance image quality, significantly improve the robustness of the model, ensure stable operation under various complex environmental conditions, and greatly enhance the reliability of the detection results.

[0065] Furthermore, detection performance has been comprehensively improved. The batch normalization fusion module accurately extracts key features, enhancing model robustness and detection performance; the local feature fusion module deeply extracts features, optimizing computational efficiency; the self-feedback feature fusion module fuses features at multiple levels, capturing complex stress characteristics and improving detection accuracy and generalization ability; the attention-enhanced convolution module focuses on key regions, improving perception accuracy; and the feature map stress analysis module comprehensively analyzes multiple aspects of information to determine the stress state of the utility pole. This application can better capture the complex features of the stress state of utility poles, further improving detection accuracy and the model's generalization ability.

[0066] Finally, efficient detection was achieved while reducing computational complexity. The utility pole stress state analysis model designed in this application implements multi-stage feature fusion and enhancement. This design optimizes the model's ability to perceive complex stress states and improves the extraction of key features, thereby achieving more efficient and accurate stress state judgment. Simultaneously, the self-feedback feature fusion module performs multiple feature fusions and dimensionality compressions through fully connected layers and pooling operations, effectively reducing computational overhead and memory consumption, minimizing the interference of redundant features on the detection results, and further enhancing the model's robustness and generalization ability. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0068] Figure 1 This is a flowchart of a method for monitoring and early warning of the stress state of a utility pole, as disclosed in an embodiment of this application.

[0069] Figure 2 This is a structural schematic diagram of a force state analysis model for a utility pole disclosed in an embodiment of this application;

[0070] Figure 3 This is a schematic diagram of the structure of a batch normalization fusion module disclosed in an embodiment of this application;

[0071] Figure 4 This is a schematic diagram of the structure of a local feature fusion module disclosed in an embodiment of this application;

[0072] Figure 5 This is a schematic diagram of the structure of a self-feedback feature fusion module disclosed in an embodiment of this application;

[0073] Figure 6 This is a schematic diagram of a power pole stress state monitoring and early warning device disclosed in an embodiment of this application;

[0074] Figure 7 This is a hardware structure block diagram of a power pole stress state monitoring and early warning device disclosed in an embodiment of this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0076] This application can be used in a wide variety of general-purpose or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0077] The following section introduces the solution proposed in this application. The technical solution is as follows, and details are provided below.

[0078] Figure 1 This is a flowchart of a method for monitoring and early warning of the stress state of a utility pole, as disclosed in an embodiment of this application.

[0079] like Figure 1 As shown, the method may include:

[0080] Step S1: Acquire normal utility pole images and tilted utility pole images from different angles captured by the camera, and label the force values ​​of the utility poles in each image to form a utility pole image dataset.

[0081] Specifically, drone aerial photography and camera shots were used to acquire images of normal and tilted utility poles from different angles. This diverse shooting perspective comprehensively covers various forms of utility poles in real-world scenarios, providing a rich data foundation for subsequent model training. Force sensors were used to accurately measure the forces acting on each utility pole. This step is crucial because accurate force labeling is key to subsequent analysis of the pole's stress state. Image annotation tools (such as LabelMe) were used to label the acquired utility pole images, specifying the force value for each pole. Through annotation, the images are linked to the actual force information, forming a complete utility pole image dataset containing images and their corresponding force labels, which can be used for subsequent model training and analysis.

[0082] Step S2: Perform histogram equalization, noise removal, affine transformation, rotation transformation, and noise addition operations on the utility pole image dataset, and divide it into training set, validation set, and test set according to the ratio.

[0083] Specifically, histogram equalization is performed on the utility pole images to enhance image contrast and optimize brightness distribution. This process makes details in the image clearer and more apparent, helping subsequent models to better recognize and learn features within the image.

[0084] Noise is generated during image acquisition due to various factors such as changes in lighting, shooting angle, and weather conditions. Noise removal can effectively reduce this noise, making the image clearer and improving image quality, thereby providing better data for model training.

[0085] Affine transformations such as translation, scaling, and shearing are used to process images of utility poles. These operations generate images at different angles and scales, greatly expanding the diversity of the dataset and enabling the model to learn the features of utility poles under different geometric shapes, thus enhancing the model's generalization ability.

[0086] The image is rotated and transformed at different angles to simulate images of utility poles taken from various angles. This helps improve the model's adaptability to angle changes, enabling it to accurately analyze and judge images of utility poles taken from different angles.

[0087] Artificial noise is added to the image to simulate the noise levels that may exist in utility pole images under different environments. This improves the model's robustness to noise interference, allowing it to maintain good performance even when faced with images in complex environments in real-world applications.

[0088] The processed utility pole image dataset is divided into training, validation, and test sets according to a preset ratio, such as 7:2:1. This reasonable dataset division ensures the training set is large enough to adequately train the model; the validation set is used to evaluate the model's training effectiveness and adjust model parameters during training; and the test set is used to ultimately evaluate the model's performance, verifying its generalization ability and accuracy.

[0089] Step S3: Design a stress state analysis model for utility poles, train the model using the training set, and verify the training effect and model performance using the validation set and the test set, respectively.

[0090] Specifically, a model for analyzing the stress state of utility poles was designed. Through a series of feature fusion, extraction, and enhancement operations, the model derives stress results from utility pole images. Next, the model was trained using a training set. A large number of utility pole images with accurate stress annotations were input into the model batch by batch. The model predicted the stress results based on the input and output. By comparing the results with the actual labeled stress values, the model's parameters were continuously adjusted using optimization algorithms, iterating repeatedly to improve prediction accuracy. Then, the model's performance was validated using a validation set. The validation set data was independent of the training set and was periodically input into the model during training. The predicted results were compared with the actual annotations, and the model's training progress was assessed based on changes in error and performance metrics. If overfitting or other issues occurred, the training strategy was adjusted promptly. Finally, the optimized model was validated using a test set. The test set, which was not used in training and had a data distribution close to real-world application scenarios, was input into the model. A comprehensive set of performance metrics, including root mean square error, mean absolute error, and accuracy, was calculated to accurately evaluate the model's performance in real-world scenarios and determine whether it meets the actual needs of utility pole stress state analysis.

[0091] The stress state analysis model for the utility pole includes:

[0092] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0093] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0094] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0095] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0096] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0097] The stress state analysis model for utility poles consists of multiple modules that work together to achieve accurate analysis and early warning of the stress state of utility poles.

[0098] The batch normalization fusion module first performs two convolutions and batch normalizations on the input feature map, adds the results, performs average pooling, activates it with a sigmoid function, and then multiplies it with the original feature map to generate a fused feature map. The local feature fusion module performs a double convolution on the fused feature map, activates it with ReLU, performs average pooling, activates it with a sigmoid function, and then multiplies it with the original convolution result to output a map that enhances local features. The self-feedback feature fusion module processes the input features... Figure 3 The feature map is divided into three sub-feature maps after initial addition, average pooling, fully connected processing, and ReLU activation. Each sub-feature map is then multiplied by the original feature map and added together to generate a self-feedback feature map. An attention-enhanced convolution module enhances the channel and spatial attention of the feature map, followed by dimensionality reduction convolution to generate a dimensionality-reduced feature map. A feature map stress analysis module analyzes the deformation gradient, texture changes, and edge offsets of the input feature map, and regresses the output stress state detection result.

[0099] Step S4: Input the real-time acquired image of the utility pole to be monitored into the trained and verified force state analysis model of the utility pole to obtain the current force state detection result output by the force state analysis model of the utility pole, and perform early warning detection and output an abnormal early warning signal in combination with the preset threshold.

[0100] Specifically, the real-time acquisition of images of the utility poles to be monitored typically utilizes high-definition cameras, drones, and other aerial photography equipment to ensure that the images clearly show the overall shape of the pole and details of its surrounding environment. The acquired images are then rapidly transmitted to a system equipped with a trained and validated stress state analysis model for the utility pole.

[0101] The model follows a predetermined modular processing flow, starting with a batch normalization fusion module to extract and fuse image features. This is followed by a progressive process involving a local feature fusion module, a self-feedback feature fusion module, and an attention-enhanced convolution module, all working in depth to mine and optimize image features. Finally, the feature map force analysis module outputs the current stress state detection result of the utility pole. This result may be presented in the form of specific force values, force levels, etc.

[0102] Next, the system enters the early warning detection phase, comparing the stress state detection results with pre-set thresholds. These thresholds are not arbitrary but are derived from rigorous data analysis and expert review, based on the pole's material, specifications, design load-bearing capacity, and historical data from numerous poles under different environmental and stress conditions. If the detection results exceed the pre-set threshold range, the system will immediately trigger an abnormality warning signal. The warning signal output can take various forms, such as sending SMS reminders to relevant personnel's mobile phones, displaying a prominent red warning window on the monitoring center's screen, and accompanied by a loud alarm, so that relevant personnel can notice the issue immediately and promptly inspect and maintain the pole to prevent safety accidents caused by abnormal pole stress.

[0103] One optional implementation involves using a stress detection result (R) from the utility pole stress state analysis model. The early warning program sets two threshold levels, P and Q, where P > Q. In this scheme, P can be set to 1.5 times the normal stress on the utility pole, and Q can be set to 1.2 times the normal stress on the utility pole. These two thresholds are determined by the implementer based on a comprehensive consideration of factors such as the actual load-bearing capacity of the utility pole, safety standards, and past operating data.

[0104] First, R is compared with the threshold P. If R is greater than P, it means that the force currently borne by the utility pole is far more than 1.5 times the normal force, and it is in a relatively dangerous state. At this time, the early warning module will immediately issue a red warning signal to remind relevant personnel that the force on the utility pole is seriously abnormal and that countermeasures need to be taken urgently.

[0105] If R is less than P, then R needs to be further compared with the threshold Q. If R is greater than Q, it indicates that the force on the utility pole is between 1.2 and 1.5 times the normal force. Although it has not reached the severity of a red warning, there is still an abnormal force situation. At this time, the warning module will issue a yellow warning signal, prompting relevant personnel to pay attention to the force status of the utility pole and arrange inspection and maintenance work in a timely manner.

[0106] If R is less than Q, it indicates that the stress on the utility pole is within or close to the normal range, and no warning conditions have been triggered. Therefore, the warning module will not issue a warning signal, meaning that the current operating state of the utility pole is relatively safe and stable. The entire warning process strictly follows this comparison strategy, achieving accurate early warning of the stress state of the utility pole through clear and explicit threshold comparisons, effectively ensuring the safe operation of the utility pole.

[0107] As can be seen from the above technical solutions, the utility pole stress state monitoring and early warning method and related equipment provided in this application construct a utility pole image dataset, perform data preprocessing, design and train a utility pole stress state analysis model network, and finally apply it to real-time monitoring and early warning of abnormalities in the stress state of utility poles.

[0108] First, this application significantly reduces reliance on sensors. It utilizes image analysis technology based on a stress state analysis model for utility poles, using captured image data to monitor the stress state of the poles. This eliminates excessive dependence on physical sensors, reduces detection costs, and makes the detection scheme more flexible and economical.

[0109] Secondly, it significantly improves environmental adaptability. Traditional force detection methods are highly susceptible to interference from environmental factors, which in turn affects the accuracy of the detection results. This application employs image preprocessing and data augmentation techniques, such as histogram equalization, affine transformation, and rotation transformation, to effectively enhance image quality, significantly improve the robustness of the model, ensure stable operation under various complex environmental conditions, and greatly enhance the reliability of the detection results.

[0110] Furthermore, detection performance has been comprehensively improved. The batch normalization fusion module accurately extracts key features, enhancing model robustness and detection performance; the local feature fusion module deeply extracts features, optimizing computational efficiency; the self-feedback feature fusion module fuses features at multiple levels, capturing complex stress characteristics and improving detection accuracy and generalization ability; the attention-enhanced convolution module focuses on key regions, improving perception accuracy; and the feature map stress analysis module comprehensively analyzes multiple aspects of information to determine the stress state of the utility pole. This application can better capture the complex features of the stress state of utility poles, further improving detection accuracy and the model's generalization ability.

[0111] Finally, efficient detection was achieved while reducing computational complexity. The utility pole stress state analysis model designed in this application implements multi-stage feature fusion and enhancement. This design optimizes the model's ability to perceive complex stress states and improves the extraction of key features, thereby achieving more efficient and accurate stress state judgment. Simultaneously, the self-feedback feature fusion module performs multiple feature fusions and dimensionality compressions through fully connected layers and pooling operations, effectively reducing computational overhead and memory consumption, minimizing the interference of redundant features on the detection results, and further enhancing the model's robustness and generalization ability.

[0112] In some embodiments of this application, combined with Figure 2 This section introduces the stress state analysis model for utility poles, which may include:

[0113] The stress state analysis model for utility poles consists of two batch normalization fusion modules, two local feature fusion modules, a self-feedback feature fusion module, an attention-enhanced convolution module, an upsampling module, a stitching module, and a feature map stress analysis module.

[0114] First batch of normalization fusion module:

[0115] The input utility pole image is subjected to two convolution operations, batch normalized, and then added together. After average pooling and activation by the sigmoid function, the first fused feature map is generated by dot product with the original input utility pole image.

[0116] Second batch normalization fusion module:

[0117] The first fused feature map F6 is subjected to two convolution operations, and after batch normalization, the two parts are added together, average pooling is performed, and the sigmoid function is activated. The second fused feature map is generated by dot product with the first fused feature map.

[0118] First local feature fusion module:

[0119] The second fused feature map is subjected to a double convolution operation in sequence to obtain the first convolution result. The result is then activated by ReLU, average pooling and sigmoid, and multiplied by the first convolution result to output the first enhanced feature map.

[0120] Self-feedback feature fusion module:

[0121] The first enhanced feature map is subjected to three self-addition operations, and then three sub-feature maps are generated by average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the first enhanced feature map and added to generate a self-feedback feature map.

[0122] Attention-enhanced convolutional module:

[0123] Channel and spatial attention enhancements are applied to the self-feedback feature map, and a dimensionality-reduced feature map is generated through dimensionality-reducing convolution.

[0124] Second local feature fusion module:

[0125] The reduced feature map is subjected to double convolution operations in sequence to obtain the second convolution result. The result is then multiplied by the second convolution result through ReLU activation, average pooling and sigmoid activation to output the second enhanced feature map.

[0126] Upsampling module:

[0127] The second enhanced feature map is upsampled by bilinear interpolation to generate an upsampled feature map.

[0128] splicing module:

[0129] The upsampled feature map and the first fused feature map are concatenated along the channel dimension, and the number of channels is adjusted to generate a concatenated feature map.

[0130] Feature map force analysis module:

[0131] The deformation gradient, texture change, and edge offset in the stitched feature map are analyzed, and the stress state detection result is regressed and output.

[0132] Specifically, the input image of a utility pole, F1, first passes through two batch normalization and fusion modules (first batch normalization and fusion module, second batch normalization and fusion module) to obtain the second fused feature map, F6. Each module performs two convolutions on the input, performs batch normalization, adds them together, performs average pooling, sigmoid activation, and then multiplies with the input, sequentially generating the first and second fused feature maps, gradually extracting and fusing image features.

[0133] The first local feature fusion module performs a double convolution on the second fused feature map F6, followed by ReLU, average pooling, and sigmoid activation, and then multiplies the result by the convolution to output the first enhanced feature map F11. The self-feedback feature fusion module adds the first enhanced feature map F11 three times, followed by average pooling, fully connected layers, ReLU activation, and segmentation operations to generate three sub-feature maps. These sub-feature maps are then multiplied by the first module and added together to obtain the self-feedback feature map F18, which enhances the feature representation.

[0134] The attention-enhancing convolution module enhances the self-feedback feature map F18 with channel and spatial attention, and then performs dimensionality reduction convolution to generate a dimensionality-reduced feature map F19. The second local feature fusion module processes the dimensionality-reduced feature map F19 and outputs the second enhanced feature map F20. The upsampling module upsamples the second enhanced feature map F20 using bilinear interpolation, and the concatenation module concatenates the upsampled feature map F21 with the first fused feature map F11 along the channel dimension (to obtain F22) and adjusts the number of channels to obtain the concatenated feature map F23.

[0135] The feature map stress analysis module analyzes the deformation gradient, texture changes and edge offset of the spliced ​​feature map F23, and regresses to output the stress state detection result R.

[0136] In some embodiments of this application, combined with Figure 3 The processing procedure of the batch normalization fusion module is described, which may include:

[0137] The input feature maps are subjected to convolution operations with kernel sizes of 1 and 3 respectively to generate the first convolution feature map and the second convolution feature map;

[0138] The first convolutional feature map and the second convolutional feature map are batch normalized respectively, and then the sum is used to generate the first pooled feature map by average pooling.

[0139] The pooled feature map is activated by sigmoid to generate attention weights;

[0140] The attention weights are multiplied element-wise with the original input feature map to output a fused feature map.

[0141] Specifically, this module performs two different convolution operations (Conv) on the input feature map F1 (i.e., the feature map of the utility pole in the scenario of force analysis of the utility pole). One is a convolution with a kernel size of 1 and a stride of 1, and the other is a convolution with a kernel size of 3 and a stride of 1.

[0142] Convolution with a kernel size of 1: This convolution operation is mainly used to extract local features from the feature map of utility poles. Because the kernel size is 1, it enhances the representational power of the features while maintaining spatial resolution. Each convolution kernel is element-wise multiplied with the corresponding position in the input feature map and the results are summed to generate the first convolutional feature map. This convolution kernel can be viewed as a linear combination of the channels of the input feature map, enabling the recombination and refinement of features without changing the spatial size of the feature map.

[0143] Convolution with a kernel size of 3: This operation aims to extract broader local features from the utility pole feature map. A kernel size of 3 means it can cover a larger local area of ​​the input feature map, thus capturing richer contextual information. Similarly, the second convolutional feature map generated by the convolution operation can extract more complex and global features compared to the case with a kernel size of 1.

[0144] Batch normalization (BN) was applied to the first and second convolutional feature maps respectively, resulting in two feature maps, F2 and F3. Batch normalization is a widely used technique in deep learning that maintains a stable data distribution during training, helping to accelerate model convergence and improve training efficiency and stability. After batch normalization, the data in the feature maps is adjusted to a distribution with a mean of 0 and a variance of 1, reducing data distribution fluctuations and making the model more adaptable to data of different scales and distributions.

[0145] The two feature maps (F2 and F3) after batch normalization are added together, and then average pooling (AVGPooling) is performed to obtain the first pooled feature map F4. Average pooling is a downsampling operation that divides the input feature map into multiple non-overlapping local regions, averages the elements in each region, and obtains a new feature value, thus generating the first pooled feature map. By using average pooling, the size of the feature map can be reduced, the computational complexity can be lowered, and the average information of each local region in the feature map can be preserved, which enhances the robustness of the features to a certain extent and makes the model focus more on important features.

[0146] The Sigmoid activation function is applied to the first pooling feature map F4. Through Sigmoid activation, each element in the first pooling feature map F4 is mapped to the interval between 0 and 1, generating attention weights F5. These attention weights can be seen as a measure of the importance of each position in the original input feature map; the closer the value is to 1, the more important the feature at that position, and the closer the value is to 0, the less important the feature at that position.

[0147] The batch normalization fusion module processes the input feature map to obtain the attention weight F5 using the following formula:

[0148]

[0149]

[0150] In the formula, The input is a feature image of a utility pole. It is a convolution with a kernel size of 1 and a stride of 1. It is a convolution with a kernel size of 3 and a stride of 1. It is an average pooling operation. It is an activation function. It outputs a feature image of a utility pole. It is element-wise multiplication.

[0151] The generated attention weights F5 are multiplied element-wise with the original input feature map F1, outputting a fused feature map F6. Element-wise multiplication means multiplying the attention weights with corresponding elements in the original input feature map, thus highlighting important features and suppressing unimportant ones. After multiplication, the fused feature map is output. This fused feature map combines information from the original input feature map and the importance information represented by the attention weights, enabling subsequent models to focus more on features that significantly influence the stress state analysis of utility poles, improving model performance and accuracy.

[0152] In practical applications, any utility pole image F1 from the utility pole image dataset is used as the input utility pole feature map and fed into the batch normalization fusion module for the above processing. Finally, the fused feature map F6 is output, providing higher quality feature information for subsequent model analysis.

[0153] In some embodiments of this application, combined with Figure 4 The processing procedure of the local feature fusion module is described, which may include:

[0154] A third convolutional feature map is generated by performing a convolution operation with a kernel size of 1 on the input feature map;

[0155] The third convolutional feature map is subjected to a convolution operation with a kernel size of 3 to generate a fourth convolutional feature map, which is then activated by ReLU to generate an activation feature map.

[0156] The activated feature map is subjected to global average pooling, and then activated by sigmoid to generate the first channel weights;

[0157] The fourth convolutional feature map is multiplied element-wise based on the weights of the first channel to output an enhanced feature map.

[0158] Specifically, the input feature map F6 first undergoes a convolution operation (Conv) with a kernel size of 1 and a stride of 1. This extracts local features from the input feature map while maintaining the spatial dimension of the feature map, generating a third convolutional feature map F7. This convolution operation is equivalent to linearly combining the channels of the input feature map, enhancing the expressive power of the features.

[0159] A convolution operation with a kernel size of 3 and a stride of 1 is performed on the third convolutional feature map F7, which can extract local features over a larger range, generating the fourth convolutional feature map F8. Next, the ReLU function is used to activate the fourth convolutional feature map F8, generating the activated feature map F9. The ReLU function introduces non-linearity, removes negative features, and accelerates model convergence.

[0160] Global average pooling (AVG pooling) is performed on the activation feature map F9 to compress the feature map in spatial dimensions, obtaining the average value for each channel and thus reducing the size of the feature map. Then, the result of average pooling is processed by the sigmoid activation function, which compresses the output value to between 0 and 1, generating the first channel weight F10, which can be regarded as a measure of the importance of each channel.

[0161] Based on the generated first channel weight F10, an element-wise dot product operation is performed on the fourth convolutional feature map F8. This enhances the features of important channels while suppressing the features of less important channels, ultimately outputting an enhanced feature map F11, which helps to more accurately determine the stress state of the utility pole. The transformation formula is as follows:

[0162]

[0163]

[0164] In the formula, It is the input feature map of the utility pole. ReLU is an activation function, and AvgPool is the average pooling operation. It is a convolution operation with a kernel size of 1 and a stride of 1. It is a convolution operation with a kernel size of 3 and a stride of 1.

[0165] In some embodiments of this application, combined with Figure 5 The processing procedure of the self-feedback feature fusion module is described, which may include:

[0166] Perform three increment operations on the input feature map to generate an incremented feature map;

[0167] The self-added feature map is subjected to average pooling and then size compression through a fully connected layer to generate a compressed feature map;

[0168] After ReLU activation, the compressed feature map is expanded by a fully connected layer to generate an expanded feature map.

[0169] The extended feature map is divided into three sub-feature maps along the channel dimension. Each sub-feature map is activated by sigmoid and then multiplied element-wise with the original input feature map. The sub-feature maps are then added together to generate a self-feedback feature map.

[0170] Specifically, the input feature map F11 undergoes three auto-addition (ADD) operations, meaning the feature map itself is added three times consecutively to generate an auto-addition feature map F12. This step strengthens the inherent feature information in the feature map by superimposing its own features.

[0171] Performing average pooling (AVG pooling) on ​​the self-added feature map reduces the spatial size of the feature map, lowers the computational cost, and preserves the average information of the region.

[0172] Next, the feature map after average pooling is compressed by a fully connected layer (FC) to adjust its dimensions and generate a compressed feature map F13, making the features more abstract and representative.

[0173] The compressed feature map F13 is activated using the ReLU activation function. Then, it is expanded by a fully connected layer (FC) to generate an expanded feature map F14, which is used for subsequent feature segmentation and fusion.

[0174] The extended feature map is split into three sub-feature maps along the channel dimension. Each of these three sub-feature maps is then processed using the sigmoid activation function. The sigmoid function compresses the output value to between 0 and 1, generating a weight-like value to represent the importance of each channel feature.

[0175] The three activated sub-feature maps (F15, F16, and F17) are each element-wise multiplied with the original input feature map F11 to highlight important features and suppress unimportant features. Finally, the results of the multiplications are summed (ADD) to generate the self-feedback feature map F18. This multi-stage feature fusion method combines feature information from different dimensions and levels, enhancing the model's ability to extract key features of the stress state of utility poles and improving detection accuracy. The transformation formula is as follows:

[0176]

[0177]

[0178]

[0179] In the formula, The input is a feature image of a utility pole. It is average pooling. ReLU is an activation function, FC is a fully connected layer, split is a splitting operation, and ADD is an addition operation. It outputs a feature diagram of a utility pole.

[0180] In some embodiments of this application, the processing procedure of the attention-enhanced convolutional module is described, which may specifically include:

[0181] The input feature map is subjected to global average pooling, and then the weights of the second channel are generated by passing through a fully connected layer and sigmoid activation.

[0182] Channel compression is performed on the input feature map, and spatial weights are generated through convolution and sigmoid activation;

[0183] The second channel weight is multiplied element-wise with the spatial weight to generate a double-enhanced feature map;

[0184] The dual-enhanced feature map is subjected to convolutional dimensionality reduction to output a dimensionality-reduced feature map.

[0185] Specifically, global average pooling is performed on the input feature map to compress it spatially, obtaining the average value for each channel and thus reducing the feature map size. Next, the result of average pooling is processed through a fully connected layer, and then activated by a sigmoid function to compress the output value to between 0 and 1, generating the second channel weights. These weights represent the importance of each channel, facilitating the subsequent highlighting of key channel features.

[0186] The input feature map undergoes channel compression to reduce the number of channels. Then, spatial features are further extracted through convolution, followed by sigmoid activation to generate spatial weights. These weights reflect the importance of different spatial locations in the feature map.

[0187] The generated second channel weights are multiplied element-wise with the spatial weights to combine the importance information from both the channel and spatial dimensions, generating a double-enhanced feature map. This operation further strengthens the features in the feature map that are important for both the channels and the spatial locations.

[0188] Convolutional dimensionality reduction is performed on the double-enhanced feature map. By setting appropriate convolution kernels and strides, the dimensionality of the feature map is reduced while retaining key feature information, ultimately outputting a dimensionality-reduced feature map. The dimensionality-reduced feature map is easier for subsequent modules to process, while also reducing computational complexity and improving model efficiency. Through this series of operations, the attention-enhanced convolution module effectively enhances the feature representation of the input feature map, providing higher-quality feature data for the accurate detection of the stress state of the utility pole.

[0189] The following describes a utility pole stress state monitoring and early warning device provided in the embodiments of this application. The utility pole stress state monitoring and early warning device described below can be referred to in correspondence with the utility pole stress state monitoring and early warning method described above.

[0190] See Figure 6 , Figure 6 This is a schematic diagram of a power pole stress state monitoring and early warning device disclosed in an embodiment of this application.

[0191] like Figure 6 As shown, the utility pole stress state monitoring and early warning device may include:

[0192] The data acquisition unit 110 is used to acquire normal utility pole images and tilted utility pole images from different angles captured by the shooting device, and to label the force values ​​of the utility poles in each image to form a utility pole image dataset.

[0193] The data classification unit 120 is used to perform histogram equalization, noise removal, affine transformation, rotation transformation and noise addition operations on the utility pole image dataset, and divide it into training set, validation set and test set according to the proportion.

[0194] The model training unit 130 is used to design a stress state analysis model for utility poles, train the model using the training set, and verify the training effect and model performance using the validation set and the test set, respectively. The stress state analysis model for utility poles includes:

[0195] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0196] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0197] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0198] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0199] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0200] The state detection unit 140 is used to input the real-time acquired image of the utility pole to be monitored into the training and verification of the utility pole stress state analysis model, obtain the current stress state detection result output by the utility pole stress state analysis model, and perform early warning detection and output an abnormal early warning signal in combination with the preset threshold.

[0201] As can be seen from the above technical solutions, the utility pole stress state monitoring and early warning method and related equipment provided in this application construct a utility pole image dataset, perform data preprocessing, design and train a utility pole stress state analysis model network, and finally apply it to real-time monitoring and early warning of abnormalities in the stress state of utility poles.

[0202] First, this application significantly reduces reliance on sensors. It utilizes image analysis technology based on a stress state analysis model for utility poles, using captured image data to monitor the stress state of the poles. This eliminates excessive dependence on physical sensors, reduces detection costs, and makes the detection scheme more flexible and economical.

[0203] Secondly, it significantly improves environmental adaptability. Traditional force detection methods are highly susceptible to interference from environmental factors, which in turn affects the accuracy of the detection results. This application employs image preprocessing and data augmentation techniques, such as histogram equalization, affine transformation, and rotation transformation, to effectively enhance image quality, significantly improve the robustness of the model, ensure stable operation under various complex environmental conditions, and greatly enhance the reliability of the detection results.

[0204] Furthermore, detection performance has been comprehensively improved. The batch normalization fusion module accurately extracts key features, enhancing model robustness and detection performance; the local feature fusion module deeply extracts features, optimizing computational efficiency; the self-feedback feature fusion module fuses features at multiple levels, capturing complex stress characteristics and improving detection accuracy and generalization ability; the attention-enhanced convolution module focuses on key regions, improving perception accuracy; and the feature map stress analysis module comprehensively analyzes multiple aspects of information to determine the stress state of the utility pole. This application can better capture the complex features of the stress state of utility poles, further improving detection accuracy and the model's generalization ability.

[0205] Finally, efficient detection was achieved while reducing computational complexity. The utility pole stress state analysis model designed in this application implements multi-stage feature fusion and enhancement. This design optimizes the model's ability to perceive complex stress states and improves the extraction of key features, thereby achieving more efficient and accurate stress state judgment. Simultaneously, the self-feedback feature fusion module performs multiple feature fusions and dimensionality compressions through fully connected layers and pooling operations, effectively reducing computational overhead and memory consumption, minimizing the interference of redundant features on the detection results, and further enhancing the model's robustness and generalization ability.

[0206] The utility pole stress state monitoring and early warning device provided in this application embodiment can be applied to utility pole stress state monitoring and early warning equipment. Figure 7 The hardware structure block diagram of the utility pole stress condition monitoring and early warning device is shown. (Refer to...) Figure 7 The hardware structure of the power pole stress condition monitoring and early warning device may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0207] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0208] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0209] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0210] The memory stores a program, which the processor can call. The program is used for:

[0211] Acquire images of normal utility poles and tilted utility poles from different angles captured by a camera, and label the force values ​​of the utility poles in each image to form a utility pole image dataset;

[0212] The utility pole image dataset is subjected to histogram equalization, noise removal, affine transformation, rotation transformation, and noise addition operations, and is divided into training set, validation set, and test set according to the proportions.

[0213] Design a stress state analysis model for utility poles, and train the model using the training set. Validate the training effect and model performance using the validation set and test set, respectively. The stress state analysis model for utility poles includes:

[0214] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0215] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0216] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0217] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0218] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0219] The real-time acquired images of the utility pole to be monitored are input into the trained and validated force state analysis model of the utility pole to obtain the current force state detection result output by the force state analysis model of the utility pole. The model is then combined with the preset threshold to perform early warning detection and output an abnormal early warning signal.

[0220] Optionally, the refined and extended functions of the program can be referred to the above description.

[0221] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0222] Acquire images of normal utility poles and tilted utility poles from different angles captured by a camera, and label the force values ​​of the utility poles in each image to form a utility pole image dataset;

[0223] The utility pole image dataset is subjected to histogram equalization, noise removal, affine transformation, rotation transformation, and noise addition operations, and is divided into training set, validation set, and test set according to the proportions.

[0224] Design a stress state analysis model for utility poles, and train the model using the training set. Validate the training effect and model performance using the validation set and test set, respectively. The stress state analysis model for utility poles includes:

[0225] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0226] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0227] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0228] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0229] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0230] The real-time acquired images of the utility pole to be monitored are input into the trained and validated force state analysis model of the utility pole to obtain the current force state detection result output by the force state analysis model of the utility pole. The model is then combined with the preset threshold to perform early warning detection and output an abnormal early warning signal.

[0231] Optionally, the refined and extended functions of the program can be referred to the above description.

[0232] This application also provides a computer program product, including a computer program, wherein the computer program is executed by a processor using the following method:

[0233] Acquire images of normal utility poles and tilted utility poles from different angles captured by a camera, and label the force values ​​of the utility poles in each image to form a utility pole image dataset;

[0234] The utility pole image dataset is subjected to histogram equalization, noise removal, affine transformation, rotation transformation, and noise addition operations, and is divided into training set, validation set, and test set according to the proportions.

[0235] Design a stress state analysis model for utility poles, and train the model using the training set. Validate the training effect and model performance using the validation set and test set, respectively. The stress state analysis model for utility poles includes:

[0236] The batch normalization fusion module is used to perform two convolution operations on the input feature map, add them after batch normalization, perform average pooling and activate it with the sigmoid function, and generate a fused feature map by dot product with the original input feature map.

[0237] The local feature fusion module is used to perform double convolution operations on the input feature map sequentially to obtain the original convolution result, and then multiply the original convolution result by ReLU activation, average pooling and sigmoid activation to output an enhanced feature map.

[0238] The self-feedback feature fusion module is used to perform three self-addition operations on the input feature map, and generate three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the original input feature map and added to generate the self-feedback feature map.

[0239] The attention-enhancing convolution module is used to enhance the channel and spatial attention of the input feature map and generate a dimensionality-reduced feature map through dimensionality-reducing convolution.

[0240] The feature map stress analysis module is used to analyze the deformation gradient, texture change and edge offset in the input feature map and regress the stress state detection results.

[0241] The real-time acquired images of the utility pole to be monitored are input into the trained and validated force state analysis model of the utility pole to obtain the current force state detection result output by the force state analysis model of the utility pole. The model is then combined with the preset threshold to perform early warning detection and output an abnormal early warning signal.

[0242] Optionally, the refined and extended functions of the program can be referred to the above description.

[0243] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0244] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0245] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring and early warning of the stress state of utility poles, characterized in that, include: Acquire images of normal utility poles and tilted utility poles from different angles captured by a camera, and label the force values ​​of the utility poles in each image to form a utility pole image dataset; The utility pole image dataset is subjected to histogram equalization, noise removal, affine transformation, rotation transformation, and noise addition operations, and is divided into training set, validation set, and test set according to the proportions. Design a stress state analysis model for utility poles, and train the model using the training set. Use the validation set and the test set to verify the training effect and model performance of the model, respectively. The stress state analysis model for utility poles consists of two batch normalization fusion modules, two local feature fusion modules, a self-feedback feature fusion module, an attention-enhanced convolution module, an upsampling module, a stitching module, and a feature map stress analysis module. The first batch normalization fusion module performs two convolution operations on the input utility pole image, adds them after batch normalization, performs average pooling and activates them with the sigmoid function, and generates the first fused feature map by dot product with the original input utility pole image. The second batch normalization fusion module performs two convolution operations on the first fused feature map, adds them after batch normalization, performs average pooling and activates them with the sigmoid function, and multiplies them with the first fused feature map to generate the second fused feature map. The first local feature fusion module performs double convolution operations on the second fused feature map to obtain the first convolution result. After ReLU activation, average pooling and sigmoid activation, the first convolution result is multiplied by the first convolution result to output the first enhanced feature map. The self-feedback feature fusion module performs three self-addition operations on the first enhanced feature map, and generates three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the first enhanced feature map and added to generate the self-feedback feature map. The attention-enhanced convolutional module enhances the channel and spatial attention of the self-feedback feature map and generates a dimensionality-reduced feature map through dimensionality-reducing convolution. The second local feature fusion module performs double convolution operations on the reduced feature map to obtain the second convolution result. After ReLU activation, average pooling and sigmoid activation, the second convolution result is multiplied by the second convolution result to output the second enhanced feature map. The upsampling module performs bilinear interpolation upsampling on the second enhanced feature map to generate an upsampled feature map; The stitching module stitches the upsampled feature map and the first fused feature map along the channel dimension and adjusts the number of channels to generate a stitched feature map; The feature map stress analysis module analyzes the deformation gradient, texture change and edge offset in the spliced ​​feature map, and regresses the stress state detection result. The real-time collected images of the utility poles to be monitored are input into the trained and validated force state analysis model of the utility poles to obtain the current force state detection result output by the force state analysis model of the utility poles. The model is then combined with a preset threshold to perform early warning detection and output an abnormal early warning signal.

2. The method according to claim 1, characterized in that, The processing procedure of the batch normalization fusion module includes: The input feature map is subjected to convolution operations with kernel sizes of 1 and 3 respectively to generate a first convolution feature map and a second convolution feature map. The input feature map is either the input utility pole image or the first fused feature map. The first convolutional feature map and the second convolutional feature map are batch normalized respectively, and then the sum is used to generate the first pooled feature map by average pooling. The pooled feature map is activated by sigmoid to generate attention weights; The attention weights are multiplied element-wise with the input feature map to output a fused feature map.

3. The method according to claim 1, characterized in that, The processing procedure of the local feature fusion module includes: A third convolutional feature map is generated by performing a convolution operation with a kernel size of 1 on the input feature map, wherein the input feature map is the second fused feature map or the dimensionality-reduced feature map; The third convolutional feature map is subjected to a convolution operation with a kernel size of 3 to generate a fourth convolutional feature map, which is then activated by ReLU to generate an activation feature map. The activated feature map is subjected to global average pooling, and then activated by sigmoid to generate the first channel weights; The fourth convolutional feature map is multiplied element-wise based on the weights of the first channel to output an enhanced feature map.

4. The method according to claim 1, characterized in that, The processing procedure of the self-feedback feature fusion module includes: Perform three increment operations on the first enhanced feature map to generate an incremented feature map; The self-added feature map is subjected to average pooling and then size compression through a fully connected layer to generate a compressed feature map; After ReLU activation, the compressed feature map is expanded by a fully connected layer to generate an expanded feature map. The extended feature map is divided into three sub-feature maps along the channel dimension. Each sub-feature map is activated by sigmoid and then multiplied element-wise with the first enhanced feature map. The sub-feature maps are then added together to generate a self-feedback feature map.

5. The method according to claim 1, characterized in that, The processing procedure of the attention-enhanced convolutional module includes: The self-feedback feature map is subjected to global average pooling, and then the second channel weights are generated by passing it through a fully connected layer and sigmoid activation. Channel compression is performed on the self-feedback feature map, and spatial weights are generated through convolution and sigmoid activation; The second channel weight is multiplied element-wise with the spatial weight to generate a double-enhanced feature map; The dual-enhanced feature map is subjected to convolutional dimensionality reduction to output a dimensionality-reduced feature map.

6. A device for monitoring and early warning of the stress state of a utility pole, characterized in that, include: The data acquisition unit is used to acquire images of normal utility poles and tilted utility poles from different angles captured by the camera, and to label the force values ​​of the utility poles in each image to form a utility pole image dataset. The data classification unit is used to perform histogram equalization, noise removal, affine transformation, rotation transformation and noise addition operations on the utility pole image dataset, and divide it into training set, validation set and test set according to the proportion. The model training unit is used to design a stress state analysis model for utility poles, and to train the model using the training set. The validation set and the test set are used to verify the training effect and model performance of the stress state analysis model for utility poles, respectively. The stress state analysis model for utility poles consists of two batch normalization fusion modules, two local feature fusion modules, a self-feedback feature fusion module, an attention-enhanced convolution module, an upsampling module, a stitching module, and a feature map stress analysis module. The first batch normalization fusion module performs two convolution operations on the input utility pole image, adds them after batch normalization, performs average pooling and activates them with the sigmoid function, and generates the first fused feature map by dot product with the original input utility pole image. The second batch normalization fusion module performs two convolution operations on the first fused feature map, adds them after batch normalization, performs average pooling and activates them with the sigmoid function, and multiplies them with the first fused feature map to generate the second fused feature map. The first local feature fusion module performs double convolution operations on the second fused feature map to obtain the first convolution result. After ReLU activation, average pooling and sigmoid activation, the first convolution result is multiplied by the first convolution result to output the first enhanced feature map. The self-feedback feature fusion module performs three self-addition operations on the first enhanced feature map, and generates three sub-feature maps through average pooling, fully connected layer processing, ReLU activation and segmentation operations. These sub-feature maps are then multiplied by the first enhanced feature map and added to generate the self-feedback feature map. The attention-enhanced convolutional module enhances the channel and spatial attention of the self-feedback feature map and generates a dimensionality-reduced feature map through dimensionality-reducing convolution. The second local feature fusion module performs double convolution operations on the reduced feature map to obtain the second convolution result. After ReLU activation, average pooling and sigmoid activation, the second convolution result is multiplied by the second convolution result to output the second enhanced feature map. The upsampling module performs bilinear interpolation upsampling on the second enhanced feature map to generate an upsampled feature map; The stitching module stitches the upsampled feature map and the first fused feature map along the channel dimension and adjusts the number of channels to generate a stitched feature map; The feature map stress analysis module analyzes the deformation gradient, texture change and edge offset in the spliced ​​feature map, and regresses the stress state detection result. The state detection unit is used to input the real-time acquired images of the utility pole to be monitored into the trained and verified force state analysis model of the utility pole, obtain the current force state detection result output by the force state analysis model of the utility pole, and perform early warning detection and output abnormal early warning signal in combination with preset threshold.

7. A device for monitoring and early warning of the stress state of utility poles, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the method for monitoring and early warning of the stress state of utility poles as described in any one of claims 1-5.

8. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the method for monitoring and early warning of the stress state of utility poles as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is run by the processor, it executes each step of the method for monitoring and early warning of the stress state of utility poles as described in any one of claims 1-5.

Images