Drop-out fuse state monitoring method based on multi-size features and deep learning

Abstract

The invention discloses a drop-out fuse state monitoring method based on multi-size features and deep learning, and the method comprises the steps: S1, forming a data set for obtained drop-out fuse pictures, and dividing the data set into a training set and a test set; s2, adding an improved receptive field block and a coordinate attention module to a YOLOx backbone network, adding an adaptive spatial feature fusion module to PANet, carrying out secondary fusion on features of different scales, then introducing a loss function of weighting loss and positioning loss fusion, and finally carrying out lightweight improvement, constructing a state monitoring model, and carrying out state monitoring. Performing training verification on the constructed state monitoring model through the training set and the test set; s3, identifying a drop-out fuse picture acquired in real time by adopting the trained and verified state monitoring model; according to the image data collected by the application, the construction of a special database is realized, the YOLOx is improved and lightweight operation is carried out, and the complexity of the model is reduced, so that the method is suitable for an embedded platform of an electric unmanned aerial vehicle, and the detection effect of the drop-out fuse is improved.

Description

A method for monitoring the status of drop-out fuses based on multi-size features and deep learning Technical Field

[0001] This invention relates to the field of image processing technology in computer vision, and in particular to a method for monitoring the status of drop fuses based on multi-size features and deep learning. Background Technology

[0002] Drop-out switches, as a critical outdoor high-voltage protection device, are widely used in 10kV distribution network systems, including the primary side of pole-mounted distribution transformers, the branch side of lines, and the incoming side of user lines, undertaking important backup protection functions. However, the safety hazards of the porcelain insulation of drop-out switches cannot be ignored. Due to long-term operation in rural areas, the porcelain skirts of drop-out switches may experience flashover due to the accumulation of dirt; at the same time, long-term operation may also lead to structural aging, which can damage the insulation under external forces. These conditions can all lead to a decrease in the insulation strength of the drop-out switch, and may cause short-circuit faults under certain conditions, resulting in power outages in some areas, damage to transformer equipment, and even endangering personal safety. Therefore, regular inspection and maintenance of the porcelain insulation of drop-out switches in pole-mounted substations is crucial.

[0003] Currently, drone inspections, thanks to their advanced technologies such as flight modules and GPS, have achieved wider coverage and higher inspection efficiency. When unforeseen circumstances in remote areas make manual inspections difficult, drones can quickly reach the target area, accurately collect and intelligently analyze data, increasing efficiency by dozens of times compared to traditional manual inspections.

[0004] In current monitoring models, two-stage object detection algorithms such as Faster R-CNN perform well in terms of recognition accuracy, but they consume a lot of hardware resources and have a relatively long computation time. In contrast, regression algorithms such as YOLO and SSD adopt an integrated detection framework, which significantly improves computational efficiency, but is slightly inferior in detection accuracy. However, most current monitoring models are trained on general datasets. In professional applications, model performance may decline due to various factors. Therefore, in professional scenarios such as defect detection in the power industry, it is necessary to build a dataset specific to this field and design targeted model optimization strategies to ensure the accuracy and reliability of the detection algorithm. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing an image quality optimization system based on multi-scale feature decoupling and dynamic fusion. It addresses the challenge of detecting small targets such as dropped fuses in complex backgrounds using a dedicated dataset.

[0006] To address the aforementioned technical problems, the present invention provides a technical solution: a method for monitoring the status of drop-out fuses based on multi-size features and deep learning, comprising the following steps:

[0007] S1. Perform data augmentation on the acquired images of the fallen fuse to form a dataset, and divide the dataset into a training set and a test set;

[0008] S2. First, add the improved receptive field block and coordinate attention module to the YOLOx backbone network. Then, add the adaptive spatial feature fusion module to the path aggregation network and perform secondary fusion on features of different scales to fully fuse shallow and deep features. Then, introduce a loss function that fuses weighted loss and localization loss. Finally, perform lightweight improvements to build a state monitoring model. Train and verify the built state monitoring model using training and test sets.

[0009] S3. The trained and validated state monitoring model is used to identify the real-time acquired images of the drop fuse.

[0010] Furthermore, in step S1, the method for obtaining images of the drop fuse is to acquire multi-angle and multi-state images of the drop fuse using a drone.

[0011] Furthermore, in step S1, the data augmentation method for the images of the fallen fuse is as follows: by performing contrast adjustment, scaling, Gaussian blur, brightness adjustment, saturation adjustment, Mosaic data augmentation and Mixup data augmentation operations, the images of the fallen fuse in multiple scenarios are simulated to expand the dataset, and the dataset is labeled using Labelme annotation software.

[0012] Furthermore, in step S2, the secondary fusion of features at different scales is performed as follows: (1) An improved receptive field block is added after the shallow feature layer CSP1_3, and a dilated convolutional layer is introduced to increase the receptive field of the network; (2) Three coordinate attention modules are added after the effective feature layers CSP1_3, CSP1_3 and CSP2_1 of the backbone feature extraction network, so that the backbone network can better acquire the spatial orientation feature information of the target; (3) The three generated effective feature maps P3, P4 and P5 are fused for the first time using the feature pyramid to generate three feature maps P3_out, P4_out and P5_out at different scales. Then, the adaptive spatial feature fusion module is used for the second fusion to generate the final fused feature map.

[0013] Furthermore, in step S2, the process of introducing a loss function that fuses weighted loss and localization loss is as follows: Based on the localization loss function, the bounding box evaluation method based on the weighted loss function with Wasserstein distance normalization is used to model the target detection box as a two-dimensional Gaussian distribution parameter space, and achieve more accurate box matching through probability distribution similarity measurement.

[0014] Furthermore, in step S2, the lightweight improvement process is as follows: (1) replace the YOLOx backbone network with the lightweight ShuffleNetV2 network; (2) expand the convolution kernel and prune the convolutional layers of ShuffleNetV2; (3) add an efficient channel attention module to PANet to make the network pay more attention to key channels with a small computational cost, so that the features are more fully integrated.

[0015] Furthermore, the specific pruning process is as follows: the LAMP pruning method based on layer adaptive amplitude is adopted. By performing layer adaptive recalibration of the magnitude of the weight parameters, weights with low LAMP scores are pruned first. LAMP automatically determines the layer sparsity until the global sparsity requirement is met.

[0016] To address the aforementioned technical problems, another technical solution provided by this invention is: an image quality optimization system based on multi-scale feature decoupling and dynamic fusion, characterized by comprising:

[0017] Data processing module: Performs data augmentation on the acquired images of the fallen fuse to form a dataset, and divides the dataset into training set and test set;

[0018] The monitoring model construction module adds an improved receptive field block and coordinate attention module to the YOLOx backbone network, adds an adaptive spatial feature fusion module to PANet, performs secondary fusion of features at different scales to fully integrate shallow and deep features, introduces a loss function that combines weighted loss and localization loss, and finally performs lightweight improvements to build a state monitoring model. The built state monitoring model is then trained and validated using training and testing sets.

[0019] Real-time monitoring module: The trained and validated state monitoring model is used to identify the real-time images of the fallen fuse.

[0020] To solve the above-mentioned technical problems, another technical solution provided by the present invention is: an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the steps of the method described in any one of the above claims.

[0021] To solve the above-mentioned technical problems, another technical solution provided by the present invention is: a computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described in any one of the preceding claims.

[0022] The beneficial effects of this invention are as follows:

[0023] 1. This application utilizes image data collected by UAVs to construct a dataset targeting defects in the insulation of drop-out circuit breakers. Multiple data augmentation methods are employed to expand the dataset, and Mosaic and Mixup image aliasing techniques are introduced to enhance data diversity. This enables the construction of a dedicated database, improving detection accuracy. YOLOx is improved by introducing receptive field blocks and coordinate attention modules to increase the network's receptive field and enable the backbone network to better acquire target spatial orientation features. A second fusion is then performed using an adaptive spatial feature fusion module to generate the final fused feature map. Finally, a weighted loss function and a localization loss function are combined to improve the detection accuracy of small targets. Lightweight improvements are then implemented to reduce model complexity, making it more suitable for embedded power UAV platforms, effectively improving the detection and defect identification of drop-out circuit breakers.

[0024] 2. This application is based on the YOLOx algorithm. An improved receptive field block is added after the shallow feature layer of the backbone network to increase the network's receptive field. A coordinate attention module is added to better acquire spatial orientation feature information of the target and improve the target localization accuracy. After the path aggregation network fuses features for the first time, an adaptive spatial feature fusion module is added. By assigning adaptive weight parameters to features of different scales, efficient re-fusion of multi-scale shallow and deep features is achieved. The weighted loss function and the localization loss function are combined to improve the detection accuracy of small targets, effectively improving the model's detection accuracy for small targets and achieving effective detection of small targets.

[0025] 3. This application adopts the LAMP pruning method based on layer adaptive amplitude, which successfully reduces the size of the improved model and lowers the computational requirements of mobile devices. While maintaining the accuracy of target recognition, it constructs a lightweight target detection network framework and defect detection and recognition system by compressing model parameters and improving algorithm efficiency, effectively reducing the workload of power grid inspection and improving inference efficiency. An efficient channel attention module is added to the neck feature fusion part. This mechanism can optimize the model's attention distribution on target features, generate target-oriented feature representations, significantly improve the model's ability to identify defect features, and enable the network to better focus on important areas, thereby improving target detection accuracy with a small computational cost.

[0026] To make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only thirteen of the drawings in this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 is a flowchart of the method described in this invention;

[0029] Figure 2 shows the improved YOLOx network structure;

[0030] Figure 3 shows the improved receptive field block structure.

[0031] Figure 4 is a structural diagram of the coordinate attention module;

[0032] Figure 5 shows the structure of the adaptive spatial feature fusion module;

[0033] Figure 6 is a schematic diagram of Wasserstein distance calculation;

[0034] Figure 7 shows the YOLOx network structure after the lightweighting operation;

[0035] Figure 8 is a flowchart of ShuffleNetV2;

[0036] Figure 9 is a structural diagram of the high-efficiency channel attention module;

[0037] Figure 10 is a flowchart of LAMP pruning;

[0038] Figure 11 shows the PR curve of the improved model;

[0039] Figure 12 shows the loss variation curves of the original model and the improved model;

[0040] Figure 13 shows the accuracy index curves of the original model and the improved model. Detailed Implementation

[0041] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the invention. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the invention.

[0042] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0043] The names of the messages or information exchanged between the multiple devices in the embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0044] Example

[0045] As shown in Figure 1-3, a method for monitoring the status of drop-out fuses based on multi-size features and deep learning includes the following steps:

[0046] S1. Perform data augmentation on the acquired images of the fallen fuse to form a dataset, and divide the dataset into a training set and a test set;

[0047] S2. First, the improved Receptive Field Block (RFB) and CoordAttention (CA) modules are added to the YOLOx backbone network. Then, the Adaptively Spatial Feature Fusion (ASFF) module is added to the PathAggregation Network (PANet) to perform secondary fusion of features at different scales, so that shallow and deep features are fully fused. Then, a loss function that combines Normalized Wasserstein Distance (NWD) and CompleteIntersection over Union (CIoU) is introduced. Finally, lightweight improvements are made to build a state monitoring model. The built state monitoring model is trained and validated using training and test sets.

[0048] S3. The trained and validated state monitoring model is used to identify the real-time acquired images of the drop fuse.

[0049] In step S1, the method for obtaining images of the drop fuse is to acquire multi-angle and multi-state images of the drop fuse using a drone.

[0050] Personnel used drones to obtain images of insulation defects in drop-out switches in a certain area, and obtained multiple images by adjusting the drone's shooting angle and distance.

[0051] In step S1, the data augmentation of the images of the fallen fuse is performed by: simulating photos of fallen fuses in multiple scenarios through contrast adjustment, scaling, Gaussian blur, brightness adjustment, saturation adjustment, Mosaic data augmentation, and Mixup data augmentation operations, thereby expanding the dataset, and labeling the dataset using Labelme annotation software.

[0052] Contrast adjustment: First, calculate the global average brightness of the image as a baseline. Then, subtract this average from each pixel value. Next, multiply the resulting pixel value by a randomly determined factor. Finally, add the original average back to complete the contrast adjustment.

[0053] Scaling: In image processing, scaling refers to changing the size of an image, including both enlarging and shrinking.

[0054] Gaussian blur: Gaussian blur reduces noise and details by smoothing the image, which reduces interference, highlights key features, and improves recognition accuracy and robustness. It is more stable when processing complex images.

[0055] Brightness adjustment: Brightness adjustment is also crucial for image recognition. It can improve the accuracy of image processing and the generalization performance of the model. By using the random_brightness function in the HSV color space to randomly adjust the image brightness, the diversity of training data can be increased, and the model's adaptability to different lighting conditions can be enhanced.

[0056] Saturation Adjustment: By using the random_saturation function in the HSV color space to randomly modify the values ​​of the saturation channels, the saturation of the image is randomly adjusted, enriching the diversity of image data and enhancing the model's generalization ability in complex scenes.

[0057] Mixup data augmentation: Mixup data augmentation technology operates based on the principle of overlay. Its core concept is to randomly select two original images and merge them according to a predetermined ratio to generate new image samples, thereby enhancing the generalization ability of the model.

[0058] Mosaic data augmentation: The Mosaic data augmentation method randomly selects four images from the training set, normalizes their size, and then stitches them together in a corner-by-corner arrangement to generate a large image. A region of the target size is then randomly cropped from this large image to serve as the final composite image. The generated image not only has richer background information but also increases the number of samples input into the model for each training iteration through stitching, thus increasing the batch size. This helps reduce the dependence on a specific batch size when performing batch normalization.

[0059] During the labeling process, two main defect types were clearly distinguished and marked: breakage and flashover.

[0060] In step S2, as shown in Figure 2, the secondary fusion of features at different scales is performed as follows: (1) An improved receptive field block (RFB) is added after the shallow feature layer CSP1_3, and a dilated convolutional layer is introduced to increase the receptive field of the network; (2) Three coordinate attention (CA) modules are added after the effective feature layers CSP1_3, CSP1_3 and CSP2_1 of the backbone feature extraction network, so that the backbone network can better acquire the spatial orientation feature information of the target; (3) The three effective feature maps P3, P4 and P5 generated are fused for the first time using the feature pyramid to generate three feature maps P3_out, P4_out and P5_out at different scales. Then, the adaptive spatial feature fusion (ASFF) module is used for the second fusion to generate the final fused feature map.

[0061] The improved receptive field block (RFB) is as follows: As shown in Figure 3, based on the RFB, the original 3×3 convolutional layer branch is transformed into two 1×3 and 3×1 convolutional layer branches by replacing the 5×5 convolutional layer in the RFB with a 3×3 convolutional layer, and replacing the 3×3 convolutional layer in the RFB with 1×3 and 3×1 convolutional layers, which can reduce the computational load of the model.

[0062] As shown in Figure 4, the operation process of coordinate attention (CA) mainly consists of two steps: coordinate information embedding and coordinate attention generation.

[0063] Coordinate information embedding: For the input feature map C×H×W, two one-dimensional global average pooling operations are applied to the input feature map, respectively, along the horizontal and vertical directions, to obtain feature maps with spatial information of size C×H×1 and C×1×W. and As shown in the following formula:

[0064]

[0065] in, This represents the output of the c-th channel with height h. The width is The output of the c-th channel has dimensions of C×H×1 and C×1×W.

[0066] Coordinate attention generation: The generated feature maps with spatial location information are concatenated to obtain a feature map of size C×1×(H+W). Then, a 1×1 convolution is used to compress the channel dimension from C to C / r dimensions at a compression ratio r. Finally, a BN layer and the ReLU function are used for non-linear activation to obtain an intermediate feature map of size C / r×1×(H+W). As shown in the following formula:

[0067]

[0068] in, For the characteristic of the join operation, For a convolution with a kernel size of 1×1, B(.) is a set of BN layers and non-linear activation functions ReLU.

[0069] The obtained intermediate feature maps Decomposed along the spatial dimension into horizontal attention tensors and vertical attention tensor Then, the channel dimension of the two spatial tensors is increased from C / r to C dimensions through two sets of 1×1 convolutions, and non-linear activation is performed using the Sigmoid function to obtain feature maps of sizes C×H×1 and C×1×W, respectively. and As shown in the following formula:

[0070]

[0071] in, It is the Sigmoid activation function. and All are 1×1 convolution operations.

[0072] Finally, the output feature map will be... and By multiplying the weighted coefficients x onto the initial input features, a new coordinate-aware feature map Y is generated, as shown in the following equation:

[0073] .

[0074] The Adaptive Spatial Feature Fusion (ASFF) module assigns adaptive weight parameters to features at different scales, making it easier for low-level feature information to be transmitted to high-level features, enabling the full fusion of shallow and deep features, improving the network's ability to retain detailed information in feature maps, achieving efficient fusion of multi-scale features, and effectively improving the problem of difficult detection of small targets at long distances in power distribution line images, as shown in Figure 5.

[0075] The calculation formula for the fusion process of Adaptive Spatial Feature Fusion (ASFF) is shown below:

[0076]

[0077] in, Represents the fused feature map. The feature vectors representing the feature map This represents the feature map of the Level 1 layer. Representative at the The fusion weights learned from the layer feature maps are processed by the Softmax function to make... ,and By updating the fusion weights through backpropagation of the error gradient, the multi-scale feature fusion of the model is more complete, thereby improving the model's detection accuracy.

[0078] In step S2, based on the localization loss function (CIoU), the bounding box evaluation method based on Wasserstein distance normalization weighted loss (NWD) models the target detection box as a two-dimensional Gaussian distribution parameter space, and achieves more accurate box matching through probability distribution similarity measurement.

[0079] Since the broken and flashover problems of drop fuses are relatively small targets in images, in order to further improve the model's recognition of small targets, the weighted loss function (NWD) and the localization loss function (CIoU) are integrated in YOLOvx. By reducing the localization error of small targets, the model's detection accuracy for small targets is effectively improved.

[0080] The CIoU (Correction-in-Unit) loss function is highly sensitive to positional deviations of small targets. Even small deviations can lead to a significant decrease in IoU, thus affecting the model's detection accuracy. In this application, the task of identifying flashover defects in the insulation of drop-out switches can be considered as a small target detection problem. For small target detection tasks (when some drop-out switch insulation defects manifest as flashover defects), bounding box offsets cause a significant decrease in the Cross-Union Ratio (CIU). For regular-sized targets, the CIU change is relatively small when bounding box offsets occur. However, the CIoU loss function is more sensitive to targets with low pixel proportions. Even small positioning deviations can cause large fluctuations in the CIU value, thus affecting the overall performance of the detection system.

[0081] To address the limitations of the traditional localization loss function (CIoU) in detecting small targets in complex scenes, a new NWD bounding box evaluation method based on Wasserstein distance normalization is proposed. The target detection box is modeled as a two-dimensional Gaussian distribution parameter space, and more accurate box matching is achieved through probability distribution similarity measurement.

[0082] Figure 6 illustrates the Wasserstein distance calculation.

[0083] Specifically, the parameters for the rectangular detection box are: Define its spatial properties; the coordinates of its center point are... The rectangular region has a width of w and a height of h; the geometric characteristics of its maximum inscribed ellipse are shown in the following formula:

[0084]

[0085] Ellipse geometric center coordinates Corresponding center point of the rectangle Its principal axis radius and These correspond to half the width and half the height of the rectangle, respectively, and the specific relationship is as follows: .

[0086] The probability density function of a two-dimensional Gaussian distribution is shown in the following equation:

[0087]

[0088] In a two-dimensional Gaussian distribution, the coordinate vectors x and y, the mean vector μ, and the covariance matrix are... Corresponding spatial positions Distribution center and morphological parameters. When When the ellipse is defined as shown in the following formula, it is equivalent to a Gaussian distribution contour with equal probability density. Therefore, the rectangular detection box parameters... It can be transformed into a two-dimensional Gaussian distribution. :

[0089]

[0090] The geometric distribution characteristics of the bounding boxes and detection boxes can be modeled as two-dimensional Gaussian probability distributions based on the equation of an ellipse. , To indicate, among which for Norms. Then their second-order Wasserstein distance is expressed by the following equation:

[0091]

[0092] In the detection box parameters With annotation box parameters Under the distribution modeling framework, the corresponding Gaussian distributions of the two are and The squared Wasserstein distance It can be analyzed as a linear combination of center offset and morphological difference, expressed as follows:

[0093]

[0094] By standardizing the bounding boxes to correspond to a Gaussian model distribution, NWD establishes a similarity evaluation system unaffected by target size, optimizing the similarity metric. Its advantages include: firstly, eliminating scale sensitivity and achieving stable distribution distance calculation; secondly, accurately quantifying distribution similarity even if target boxes do not overlap or contain each other. The core mathematical expression of this method is shown below, and it possesses closed-form solution properties:

[0095]

[0096] In step S2, as shown in Figure 7, the lightweight improvement process is as follows: (1) Replace the backbone network with the lightweight ShuffleNetV2 network; (2) Expand the convolution kernel and prune the convolutional layers of ShuffleNetV2; (3) Add an efficient channel attention (ECA) module to PANet to make the network pay more attention to key channels with a small computational cost, so that the features are more fully integrated.

[0097] ShuffleUnit is the basic building block of ShuffleNetV2, consisting of Unit1 and Unit2. As shown in Figure 8, Unit1 is the basic module of ShuffleNetV2, and Unit2 is the downsampling module of ShuffleNetV2. Both modules use channel merging. Here, DWConv represents a depthwise convolutional layer, performing convolution operations on the feature map of each channel; PWConv represents a pointwise convolutional layer, performing convolution operations on the feature map of each pixel; Conv represents a regular convolutional layer, performing convolution operations on the feature map; Channel Split represents splitting the feature map along the channel dimension, allowing different branches to process different feature information independently; Channel Shuffle represents the channel shuffling module, randomly shuffling the feature map along the channel dimension and then integrating it, fusing the feature information between channels of different branches; Concat represents concatenating the feature map along the channel direction, merging the feature information processed by different branches together.

[0098] As shown in Figure 8(a), the Unit1 basic module. Unit1 uses only a two-branch structure, connecting the feature maps on the two branches along the channel dimension to ensure that the number of channels in the output feature map is the same as the number of channels in the input feature map, thereby reducing the cost of memory access time. At the same time, Unit1 does not use element-wise addition operations, further reducing the time consumption of memory access.

[0099] As shown in Figure 8(b), the Unit2 downsampling module uses the convolution stride to perform downsampling operations. When the stride is 2, the height and width of the output feature map are half of the input feature map; while when the stride is 1, the size of the output feature map is the same as the input feature map.

[0100] The Efficient Channel Attention (ECA) module is a channel attention module for deep CNNs. This module proposes a local cross-channel interaction strategy that does not require dimensionality reduction, effectively avoiding the impact of dimensionality reduction on channel attention learning performance. Furthermore, appropriate cross-channel interaction can significantly reduce model complexity while maintaining performance. Compared to fully connected layers, the ECA module establishes connections between local channels through a 1D convolutional structure, highlighting important features and thus improving performance. This addresses the performance degradation caused by dimensionality reduction in the first fully connected layer of the activation module in the SENet architecture.

[0101] The ECA module performs pooling and feature extraction on the input feature layer using global average pooling and 1D convolution. It then uses a sigmoid activation function to obtain the weights for each feature point, which are multiplied by the input feature layer to generate a new feature layer. Here, GAP represents global average pooling. This represents the Sigmoid activation function. Represents the dot product. The kernel size is shown in Figure 9.

[0102] The ECA module employs grouped convolution, which increases the channel dimension... With kernel size Establish a proportional mapping relationship, as shown in the following formula. Where, and The values ​​are 2 and 1 respectively.

[0103]

[0104] Among them, the LAMP pruning method based on layer adaptive amplitude is adopted. By performing layer adaptive recalibration of the magnitude of the weight parameters, weights with low LAMP scores are pruned first. LAMP automatically determines the layer sparsity until the global sparsity requirement is met. The iterative process of LAMP pruning is shown in Figure 10.

[0105] Its advantages lie in that no additional hyperparameter tuning is required, complex calculation processes are avoided, and it significantly outperforms traditional pruning strategies in diverse image classification tasks, with stronger universality. The design idea of LAMP is as follows:

[0106] Since the dimensions of the weights in the fully connected layer and the convolutional layer are different, each weight tensor is flattened into a one-dimensional vector W. By calculating the magnitude of each vector W and arranging |W| in ascending order. u and v represent the indices of the weight magnitudes |W|, and W(u) and W(v) represent the weight terms mapped by the indices u and v. Since |W| is arranged in ascending order, so u < v, then W(u) < W(v). At this time, the definition of the LAMP score is shown in the following formula:

[0107]

[0108] According to the above formula, the numerator represents the weight magnitude, and the denominator represents the cumulative sum of all weights higher than the weight at index u in the current layer, quantifying the importance of the connection corresponding to index u. Since the weights are arranged in descending order of indices, the magnitude of W[u] increases as the index u increases, which results in a decrease in the number of weights covered by the denominator. The finally derived threshold expression is shown as follows:

[0109]

[0110] According to the above formula, the LAMP score measures the importance between the weight terms and the channels. During pruning, the weights with low LAMP scores are preferentially pruned. Different from local pruning, LAMP automatically determines the hierarchical sparsity until the global sparsity requirement is met. The LAMP score ensures that there is at least one optimal channel with a score of 1 in each layer, avoiding the problem of layer failure.

[0111] Next, the validation set obtained in this application is used to validate this application.

[0112] The experimental results show that the precision of the model disclosed in this application has increased from 91.3% to 92.3%, and the recall rate has increased from 79.9% to 82.0%. It can be found from the P-R curve shown in Figure 11 that the AP value of the pollution flash category has increased by 4% to 84.9%. This experimental data fully proves that the improvement strategy of this application has a significant improvement effect on the detection effect of small targets. In addition, the mean average precision has increased by 2.2% to reach 89.1%, which is better than the results of each single improvement.

[0113] It can be seen from the comparison of experimental data that the model of this application has achieved significant improvements in various evaluation indicators and obtained the highest mAP value, indicating that the fusion improvement scheme of this research is effective.

[0114] Figures 12 and 13 show a comparison of the training result curves of the original YOLOvx model and the improved fusion model. The loss curves and accuracy metric curves of the two models are also displayed.

[0115] The 20 experimental curves mentioned above cover key parameters such as bounding box regression loss, distribution focusing loss, and classification loss in the training and validation data, as well as the dynamic changes in performance metrics such as mAP@0.5 and mAP@0.5:0.95. The loss function curve represents the average trend of the object detection loss; a lower value generally corresponds to better detection performance. The mAP metric is quantified by the area under the PR curve. Experimental results show that the improved YOLOvx algorithm in this application exhibits significant advantages in both loss convergence speed and accuracy.

[0116] Based on the same inventive concept, this invention also provides a drop-out fuse status monitoring system based on multi-size features and deep learning, characterized in that it includes:

[0117] Data processing module: Performs data augmentation on the acquired images of the fallen fuse to form a dataset, and divides the dataset into training set and test set;

[0118] The monitoring model construction module adds the improved receptive field block and coordinate attention module to the YOLOx backbone network, adds the adaptive spatial feature fusion module to PANet, performs secondary fusion of features at different scales to fully integrate shallow and deep features, then introduces a loss function that combines weighted loss and localization loss, and finally performs lightweight improvements to build a state monitoring model. The built state monitoring model is then trained and validated using training and testing sets.

[0119] Real-time monitoring module: The trained and validated state monitoring model is used to identify the real-time images of the fallen fuse.

[0120] In this application, the other technical features of the drop fuse status monitoring system based on multi-size features and deep learning are the same as those disclosed in the above method embodiments, and will not be repeated here.

[0121] Based on the same inventive concept, the present invention also provides an electronic device, which may include a processor, a memory, a display screen, an input device, and a peripheral terminal port.

[0122] Furthermore, the processor can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0123] Furthermore, memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. The memory in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0124] Furthermore, the peripheral terminal interface can be used to connect at least one I / O (Input / Output) related peripheral terminal to the processor and memory. In some embodiments, the processor, memory, and peripheral terminal interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, memory, and peripheral terminal interface can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0125] Furthermore, the display screen is used to display a UI (User Interface). This UI can include graphics, text, icons, videos, and any combination thereof. When the display screen is a touch screen, it also has the ability to collect touch signals on or above the surface of the display. These touch signals can be input as control signals to the processor for processing. In this case, the display screen can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there can be one display screen, located on the front panel of the electronic device; in other embodiments, there can be at least two display screens, respectively located on different surfaces of the electronic device or in a folded design; in still other embodiments, the display screen can be a flexible display screen, located on a curved or folded surface of the electronic device. Furthermore, the display screen can even be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0126] Furthermore, input devices are used to input data and information into a computer, serving as a bridge for communication between the computer and the user or other devices. Input devices are one of the main devices for information exchange between the user and the computer system. These include keyboards, mice, cameras, scanners, light pens, handwriting input tablets, joysticks, and voice input devices. An input device is a device used by a person or external device to interact with a computer, inputting raw data and programs that process that data into the computer. Computers can receive various types of data, including both numerical and non-numerical data such as graphics, images, and sounds, all of which can be input into the computer through different types of input devices for storage, processing, and output.

[0127] Based on the same inventive concept, this application also provides a computer-readable storage medium storing a computer program that can be executed by a processor to implement the above-described method.

[0128] The technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a readable computer storage medium and includes several instructions / computer programs to cause an Internet of Things device (which may be a personal computer, server, or network terminal, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as electronic terminals such as computers, mobile phones, laptops, tablets, and cameras that have the aforementioned storage media.

[0129] The description of the execution process of program data in a computer-readable storage medium can be found in the descriptions in the various method embodiments of this application above, and will not be repeated here.

[0130] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for monitoring the status of drop-out fuses based on multi-size features and deep learning, comprising the following steps: S1. Data augmentation is performed on the acquired images of the fallen fuse to form a dataset, which is then divided into a training set and a test set. S2. First, the improved receptive field block and coordinate attention module are added to the YOLOx backbone network. Then, the adaptive spatial feature fusion module is added to the path aggregation network. Secondary fusion is performed on features of different scales to fully integrate shallow and deep features. Then, a loss function that combines weighted loss and localization loss is introduced. Finally, lightweight improvements are made to build a state monitoring model. The built state monitoring model is trained and validated using the training set and the test set. S3. The trained and validated state monitoring model is used to identify the real-time acquired images of the drop fuse.

2. The method and system for monitoring the status of drop-out fuses based on multi-size features and deep learning according to claim 1, characterized in that: In step S1, the method for obtaining images of the drop fuse is to acquire multi-angle and multi-state images of the drop fuse using a drone.

3. The method and system for monitoring the status of drop-out fuses based on multi-size features and deep learning according to claim 1, characterized in that: In step S1, the data augmentation method for the images of the fallen fuse is as follows: by performing contrast adjustment, scaling, Gaussian blur, brightness adjustment, saturation adjustment, Mosaic data augmentation and Mixup data augmentation operations, the images of the fallen fuse in multiple scenarios are simulated to expand the dataset, and the dataset is labeled using Labelme annotation software.

4. The method and system for monitoring the status of drop-out fuses based on multi-size features and deep learning according to claim 1, characterized in that: In step S2, the secondary fusion of features at different scales is performed as follows: (1) An improved receptive field block is added after the shallow feature layer CSP1_3, and a dilated convolutional layer is introduced to increase the receptive field of the network; (2) Three coordinate attention modules are added after the effective feature layers CSP1_3, CSP1_3 and CSP2_1 of the backbone feature extraction network, so that the backbone network can better acquire the spatial orientation feature information of the target; (3) The three generated effective feature maps P3, P4 and P5 are fused for the first time using the feature pyramid to generate three feature maps P3_out, P4_out and P5_out at different scales. Then, the adaptive spatial feature fusion module is used for the second fusion to generate the final fused feature map.

5. The method and system for monitoring the status of drop-out fuses based on multi-size features and deep learning according to claim 1, characterized in that: In step S2, the process of introducing a loss function that fuses weighted loss and localization loss is as follows: Based on the localization loss function, the bounding box evaluation method based on the weighted loss function with Wasserstein distance normalization is used to model the target detection box as a two-dimensional Gaussian distribution parameter space, and achieve more accurate box matching through probability distribution similarity measurement.

6. The method and system for monitoring the status of drop-out fuses based on multi-size features and deep learning according to claim 1, characterized in that: in In step S2, the lightweight improvement process is as follows: (1) Replace the YOLOx backbone network with the lightweight ShuffleNetV2 network; (2) Expand the convolution kernel and prune the convolutional layers of ShuffleNetV2; (3) Add an efficient channel attention module to PANet to make the network pay more attention to key channels with a small computational cost, so that the features are more fully integrated.

7. The method and system for monitoring the status of drop-out fuses based on multi-size features and deep learning according to claim 6, characterized in that: The specific pruning process is as follows: The LAMP pruning method based on layer adaptive amplitude is adopted. By recalibrating the amplitude of the weight parameters at the layer level, weights with low LAMP scores are pruned first. LAMP automatically determines the layer sparsity until the global sparsity requirement is met.

8. An image quality optimization system based on multi-scale feature decoupling and dynamic fusion, characterized by: include: Data processing module: Performs data augmentation on the acquired images of the fallen fuse to form a dataset, and divides the dataset into training set and test set; The monitoring model construction module adds the improved receptive field block and coordinate attention module to the YOLOx backbone network, adds the adaptive spatial feature fusion module to PANet, performs secondary fusion of features at different scales to fully integrate shallow and deep features, then introduces a loss function that combines weighted loss and localization loss, and finally performs lightweight improvements to build a state monitoring model. The built state monitoring model is then trained and validated using training and testing sets. Real-time monitoring module: The trained and validated state monitoring model is used to identify the real-time images of the fallen fuse.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images