Fault indicator damp detection method based on multi-branch feature fusion and related device

Abstract

The invention discloses a fault indicator damp detection method based on multi-branch feature fusion and a related device, and belongs to the technical field of power system equipment fault state monitoring, and the method comprises the steps: obtaining an image of a fault indicator collection unit; identifying a candidate area in the image of the fault indicator acquisition unit; extracting features of the candidate region, wherein the features of the candidate region comprise an intensity uniformity matrix, a correlation matrix, a color saturation standard deviation, a boundary regularity index and a condensation region distribution density; training a multi-branch convolutional neural network model by taking the features of the candidate region as input to obtain a trained damp detection model; and extracting features of a candidate area of the image of the acquisition unit of the to-be-detected fault indicator, and inputting the features of the candidate area into the trained damp detection model to obtain a damp detection result of the to-be-detected fault indicator. According to the method, the detection robustness in a complex scene is improved, and the adaptability is high.

Description

Technical Field

[0001] The present invention belongs to the technical field of fault status monitoring of power system equipment, and particularly relates to a method and related device for detecting moisture ingress of a fault indicator based on multi-branch feature fusion. Background Art

[0002] A fault indicator is an indicating device installed on a distribution line for detecting short-circuit faults and grounding faults occurring on the line. By uploading fault waveforms and flag signals to the master station, grounding judgment and fault section judgment are realized. With the construction and development of the smart grid, it has been widely used in the distribution network due to its low cost and convenient installation.

[0003] Common faults of fault indicators include problems such as abnormal waveforms, failed waveform recording, disconnection, etc. Among them, abnormal waveforms are mainly caused by outdoor rain erosion and the influence of day-night temperature difference, and the acquisition unit of the fault indicator is affected by moisture and corrodes the internal circuit. The grounding fault judgment algorithm of the fault indicator needs to rely on fault waveforms and comprehensively judge in combination with the line topology. Abnormal waveforms sent to the master station will greatly reduce the judgment accuracy for grounding faults, significantly increasing the operation and maintenance cost and fault recovery time of distribution automation equipment.

[0004] Therefore, it is necessary to regularly detect the moisture ingress situation in the acquisition unit of the fault indicator. At present, the detection of water ingress and moisture in the acquisition unit of the fault indicator is usually achieved by field personnel going to the site to climb the pole and disassemble the acquisition unit of the fault indicator one by one for inspection. The entire process of climbing the pole and disassembling the acquisition unit requires the cooperation of three people, one to climb the pole and disassemble, one to be responsible for transmitting the insulating pole and inspecting the acquisition unit, and one to supervise. The whole process takes about half an hour, requires high-intensity work from the operators, and has a certain degree of danger. Summary of the Invention

[0005] The purpose of the present invention is to provide a method and related device for detecting moisture ingress of a fault indicator based on multi-branch feature fusion to solve the technical problems of inconvenient operation, long time consumption, and high danger in the working environment of existing technologies.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions: In a first aspect, the present invention provides 1. A method for detecting moisture ingress of a fault indicator based on multi-branch feature fusion, including: Obtaining an image of the acquisition unit of the fault indicator; Identifying candidate regions in the image of the acquisition unit of the fault indicator; Extracting features of the candidate regions, where the features of the candidate regions include an intensity uniformity matrix, a correlation matrix, a standard deviation of color saturation, a boundary regularity index, and a condensation region distribution density; Taking the features of the candidate regions as inputs, train a multi-branch convolutional neural network model to obtain a trained moisture detection model; Extract the features of the candidate regions in the image collected by the fault indicator to be detected, and input the features of the candidate regions into the trained moisture detection model to obtain the moisture detection result of the fault indicator to be detected.

[0007] Further, the recognition of the candidate regions in the image of the fault indicator acquisition unit includes the following steps: Obtain the image of the fault indicator acquisition unit; the image includes images under illumination conditions, different angles, and backgrounds; Perform gradient calculation on the image of the fault indicator acquisition unit to generate a gradient image, use the watershed algorithm for image segmentation to obtain the boundary contours of each region, and perform connected component analysis on the segmentation result to obtain candidate regions; Calculate the area and boundary regularity index BRI of each candidate region; screen out the water accumulation regions based on the area and boundary regularity index BRI of each candidate region; For all pixel points in the candidate regions other than the water accumulation regions, calculate the standard deviation of the polarization angle, polarization intensity, polarization consistency index PCI, and polarization intensity ratio PIR; Based on the polarization consistency index PCI, segment to obtain an optimized foreground and background label map; Screen out the condensation regions based on the standard deviation of the polarization angle, polarization intensity, boundary regularity index BRI, polarization consistency index PCI, and polarization intensity ratio PIR.

[0008] Further, the extraction of the features of the candidate regions includes the following steps: Extract the intensity uniformity matrix and correlation matrix in the gray-level co-occurrence matrix features of the candidate regions as the inputs of the texture feature branch of the multi-branch convolutional neural network model; Select the standard deviation of the color saturation in the color features as the input of the color feature branch of the multi-branch convolutional neural network model; Select the boundary regularity index BRI and the condensation region distribution density DLD as the inputs of the global feature branch of the multi-branch convolutional neural network model; Take the inputs of the texture feature branch, the inputs of the color feature branch, and the inputs of the global feature branch as the inputs of the multi-branch convolutional neural network model, extract the local features of all the inputs, and then merge them in the feature fusion layer to obtain a feature vector.

[0009] Further, the training of the multi-branch convolutional neural network model with the features of the candidate regions as inputs to obtain a trained moisture detection model includes: Cut the image of the fault indicator acquisition unit into sizes of , small patches with a step size of 25% of the original patch size are used to extract the feature vectors of the small patches as the input to train a multi-branch CNN model. The output of the multi-branch CNN model calculates the class probability distribution through the Softmax function. During training, the categorical cross-entropy loss function is used to measure the difference between the prediction and the true label, and the parameters are adjusted through the Adadelta optimization algorithm. The training is judged to converge by monitoring the validation loss, and a trained moisture detection model is obtained.

[0010] Furthermore, the multi-branch convolutional neural network model includes an input layer, a feature fusion layer, a global processing layer, and an output layer connected in sequence, and the input layer includes three branches.

[0011] Furthermore, the image of the fault indicator to be detected is obtained by a multi-spectral imaging module installed on the distribution network fault indicator.

[0012] In a second aspect, the present invention provides a multi-spectral imaging module to acquire images of the fault indicator acquisition unit; a candidate region recognition module for identifying candidate regions in the image of the fault indicator acquisition unit; a feature extraction module for extracting features of the candidate regions, where the features of the candidate regions include an intensity uniformity matrix, a correlation matrix, a color saturation standard deviation, a boundary regularity index, and a condensation region distribution density; a classification module for extracting features of candidate regions in the image of the acquisition unit of the fault indicator to be detected, and inputting the features of the candidate regions into the trained moisture detection model to obtain the moisture detection result of the fault indicator to be detected.

[0013] In a third aspect, the present invention provides an electronic device, including: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for detecting moisture in a fault indicator based on multi-branch feature fusion according to any one of the first aspects of the present invention.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, it implements the method for detecting moisture in a fault indicator based on multi-branch feature fusion according to any one of the first aspects of the present invention.

[0015] Fifth aspect, the present invention provides a computer program product, including a computer program which, when executed by a processor, implements the moisture detection method for fault indicators based on multi-branch feature fusion according to any one of the first aspects of the present invention.

[0016] Compared with the prior art, the present invention has at least the following beneficial technical effects: 1) Improving the accuracy of ROI extraction by combining polarization characteristics: By studying the polarization characteristic differences among the aging area, the condensation (water droplet) area, and the water accumulation area, the present invention optimizes the image using the polarization consistency index (PCI) and the polarization intensity ratio (PIR). Due to the different manifestations of different moisture types (such as condensation and water accumulation) in the polarization angle distribution and polarization intensity, the present invention accurately distinguishes through these differences, significantly improving the accuracy of ROI area extraction, especially in the case where the aging area of the transparent bottom cover interferes with the detection.

[0017] 2) Optimizing foreground and background segmentation based on gradient and morphological operations: By performing gradient calculation, threshold segmentation, and morphological operations on the image F, the present invention can effectively separate the foreground and background of the moisture area. Using the watershed algorithm for boundary contour extraction and combining connected component analysis, the boundary point set of the candidate area is extracted. Through geometric analysis and the calculation of the boundary regularity index (BRI), the segmentation effect is further improved, successfully eliminating the interference of scattered light in the aging area, and thus more accurately identifying the ROI area.

[0018] 3) Integrating multi-branch network features to improve detection accuracy and robustness: Using a multi-channel branch CNN model to process multi-modal features such as texture, color, shape, and global statistical features in parallel, by extracting local features (such as intensity uniformity, correlation index, etc.) of each ROI area and combining full-image features (such as boundary regularity index (BRI) and condensation area distribution density (DLD), etc.), various types of feature information are comprehensively mined and fused for classification, improving the detection robustness in complex scenarios and the adaptability of the model. At the same time, through quantization indicators such as the condensation index, the quantitative analysis of the condensation area distribution density and severity is refined, providing a scientific basis for the diagnosis and classification of moisture faults. Description of the Drawings

[0019] Figure 1 It is a schematic diagram of the feature fusion process of the multi-branch CNN; Figure 2 It is a flowchart of the moisture detection method for fault indicators; Figure 3 It is a structural block diagram of the moisture detection device for fault indicators based on multi-branch feature fusion provided by the embodiment of the present invention; Figure 4 It is a block diagram of an electronic device according to an embodiment of the present invention. Specific Embodiments

[0020] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned accompanying drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0022] The present invention will be further described in detail below with reference to the accompanying drawings: Embodiment 1 Referring to Figure 1 , this embodiment provides a moisture detection method for a fault indicator based on multi-branch feature fusion, including the following steps: Step S001, obtain the image ROI (Region of Interest) area, and the ROI area includes areas where condensation (water droplets) or water accumulation may exist; The moisture phenomenon in the fault indicator acquisition unit is usually divided into two cases. One is the formation of water mist affected by the temperature difference and air humidity inside and outside the fault indicator acquisition unit, which will eventually converge into condensation and adhere to the transparent bottom cover of the fault indicator acquisition unit. The other is the infiltration of water at the connection between the housing and the transparent bottom cover of the fault indicator acquisition unit due to external force and rain and snow erosion, resulting in water accumulation sinking into the transparent bottom cover of the fault indicator acquisition unit. In order to reduce the influence of the aging area of the transparent bottom cover on the detection of moisture conditions, research is carried out on the polarization characteristic differences among the aging area, the condensation (water droplet) area, and the water accumulation area. Different surfaces or media have different response characteristics to polarized light, and polarization information can be used to analyze regional characteristics.

[0023] The surface structure of the aging area is uniform and scatters light weakly. The scattering characteristics of each pixel point to the light source are similar, without obvious differences between the center and the edge. The overall reflection intensity is low and changes little. Therefore, its polarization angle varies slightly within the local area and has a low variance. The polarization intensity is evenly distributed without obvious local peaks. As a non-uniform medium, due to its irregular structure, the polarization direction of the scattered light by the water droplet will change with the shape and size of the water droplet. Especially at the center of the water droplet, the reflection and refraction of light are more complex, resulting in a significant change in the polarization angle distribution within the local area and a high variance. And due to the enhanced effect of internal refraction and reflection within the water droplet, the reflection intensity at the center area of the water droplet is the strongest, and the polarization intensity gradually weakens at the edge area. The spatial distribution of the polarization intensity shows significant local maxima. In view of the formation mechanism of the water accumulation area, the water droplets gather at the bottom of the transparent bottom cover of the fault indicator under the action of gravity. Due to the stability of its structure, the water accumulation liquid surface is usually flat, and the reflection of light is relatively uniform. The polarization angles in the water accumulation area are similar, and the change in the polarization angle within the local area is small, similar to the aging area, both having a low standard deviation of the polarization angle.

[0024] To simulate the actual scenario of manually observing the moisture condition of the acquisition unit of the fault indicator, it is necessary to use a camera to take pictures under strong light, low light and backlight conditions respectively. On the premise of ensuring that the moisture area can be observed, images of the acquisition unit of the fault indicator are obtained from different angles and include different backgrounds (such as interference objects like power facilities, tree branches, etc.) to enhance the sample diversity and robustness. Calculate the gradient of the image F obtained by the camera to generate a gradient image, generate a foreground marker map and a background marker map through threshold segmentation and morphological operations, use the watershed algorithm for image segmentation to obtain the boundary contours of each area, and perform connected component analysis on the segmentation results to obtain candidate regions , extract the set of boundary points of the candidate region and denote it as , where is the abscissa of the i-th pixel point in the candidate region, is the ordinate of the i-th pixel point in the candidate region.

[0025] Calculate the area A of the connected component using the polygon area formula:

[0026] where, is the total number of boundary points, and the boundary points are arranged in a clockwise or counterclockwise order, where is the abscissa of the (i + 1)-th pixel point in the candidate region, is the ordinate of the (i + 1)-th pixel point in the candidate region.

[0027] Calculate the geometric center of the boundary points

[0028]

[0029] Calculate the distance from each boundary point to the geometric center :

[0030] Obtain the average radius from the boundary point to the geometric center :

[0031] Calculate the circle fitting deviation, that is, the radius deviation of all boundary points :

[0032] Substitute the deviation into the formula to calculate the boundary regularity index :

[0033] The boundary regularity index (BRI) is used to quantify the regularity degree of the region boundary, especially its degree of approaching a circle. Since the structure of the transparent bottom cover of the fault indicator is circular, the water accumulation area formed at the bottom also shows the characteristic that the boundary is approximately circular, with a high boundary regularity, while the boundary of the aging area is often blurred and has a low regularity. The radius deviation is smaller, the larger the BRI, and approaching 1 indicates a high boundary regularity; if the boundary is irregular (such as the aging area, condensation point), is larger and the BRI approaches 0; some condensation points are approximately circular but have a small area. Therefore, for the candidate regions obtained by solving the BRI values and areas set thresholds. The boundary regularity index indicates approaching a circle, and the area indicates meeting the minimum area for water accumulation formation. Screen out the regions that meet the conditions and mark the set of their boundary points. Thus, the water accumulation area J is obtained.

[0034] For all pixel points of the candidate regions (denoted as ) outside the water accumulation area J, calculate the standard deviation and mean of the polarization intensity and polarization angle.

[0035] Polarization intensity The calculation formula is:[[]]

[0036] Polarization angle The calculation formula is:[[]]

[0037] Where:[[]] Total intensity, quantity diagonal direction polarization component

[0038] Calculate the standard deviation of the polarization angles of the pixel points within the candidate region by the following formula :

[0039] Calculate the mean value of the polarization angles of the pixel points within the candidate region by the following formula

[0040] where is the number of pixel points is the pixel point polarization angle

[0041] According to the standard deviation and mean value of the polarization angles of the pixel points within each candidate region obtain the polarization consistency index PCI:

[0042] is an adjustment parameter, and the empirical value is taken as 10 -6 .

[0043] According to the maximum value and mean value of the polarization intensities of the pixel points within each candidate region obtain the polarization intensity ratio PIR:

[0044] where is the maximum value of the polarization intensity within the candidate region is the mean value of the polarization intensity within the candidate region

[0045] The polarization consistency index is used to reflect the polarization angle distribution of each candidate region . For the aging and whitening region, due to the uniform distribution of scattered light, the standard deviation of its polarization angle is small, showing low polarization consistency and a low PCI value. If the PCI value of the candidate region is lower than a certain threshold, then set the pixel intensity value of this block to 0, and perform threshold segmentation in this way to obtain the marked maps of the optimized foreground (highlighted region) and background (lowlight region), and perform boundary contour segmentation detection again to obtain the processed candidate region, eliminating the interference of the scattered light in the aging region on the detection of internal condensation points

[0046] The polarization intensity ratio is used to reflect the polarization intensity distribution of the region. In the water droplet region, the reflection ability of the center region of the water droplet is the strongest, and the polarization intensity ratio is usually higher than that of the surrounding regions. For each candidate region Calculate the PIR values of all pixels in the individual calculation region. Here, the block with the highest PIR value in the region is regarded as the center of the water droplet, and this is used as the initial seed point for the region growing algorithm. Starting from the seed point, the region growing algorithm (Region Growing Algorithm) gradually checks the PIR values of adjacent pixels. If it meets the basic condition of being greater than a certain threshold, then this pixel is added to the growing region, continuously expanding to the four-neighborhood or eight-neighborhood, checking the pixels that meet the conditions and adding them to the growing region. The expansion process stops when the conditions are not met or the boundary is reached. The growing region is the set of all pixels that meet the PIR value conditions, forming the complete contour of the condensation (water droplet) region. Finally, extract the contour of the growing region as the final ROI region, further refining the segmentation region, increasing the accuracy of the boundary of the water droplet region, and ensuring more accurate subsequent feature extraction and analysis. Thus, the condensation region L is obtained.

[0047] The condensation region L refers to the region identified as water droplets in the candidate region. These regions are screened by features such as polarization characteristic analysis, boundary regularity analysis, polarization consistency index (PCI), and polarization intensity ratio (PIR), and finally further refined by the region growing algorithm to clearly extract the boundary contour of the water droplets.

[0048] Specifically, the condensation region L includes: 1. Regions that meet the polarization characteristic differences: • The polarization angle varies significantly in the local region with a high variance.

[0049] • The spatial distribution of the polarization intensity shows local maxima (such as the strongest polarization intensity at the center of the water droplet and gradually weakening at the edges).

[0050] 2. Irregular small regions screened by the boundary regularity index (BRI): • The boundary regularity index is low, showing the irregular boundary characteristics unique to water droplets.

[0051] 3. Regions screened by the polarization consistency index (PCI): • The PCI value is relatively high, reflecting the obvious change in the polarization angle inside the water droplet region.

[0052] 4. Regions screened by the polarization intensity ratio (PIR): • The PIR value forms a local peak in the region, thereby determining the center point of the water droplet and expanding through the region growing algorithm to form a complete water droplet region.

[0053] In summary, the condensation area L ultimately refers to those water drop areas with significant polarization characteristics, low boundary regularity, high polarization consistency, polarization intensity ratio forming a local peak, and expanded by the regional growing algorithm. These areas are clearly distinguished from aging areas and water accumulation areas, ensuring the accuracy of subsequent feature extraction and analysis.

[0054] Through the above steps, the water accumulation area J and the condensation area L are clearly segmented, and the influence of the aging area is significantly reduced, thereby ensuring the accuracy of subsequent feature extraction.

[0055] Step S002: Feature extraction and processing Traditional deep learning neural networks usually use a single input and are trained only through global image features, texture features, etc. This method lacks the ability to process multi-dimensional and multi-modal features of complex data sets and cannot cope with different feature types. It usually relies on global features or performs feature fusion through some simple feature pooling methods, which makes it difficult to fully explore the unique information of each feature and lacks fine-grained processing and collaborative utilization of different feature sources, which may lead to the loss of key information, especially in complex scenes or tasks. Compared with the original network, the multi-branch network has significant advantages, processing different types of features at the same time, and improving the expressiveness of features. The feature fusion layer effectively integrates the outputs of each branch network, reduces information loss, and enhances the comprehensive understanding of different features. Through diversified feature representation and learning strategies, the multi-branch network can better cope with new data and complex scenes, improve the robustness of the model, process different features in parallel, avoid redundant calculations, and improve the efficiency of the model. Therefore, when facing multi-modal data and complex tasks, the multi-branch network can better play its advantages and perform better in performance and efficiency than the traditional original network.

[0056] In order to construct a multi-branch convolutional neural network, it is necessary to first define different input modules. The input modules are used to receive different types of features and pass them to their respective branch networks for processing. Here, the intensity uniformity and correlation indexes in the grayscale co-occurrence matrix features of the candidate region are selected as the input of the texture feature branch.

[0057] Among them, the gray level co-occurrence matrix (GLCM) is calculated based on the gray distribution relationship of the image. It describes the spatial co-occurrence relationship of the gray values ​​between adjacent pixels in the image and is used to extract the texture features of the image.

[0058] In this method, GLCM is calculated based on the candidate region (ROI region) in the fault indicator acquisition unit image. The candidate region includes possible aging regions, condensation (water droplet) regions, and water accumulation regions. GLCM extracts the frequency of occurrence of neighboring pixel pairs from the pixel grayscale values ​​of these regions and establishes the co-occurrence matrix between grayscale levels.

[0059] The construction process of the GLCM includes: 1. Select the relative positions of pixel pairs: Such as the horizontal direction, vertical direction, 45° direction, 135° direction, etc., to define the distance and direction between adjacent pixels.

[0060] 2. Calculate the occurrence frequency of gray-level combinations: The statistical indicator acquisition unit counts the number of occurrences of different gray-level combinations in adjacent pixel pairs (such as pixel A and pixel B) in the image.

[0061] 3. Generate a matrix: Map these gray-level combinations and their occurrence frequencies into a matrix, where each element in the matrix represents the occurrence frequency of a specific gray-level combination in the image.

[0062] In this process, each candidate region in the fault indicator image will generate a corresponding GLCM, which serves as the basis for subsequent calculations of intensity uniformity and correlation metrics. These texture features are crucial for discriminating aging regions, condensation regions, and water accumulation regions.

[0063] The correlation of the gray-level co-occurrence matrix measures the degree of linear dependence between gray values in the image. A high correlation indicates that the texture of the image is more regular and the changes between pixel values are more consistent. Correlation The calculation formula is:

[0064] Where, is an element of the gray-level co-occurrence matrix, is the gray level and is the mean value, is the standard deviation.

[0065] The intensity uniformity of the gray-level co-occurrence matrix measures the difference between adjacent pixel values in the image, and the calculation formula is:

[0066] Select the standard deviation of color saturation in the color features as the input of the color feature branch. The standard deviation of color saturation is used to measure the distribution of color saturation within the region:

[0067] Where: is the The saturation value of each pixel. The standard deviation of saturation is extracted as the color feature of the candidate region and used for analysis in the color feature branch of the multi-branch network to help distinguish aging regions, condensation regions, and water accumulation regions.

[0068] Among them, the saturation value S of each pixel point is calculated by the following formula:

[0069] Where are the red, green, and blue channel values of the pixel, is the color intensity difference of the pixel, The is the brightness value of the pixel.

[0070] The average saturation of all pixels in the target region:

[0071] The total number of pixels in the target region.

[0072] The extraction of global features does not depend on the ROI region, but directly calculates statistical features from the entire image as the input of the global feature branch, including the boundary regularity index (BRI) and the condensation region distribution density (DLD), where the latter is the average distance between the centers of all condensation region L:

[0073] Where: is the average distance of the center points of the condensation regions, is the Euclidean distance between the center points of the condensation regions.

[0074] Since the value ranges of different features are different, in order to avoid the excessive influence of certain features on the results in subsequent analysis, all features need to be normalized. Here, the min-max normalization method is adopted:

[0075] Since the convolutional neural network (CNN) has the ability to process multi-feature inputs, and can effectively capture local features in the data and reduce the number of model parameters, therefore, the present invention will use CNN as the main part of the multi-branch network channel, and construct a suitable input matrix for the above characteristics to make full use of the feature extraction ability of CNN.

[0076] The ROI region is divided into grids of a fixed size, divided into grids, and the intensity uniformity and correlation are calculated for each grid: Intensity uniformity matrix: Calculate the intensity uniformity for each grid to obtain the intensity uniformity matrix

[0077]

[0078] Among them is the intensity uniformity of the nth grid.

[0079] Correlation matrix: Calculate the correlation for each grid to obtain the correlation matrix

[0080]

[0081] Where is the correlation of the nth grid.

[0082] Color saturation standard deviation matrix: Calculate the color saturation standard deviation for each grid to obtain the color saturation standard deviation matrix

[0083]

[0084] Where is the color saturation standard deviation of the nth grid.

[0085] The intensity uniformity matrix, correlation matrix, and color saturation standard deviation matrix are used as input features for the training of the multi-branch CNN model and the detection of the damp area. They respectively reflect the detailed differences in texture, structure, and color of the candidate region (ROI region), helping the model to comprehensively understand and distinguish the aging region, condensation region, and water accumulation region.

[0086] In the multi-branch CNN model, these three matrices are respectively used as the input data for the texture feature branch, color feature branch, and the global feature is the shape feature branch of the model. Extract the local features in these three matrices through CNN, and then merge them in the feature fusion layer to form a more comprehensive regional feature representation, thereby improving the model's classification and recognition ability for condensation, water droplets, water accumulation, and aging regions.

[0087] The specific functions are as follows: 1. Intensity Uniformity Matrix • Function: Describe the smoothness of the change in pixel gray values in the local area of the image. The higher the intensity uniformity, the smaller the change in pixel values within the region, and the smoother the texture.

[0088] • Use: In the candidate region, the aging region usually shows a higher intensity uniformity, while the condensation and water accumulation regions have a lower uniformity due to the irregular surface and different reflections.

[0089] 2. Correlation Matrix • Function: Measure the gray correlation between adjacent pixels in the image, reflecting the regularity of the local texture of the image. High correlation indicates regular pixel changes, while low correlation indicates random or complex textures.

[0090] • Usage: In the aging area, due to surface aging, the scattered light is evenly distributed, resulting in a relatively high correlation; in the water droplet area, due to the irregular structure, the pixel changes are drastic, resulting in a relatively low correlation.

[0091] 3. Saturation Standard Deviation Matrix • Function: Describe the degree of dispersion of pixel saturation within the area. The higher the saturation standard deviation, the more obvious the color change within the area.

[0092] • Usage: In the water accumulation area, due to the uniform light reflection, the saturation change is small, resulting in a low standard deviation; in the condensation area, due to the complex structure of the water droplets, the refraction and reflection of light lead to large color differences, resulting in a high saturation standard deviation.

[0093] Step S003, input the extracted features for training to provide a decision basis for judging the moisture condition.

[0094] Here, a multi-branch CNN model is adopted, and each channel branch is responsible for obtaining different features in the image of the fault indicator acquisition unit. Figure 1 It is a schematic diagram of the feature fusion process of the multi-branch CNN.

[0095] The input of the multi-branch CNN model adopted here is the features of the image. The image is a sample image taken before training and with a known moisture level. These sample images are usually labeled with the types of moisture areas (such as condensation, water accumulation, aging, normal area) and used as the training set for model learning. These images provide the texture, color, and shape features required by the model to help the model establish the recognition ability for different moisture conditions during training.

[0096] Specifically, the differences among the three branches are as follows: 1. Branch 1: Based on the Gray-Level Co-Occurrence Matrix (GLCM) features of the image • Input: GLCM feature maps such as intensity uniformity matrix, correlation matrix, etc. • Feature type: Texture feature • Function: Extract the surface texture information of the image, such as smoothness, roughness, stripes, etc. These features are very crucial for judging the states such as condensation, frosting, and moisture.

[0097] • Convolution kernel design: More inclined to capture the subtle differences between local pixels, and more sensitive to edges and texture directions.

[0098] 2. Branch 2: Based on color features (such as saturation matrix, brightness matrix) • Input: Color channel information such as saturation matrix, brightness distribution map, etc. • Feature type: Color feature • Function: Analyze the changing trends of colors in the image. For example, the color in the damp area may be darker or lighter, and the saturation may be higher or lower. These information are important bases for detecting humidity and dew condensation degree.

[0099] • Convolution kernel design: Pay more attention to color distribution and changes, and be more sensitive to features such as color gradients and saturation fluctuations.

[0100] 3. Branch 3: Based on spatial distribution features (such as global features after grid division) • Input: The distribution map after calculating features such as intensity uniformity and correlation for each grid by dividing the ROI area into grids of fixed size (such as 32×32) • Feature type: Spatial distribution feature • Function: Analyze the global distribution law of features on the entire image, and judge which areas are more likely to form dew or wet areas. Through grid processing, the overall shape, size and position distribution of the damp area can be captured.

[0101] • Convolution kernel design: May adopt a convolution kernel with a larger receptive field, pay attention to global features, and be more sensitive to regional changes and overall trends.

[0102] Figure 1 Among them, Branch 1 is the texture feature, using the convolutional layer to extract the local texture features of the image; Branch 2 is the color feature, extracting the color information and regional features in the image through different convolution kernels; Branch 3 is the shape information and regional distribution, extracting the shape features of each ROI area in the image. Finally, the outputs of these branches are merged together through a feature fusion layer (such as concatenation or weighted average) to form a unified feature vector, and then sent to the global processing layer for classification and regression.

[0103] The image captured by the dew detection device is sliced into sizes of , small patches with a step size of 25% of the original patch size are used to extract the features of these small patches. The features of these small patches are used as input to train a multi-branch CNN model. The multi-branch CNN model outputs the class probability distribution calculated by the Softmax function. During training, the categorical cross-entropy loss function is used to measure the difference between the prediction and the true label. The parameters are adjusted by the Adadelta optimization algorithm. The multi-branch CNN model is trained with image features in a batch size of 256, with a maximum of 1000 iterations and a learning rate set to 1.0. The training is determined to converge by monitoring the validation loss, with the goal of achieving the optimal classification performance of the multi-branch CNN model.

[0104] The image has dimensions of , and each channel represents a spectral dimension. The image is divided into small patches of size , and each patch retains only the information of a single spectral channel (i.e., ). When slicing, a step size of 25% of the patch size (i.e., 16×16) is used to ensure overlap between patches, covering more local features. These small patches can come from: different band channels of the original multi-spectral image , the previously mentioned feature maps (such as intensity uniformity and correlation matrix calculated by GLCM), and local image segments extracted from ROI (growing area, dew area).

[0105] Among them, the image captured by the dew detection device is not an ordinary RGB image, but a multi-channel image (C spectral channels). For example, it may contain information in bands such as visible light, infrared, and ultraviolet. These spectral information can more comprehensively reflect the physical state of the object surface, such as humidity, temperature, color change, etc.

[0106] Among them, the label is the moisture category, including dry area, slight dew area, obvious dew area, and severely moisture-affected area.

[0107] The cross-entropy loss function is used in the training process of the multi-branch CNN model as the optimization objective to measure the difference between the class distribution predicted by the model and the true distribution, and its definition is:

[0108] where: M is the total number of samples; is the number of classes; is the indicator function of the actual class label. If the true class of the th pixel is , then , otherwise is the probability that the model predicts the th pixel belongs to class ​Probability.

[0109] Use the Adadelta optimization algorithm to iteratively update the model parameters of the multi-branch CNN model, combined with adaptive learning rate adjustment. The optimization objective is to minimize the loss function:

[0110] Where: are the model parameters of the neural network, is the base learning rate, is the gradient, is the historical moving average of the parameter update; is the historical moving average of the squared gradient; is a smoothing factor to prevent division by zero.

[0111] Step S004, as Figure 2 shown, in the actual detection stage, the present invention uses the trained multi-branch CNN model to automatically identify the moisture type (condensation or water accumulation) and the moisture area in the fault indicator image collected by photographing, and calculates the ratio of the condensation area to the total area of all ROIs to obtain the condensation index inside the fault indicator. This method realizes the accurate detection and quantitative analysis of the condensation and water accumulation areas, and can effectively distinguish different types of moisture phenomena. During the detection process, when the moisture type is water accumulation, it indicates that a relatively serious fault has occurred in the fault indicator acquisition unit, and the field personnel need to promptly disassemble and replace the fault indicator acquisition unit of this phase to ensure the normal operation of the equipment. When the moisture type is condensation, if the condensation index is higher than a certain threshold and the wave recording function is abnormal, the field personnel also need to promptly disassemble and replace the fault indicator acquisition unit of this phase. If the condensation index is higher than a certain threshold and the wave recording function is normal, it means that the condensation degree is relatively serious and may affect the normal operation of the fault indicator. At this time, the field personnel should promptly remove the transparent bottom cover of the fault indicator acquisition unit of this phase for drying and add a moisture-proof agent to prevent the condensation from further developing and ensure the long-term stable operation of the equipment. This method can not only accurately identify the moisture type of the fault indicator, but also quantify the degree of condensation through the condensation index, providing a scientific decision-making basis for maintenance personnel, thereby improving the fault diagnosis and maintenance efficiency and ensuring the safety and reliability of the equipment.

[0112] In the actual detection stage, the input of the multi-branch CNN model is the texture features, color features, shape information, and regional distribution features of the real-time image to be recognized. These images come from the multi-spectral imaging module installed on the distribution network fault indicator. The multi-branch CNN model will automatically detect and classify the moisture areas in these real-time images based on the features learned in the training stage.

[0113] Embodiment 2 Please refer toFigure 3 , in this embodiment, a moisture detection device for a fault indicator based on multi-branch feature fusion is provided, including: A multi-spectral imaging module to obtain an image of the acquisition unit of the fault indicator; A candidate region recognition module for recognizing candidate regions in the image of the acquisition unit of the fault indicator; A feature extraction module for extracting features of the candidate region, where the features of the candidate region include an intensity uniformity matrix, a correlation matrix, a color saturation standard deviation, a boundary regularity index, and a condensation region distribution density; A classification module for extracting features of candidate regions in the image of the acquisition unit of the fault indicator to be detected, inputting the features of the candidate region into a trained moisture detection model, and obtaining a moisture detection result of the fault indicator to be detected.

[0114] All relevant contents of each step involved in the embodiment of the foregoing moisture detection method for a fault indicator based on multi-branch feature fusion can be cited in the function description of the corresponding functional modules of the moisture detection device for a fault indicator based on multi-branch feature fusion in the embodiment of the present invention, and will not be elaborated here.

[0115] Embodiment 3 Refer to Figure 4, this embodiment provides an electronic device, which includes a processor and a memory, and the processor is connected to the memory through a bus; the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, and is suitable for implementing one or more instructions. Specifically, it is suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the moisture detection method of the fault indicator based on multi-branch feature fusion. The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, Figure 4 only one line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0116] Embodiment 4 This embodiment provides a storage medium, specifically a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in an electronic device, used to store programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the electronic device and, of course, the extended storage medium supported by the electronic device. The computer-readable storage medium provides a storage space, and this storage space stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are stored in this storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. One or more instructions stored in the computer-readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the method for detecting moisture ingress of a fault indicator based on multi-branch feature fusion in the above embodiment.

[0117] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, apparatus, or computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (apparatuses), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0119] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and this instruction device implements the functions in Figure 1 one flow or multiple flows and / or blocks Figure 1The functions specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps for the functions specified in one Figure 1 one process or more processes and / or boxes Figure 1 step for the functions specified in one or more boxes.

[0121] Embodiment 5 This embodiment provides a computer program product, including a non-volatile computer-readable storage medium. The non-volatile computer-readable storage medium stores the computer program product. When the computer program is executed by a processor, it implements the steps of the methods described in various embodiments of the present application.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0123] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: still can modify the specific implementation manners of the present invention or make equivalent replacements. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A method for detecting moisture in fault indicators based on multi-branch feature fusion, characterized in that: include: acquiring an image of a fault indicator acquisition unit; identifying a candidate region in the image of the fault indicator acquisition unit; Extracting features of the candidate area, wherein the features of the candidate area include an intensity uniformity matrix, a correlation matrix, a color saturation standard deviation, a boundary regularity index, and a condensation area distribution density; The multi-branch convolutional neural network model is trained with the features of the candidate area as input to obtain a trained moisture detection model; The features of the candidate area of ​​the image of the acquisition unit of the fault indicator to be detected are extracted, and the features of the candidate area are input into the trained moisture detection model to obtain the moisture detection result of the fault indicator to be detected.

2. The method for detecting moisture in fault indicators based on multi-branch feature fusion according to claim 1, characterized in that: The identifying of the candidate area in the image of the fault indicator acquisition unit comprises the following steps: Acquire an image of the fault indicator acquisition unit; the image includes images under lighting conditions, different angles and backgrounds; Perform gradient calculation on the image of the fault indicator acquisition unit to generate a gradient image, use the watershed algorithm to segment the image to obtain the boundary contour of each area, perform connected domain analysis on the segmentation result to obtain the candidate area; Calculate the area and boundary regularity index (BRI) of each candidate area; screen out the waterlogged area based on the area and boundary regularity index (BRI) of each candidate area; For all pixels in the candidate area outside the waterlogged area, calculate the standard deviation of polarization angle, polarization intensity, polarization consistency index PCI and polarization intensity ratio PIR; The optimized foreground and background markers are obtained based on the polarization consistency index PCI segmentation; The condensation area is screened out based on the standard deviation of polarization angle, polarization intensity, boundary regularity index BRI, polarization consistency index PCI and polarization intensity ratio PIR.

3. The method for detecting moisture in fault indicators based on multi-branch feature fusion according to claim 1, characterized in that: The extracting of the feature of the candidate region comprises the following steps: Extracting the intensity uniformity matrix and the correlation matrix from the gray level co-occurrence matrix features of the candidate region as inputs to the texture feature branch of the multi-branch convolutional neural network model; The color saturation standard deviation in the color feature is selected as the input of the color feature branch of the multi-branch convolutional neural network model; The boundary regularity index BRI and the condensation area distribution density DLD are selected as the input of the global feature branch of the multi-branch convolutional neural network model; The input of the texture feature branch, the input of the color feature branch and the input of the global feature branch are used as the input of the multi-branch convolutional neural network model, the local features of all the inputs are extracted, and then merged in the feature fusion layer to obtain a feature vector.

4. The method for detecting moisture in fault indicators based on multi-branch feature fusion according to claim 1, characterized in that: The multi-branch convolutional neural network model is trained by taking the features of the candidate area as input to obtain a trained moisture detection model, which includes: The image of the fault indicator acquisition unit is divided into , a small patch with a step size of 25% of the original patch size, and the feature vector of the small patch is extracted as input to train the multi-branch CNN model. The category probability distribution of the multi-branch CNN model output is calculated by the Softmax function. The classification cross entropy loss function is used in training to measure the difference between the prediction and the true label. The parameters are adjusted by the Adadelta optimization algorithm. The convergence of the training is judged by monitoring the verification loss, and a trained moisture detection model is obtained.

5. The method for detecting moisture in fault indicators based on multi-branch feature fusion according to claim 1, characterized in that: The multi-branch convolutional neural network model includes an input layer, a feature fusion layer, a global processing layer and an output layer connected in sequence, and the input layer includes three branches.

6. The method for detecting moisture in fault indicators based on multi-branch feature fusion according to claim 1, characterized in that: The image of the fault indicator to be detected is obtained by taking a photo with a multi-spectral imaging module installed on the distribution network fault indicator.

7. A fault indicator moisture detection device based on multi-branch feature fusion, characterized in that: include: A multispectral imaging module to acquire images of the fault indicator acquisition unit; A candidate region identification module, used to identify candidate regions in the image of the fault indicator acquisition unit; A feature extraction module, used to extract features of the candidate area, wherein the features of the candidate area include an intensity uniformity matrix, a correlation matrix, a color saturation standard deviation, a boundary regularity index, and a condensation area distribution density; The classification module is used to extract the features of the candidate area of ​​the image of the acquisition unit of the fault indicator to be detected, input the features of the candidate area into the trained moisture detection model, and obtain the moisture detection result of the fault indicator to be detected.

8. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the fault indicator moisture detection method based on multi-branch feature fusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting moisture in a fault indicator based on multi-branch feature fusion according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for detecting moisture in a fault indicator based on multi-branch feature fusion according to any one of claims 1 to 6 are implemented.

Images