Infrared image power equipment fault detection method, device, equipment and medium

Abstract

The present invention relates to a method, apparatus, device, and medium for detecting faults in power equipment using infrared images. The method comprises the following steps: obtaining text data of the power equipment's permitted environment as prompt words, performing word-unit analysis on the prompt words, and extracting features using a natural language model to obtain auxiliary features of the prompt words; obtaining an infrared image of the power equipment and inputting it, along with the auxiliary features of the prompt words, into a fault detection model based on a Unet structure, outputting a fault detection result. The fault detection model comprises a feature encoding module and a feature decoding module. The feature encoding module dynamically generates convolution kernels based on the auxiliary features of the prompt words to extract fault features. The feature decoding module matches the auxiliary features of the prompt words with the row features and column features of a low-resolution feature map for fault features of different scales, generating a target positioning attention map for auxiliary feature extraction. Compared with the prior art, the present invention has advantages such as improved detection accuracy.

Description

Technical Field

[0001] The present invention relates to power equipment fault detection, and in particular to an infrared image power equipment fault detection method, device, equipment and medium. Background Art

[0002] By capturing an object's thermal radiation, infrared images can effectively reveal the temperature distribution on its surface and within its interior. Because infrared imaging does not rely on visible light, it offers significant advantages in many fields, such as industrial equipment monitoring, building inspection, medical imaging, and military reconnaissance. In the industrial sector, infrared images are widely used for tasks such as equipment fault detection, quality control, equipment maintenance, and preventive diagnosis. In particular, in fault detection of electrical and mechanical equipment, infrared images can effectively detect temperature anomalies and thus identify potential equipment problems. With the development of deep learning technology, image segmentation and object detection have become important research directions in computer vision. Image segmentation technology is widely used for automated analysis and diagnosis in fields such as medical imaging, remote sensing imaging, and industrial inspection. However, existing infrared image fault detection technology still has some challenges.

[0003] In existing infrared image fault detection technology systems, the lack of distinct infrared image features and the susceptibility of the model encoding process to background noise are key factors limiting detection accuracy. From a physical perspective, infrared images are based on the thermal radiation characteristics of objects. Thermal anomaly signals generated in faulty areas are often extremely weak, resulting in low contrast in the image. For example, in the early detection of faults in industrial equipment, thermal changes caused by minor electrical shorts or mechanical wear appear only as subtle grayscale differences in infrared images, making them difficult to distinguish from normal areas. This poses a significant challenge to extracting fault features.

[0004] Furthermore, interference from background noise further exacerbates this dilemma. Background noise comes from a wide range of sources, including dynamic changes in ambient temperature, radiation interference from other surrounding heat sources, and electronic noise from the infrared imaging equipment itself. This noise is indiscriminately mixed into the input signal during the model's encoding phase, interfering with the convolutional neural network's effective extraction of fault features. Because the convolutional and pooling layers struggle to accurately filter key fault-related features from the vast amount of background information when processing these complex signals, the encoded feature representation is biased, making it impossible to provide a reliable basis for subsequent fault classification and judgment, significantly reducing the accuracy and reliability of fault detection.

[0005] For example, in fault detection on high-voltage transmission lines, partial discharge of insulators or poor contact at joints may only appear as a tiny hotspot on an infrared image. Since the features of the fault target cannot be accurately extracted during the encoding phase, it is difficult for the model to accurately restore its location information from multiple layers of complex feature mapping during decoding. Moreover, other heat sources in the background, such as thermal radiation from adjacent towers and thermal reflection from the surrounding environment, will intertwine with the thermal signals of the fault target and interfere with the model's judgment. The model may mistakenly identify these background thermal signals as fault targets, or may experience large deviations when locating the fault location. This inaccurate positioning problem makes it difficult for maintenance personnel to quickly find the fault point based on the detection results, which not only delays maintenance time, but may also cause the fault range to expand, affecting the stable operation of the power system.

[0006] Patent CN117011669A develops a method and system for detecting small infrared targets. This system uses convolution to downsample a single-channel image to obtain multi-channel feature maps at different scales. Details are extracted through convolution operations, and a sparse sampling-related attention mechanism is used to transform the feature maps. After bilinear interpolation upsampling, the multiple feature maps are fused to convert the features into pixel-wise binary classification probabilities. However, the sparse sampling attention mechanism is affected by the complex background environment and noise in infrared images, making it difficult for the model to focus on specific areas related to faults, which is not conducive to the detection and location of small targets in scenarios with insufficient information.

[0007] Patent CN119399450A constructs a processing system based on multiple network models. It utilizes a low-rank background estimation network model, a sparse target extraction network model, and an image reconstruction network model to work together. Combined with the Res Blocks network model and the Collaborative Attention network model, it processes and extracts background and targets from images through operations such as convolutional residual blocks and a collaborative attention mechanism. However, collaborative attention struggles to accurately capture fault-related features in complex power system infrared images, and convolutional neural networks struggle to accurately identify faults without prior knowledge of the power system.

[0008] Patent CN119831954A discloses a method and system for detecting power equipment defects based on a large multimodal model. This method constructs a self-attention device fault detection model and sets basic tasks for training. This method has achieved some success in power equipment defect detection, but it fails to fully utilize text information to assist infrared image fault detection, resulting in limited detection performance in complex scenarios.

[0009] In summary, the main problems existing in the existing technology include: First, the fault thermal signals in infrared images are often weak and have low contrast with normal areas, resulting in unclear fault features and affecting the accuracy of fault detection. Second, due to background noise interference, it is difficult to accurately extract fault-related features during the model encoding stage, which affects the accuracy and reliability of fault detection. Finally, in application scenarios such as power systems, the fault targets are usually small in infrared images. Due to feature extraction bias in the encoding stage and interference from complex background heat sources, the model has difficulty accurately restoring the fault target location from the feature map. The decoding stage suffers from low fault location accuracy, which can easily delay maintenance and affect the stable operation of the system. In addition, most existing infrared image fault detection methods fail to effectively utilize prior knowledge and environmental information about power equipment and lack a mechanism to organically combine text information with image features, resulting in limited fault identification and location capabilities in complex scenarios. Therefore, how to effectively use text information to guide fault feature extraction and target precise location during feature encoding and decoding remains an urgent problem to be solved. Summary of the Invention

[0010] The purpose of the present invention is to provide an infrared image power equipment fault detection method, device, equipment and medium in order to overcome the defects of the above-mentioned prior art. The prompt word auxiliary feature is introduced in the encoding stage to alleviate the problems of high noise and unclear features of infrared images, and improve the ability to extract fault features; at the same time, by introducing the prompt word auxiliary feature in the decoding stage, a target positioning attention map is generated, thereby improving the decoder's fault positioning ability.

[0011] The purpose of the present invention can be achieved by the following technical solutions:

[0012] According to a first aspect of the present invention, a method for detecting faults in power equipment using infrared images is provided, the method comprising the following steps:

[0013] Acquire power equipment permission environment text data as prompt words, perform word element analysis on the prompt words, and use a natural language model to perform feature extraction to obtain auxiliary features of the prompt words;

[0014] An infrared image of electric power equipment is acquired and input into a Unet-based fault detection model together with the prompt word auxiliary features to output the fault detection results. The fault detection model includes a feature encoding module and a feature decoding module. The feature encoding module dynamically generates a convolution kernel based on the prompt word auxiliary features to extract fault features. The feature decoding module matches the prompt word auxiliary features with the row features and column features of the low-resolution feature map for fault features of different scales to generate a target positioning attention map for auxiliary feature extraction.

[0015] As an optimal technical solution, the prompt word is parsed to obtain a word unit, and the word unit embedding is obtained using a linear mapping layer. A position embedding matrix is ​​constructed to obtain a position embedding according to the position of the word unit, and the word unit embedding and position embedding are added as the input of the natural language model.

[0016] As a preferred technical solution, the natural language model adopts a bidirectional encoding representation model pre-trained on a large-scale language dataset, and outputs the vector corresponding to the CLS head of the bidirectional encoding representation model as an auxiliary feature of the prompt word.

[0017] As a preferred technical solution, the feature encoding module includes multiple encoding blocks, each encoding block includes a first convolution layer, a first batch normalization and activation layer, a prompt auxiliary convolution layer, a second batch normalization and activation layer and a downsampling layer connected in sequence. The input features of each encoding block are processed by the first convolution layer and the first batch normalization and activation layer to obtain preliminary features. The prompt auxiliary convolution layer uses a convolution kernel dynamically generated based on the auxiliary features of the prompt word to perform a convolution operation on the preliminary features to generate intermediate features. The intermediate features are processed by the downsampling layer and output.

[0018] As a preferred technical solution, the dynamic generation of the convolution kernel of the hint-assisted convolution layer is specifically as follows:

[0019] Performing average pooling and maximum pooling on the preliminary features respectively to obtain average pooling features and maximum pooling features;

[0020] The average pooling feature, the maximum pooling feature and the prompt word auxiliary feature are spliced ​​together and a convolution kernel is dynamically generated through a multi-layer perceptron.

[0021] As an optimal technical solution, the feature decoding module includes multiple decoding blocks, the number of which is the same as the number of encoding blocks. The feature output to the current decoding block by the previous decoding block or the last encoding block is used as the first input feature, and the intermediate feature of the corresponding encoding block is used as the second input feature to perform high-resolution restoration of the input feature. The decoding block includes an upsampling layer, a second convolutional layer, a third batch normalization and activation layer, a target positioning attention layer and a third convolutional layer connected in sequence. After the first input feature is processed by the upsampling layer, it is spliced ​​with the second input feature, input into the second convolutional layer, and processed by the third batch normalization and activation layer to obtain a low-resolution feature map. The target positioning attention layer matches the auxiliary features of the prompt word with the row features and column features of the low-resolution feature map for fault features of different scales to generate a target positioning attention map. The third convolutional layer performs feature extraction processing on the low-resolution feature map based on the target positioning attention map and outputs it.

[0022] As a preferred technical solution, the target positioning attention layer performs the following steps:

[0023] Slice the low-resolution feature map in rows and columns:

[0024] ,

[0025] ,

[0026] in, is a low-resolution feature map, W 、 H 、 C in are the width, height and number of channels of the input power equipment infrared image, is the row slicing result, is the column slicing result, The shape of the feature is ;

[0027] Perform dot product on the results of row slicing and column slicing with the auxiliary features of the prompt word to obtain row attention and column attention:

[0028] ,

[0029] ,

[0030] in, For the i The attention of the line, For the j Column-by-column attention, It is the auxiliary feature of the prompt word;

[0031] Perform outer product of the row attention and column attention to generate the target localization attention map:

[0032] ,

[0033] in, Localize the attention map for the target, is the row attention vector for all rows, is the column attention vector for all columns.

[0034] According to a second aspect of the present invention, there is provided an infrared image power equipment fault detection device, the device comprising:

[0035] A data acquisition module is used to obtain text data of the permitted environment of the power equipment as prompt words and to obtain infrared images of the power equipment;

[0036] A prompt word auxiliary feature extraction module, used to perform word grammar analysis on the prompt word and extract features using a natural language model to obtain prompt word auxiliary features;

[0037] The fault detection module is used to input the infrared image of the power equipment and the prompt word auxiliary features into a fault detection model based on the Unet structure, and output the fault detection results. The fault detection model includes a feature encoding module and a feature decoding module, wherein the feature encoding module dynamically generates a convolution kernel based on the prompt word auxiliary features to extract fault features. In the feature decoding module, the prompt word auxiliary features are matched with the row features and column features of the low-resolution feature map for fault features of different scales, and a target positioning attention map is generated for auxiliary feature extraction.

[0038] According to a third aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method when executing the program.

[0039] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The present invention extracts features related to fault detection from prompt words through a natural language model and uses them as auxiliary features of prompt words. This introduces additional prior knowledge into the image model in an environment with high data noise and unclear features, thereby improving the prediction accuracy of the model.

[0042] (2) By introducing prompt word information, the problem of high noise and unclear features in infrared images is alleviated, and the ability to extract fault features in the encoding stage is improved. This invention uses the prompt word auxiliary features extracted by the natural language model to dynamically generate convolution kernels in the feature encoding module, thereby enhancing the ability to extract fault features and effectively solving the problem of unclear features in infrared images caused by weak fault thermal signals and low contrast with normal areas.

[0043] (3) By introducing the cue word auxiliary features in the decoding stage, a target localization attention map is generated, thereby improving the decoder's ability to locate faults. In the feature decoding module, the present invention matches the cue word auxiliary features with the row features and column features of the low-resolution feature map to generate a target localization attention map. This effectively solves the problem of low fault localization accuracy caused by the small size of the fault target in the infrared image. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 Schematic diagram of the model structure of the present invention;

[0046] Figure 3 A schematic diagram of the process of obtaining auxiliary features of prompt words in the present invention;

[0047] Figure 4 Schematic diagram of the process of dynamically generating convolution kernels of the present invention;

[0048] Figure 5 A schematic diagram of the process of generating the target positioning attention map of the present invention;

[0049] Figure 6 A visual comparison of detection results of different models in one embodiment. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0051] Unless otherwise defined, the technical or scientific terms used in this application should have the ordinary meaning understood by a person of ordinary skill in the technical field to which this application belongs. The words "one", "a", "the" and the like used in this application do not indicate a limit on quantity and may indicate the singular or plural. The terms "include", "comprise", "have" and any variations thereof used in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units that are inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The word "multiple" used in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0052] Example 1

[0053] In order to solve the problems of insufficient feature extraction in the encoding stage and inaccurate fault location in the decoding stage caused by high data noise and unclear features in the fault detection task in infrared images, this embodiment proposes a fault detection model that integrates multimodal text information. This model can alleviate the problems of high noise and unclear features in infrared images by introducing prompt word information, thereby improving the ability to extract fault features in the encoding stage. At the same time, by introducing prompt word auxiliary features in the decoding stage, a target positioning attention map is generated, thereby improving the decoder's ability to locate faults.

[0054] like Figure 1 As shown, this embodiment provides an infrared image power equipment fault detection method, which includes the following steps:

[0055] S1, obtain the power equipment permission environment text data as prompt words, perform word element analysis on the prompt words, and use the natural language model to extract features to obtain auxiliary features of the prompt words.

[0056] In this embodiment, the permitted environment text data for the power equipment is first obtained as a prompt. This permitted environment text data refers to textual information describing the normal operating environment conditions for the power equipment, such as "The normal operating temperature range for transformers is -20°C to 40°C" and "The normal operating humidity for switchgear does not exceed 85%." This textual data can be obtained from the power equipment's technical manuals, specifications, standards, or expert knowledge bases.

[0057] like Figure 3 As shown, the obtained prompt word is parsed by word grammar. Word grammar parsing is the process of decomposing text data into basic language units. The prompt word is parsed to obtain word grammar. The process is shown in formula (1), where Represents the input word, Represents the parsed token.

[0058]

[0059] Then, based on the obtained word T , using the linear mapping layer Get word embedding. At the same time, in order to facilitate the model to perceive the position of the token in the sequence, build a position embedding matrix , according to the position of the word Get location embedding ,in, d is the embedding feature dimension. The word embedding and position embedding are added together as the input of the natural language model. The process is shown in formula (2):

[0060]

[0061] Specifically, the word-meta parsing process first segments the text into words or subwords. For example, the sentence "The normal operating temperature range of the transformer is -20°C to 40°C" is segmented into words such as [transformer, normal, operating, temperature, range, is, -20°C, to, 40°C]. Each word-meta is then converted into a fixed-dimensional vector representation, or word-meta embedding, through a linear mapping layer. Simultaneously, a positional embedding matrix is ​​constructed, assigning each word-meta a positional encoding based on its position in the sequence to preserve sequential information between words. Finally, the word-meta embedding and the corresponding positional embedding are added together to obtain a comprehensive representation containing both semantic and positional information, which serves as the input to the natural language model.

[0062] In this embodiment, the natural language model uses a bidirectional encoder representation model (Bidirectional Encoder Representations from Transformers, BERT) pre-trained on a large-scale language dataset, and outputs the vector corresponding to the CLS head of the bidirectional encoder representation model as an auxiliary feature of the prompt word. The bidirectional encoder representation model is a deep learning model that can simultaneously consider the contextual information before and after the text. By pre-training on a large-scale language dataset, it can effectively capture the semantic features of the text. A special classification tag [CLS] is added at the beginning of the input sequence of the model. The final hidden state vector corresponding to the tag can be used as an aggregate representation of the entire sequence. In this embodiment, the vector corresponding to this CLS head is used as the auxiliary feature of the prompt word. Output as the auxiliary feature of the prompt word for subsequent fault detection tasks, namely:

[0063]

[0064] By integrating the auxiliary features of the prompt words, the model can make up for the lack of relevant prior knowledge in the image, thereby effectively assisting subsequent encoding and decoding tasks.

[0065] S2, obtains the infrared image of the power equipment and inputs it into the fault detection model based on the Unet structure together with the prompt word auxiliary features, and outputs the fault detection results.

[0066] The fault detection model uses UNet as its underlying network architecture, acquiring multi-scale information through layer-by-layer downsampling and upsampling, effectively capturing both local features and global information in the image. Specifically, faults in infrared images include both large-scale, overall defects and small-scale, local details. To capture this information simultaneously at a single resolution, the network employs multi-scale pooling, enabling the convolution kernel to capture features at different scales.

[0067] The fault detection model includes a feature encoding module and a feature decoding module, wherein the feature encoding module includes L encoding blocks, each of which extracts local features of the image through convolution operations, and reduces the resolution of the feature map through pooling operations, expands the receptive field, and captures global information; at the same time, a convolution kernel is dynamically generated based on the auxiliary features of the prompt word to extract fault features and improve the ability of the encoder to capture key features of infrared images. In the feature decoding module, the low-resolution feature map is upsampled through a deconvolution operation to restore high-resolution detail features. In this process, the decoding layer not only restores the image details, but also retains multi-scale information by fusing with the high-level features in the feature encoding module, and ultimately generates accurate fault detection results. At the same time, the present invention uses a natural language model to assist in fault location. For different scale features, the prompt word auxiliary features of the natural language model are matched with the row features and column features of the feature map respectively to generate a target positioning attention map, thereby effectively improving the model's sensitivity to the fault location.

[0068] like Figure 2 As shown in Figure 1, each encoding block includes a first convolutional layer, a first batch normalization and activation layer, a prompt-assisted convolutional layer, a second batch normalization and activation layer, and a downsampling layer, all connected in sequence. The input features of each encoding block are processed by the first convolutional layer and the first batch normalization and activation layer to obtain preliminary features. The prompt-assisted convolutional layer uses a convolution kernel dynamically generated based on the auxiliary features of the prompt word to perform a convolution operation on the preliminary features to generate intermediate features. The intermediate features are processed by the downsampling layer and then output. Encoding blocks 2 to L, except encoding block 1, all use the output of the previous encoding block as input features. Encoding block 1 uses the infrared image of the power equipment as input features. The overall structure of the encoding process is shown in Formula (4).

[0069]

[0070] in, represents the operation of encoding blocks, Indicates the l The input features of the encoding blocks, Indicates the l The output features of the encoding block.

[0071] In the feature encoding module, the first convolutional layer is Convolution extracts general features, and then the first batch of normalization and activation layers are used to improve the stability of training and obtain preliminary features. , as shown in formula (5).

[0072]

[0073] in, represents convolution, represents batch normalization, Represents the activation function.

[0074] Subsequently, in order to accurately capture fault-related features in infrared data with complex backgrounds and unclear features, the prompt-assisted convolution layer of this embodiment uses the prompt word auxiliary features of the natural language model and the current feature map to dynamically generate a convolution kernel for feature extraction. Specifically, Figure 4 As shown, the following steps are included:

[0075] Average pooling and max pooling are performed on the preliminary features to obtain average pooled features and max pooled features, respectively. Global pooling average pools the entire feature map to obtain global information for each channel, while max pooling extracts the most salient features for each channel. These two pooling operations respectively obtain global statistical information and local saliency information of the features, and can describe the characteristics of the feature map from different perspectives.

[0076] The average pooling features, maximum pooling features, and prompt word auxiliary features are spliced ​​together and used to dynamically generate convolution kernels through a multilayer perceptron (MLP). This allows the convolution operation to be dynamically adjusted according to the input image and prompt word information, thereby improving the pertinence and effectiveness of feature extraction.

[0077] The above process is shown in formula (6):

[0078]

[0079] in, is the maximum pooling, is average pooling, MLP is multi-layer perceptron, represents the convolution operation, is an intermediate feature.

[0080] Finally, for the intermediate features conduct The pooling operation is performed to downsample and the result is input to the next encoding block, as shown in formula (7).

[0081]

[0082] The feature decoding module consists of L decoding blocks. Decoding block L, corresponding to encoding block L, uses the output features of encoding block L as its first input features. Decoding blocks 1-L-1, corresponding to encoding blocks 1-L-1, use the output features of decoding blocks 2-L as their first input features. Furthermore, decoding blocks 1-L use the intermediate features of their corresponding encoding blocks 1-L as their second input features, restoring the input features at high resolution. In infrared small target detection, the decoder faces difficulties in locating fault locations. Small infrared targets are small and have less distinct features, making them difficult to discern against complex backgrounds. When processing the encoded feature information, the decoder must convert abstract features into specific target locations. However, the spatial information of small targets is easily lost during the feature mapping process, making it difficult for the decoder to accurately reconstruct their spatial location. Furthermore, infrared images contain numerous interferences, such as similar temperature regions in the background. These can easily lead the decoder to misidentify interference as the target, or cause the positioning results to deviate from the true location, making it difficult to accurately pinpoint the fault location.

[0083] By learning from massive amounts of text data, natural language models can extract accurate and rich feature information from descriptive prompts. These features cover key aspects such as target characteristics and scene associations. When integrated into the decoder for infrared small target detection, these features serve as powerful positioning guidance, helping the decoder accurately match the location information corresponding to the fault target in a complex feature space. This effectively overcomes positioning challenges caused by small targets and background interference, significantly improving fault location accuracy.

[0084] In this embodiment, Figure 2 As shown in the figure, the decoding block includes an upsampling layer, a second convolutional layer, a third batch normalization and activation layer, a target positioning attention layer and a third convolutional layer connected in sequence. After the first input feature is processed by the upsampling layer, it is spliced ​​with the second input feature, input into the second convolutional layer, and processed by the third batch normalization and activation layer to obtain a low-resolution feature map. The target positioning attention layer matches the auxiliary features of the prompt word with the row features and column features of the low-resolution feature map for fault features of different scales to generate a target positioning attention map. The third convolutional layer extracts features from the low-resolution feature map based on the target positioning attention map and outputs it.

[0085] No. l (1≤ l < L ) decoding blocks before the output of the previous decoding block and the intermediate features of the corresponding encoding block As input, the result is passed to the next decoding block, as shown in formula (8).

[0086]

[0087] in, Represents the operation of decoding a block.

[0088] The decoding block also consists of two layers of convolution. First, the feature Upsampling , to obtain larger scale features , and compare it with the intermediate features of the corresponding encoding block The concatenation is performed and batch normalization and activation are performed as shown in formula (9).

[0089]

[0090] In order to use the auxiliary features of the prompt word for auxiliary positioning, this embodiment proposes a target positioning attention layer based on the prompt word assistance, such as Figure 5 As shown, it performs the following steps:

[0091] The low-resolution feature map is sliced ​​in rows and columns, as shown in formula (10):

[0092]

[0093] in, is a low-resolution feature map, W 、 H 、 C in are the width, height and number of channels of the input power equipment infrared image, is the row slicing result, is the column slicing result, The shape of the feature is .

[0094] The results of row slicing and column slicing are dot-producted with the auxiliary features of the prompt word to obtain row attention and column attention, as shown in formula (11):

[0095]

[0096] in, For the i The attention of the line, For the j Column-by-column attention, It is the auxiliary feature of the prompt word.

[0097] Perform outer product of row attention and column attention to generate target localization attention map, as shown in formula (12):

[0098]

[0099] in, Localize the attention map for the target, is the row attention vector for all rows, , is the column attention vector for all columns, .

[0100] The row and column slicing operations extract features from the low-resolution feature map by row and column, respectively, to obtain feature vectors for each row and column. By calculating the similarity (dot product) between these feature vectors and the auxiliary features of the prompt word, we can obtain row and column attention, indicating the degree of match between each row and column and the fault characteristics described by the prompt word. Combining the row and column attention using an outer product operation generates a two-dimensional target localization attention map. This attention map highlights areas where faults may exist, guiding subsequent feature extraction to focus on these areas.

[0101] Finally, the third convolutional layer extracts features from the low-resolution feature map based on the target positioning attention map and outputs it, further enhancing the feature representation of the fault area, suppressing the interference of the non-fault area, and improving the accuracy of fault detection. The process is shown in formula (13):

[0102]

[0103] By using prompt word features to assist in positioning, the model can accurately locate the fault area and allocate more attention to it, thereby effectively improving the accuracy of fault location.

[0104] By introducing additional prompt word information, the network of the present invention can more accurately process fault information in infrared images under complex environments and achieve precise positioning, thereby improving the accuracy of fault detection and segmentation.

[0105] Through the above process, the fault detection model outputs fault detection results, including information such as fault type, fault location, and fault severity. These results can help power maintenance personnel promptly identify equipment failures and take appropriate repair measures to ensure the safe and stable operation of the power system.

[0106] Example 2

[0107] This embodiment was extensively experimented with a cable fault detection dataset. The experimental results are shown in Table 1. The proposed method achieved an Intersection over Union (IoU) of 0.6042 and a Probability of Detection (PD) of 0.7816, significantly higher than those of other methods. Its false alarm rate (FA) was only 0.0029, lower than that of other methods, demonstrating its significant advantages in fault detection performance.

[0108] Table 1 Prediction indicators of different models on the cable infrared fault detection dataset

[0109]

[0110] The visualization effects of the detection results of different methods are as follows Figure 6 As shown in the figure, compared with other models, the targets detected by the present invention are more complete. There is basically no obvious break in the lines of the target object in the figure, and the overlap with the real label is high. At the same time, there are fewer noise points and false detection areas in the detection results, and the background is basically black, which shows that it can better distinguish between the target and the background, reduce the false alarm rate, and provide reliable protection for the stable operation of the power system.

[0111] Example 3

[0112] The above is an introduction to a method embodiment. The following further illustrates the solution of the present invention through an apparatus embodiment.

[0113] This embodiment provides an infrared image power equipment fault detection device, which includes:

[0114] A data acquisition module is used to obtain text data of the permitted environment of the power equipment as prompt words and to obtain infrared images of the power equipment;

[0115] A prompt word auxiliary feature extraction module, used to perform word grammar analysis on the prompt word and extract features using a natural language model to obtain prompt word auxiliary features;

[0116] The fault detection module is used to input the infrared image of the power equipment and the prompt word auxiliary features into a fault detection model based on the Unet structure, and output the fault detection results. The fault detection model includes a feature encoding module and a feature decoding module, wherein the feature encoding module dynamically generates a convolution kernel based on the prompt word auxiliary features to extract fault features. In the feature decoding module, the prompt word auxiliary features are matched with the row features and column features of the low-resolution feature map for fault features of different scales, and a target positioning attention map is generated for auxiliary feature extraction.

[0117] In this embodiment, the infrared image power equipment fault detection device includes three main modules: a data acquisition module, a prompt word auxiliary feature extraction module, and a fault detection module.

[0118] The data acquisition module is responsible for acquiring two types of data: text data on the permitted operating environment of power equipment and infrared images of power equipment. This text data describes the normal operating environment conditions of power equipment, such as temperature range, humidity requirements, and load capacity. This text data can be obtained from the power equipment's technical manuals, standards, or expert knowledge bases and used as prompts for subsequent processing. Infrared images of power equipment are captured using infrared thermal imaging equipment and reflect the surface temperature distribution of the equipment. The data acquisition module can connect to the infrared thermal imaging equipment via a wired or wireless network to acquire infrared images of the power equipment in real time, or it can read historical infrared image data from an image database.

[0119] The prompt word auxiliary feature extraction module receives the prompt word provided by the data acquisition module, performs lemma analysis on it, and uses a natural language model to extract features to obtain prompt word auxiliary features. Lexical lemma analysis is the process of breaking down text data into basic language units, including operations such as word segmentation and tokenization. The natural language model uses a bidirectional encoding representation model pre-trained on a large-scale language dataset, such as BERT (Bidirectional Encoder Representations from Transformers). This model can understand the semantic information of the text and convert the text into a high-dimensional vector representation. The prompt word auxiliary feature extraction module outputs the vector corresponding to the CLS head of the bidirectional encoding representation model as the prompt word auxiliary feature to guide the subsequent fault detection process.

[0120] The fault detection module is the core of the device. It inputs infrared images of power equipment and cue-word auxiliary features into a Unet-based fault detection model and outputs fault detection results. The fault detection model consists of a feature encoding module and a feature decoding module. The feature encoding module dynamically generates convolution kernels based on the cue-word auxiliary features to extract fault features. The feature decoding module matches the cue-word auxiliary features with the row and column features of the low-resolution feature map for fault features at different scales, generating a target localization attention map for auxiliary feature extraction.

[0121] The feature encoding module consists of multiple encoding blocks, each of which consists of a first convolutional layer, a first batch normalization and activation layer, a cue-assisted convolutional layer, a second batch normalization and activation layer, and a downsampling layer. The cue-assisted convolutional layer is an innovative design that dynamically generates convolution kernels using cue word auxiliary features. This convolution operation is performed on the preliminary features to generate intermediate features. This dynamic generation of convolution kernels enables the model to adjust its feature extraction strategy based on different cue word information, improving its ability to identify specific fault types.

[0122] The feature decoding module consists of multiple decoding blocks, the same number as the encoding blocks. Each decoding block consists of an upsampling layer, a second convolutional layer, a third batch normalization and activation layer, a target localization attention layer, and a third convolutional layer. The target localization attention layer is another innovation. It matches the auxiliary features of the cue word with the row and column features of the low-resolution feature map to generate a target localization attention map. This attention mechanism guides the model's focus on areas where faults may exist, improving fault detection accuracy.

[0123] The fault detection module ultimately outputs fault detection results, including information such as fault type, location, and severity. These results can be displayed visually on a display device or transmitted to a remote monitoring center via the network, helping power maintenance personnel promptly identify equipment faults and take appropriate repair measures.

[0124] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0125] Example 4

[0126] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the program, the infrared image power equipment fault detection method as described in Example 1 is implemented.

[0127] In this embodiment, the electronic device may be a server, a workstation, a personal computer, a notebook computer, a tablet computer, or an embedded device, etc. The electronic device includes two main hardware components: a memory and a processor.

[0128] Memory is used to store computer programs and data. Memory can include non-volatile storage media and volatile storage media. Non-volatile storage media can include read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other types of non-volatile storage devices. Volatile storage media can include random access memory (RAM), which acts as external cache memory. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DRRAM).

[0129] The computer program stored in the memory includes program code for implementing a method for detecting power equipment faults using infrared images. This program code includes a module for obtaining text data from the power equipment's permitted environment as prompt words, a module for performing lexical analysis on the prompt words, a module for extracting features using a natural language model, a module for acquiring infrared images of the power equipment, and a module for a fault detection model based on a Unet structure. These modules are stored in the memory as program code and loaded and executed by the processor.

[0130] The processor may be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor is responsible for executing the computer program stored in the memory to implement each step of the infrared image power equipment fault detection method.

[0131] When the electronic device starts up and runs the infrared imaging power equipment fault detection program, the processor first retrieves the power equipment's permitted environment text data from memory or an external data source as prompt words. It then performs lemma analysis on the prompt words and uses a natural language model to extract features to obtain auxiliary features for the prompt words. The processor then acquires an infrared image of the power equipment, either in real time from a connected infrared camera or by reading a saved infrared image from memory or an external storage device.

[0132] The processor inputs the infrared image of the power equipment and the cue word auxiliary features into a Unet-based fault detection model, which is stored in memory as program code. The fault detection model includes a feature encoding module and a feature decoding module. In the feature encoding module, the processor dynamically generates convolution kernels based on the cue word auxiliary features to extract fault features. In the feature decoding module, the processor matches the cue word auxiliary features with the row and column features of the low-resolution feature map for fault features at different scales, generating a target localization attention map for auxiliary feature extraction.

[0133] Finally, the processor uses the fault detection model to output fault detection results, including information such as fault type, fault location, and fault severity. These results can be displayed on the electronic device's display or transmitted to other devices or systems via a network interface for reference by power maintenance personnel.

[0134] Example 5

[0135] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the infrared image power equipment fault detection method as described in the first embodiment.

[0136] In this embodiment, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable storage media may include, but are not limited to: magnetic storage devices such as hard disks, floppy disks, and magnetic tapes; optical storage devices such as compact disks (CDs), digital versatile disks (DVDs), and Blu-ray disks; solid-state storage devices such as solid-state drives (SSDs), flash drives, and memory cards; or any combination of the foregoing.

[0137] The computer-readable storage medium stores a computer program containing program code for implementing the infrared image power equipment fault detection method. When the program is executed by a processor, the processor performs the following operations:

[0138] The system obtains text data describing the permitted operating environment of power equipment as prompt words, performs lemma analysis on the prompt words, and uses a natural language model to extract features to obtain auxiliary features for the prompt words. The prompt words can be text information describing the normal operating environment conditions of the power equipment. The natural language model uses a bidirectional encoding representation model pre-trained on a large-scale language dataset and outputs the vector corresponding to the CLS head as the auxiliary features of the prompt words.

[0139] An infrared image of power equipment is acquired and fed into a Unet-based fault detection model along with auxiliary features derived from prompt words. The model then outputs fault detection results. The fault detection model consists of a feature encoding module and a feature decoding module. The feature encoding module dynamically generates convolution kernels based on the auxiliary features derived from prompt words to extract fault features. The feature decoding module matches the auxiliary features derived from prompt words with the row and column features of a low-resolution feature map for fault features at different scales. This generates a target localization attention map for auxiliary feature extraction.

[0140] In this way, when the program stored on the computer-readable storage medium is executed by the processor, the infrared image power equipment fault detection method can be implemented, thereby improving the accuracy and efficiency of power equipment fault detection.

[0141] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for detecting faults in power equipment using infrared images, characterized in that: The method comprises the following steps: Acquire power equipment permission environment text data as prompt words, perform word element analysis on the prompt words, and use a natural language model to perform feature extraction to obtain auxiliary features of the prompt words; Acquire an infrared image of the power equipment and input it, along with the prompt word auxiliary features, into a Unet-based fault detection model to output a fault detection result. The fault detection model includes a feature encoding module and a feature decoding module. The feature encoding module dynamically generates convolution kernels based on the prompt word auxiliary features to extract fault features. The feature decoding module matches the prompt word auxiliary features with the row features and column features of a low-resolution feature map for fault features of different scales to generate a target positioning attention map for auxiliary feature extraction. The feature encoding module includes multiple encoding blocks, each of which includes a first convolutional layer, a first batch normalization and activation layer, a prompt auxiliary convolutional layer, a second batch normalization and activation layer, and a downsampling layer connected in sequence. The input features of each encoding block are processed by the first convolutional layer and the first batch normalization and activation layer to obtain preliminary features. The prompt auxiliary convolutional layer uses a convolution kernel dynamically generated based on the auxiliary features of the prompt word to perform a convolution operation on the preliminary features to generate intermediate features. The intermediate features are processed by the downsampling layer and then output; The dynamic generation of the convolution kernel of the hint-assisted convolution layer is specifically as follows: Performing average pooling and maximum pooling on the preliminary features respectively to obtain average pooling features and maximum pooling features; The average pooling feature, the maximum pooling feature and the prompt word auxiliary feature are spliced ​​together and a convolution kernel is dynamically generated through a multi-layer perceptron; The feature decoding module includes multiple decoding blocks, the number of which is the same as the number of encoding blocks. The feature output to the current decoding block by the previous decoding block or the last encoding block is used as the first input feature, and the intermediate feature of the corresponding encoding block is used as the second input feature to perform high-resolution restoration on the input feature. The decoding block includes an upsampling layer, a second convolutional layer, a third batch normalization and activation layer, a target positioning attention layer and a third convolutional layer connected in sequence. After the first input feature is processed by the upsampling layer, it is spliced ​​with the second input feature, input into the second convolutional layer, and processed by the third batch normalization and activation layer to obtain a low-resolution feature map. The target positioning attention layer matches the auxiliary features of the prompt word with the row features and column features of the low-resolution feature map for fault features of different scales to generate a target positioning attention map. The third convolutional layer performs feature extraction processing on the low-resolution feature map based on the target positioning attention map and outputs it. The target localization attention layer performs the following steps: Slice the low-resolution feature map in rows and columns: ， ， in, is a low-resolution feature map, W 、 H 、 C in are the width, height and number of channels of the input power equipment infrared image, is the row slicing result, is the column slicing result, The shape of the feature is ; Perform dot product on the results of row slicing and column slicing with the auxiliary features of the prompt word to obtain row attention and column attention: ， ， in, For the i The attention of the line, For the j Column-by-column attention, It is the auxiliary feature of the prompt word; Perform outer product of the row attention and column attention to generate the target localization attention map: ， in, Localize the attention map for the target, is the row attention vector for all rows, is the column attention vector for all columns.

2. The infrared image power equipment fault detection method according to claim 1, characterized in that: The prompt word is parsed to obtain a word unit, and a linear mapping layer is used to obtain a word unit embedding. A position embedding matrix is ​​constructed to obtain a position embedding according to the position of the word unit. The word unit embedding and the position embedding are added as the input of the natural language model.

3. The infrared image power equipment fault detection method according to claim 1, characterized in that: The natural language model adopts a bidirectional encoding representation model pre-trained on a large-scale language dataset, and outputs the vector corresponding to the CLS head of the bidirectional encoding representation model as an auxiliary feature of the prompt word.

4. An infrared image power equipment fault detection device, characterized in that: The device includes: A data acquisition module is used to obtain text data of the permitted environment of the power equipment as prompt words and to obtain infrared images of the power equipment; A prompt word auxiliary feature extraction module, used to perform word grammar analysis on the prompt word and extract features using a natural language model to obtain prompt word auxiliary features; A fault detection module is configured to input the infrared image of the power equipment and the prompt word auxiliary features into a fault detection model based on a Unet structure and output a fault detection result. The fault detection model includes a feature encoding module and a feature decoding module. The feature encoding module dynamically generates a convolution kernel based on the prompt word auxiliary features to extract fault features. The feature decoding module matches the prompt word auxiliary features with the row features and column features of a low-resolution feature map for fault features of different scales to generate a target positioning attention map for auxiliary feature extraction. The feature encoding module includes multiple encoding blocks, each of which includes a first convolutional layer, a first batch normalization and activation layer, a prompt auxiliary convolutional layer, a second batch normalization and activation layer, and a downsampling layer connected in sequence. The input features of each encoding block are processed by the first convolutional layer and the first batch normalization and activation layer to obtain preliminary features. The prompt auxiliary convolutional layer uses a convolution kernel dynamically generated based on the auxiliary features of the prompt word to perform a convolution operation on the preliminary features to generate intermediate features. The intermediate features are processed by the downsampling layer and then output; The dynamic generation of the convolution kernel of the hint-assisted convolution layer is specifically as follows: Performing average pooling and maximum pooling on the preliminary features respectively to obtain average pooling features and maximum pooling features; The average pooling feature, the maximum pooling feature and the prompt word auxiliary feature are spliced ​​together and a convolution kernel is dynamically generated through a multi-layer perceptron; The feature decoding module includes multiple decoding blocks, the number of which is the same as the number of encoding blocks. The feature output to the current decoding block by the previous decoding block or the last encoding block is used as the first input feature, and the intermediate feature of the corresponding encoding block is used as the second input feature to perform high-resolution restoration on the input feature. The decoding block includes an upsampling layer, a second convolutional layer, a third batch normalization and activation layer, a target positioning attention layer and a third convolutional layer connected in sequence. After the first input feature is processed by the upsampling layer, it is spliced ​​with the second input feature, input into the second convolutional layer, and processed by the third batch normalization and activation layer to obtain a low-resolution feature map. The target positioning attention layer matches the auxiliary features of the prompt word with the row features and column features of the low-resolution feature map for fault features of different scales to generate a target positioning attention map. The third convolutional layer performs feature extraction processing on the low-resolution feature map based on the target positioning attention map and outputs it. The target localization attention layer performs the following steps: Slice the low-resolution feature map in rows and columns: ， ， in, is a low-resolution feature map, W 、 H 、 C in are the width, height and number of channels of the input power equipment infrared image, is the row slicing result, is the column slicing result, The shape of the feature is ; Perform dot product on the results of row slicing and column slicing with the auxiliary features of the prompt word to obtain row attention and column attention: ， ， in, For the i The attention of the line, For the j Column-by-column attention, It is the auxiliary feature of the prompt word; Perform outer product of the row attention and column attention to generate the target localization attention map: ， in, Localize the attention map for the target, is the row attention vector for all rows, is the column attention vector for all columns.

5. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 3 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images