Fault detection method and apparatus for converter valve, and computer device and storage medium

Abstract

The present application relates to a fault detection method and apparatus for a converter valve, and a computer device and a storage medium. The method comprises: acquiring infrared image data and visible light image data that correspond to a device in a converter valve; inputting the infrared image data and the visible light image data into a feature extraction network comprising a first convolution module and a gradient residual density module, so as to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; further inputting, into a fusion network comprising a global attention module and a second convolution module, the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data, so as to obtain a fused feature; and finally, inputting the fused feature into a target detection model, so as to obtain fault detection information of the converter valve. The solution can improve the efficiency of fault detection of converter valves.

Description

Fault detection method and device of converter valve, computer device and storage medium

[0001] The present application claims priority to the Chinese patent application No. CN202411028853.6, filed on July 30, 2024, and entitled "Fault detection method and device of converter valve, computer device and storage medium", the contents of which are hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of converter valve fault monitoring, in particular to a fault detection method and device of a converter valve, a computer device and a storage medium. BACKGROUND

[0003] In a direct current transmission system, a converter valve plays a vital role. In order to realize stable operation of the direct current transmission system, it is necessary to monitor the operating state of each device in the converter valve and ensure the normal and stable operation of each device in the converter valve.

[0004] In the conventional technology, whether each device in the converter valve has a fault is usually identified by manual inspection. Manual inspection has the problem of low efficiency in identifying whether each device in the converter valve has a fault. SUMMARY

[0005] Therefore, it is necessary to provide a fault detection method and device of a converter valve, a computer device and a storage medium to improve the fault detection efficiency of the converter valve.

[0006] In a first aspect, the present application provides a fault detection method of a converter valve, the method comprising:

[0007] obtaining infrared image data and visible light image data corresponding to a device in the converter valve;

[0008] inputting the infrared image data and the visible light image data into a feature extraction network to obtain feature images corresponding to the infrared image data and feature images corresponding to the visible light image data; wherein the feature extraction network comprises a first convolution module and a gradient residual density module;

[0009] inputting the feature images corresponding to the infrared image data and the feature images corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module;

[0010] inputting the fusion features into a target detection model to obtain fault detection information of the converter valve.

[0011] In one embodiment, the first convolution module comprises at least two convolution networks connected in sequence.

[0012] The inputting the infrared image data and the visible light image data into the feature extraction network comprises:

[0013] The output feature of the previous convolutional network is input into the next convolutional network to obtain the output feature of the next convolutional network; wherein the input feature of the first convolutional network is the infrared image data and the visible light image data, and the output feature of the last convolutional network is a first convolutional feature map corresponding to the infrared image data and a second convolutional feature map corresponding to the visible light image data;

[0014] The first convolutional feature map and the second convolutional feature map are respectively input into the gradient residual density module to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data.

[0015] In one of the embodiments, each convolutional network comprises a deep convolutional layer, a point-wise convolutional layer and an activation function layer;

[0016] The inputting the output feature of the previous convolutional network into the next convolutional network to obtain the output feature of the next convolutional network comprises:

[0017] The output feature of the previous convolutional network is input into the deep convolutional layer in the next convolutional network for deep convolution processing to obtain a deep convolutional feature map;

[0018] The deep convolutional feature map is input into the point-wise convolutional layer in the next convolutional network for point-wise convolution processing to obtain a point-wise convolutional feature map;

[0019] The point-wise convolutional feature map is input into the activation function layer in the next convolutional network to obtain the output feature of the next convolutional network.

[0020] In one of the embodiments, the gradient residual density module comprises at least two gradient residual density networks connected in sequence;

[0021] The inputting the first convolutional feature map and the second convolutional feature map into the gradient residual density module to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data comprises:

[0022] input the output feature of the previous gradient residual density network into a next gradient residual density network to obtain an output feature of the next gradient residual density network, wherein the input feature of the first gradient residual density network is the first convolution feature map and the second convolution feature map, and the output feature of the last gradient residual density network is a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data.

[0023] In one of the embodiments, each gradient residual density network comprises a first convolution function layer, a second convolution function layer, a convolution layer, a gradient convolution layer and a fusion layer.

[0024] The inputting of the output feature of the previous gradient residual density network into the next gradient residual density network to obtain the output feature of the next gradient residual density network comprises:

[0025] The output feature of the previous gradient residual density network is input into the first convolution function layer in the next gradient residual density network for convolution operation to obtain a first intermediate feature map.

[0026] The first intermediate feature map and the output feature of the previous gradient residual density network are input into the second convolution function layer in the next gradient residual density network for convolution operation to obtain a second intermediate feature map.

[0027] The second intermediate feature map and the output feature of the previous gradient residual density network are input into the convolution layer in the next gradient residual density network for convolution operation to obtain a third intermediate feature map.

[0028] The output feature of the previous gradient residual density network is input into the gradient convolution layer in the next gradient residual density network for convolution operation to obtain a fourth intermediate feature map.

[0029] The third intermediate feature map and the fourth intermediate feature map are input into the fusion layer in the next gradient residual density network for feature fusion operation to obtain the output feature of the next gradient residual density network.

[0030] In one of the embodiments, the inputting of the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into the fusion network to obtain the fusion feature comprises:

[0031] The feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data are input into the global attention module for global feature extraction to obtain a global sum feature map.

[0032] The global sum feature map is input into the second convolution module for convolution processing to obtain the fusion feature.

[0033] In one embodiment, the target detection model includes a YOLO model and a Bytetrack model.

[0034] The fusion feature is input into a target detection model to obtain the fault detection information of the converter valve.

[0035] The fusion feature is input into a YOLO model to obtain a target region image; wherein the target region image is a region image of a device with a fault in the converter valve.

[0036] The target region image is input into a Bytetrack model to obtain the fault detection information of the converter valve; wherein the fault detection information includes at least one of a faulty device, a fault type, and a fault location.

[0037] In a second aspect, the present application also provides a fault detection device for a converter valve, which includes:

[0038] An acquisition module is configured to acquire infrared image data and visible light image data corresponding to a device in a converter valve.

[0039] A feature extraction module is configured to input the infrared image data and the visible light image data into a feature extraction network to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; wherein the feature extraction network includes a first convolution module and a gradient residual density module.

[0040] A fusion module is configured to input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network to obtain a fusion feature; wherein the fusion network includes a global attention module and a second convolution module.

[0041] A detection module is configured to input the fusion feature into a target detection model to obtain fault detection information of the converter valve.

[0042] In a third aspect, the present application also provides a computer device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0043] Infrared image data and visible light image data corresponding to a device in a converter valve are acquired.

[0044] The infrared image data and the visible light image data are input into a feature extraction network to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; wherein the feature extraction network includes a first convolution module and a gradient residual density module.

[0045] input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module;

[0046] input the fusion features into a target detection model to obtain the fault detection information of the converter valve.

[0047] In a fourth aspect, the present application also provides a computer readable storage medium, the computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the following steps:

[0048] obtain infrared image data and visible light image data corresponding to a device in a converter valve;

[0049] input the infrared image data and the visible light image data into a feature extraction network to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; wherein the feature extraction network comprises a first convolution module and a gradient residual density module;

[0050] input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module;

[0051] input the fusion features into a target detection model to obtain the fault detection information of the converter valve.

[0052] In a fifth aspect, the present application also provides a computer program product, the computer program product comprises a computer program, and the computer program is executed by a processor to implement the following steps:

[0053] obtain infrared image data and visible light image data corresponding to a device in a converter valve;

[0054] input the infrared image data and the visible light image data into a feature extraction network to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; wherein the feature extraction network comprises a first convolution module and a gradient residual density module;

[0055] input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module;

[0056] input the fusion features into a target detection model to obtain the fault detection information of the converter valve.

[0057] The fault detection method, device, computer equipment and storage medium of the converter valve described above, obtain infrared image data and visible light image data corresponding to the devices in the converter valve; and input the infrared image data and the visible light image data into a feature extraction network including a first convolution module and a gradient residual density module to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; further input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network including a global attention module and a second convolution module to obtain fusion features; finally input the fusion features into a target detection model to obtain fault detection information of the converter valve. The above scheme uses a feature extraction network including a first convolution module and a gradient residual density module for feature extraction, which improves the discrimination and representativeness of the extracted features; a fusion network including a global attention module and a second convolution module is used for feature fusion, so that the fusion network pays more attention to important features; and then the fusion features are input into a target detection model to obtain fault detection information of the converter valve, which not only improves the fault detection efficiency of the converter valve, but also improves the fault detection accuracy of the converter valve. BRIEF DESCRIPTION OF DRAWINGS

[0058] Fig. 1 is a flowchart of a fault detection method of a converter valve in one embodiment;

[0059] Fig. 2 is a structural diagram of a feature extraction network in one embodiment;

[0060] Fig. 3 is a structural diagram of a fusion network in one embodiment;

[0061] Fig. 4 is a flowchart of obtaining feature images in one embodiment;

[0062] Fig. 5 is a flowchart of obtaining output features of each convolutional network in one embodiment;

[0063] Fig. 6 is a structural diagram of a gradient residual density network in one embodiment;

[0064] Fig. 7 is a flowchart of obtaining output features of the gradient residual density network in one embodiment;

[0065] Fig. 8 is a flowchart of obtaining fusion features in one embodiment;

[0066] Fig. 9 is a flowchart of YOLO model training and fault detection in one embodiment;

[0067] Fig. 10 is a flowchart of obtaining fault detection information of the converter valve in one embodiment;

[0068] Fig. 11 is a flowchart of another fault detection of the converter valve in one embodiment;

[0069] FIG. 12 is a structural block diagram of a fault detection device of a converter valve in an embodiment;

[0070] FIG. 13 is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0071] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0072] The fault detection method of the converter valve provided by the embodiments of the present application can be applied to the application scenario of detecting whether the converter valve has a fault, and is specifically applied to detecting whether each device in the converter valve has a temperature anomaly, a crack, a damage, etc. The method can be executed by a server or a terminal with a certain computing power.

[0073] In an embodiment, as shown in FIG. 1, a fault detection method of a converter valve is provided, and the method is taken as an example to be executed by a server, which includes the following steps:

[0074] S101, infrared image data and visible light image data corresponding to devices in the converter valve are acquired.

[0075] For example, the infrared image data corresponding to the devices in the converter valve can be collected by an infrared thermal imager, and the visible light image data corresponding to the devices in the converter valve can be collected by a visible light camera device. The devices in the converter valve include, but are not limited to, thyristors, thyristor control devices, damping resistors, capacitors, waterway systems, reactors and through-flow circuits.

[0076] Optionally, after the infrared image data and the visible light image data are acquired, the infrared image data and the visible light image data can be preprocessed to improve the quality of the image data and facilitate subsequent feature extraction and other operations on the images. For example, the infrared image data collected by the infrared thermal imager can be affected by factors such as environmental temperature and device drift, so temperature calibration is needed to ensure that the temperature information in the infrared image data is accurate and reliable. The visible light image data collected by the visible light camera device can have problems such as noise, blur or insufficient contrast due to different environmental conditions during collection, so image enhancement is needed to improve the quality and information readability of the visible light image data.

[0077] Exemplarily, the temperature calibration of the infrared image data can be performed based on the correspondence between the gray value and the ambient temperature. For example, a standard temperature object in the environment is measured by using the thermal imaging device, and the corresponding pixel gray value is recorded. The mapping relationship between the gray value and the temperature is established. According to the established mapping relationship, the pixel gray value in the image is converted into the corresponding temperature value. According to the heat characteristics of the standard temperature object, the relationship between the pixel gray value and the temperature is established, and the accurate extraction of the temperature information in the infrared image data is realized.

[0078] Exemplarily, the visible light image data can be enhanced by using the histogram equalization method, and the process of the image enhancement is as follows:

[0079] Firstly, the gray histogram of the visible light image data can be calculated, the visible light image data is gray processed, and the color image is converted into a gray image. The frequency of each pixel value in the image is counted to obtain the gray histogram of the gray image. It is assumed that the gray level range of the gray image is [0, L-1], wherein L is the number of gray levels of the pixel, the frequency is n(k), and k represents the gray level of the pixel.

[0080] Further, the cumulative distribution function (CDF) can be calculated, and the calculation formula of the CDF is as follows:

[0081] wherein CDF(k) is the cumulative distribution function of the pixel with the gray level k, n(j) is the frequency of the gray level j, and N is the total number of pixels of the image.

[0082] Further, the CDF is normalized as follows:

[0083] wherein CDF norm (k) is the normalized CDF(k). The normalized CDF is applied to each pixel value in the image, and the gray value of each pixel is adjusted to complete the data enhancement of the visible light image data.

[0084] It should be noted that when the infrared image data and the visible light image data corresponding to the device in the converter valve are collected, the shooting angle, distance, collection time, sampling frequency, etc. of the infrared thermal imager and the visible light camera device relative to the device need to be ensured to be the same, and the infrared image data and the visible light image data are time-synchronized and space-synchronized calibrated to ensure the correspondence and consistency between the two kinds of data, so as to better perform image fusion of the converter valve device. In this way, the collected infrared image data and visible light image data can correspond to each other, and can jointly reflect the state of the device at the same time, so as to facilitate judging whether the device has fault information according to the infrared image data and the visible light image data.

[0085] S102, input the infrared image data and the visible light image data to the feature extraction network to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data.

[0086] Referring to FIG. 2, FIG. 2 provides a structural schematic diagram of a feature extraction network, wherein the feature extraction network includes a first convolution module and a gradient residual density module. The infrared image data and the visible light image data can be input to the feature extraction network, so that the feature extraction network can perform feature extraction on the infrared image data and the infrared image data to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data. The feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data include features meaningful for converter valve device fault detection. The first convolution module used by the feature extraction network is a lightweight depth separable convolution, and the gradient residual density module used by the feature extraction network is a residual connection gradient operator. The description ability of the fusion network to fine-grained spatial details can be still enhanced under the premise of lightweight and efficient overall network. Thus, the image fusion of the infrared image data and the visible light image data can be better performed.

[0087] In addition, the depth separable convolution is an operation in a lightweight convolutional neural network, which reduces the amount of calculation and reduces the amount of model parameters by separating the depth and spatial operation of the convolution kernel. The discrimination and representativeness of the extracted features can be improved, while the computational complexity and the amount of model parameters are reduced, thereby improving the detection speed.

[0088] S103, input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data to the fusion network to obtain the fusion feature.

[0089] Referring to FIG. 3, FIG. 3 provides a structural schematic diagram of a fusion network, wherein the fusion network comprises a global attention module and a second convolution module. The feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data can be input into the fusion network to obtain the fusion feature. The global attention mechanism is introduced into the global attention module in the fusion network, so that the fusion network can better focus on the important feature image region. The second convolution module can enhance the expression ability of the fusion network, so that the obtained fusion feature contains richer and more important features.

[0090] S104, inputting the fusion feature into the target detection model to obtain the fault detection information of the converter valve.

[0091] Further, the target detection model can be obtained by training according to the sample infrared image data and the sample visible light image data, wherein the sample infrared image data and the sample visible light data include different types and severities of faults. Thus, the fusion feature can be input into the target detection model for fault detection, and then the fault detection information of the converter valve is obtained.

[0092] The above-mentioned fault detection method of the converter valve, obtains the infrared image data and the visible light image data corresponding to the device in the converter valve; and inputs the infrared image data and the visible light image data into a feature extraction network comprising a first convolution module and a gradient residual density module to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; further inputs the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network comprising a global attention module and a second convolution module to obtain a fusion feature; finally, inputs the fusion feature into a target detection model to obtain the fault detection information of the converter valve. The above-mentioned scheme uses the feature extraction network comprising the first convolution module and the gradient residual density module to perform feature extraction, which improves the discrimination and representativeness of the extracted features; uses the fusion network comprising the global attention module and the second convolution module to perform feature fusion, so that the fusion network pays more attention to important features; and then inputs the fusion feature into the target detection model to obtain the fault detection information of the converter valve, which not only improves the fault detection efficiency of the converter valve, but also improves the fault detection accuracy of the converter valve.

[0093] In some optional implementations, the first convolution module in the above-mentioned embodiments comprises at least two convolution networks connected in sequence.

[0094] Based on this, referring to FIG. 4, FIG. 4 provides a flowchart for obtaining a feature image, which specifically comprises the following steps:

[0095] S401, inputting the output feature of the previous convolution network into the next convolution network to obtain the output feature of the next convolution network.

[0096] The output features of the previous convolutional network are input into the next convolutional network for feature extraction, so as to obtain the output features of the next convolutional network. The input features of the first convolutional network are the infrared image data and the visible light image data, and the output features of the last convolutional network are the first convolutional feature map corresponding to the infrared image data and the second convolutional feature map corresponding to the visible light image data.

[0097] S402, the first convolutional feature map and the second convolutional feature map are input into the gradient residual density module respectively, to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data.

[0098] Further, the first convolutional feature map output by the last convolutional network and the second convolutional feature map output by the last convolutional network can be input into the gradient residual density module respectively for feature extraction, to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data.

[0099] In the embodiment of the application, the first convolutional module is set to include at least two convolutional networks connected in sequence, which can reduce the calculation amount in the feature extraction process, and further reduce the parameter amount of the target detection model, and improve the fault detection efficiency of the converter valve.

[0100] In some optional implementations, each convolutional network in the above embodiment can include a deep convolutional layer, a pointwise convolutional layer and an activation function layer. Specifically, the output features of the deep convolutional layer in each convolutional network are input into the pointwise convolutional layer in the convolutional network; the output features of the pointwise convolutional layer in the convolutional network are input into the activation function layer in the convolutional network.

[0101] Based on this, referring to FIG. 5, FIG. 5 provides a flowchart for obtaining the output features of each convolutional network, which specifically includes the following steps:

[0102] S501, the output features of the previous convolutional network are input into the deep convolutional layer in the next convolutional network for deep convolution processing, to obtain a deep convolutional feature map.

[0103] For example, the output features of the previous convolutional network can be input into the deep convolutional layer in the next convolutional network for deep convolution processing, to obtain a deep convolutional feature map. Specifically, the deep convolutional layer first applies a deep convolution kernel to each channel of the output features of the previous convolutional network, to generate a deep convolutional feature map with the same number of channels as the output features of the previous convolutional network. The mathematical expression of the deep convolutional layer is: i = K i * X

[0104] wherein Y iis the i-th depth convolution kernel, X is the output feature of the previous convolutional network, and * is a convolution operation. i is the i-th depth convolution kernel, X is the output feature of the previous convolutional network, and * is a convolution operation.

[0105] S502, input the depth convolution feature map into the pointwise convolution layer in the next convolutional network for pointwise convolution processing to obtain a pointwise convolution feature map.

[0106] For example, the pointwise convolution layer in the next convolutional network is to apply a 1x1 convolution kernel (i.e., a pointwise convolution kernel) to each depth convolution feature map based on the depth convolution layer for pointwise convolution processing, and to mix the information of each channel together to obtain a pointwise convolution feature map.

[0107] S503, input the pointwise convolution feature map into the activation function layer in the next convolutional network to obtain the output feature of the next convolutional network.

[0108] For example, the activation function LReLU in the activation function layer is a variant of the ReLU (Rectified Linear Unit), which has better performance, and its function expression is:

[0109] where x is the pointwise convolution feature map, a is a proportionality coefficient, and f(x) is the output feature of the next convolutional network. Thus, the pointwise convolution feature map is input into the activation function layer in the next convolutional network, and the pointwise convolution feature map is processed by the activation function LReLU to obtain the output feature of the next convolutional network.

[0110] In the embodiments of the present application, each convolutional network is provided to include a depth convolution layer, a pointwise convolution layer and an activation function layer, and each convolutional network can be considered as a depth separable convolutional network. The depth separable convolutional network is a lightweight convolutional neural network, which reduces the amount of calculation and the number of model parameters by separating the depth and spatial operations of the convolution kernel, thereby improving the fault detection efficiency of the converter valve.

[0111] In some optional implementations, the gradient residual density module in the above embodiments can include at least two gradient residual density networks connected in sequence.

[0112] Based on this, the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data are obtained, which can be achieved by the following way:

[0113] The output feature of the previous gradient residual density network is input into the next gradient residual density network to obtain an output feature of the next gradient residual density network, wherein the input feature of the first gradient residual density network is the first and second convolution feature maps, and the output feature of the last gradient residual density network is a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data.

[0114] In the embodiments of the present application, the gradient residual density module can improve the discrimination and representativeness of the extracted features, reduce the computational complexity and the number of model parameters, and thus improve the fault detection efficiency of the converter valve.

[0115] Further, referring to FIG. 6, FIG. 6 provides a structural schematic diagram of a gradient residual density network, wherein each gradient residual density network includes a first convolution function layer, a second convolution function layer, a convolution layer, a gradient convolution layer, and a fusion layer.

[0116] Based on this, referring to FIG. 7, FIG. 7 provides a flowchart of obtaining an output feature of a gradient residual density network, which specifically includes the following steps:

[0117] S701, inputting the output feature of the previous gradient residual density network into the first convolution function layer in the next gradient residual density network to perform convolution operation to obtain a first intermediate feature map.

[0118] For example, the first convolution function layer in the next gradient residual density network includes a 3x3 convolution kernel and an LReLU activation function, and inputting the output feature of the previous gradient residual density network into the first convolution function layer in the next gradient residual density network to perform convolution operation can be understood as sequentially processing the output feature of the previous gradient residual density network through the 3x3 convolution kernel and the LReLU activation function to obtain the first intermediate feature map.

[0119] S702, inputting the first intermediate feature map and the output feature of the previous gradient residual density network into the second convolution function layer in the next gradient residual density network to perform convolution operation to obtain a second intermediate feature map.

[0120] Further, the second convolution function layer in the next gradient residual density network can also include a 3x3 convolution kernel and an LReLU activation function, and inputting the first intermediate feature map and the output feature of the previous gradient residual density network into the second convolution function layer in the next gradient residual density network to perform convolution operation can be understood as sequentially processing the first intermediate feature map and the output feature of the previous gradient residual density network through the 3x3 convolution kernel and the LReLU activation function to obtain the second intermediate feature map.

[0121] S703, inputting the second intermediate feature map and the output feature of the previous gradient residual dense network into a convolution layer in the next gradient residual dense network for convolution operation to obtain a third intermediate feature map.

[0122] Further, the convolution layer in the next gradient residual dense network can be a 1x1 convolution kernel, and the second intermediate feature map and the output feature of the previous gradient residual dense network can be input into the convolution layer in the next gradient residual dense network for convolution operation to obtain the third intermediate feature map.

[0123] S704, inputting the output feature of the previous gradient residual dense network into a gradient convolution layer in the next gradient residual dense network for convolution operation to obtain a fourth intermediate feature map.

[0124] Further, the gradient convolution layer in the next gradient residual dense network includes a gradient information processing layer and a 1x1 convolution kernel. The output feature of the previous gradient residual dense network can be input into the gradient convolution layer in the next gradient residual dense network for convolution operation, which can be understood as performing gradient convolution processing on the output feature of the previous gradient residual dense network through the gradient information processing layer and the convolution kernel in sequence to obtain the fourth intermediate feature map.

[0125] S705, inputting the third intermediate feature map and the fourth intermediate feature map into a fusion layer in the next gradient residual dense network for feature fusion operation to obtain the output feature of the next gradient residual dense network.

[0126] Further, the fusion layer can be understood as a network layer for fusing the third intermediate feature map and the fourth intermediate feature map. The third intermediate feature map and the fourth intermediate feature map are input into the fusion layer in the next gradient residual dense network for feature fusion operation to obtain the output feature of the next gradient residual dense network.

[0127] In the embodiments of the present application, each gradient residual dense network is configured to include a first convolution function layer, a second convolution function layer, a convolution layer, a gradient convolution layer and a fusion layer, which combines the ideas of residual connection and dense connection and uses gradient information to enhance the expression ability of features. There is a direct connection between each layer in the gradient residual dense network, and the output of each layer is used as the input of the subsequent layer, thereby forming a dense connection structure to improve the discrimination and representativeness of the extracted features, while reducing the computational complexity and the number of model parameters, thereby improving the fault detection efficiency of the converter valve.

[0128] In some optional implementation manners, referring to FIG. 8, FIG. 8 provides a flowchart for obtaining fused features, which specifically includes the following steps:

[0129] S801, input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a global attention module for global feature extraction to obtain a global added feature map.

[0130] For example, before inputting the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into the global attention module, the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data can be fused first, for example, the fusion can be performed first through a concatenation operation to obtain a comprehensive feature. This means that the features of the infrared image data and the features of the visible light image data are spliced together in the channel direction to form a comprehensive feature representation. In this way, the information of the two kinds of images can be combined so that the subsequent detection of the fault of the converter valve can more comprehensively consider the characteristics of the two kinds of images.

[0131] Further, the comprehensive feature can be input into the global attention module for global feature extraction, wherein the global attention module can include a pooling layer, a fully connected layer, an activation function layer and a weighted addition layer. The specific implementation steps of the global attention module for global feature extraction are as follows:

[0132] First, the comprehensive feature is input into the pooling layer for global pooling operation, and the feature values in each channel of the comprehensive feature are summarized to obtain the global feature representation of each channel. The global pooling is usually realized by using average pooling or maximum pooling.

[0133] Further, a fully connected layer is used to map the global feature to a new feature space so as to adjust and weight the feature subsequently. This process can be represented as: z = ReLU (W1gpool (Z) + b1)

[0134] Wherein, z is the mapped global feature, W1 and b1 are the weights and bias of the fully connected layer, Z is the global feature, and pool (Z) is the global pooling operation.

[0135] Further, after feature mapping, the mapped global feature is input into the activation function layer, and an activation function operation is performed to calculate the weight of each channel to determine the importance of each channel.

[0136] Finally, the original feature map is input into the weighted addition layer, and a weighted addition operation is performed with the calculated weight to obtain a global added feature map.

[0137] The global attention module is configured to introduce a global attention mechanism, so that the fusion network can focus on important feature regions better. The global attention module can include a global pooling operation to capture global feature information, and then adjust the weights of the global features through operations such as convolution and activation functions, so as to ensure that the fusion network pays more attention to important features, and then obtains a global added feature map.

[0138] At S802, the global added feature map is input into the second convolution module for convolution processing to obtain a fusion feature.

[0139] For example, the second convolution module can include at least two sequentially connected convolution networks and a reconstruction network; each convolution network can include a deep convolution layer, a point-wise convolution layer, and an activation function layer. Specifically, the output features of the deep convolution layer in each convolution network are input into the point-wise convolution layer in the convolution network; the output features of the point-wise convolution layer in the convolution network are input into the activation function layer in the convolution network. The processing process of each convolution network is described in the above embodiment of FIG. 5, which will not be described here. The reconstruction network includes a deep convolution layer, a point-wise convolution layer, and an activation function layer, and the activation function of the reconstruction network can be a hyperbolic tangent function (Tanh). The expression of the Tanh function is as follows:

[0140] where a is the output feature of the last convolution network, and e is a constant. The Tanh function scales the output value to the interval [-1, 1], which is suitable for processing the case where the output range is continuous. The purpose of this step is to learn the reconstructed image through the reconstruction network so that it is as similar as possible to the original image.

[0141] Specifically, the global added feature map can be input into the convolution network in the second convolution module, and the output feature of the last convolution network can be input into the reconstruction network for image reconstruction to obtain a fusion feature.

[0142] In the embodiments of the present application, the global attention module in the fusion network introduces a global attention mechanism, so that the fusion network can focus on important feature image regions better; the second convolution module can enhance the expression ability of the fusion network, so that the obtained fusion feature contains richer and more important features, and the accuracy of the converter valve fault detection result is improved.

[0143] In some optional implementations, the target detection model in the above embodiments can include a YOLO model and a Bytetrack model; the YOLO model can be a YOLOv8 model, and the YOLOv8 model can be trained using a training data set.

[0144] Referring to FIG. 9, FIG. 9 provides a flowchart of YOLO model training and fault detection.

[0145] Specifically, multi-angle and multi-directional converter valve infrared image data and visible light image data can be collected for converter valves of various different converter stations. The diversity of the training data set is ensured, and the training data set can include fault image data of different types and different severities, as well as image data under various lighting and environmental conditions. The collected image data is labeled using the boundary box image labeling tool LabelImg, and information such as the fault location and fault type of the converter valve fault device is added to the image data. For example, it can be labeled whether the thyristor, thyristor control device, damping resistor, capacitor, water system, reactor, and through-flow circuit have overheating, cracking, or damage.

[0146] Further, the YOLOv8 model is trained using the labeled training data set. The YOLOv8 model is used to train the flow valve fault target model on the training set for training the flow valve fault target. The training stopping condition is that the set iteration number is reached or the accuracy on the validation set for converter valve fault detection parameter adjustment reaches the preset converter valve fault accuracy threshold, and the YOLOv8 model in the target detection model is obtained.

[0147] Thus, the best YOLOv8 model and Bytetrack model trained can be used to detect faults of the converter valve based on infrared image data and visible light data, so as to realize real-time monitoring of the converter valve.

[0148] Based on this, referring to FIG. 10, FIG. 10 provides a flowchart of obtaining fault detection information of a converter valve, specifically including the following steps:

[0149] S1001, input the fusion features into the YOLO model to obtain a target region image.

[0150] For example, the fusion features can be input into the YOLO model, and the YOLO model can extract the region image of the device with a fault to obtain a target region image. The target region image is a region image of a device with a fault in the converter valve.

[0151] S1002, input the target region image into the Bytetrack model to obtain fault detection information of the converter valve.

[0152] Further, the target region image can be input into the Bytetrack model. It should be noted that the target region image can be a plurality of continuous target region images corresponding to a plurality of continuous fusion features. Thus, the plurality of continuous target region images are input into the Bytetrack model, so as to track the motion and change of the region image including the faulty device in the continuous frames, and obtain the continuous change of the faulty device, i.e., obtain the fault detection information of the converter valve. The fault detection information includes at least one of the faulty device, the fault type and the fault position, facilitating analysis of the fault of the converter valve.

[0153] In the embodiments of the present application, by setting the target detection model to include the YOLO model and the Bytetrack model, the final fault detection information of the converter valve can include the faulty device, the fault type and the fault position, and the motion trajectory of the fault detection information of the faulty device in the continuous frames, etc. The fast detection and accurate tracking of the faulty device of the converter valve can be realized, and reliable support for real-time monitoring and early warning is provided.

[0154] In some optional implementations, referring to FIG. 11, FIG. 11 provides another flowchart of the fault detection of the converter valve, which specifically includes the following steps:

[0155] Step one: data acquisition. For the converter valves of a plurality of different converter stations, multi-directional and multi-angle infrared image data and visible light image data can be acquired.

[0156] Step two: data preprocessing. The acquired infrared image data and visible light image data are respectively preprocessed to improve the quality and information readability of the images.

[0157] Step three: feature extraction. Representative feature information is extracted from the preprocessed image data for subsequent state classification and fusion. A gradient operator with residual connection is used in the feature extraction stage to better improve the discrimination and representativeness of the features. A lightweight deep separable convolutional network structure is adopted to reduce the computational complexity and model parameter quantity and improve the inference speed.

[0158] Step four: data fusion. In the data fusion stage, an attention mechanism is introduced to adaptively adjust the weights of different features, so that the model pays more attention to the information useful for the fault detection task. A lightweight convolutional network structure is used as a feature extractor to ensure that the model has fewer parameter quantities and computational complexity, while meeting the real-time requirements.

[0159] Step five: precise positioning of the traveling wave. After obtaining the fusion features, the YOLOv8 model is used to detect the location and fault type of the faulty device, and to realize the real-time and accuracy of the faulty device detection. And the ByteTrack model is used to track the faulty device detected by YOLOv8, to realize the continuous tracking and location prediction of the fault information of the faulty device.

[0160] The embodiments of the present application can realize the precise positioning of the multiple fault types and fault points of the converter valve. Moreover, the multi-modal data fusion technology is introduced, and the fault device detection and the fault device tracking are combined to cope with the converter valve state monitoring task in a complex scene. In addition, the light neural network structure and the global attention mechanism are introduced in the image fusion part of the embodiments of the present application. The efficient processing of the image fusion part and the attention to the global information are realized. The fused model not only has a faster detection speed, but also can effectively capture important information in the image and improve the accurate monitoring ability of the converter valve state.

[0161] It should be understood that, although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0162] Based on the same inventive concept, the embodiments of the present application also provide a fault detection device for a converter valve, which is used to implement the above-mentioned fault detection method for a converter valve. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more fault detection device embodiments for a converter valve provided below can refer to the limitations of the fault detection method for a converter valve described above, which will not be repeated here.

[0163] In one embodiment, as shown in FIG. 12, a fault detection device for a converter valve is provided, which comprises:

[0164] The acquisition module 10 is configured to acquire infrared image data and visible light image data corresponding to the devices in the converter valve.

[0165] The feature extraction module 20 is configured to input the infrared image data and the visible light image data into a feature extraction network to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; wherein the feature extraction network comprises a first convolution module and a gradient residual density module.

[0166] The fusion module 30 is configured to input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module.

[0167] The detection module 40 is configured to input the fusion features into a target detection model to obtain the fault detection information of the converter valve.

[0168] The above-mentioned fault detection device of the converter valve acquires infrared image data and visible light image data corresponding to devices in the converter valve; and inputs the infrared image data and the visible light image data into a feature extraction network comprising a first convolution module and a gradient residual density module to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; further inputs the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network comprising a global attention module and a second convolution module to obtain fusion features; finally inputs the fusion features into a target detection model to obtain the fault detection information of the converter valve. The above-mentioned scheme uses the feature extraction network comprising the first convolution module and the gradient residual density module to perform feature extraction, thereby improving the discrimination and representativeness of the extracted features; uses the fusion network comprising the global attention module and the second convolution module to perform feature fusion, thereby making the fusion network pay more attention to important features; and then inputs the fusion features into the target detection model to obtain the fault detection information of the converter valve, thereby not only improving the fault detection efficiency of the converter valve, but also improving the fault detection accuracy of the converter valve.

[0169] In one of the embodiments, the first convolution module comprises at least two convolution networks connected in sequence; and the feature extraction module 20 specifically comprises:

[0170] The convolution unit is configured to input the output features of the previous convolution network into the next convolution network to obtain the output features of the next convolution network; wherein the input features of the first convolution network are the infrared image data and the visible light image data, and the output features of the last convolution network are a first convolution feature map corresponding to the infrared image data and a second convolution feature map corresponding to the visible light image data;

[0171] The gradient residual density unit is configured to input the first convolution feature map and the second convolution feature map into the gradient residual density module respectively to obtain the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data.

[0172] In one of the embodiments, each of the stacked networks comprises a deep convolution layer, a point-wise convolution layer and an activation function layer; and the convolution unit is specifically configured to:

[0173] The output feature of the previous stacked network is input into the deep convolution layer in the next stacked network for deep convolution processing to obtain a deep convolution feature map; the deep convolution feature map is input into the point-wise convolution layer in the next stacked network for point-wise convolution processing to obtain a point-wise convolution feature map; and the point-wise convolution feature map is input into the activation function layer in the next stacked network to obtain the output feature of the next stacked network.

[0174] In one of the embodiments, the gradient residual density module comprises at least two gradient residual density networks connected in sequence; and the gradient residual density unit is specifically configured to:

[0175] The output feature of the previous gradient residual density network is input into the next gradient residual density network to obtain the output feature of the next gradient residual density network, wherein the input feature of the first gradient residual density network is the first convolution feature map and the second convolution feature map, and the output feature of the last gradient residual density network is a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data.

[0176] In one of the embodiments, each of the gradient residual density networks comprises a first convolution function layer, a second convolution function layer, a convolution layer, a gradient convolution layer and a fusion layer; and the gradient residual density unit is specifically configured to:

[0177] The output feature of the previous gradient residual density network is input into the first convolution function layer in the next gradient residual density network for convolution operation to obtain a first intermediate feature map; the first intermediate feature map and the output feature of the previous gradient residual density network are input into the second convolution function layer in the next gradient residual density network for convolution operation to obtain a second intermediate feature map; the second intermediate feature map and the output feature of the previous gradient residual density network are input into the convolution layer in the next gradient residual density network for convolution operation to obtain a third intermediate feature map; the output feature of the previous gradient residual density network is input into the gradient convolution layer in the next gradient residual density network for convolution operation to obtain a fourth intermediate feature map; and the third intermediate feature map and the fourth intermediate feature map are input into the fusion layer in the next gradient residual density network for feature fusion operation to obtain the output feature of the next gradient residual density network.

[0178] In one of the embodiments, the fusion module 30 is specifically configured to:

[0179] The feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data are input into a global attention module for global feature extraction to obtain a global added feature map; and the global added feature map is input into a second convolution module for convolution processing to obtain a fusion feature.

[0180] In one of the embodiments, the target detection model includes a YOLO model and a Bytetrack model; and the detection module 40 is specifically configured to:

[0181] The fusion feature is input into the YOLO model to obtain a target region image; the target region image is a region image of a device with a fault in the converter valve; the target region image is input into the Bytetrack model to obtain fault detection information of the converter valve; and the fault detection information includes at least one of a faulty device, a fault type and a fault position.

[0182] The above-mentioned modules in the fault detection device of the converter valve can be realized by software, hardware and combinations thereof in whole or in part. The above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above-mentioned modules.

[0183] In one embodiment, a computer device is provided, which can be a server, and the internal structure diagram thereof can be as shown in FIG. 13. The computer device includes a processor, a memory and a network interface connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store infrared image data and visible light image data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a fault detection method of a converter valve.

[0184] Those skilled in the art can understand that the structure shown in FIG. 13 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. Specifically, the computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0185] In one embodiment, a computer device is provided, which includes a memory and a processor, and the memory stores a computer program. The processor executes the computer program to implement the fault detection method of the converter valve according to any one of the above-mentioned embodiments.

[0186] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program, which, when executed by a processor, implements the fault detection method of the converter valve according to any one of the above embodiments.

[0187] In one embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the fault detection method of the converter valve according to any one of the above embodiments.

[0188] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. Any reference to a memory, database or other medium used in the embodiments provided in the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical memory, a high-density embedded non-volatile memory, a resistive memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0189] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, as long as the combinations of technical features do not have contradictions, they shall be considered within the scope of the present disclosure.

[0190] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method of detecting a failure of a converter valve, characterized by, The method comprises: obtaining infrared image data and visible light image data corresponding to devices in a converter valve; inputting the infrared image data and the visible light image data into a feature extraction network to obtain feature images corresponding to the infrared image data and feature images corresponding to the visible light image data; wherein the feature extraction network comprises a first convolution module and a gradient residual density module; inputting the feature images corresponding to the infrared image data and the feature images corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module; inputting the fusion features into a target detection model to obtain fault detection information of the converter valve.

2. The method of claim 1, wherein, The first convolution module comprises at least two convolution networks connected in sequence; the inputting the infrared image data and the visible light image data into the feature extraction network to obtain the feature images corresponding to the infrared image data and the feature images corresponding to the visible light image data comprises: inputting output features of a previous convolution network into a next convolution network to obtain output features of the next convolution network; wherein input features of a first convolution network are the infrared image data and the visible light image data, and output features of a last convolution network are first convolution feature images corresponding to the infrared image data and second convolution feature images corresponding to the visible light image data; respectively inputting the first convolution feature images and the second convolution feature images into the gradient residual density module to obtain the feature images corresponding to the infrared image data and the feature images corresponding to the visible light image data.

3. The method of claim 2, wherein, Each convolution network comprises a depth convolution layer, a point-wise convolution layer and an activation function layer; the inputting the output features of the previous convolution network into the next convolution network to obtain the output features of the next convolution network comprises: inputting the output features of the previous convolution network into a depth convolution layer in the next convolution network for depth convolution processing to obtain a depth convolution feature image; inputting the depth convolution feature image into a point-wise convolution layer in the next convolution network for point-wise convolution processing to obtain a point-wise convolution feature image; inputting the point-wise convolution feature image into an activation function layer in the next convolution network to obtain the output features of the next convolution network.

4. The method of claim 2, wherein, The gradient residual density module comprises at least two gradient residual density networks connected in sequence; the respectively inputting the first convolution feature images and the second convolution feature images into the gradient residual density module to obtain the feature images corresponding to the infrared image data and the feature images corresponding to the visible light image data comprises: inputting output features of a previous gradient residual density network into a next gradient residual density network to obtain output features of the next gradient residual density network; wherein input features of a first gradient residual density network are the first convolution feature images and the second convolution feature images, and output features of a last gradient residual density network are the feature images corresponding to the infrared image data and the feature images corresponding to the visible light image data.

5. The method of claim 4, wherein, Each gradient residual density network comprises a first convolution function layer, a second convolution function layer, a convolution layer, a gradient convolution layer and a fusion layer; The output feature of the previous gradient residual density network is input into the first convolution function layer in the next gradient residual density network for convolution operation to obtain a first intermediate feature map; The first intermediate feature map and the output feature of the previous gradient residual density network are input into the second convolution function layer in the next gradient residual density network for convolution operation to obtain a second intermediate feature map; The second intermediate feature map and the output feature of the previous gradient residual density network are input into the convolution layer in the next gradient residual density network for convolution operation to obtain a third intermediate feature map; The output feature of the previous gradient residual density network is input into the gradient convolution layer in the next gradient residual density network for convolution operation to obtain a fourth intermediate feature map; The third intermediate feature map and the fourth intermediate feature map are input into the fusion layer in the next gradient residual density network for feature fusion operation to obtain the output feature of the next gradient residual density network. The feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data are input into the fusion network to obtain fusion features, comprising:

6. The method of claim 1, wherein, The feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data are input into the global attention module for global feature extraction to obtain a global sum feature map; The global sum feature map is input into the second convolution module for convolution processing to obtain fusion features. The target detection model comprises a YOLO model and a Bytetrack model; 7. The method of claim 1, wherein, The fusion features are input into the target detection model to obtain fault detection information of the converter valve, comprising: The fusion features are input into the YOLO model to obtain a target region image; wherein the target region image is a region image of a device with a fault in the converter valve; The target region image is input into the Bytetrack model to obtain the fault detection information of the converter valve; wherein the fault detection information comprises at least one of a fault device, a fault type and a fault position. The device comprises:

8. A failure detection device for a converter valve, characterized in that An acquisition module configured to acquire infrared image data and visible light image data corresponding to a device in a converter valve; A feature extraction module configured to input the infrared image data and the visible light image data into a feature extraction network to obtain a feature image corresponding to the infrared image data and a feature image corresponding to the visible light image data; wherein the feature extraction network comprises a first convolution module and a gradient residual density module; A fusion module configured to input the feature image corresponding to the infrared image data and the feature image corresponding to the visible light image data into a fusion network to obtain fusion features; wherein the fusion network comprises a global attention module and a second convolution module; ​ A detection module is configured to input the fusion features into a target detection model to obtain fault detection information of the converter valve. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1 to 7.

Images