A method for correcting an infrared image of a power device, a defect identification method and a device

Abstract

The application discloses a kind of power equipment infrared image correction method, defect identification method and equipment, it is related to power equipment diagnosis technical field.The correction method identifies the contour area of power equipment from visible light image, and is segmented into multiple material areas, and according to rough factor correction reference emissivity of each material area to obtain current emissivity.Utilize the internal parameter of image acquisition device and the contour area of power equipment to calculate spatial pose, further generate observation distance, determine medium transmissivity in combination with medium parameter in observation path.Finally, the original radiation brightness value of infrared image is corrected by synthesizing medium transmissivity and current emissivity, the temperature value of each coordinate point is solved, and accurate corrected infrared image is generated.Further, the defect identification method of the application generates fusion image for identifying equipment defects in combination with visible light image and corrected infrared image, and then judges fault type.

Description

Technical Field

[0001] This invention relates to the field of power equipment testing technology, and in particular to a method for correcting infrared images of power equipment, a method for identifying defects, and a device for such identification. Background Technology

[0002] In power equipment operation and maintenance, infrared thermal imaging technology is widely used for thermal fault detection due to its advantages such as non-contact operation and high real-time performance. However, the accuracy of infrared temperature measurement is highly dependent on the accurate setting of emissivity, which is affected by various factors such as the material and surface roughness of the power equipment. If a fixed empirical value is used, it is easy to cause large temperature inversion errors, leading to misjudgments or missed diagnoses. Chinese patent application CN118211125A discloses a method and system for condition-based maintenance of power equipment based on artificial intelligence. This method identifies equipment types and matches preset emissivity using a Faster R-CNN model, and combines it with operational data for fault analysis, which improves the targeting of emissivity selection to a certain extent.

[0003] Furthermore, infrared radiation is affected by atmospheric attenuation during transmission, and the observation distance and environmental parameters also significantly affect the accuracy of temperature measurement. Chinese patent application CN114821035A discloses a method for identifying distance parameters in infrared temperature measurement equipment for power devices. This method focuses on distance identification, utilizing the device pixel width in the infrared image and automatically obtaining the shooting distance through an improved monocular ranging algorithm, providing parameter support for atmospheric attenuation compensation. However, the above method does not consider the emissivity variations caused by differences in surface roughness of the same type of power equipment. Therefore, the existing technology needs further improvement. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a method for correcting infrared images of power equipment, a method for identifying defects, and a device for such identification. The method for correcting infrared images of power equipment determines the effective current emissivity based on the coarsness factor of the power equipment and then calculates the actual temperature value by combining this with the observation distance, thereby overcoming the errors caused by inaccurate emissivity settings and transmission attenuation in traditional infrared thermometry. Furthermore, this defect identification method and device combine the correction of infrared images with visible light image recognition to identify equipment defects, making it suitable for intelligent inspection of power equipment in complex environments.

[0005] The technical solution of this invention is implemented as follows:

[0006] A method for correcting infrared images of power equipment includes the following steps:

[0007] Step 1: Identify the outline area of ​​the power equipment from the visible light image, segment the outline area into at least one material area, extract the texture feature vector of the material area and generate the roughness factor of the corresponding material area;

[0008] Step 2: Determine the reference emissivity based on the material type of each material region, and correct the reference emissivity based on the roughness factor to obtain the current emissivity of the corresponding material region;

[0009] Step 3: Calculate the spatial pose of the image acquisition device relative to the power equipment based on the internal parameters of the image acquisition device and the contour area of ​​the power equipment, and generate the observation distance between the image acquisition device and the power equipment;

[0010] Step 4: Extract the medium parameters within the observation path, and calculate the medium transmittance of the observation path based on the observation distance and medium parameters;

[0011] Step 5: Based on the coordinate mapping relationship between the visible light image and the infrared image, map the current emissivity of each material region to each coordinate point in the infrared image;

[0012] Step 6: Correct the original radiance value of the corresponding coordinate point in the infrared image according to the medium transmittance and the current emissivity of each coordinate point, generate the blackbody radiance value, calculate the temperature value of each coordinate point based on the blackbody radiance value, and the temperature values ​​of multiple coordinate points form the corrected infrared image.

[0013] In this invention, in step 1, the gray-level co-occurrence matrix of the material region is calculated, and at least two of the contrast, correlation, energy and homogeneity are extracted from the gray-level co-occurrence matrix as components of the texture feature vector. The texture feature vector is then input into the roughness mapping model to obtain the roughness factor.

[0014] In this invention, in step 2, the roughness factor and reference emissivity of the material region are input into the emissivity correction model to obtain the current emissivity of the material region. The emissivity correction model is constructed as follows: based on surface statistical parameters, multiple sets of random two-dimensional rough surfaces of different material types are constructed, and a composite grayscale image of the random two-dimensional rough surfaces is generated. The roughness factor of the composite grayscale image is calculated based on the roughness mapping model. The emissivity correction model is constructed by taking the roughness factor of the composite grayscale image and the reference emissivity of the random two-dimensional rough surfaces as inputs and the equivalent emissivity of the random two-dimensional rough surfaces as outputs.

[0015] In this invention, in step 3, the contour region in the visible light image is registered with the size data of the power equipment, and the spatial pose of the image acquisition device relative to the power equipment is calculated. The spatial pose includes a rotation matrix and a translation vector, and the magnitude of the translation vector is the observation distance between the image acquisition device and the power equipment.

[0016] In this invention, in step 4, the medium parameter is the atmospheric attenuation coefficient α, and the medium transmittance δ=exp(-αd1), where d1 is the observation distance and exp() is the natural exponential function.

[0017] In this invention, in step 6, the original radiance value F of coordinate point p in the infrared image is calculated. p The corresponding blackbody radiance value F' p Then, based on the radiation temperature model, the temperature value T1 corresponding to the blackbody radiance value is calculated, and all temperature values ​​of the infrared image are mapped to the corresponding coordinate points to generate a corrected infrared image.

[0018] A method for identifying defects in power equipment includes the following steps:

[0019] Step 100: The image acquisition device simultaneously acquires visible light and infrared images of the power equipment at the same sampling coordinates, and generates a corrected infrared image of the power equipment according to the correction method for the infrared image of the power equipment;

[0020] Step 200: Input the visible light image and the corrected infrared image into the generator to generate a fused image. Input the visible light image and the fused image into the first discriminator. Input the corrected infrared image and the fused image into the second discriminator. If neither the first discriminator nor the second discriminator can distinguish the two corresponding images, output the fused image and proceed to step 300. Otherwise, update the parameters of the generator and regenerate the fused image.

[0021] Step 300: Input the fused image into the defect diagnosis model to identify defects in the power equipment.

[0022] In this invention, in step 200, the generator determines the iteration direction of the fused image through a content loss function, which includes the infrared light principal intensity loss generated by the fused image and the corrected infrared image, the infrared light auxiliary gradient loss generated by the gradient of the fused image and the gradient of the corrected infrared image, the visible light principal intensity loss generated by the fused image and the visible light image, and the visible light auxiliary gradient loss generated by the gradient of the fused image and the visible light image.

[0023] A defect identification device for implementing the defect identification method of the power equipment, comprising:

[0024] The image acquisition device is configured to acquire visible light and infrared images of power equipment;

[0025] The data acquisition device is configured to acquire the internal parameters of the image acquisition device and the medium parameters within the observation path;

[0026] The material model library is configured to store dimensional data of electrical equipment and reference emissivity for different material types;

[0027] An image correction device is configured to generate a corrected infrared image of the power equipment;

[0028] The defect identification device is configured to combine visible light images and corrected infrared images to identify defects in electrical equipment.

[0029] The present invention provides a method for correcting infrared images of power equipment, a method for identifying defects, and an apparatus for such correction. These methods offer the following advantages: The invention simultaneously acquires visible light and infrared images of the power equipment. The outline region of the power equipment is located in the visible light image and segmented into material regions. A roughness factor is calculated using the texture feature vector of each material region, and the current emissivity of the corresponding material region is determined based on this roughness factor. The observation distance between the image acquisition device and the power equipment is then calculated using the power equipment's size data and the internal parameters of the image acquisition device. The medium transmittance is calculated based on this distance. Finally, the original radiance value of the infrared image is corrected based on the current emissivity and the medium transmittance, generating a blackbody radiance value and inverting it into the actual temperature value at each coordinate point. The temperature values ​​at multiple coordinate points constitute the corrected infrared image. This invention overcomes the errors caused by inaccurate emissivity settings and transmission attenuation in traditional infrared thermometry.

[0030] Furthermore, the defect identification method of the present invention combines visible light images and corrected infrared images to generate a fused image for identifying equipment defects, thereby determining the fault type. The fused image of the present invention employs a dual discriminator adversarial mechanism, effectively fusing the texture gradient of visible light and the brightness information of infrared light, improving the defect identification accuracy and effectively overcoming the problems of variable outdoor environments and uncertainties in equipment surface conditions. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the reflection of radiated waves from a rough surface.

[0032] Figure 2 This is a flowchart of the infrared image correction method for power equipment according to the present invention;

[0033] Figure 3 This is a schematic diagram showing the standard dimensions of some electrical equipment;

[0034] Figure 4 This is a schematic diagram of the material region in the visible light image of the present invention;

[0035] Figure 5 This is a schematic diagram illustrating the reference emissivity of different material types in this invention;

[0036] Figure 6 This is a schematic diagram illustrating the acquisition of visible light images according to the present invention;

[0037] Figure 7This is a schematic diagram of constructing triangular units on the surface of power equipment according to the present invention;

[0038] Figure 8 This is a schematic diagram illustrating the relationship between the observed temperature and the actual temperature in this invention.

[0039] Figure 9 This is a schematic diagram illustrating the relationship between the image coordinate system, the image acquisition device coordinate system, and the world coordinate system of this invention.

[0040] Figure 10 This is a schematic diagram illustrating the distance error between the observed distance and the actual distance in this invention;

[0041] Figure 11 This is a flowchart of the defect identification method for power equipment according to the present invention;

[0042] Figure 12 This is the topology map for generating the fused image according to the present invention;

[0043] Figure 13 This is an evaluation curve of the defect identification effect of the present invention;

[0044] Figure 14 This is a schematic diagram of the generator of the present invention;

[0045] Figure 15 The image fusion necessity evaluation curve of the present invention;

[0046] Figure 16 This is an evaluation curve for the image fusion effect of the present invention;

[0047] Figure 17 This is a topology diagram of the attention machine module in the defect diagnosis model of the present invention;

[0048] Figure 18 This is a schematic diagram of wavelet downsampling in this invention;

[0049] Figure 19 This is a block diagram of a defect identification device for implementing the defect identification method for power equipment according to the present invention. Detailed Implementation

[0050] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0051] In infrared thermometry of power equipment, the accuracy of the imaged temperature is affected by a variety of factors. (Refer to...) Figure 1When a beam of radiation shines on the surface of an electrical device, because this surface is not smooth but filled with numerous recessed areas, the radiation wave will be reflected multiple times between the inner walls of these recessed areas. With each reflection, some energy is absorbed by the inner wall material, ultimately leading to a significant reduction in the energy of the reflected radiation detected externally. This effect manifests macroscopically as an increase in the effective emissivity of the electrical device surface. Traditional methods often use fixed or empirical emissivity, which is insufficient to reflect the true condition of the electrical device surface, resulting in temperature measurement errors. Secondly, the observation distance between the infrared thermal imager and the measured electrical device directly affects the degree of attenuation of radiant energy. Especially at long distances, the absorption and scattering of infrared waves by water vapor and dust in the atmosphere are intensified, causing the temperature reading to be lower than expected.

[0052] Furthermore, due to the high workload of inspection personnel, they often can only set an average parameter, making it impossible to make fine adjustments for different equipment and distances, further reducing diagnostic accuracy. This invention extracts texture feature vectors from visible light images to construct roughness factors for different material areas on the surface of power equipment, dynamically determines the current emissivity, and calculates the observation distance in conjunction with visible light images to compensate for the transmission of infrared radiation, ultimately generating a high-precision temperature distribution field (corrected infrared image). This method effectively solves the temperature deviation problem caused by emissivity errors and distance uncertainties in traditional infrared thermometry, improving the reliability of fault diagnosis. Example 1

[0053] Reference Figures 2 to 18 The method for correcting infrared images of power equipment according to this embodiment of the present invention includes the following steps:

[0054] Step 1: Identify the contour region of the power equipment from the visible light image, segment the contour region into at least one material region, extract the texture feature vector of the material region, and generate the corresponding roughness factor for the material region. (Refer to...) Figure 3 Based on the equipment type and model, standard shape parameters of the power equipment are obtained. Template matching or geometric fitting is then performed on the visible light image to segment the area where the power equipment is located as the contour region. (Refer to...) Figure 4 and Figure 5 Electrical equipment typically consists of multiple functional components, each made of a different material. The contour area is divided into at least one material region according to the functional components, ensuring that each material region primarily corresponds to a single material type. The gray-level co-occurrence matrix of the material region is calculated, and at least two of contrast, correlation, energy, and homogeneity are extracted from the gray-level co-occurrence matrix as components of the texture feature vector. This texture feature vector is then input into a roughness mapping model to obtain the roughness factor. A preferred method for constructing the roughness mapping model is described in Example 2.

[0055] Step 2: Determine the reference emissivity based on the material type of each material region, and correct the reference emissivity according to the roughness factor to obtain the current emissivity of the corresponding material region. (Refer to...) Figure 5 Each material region corresponds to a functional component, and each functional component corresponds to a single material type. The reference emissivity of each material region is obtained by looking up a table. The roughness factor of the material region and the reference emissivity are input into the emissivity correction model to obtain the current emissivity of the material region. The emissivity correction model is constructed as follows: based on surface statistical parameters, multiple sets of random two-dimensional rough surfaces of different material types are constructed, and a composite grayscale image of the random two-dimensional rough surfaces is generated. The roughness factor of the composite grayscale image is calculated based on the roughness mapping model. The emissivity correction model is constructed by taking the roughness factor of the composite grayscale image and the reference emissivity of the random two-dimensional rough surfaces as inputs and the equivalent emissivity of the random two-dimensional rough surfaces as outputs. The preferred method for constructing the emissivity correction model is described in Example 2.

[0056] Step 3: Calculate the spatial pose of the image acquisition device relative to the power equipment based on the internal parameters of the image acquisition device and the contour area of ​​the power equipment, and generate the observation distance between the image acquisition device and the power equipment. In this embodiment, the geometric center of the power equipment is set as the origin of the world coordinate system, where the geometric center refers to the center of the contour area. In another embodiment, a key part of the power equipment (such as the center of the conductive contact) can also be set as the origin of the world coordinate system. (Refer to...) Figure 3 and Figure 6 After calibrating the image acquisition device, the internal parameters of the image acquisition device are obtained. The internal parameters are then used to perform geometric registration between the contour area in the visible light image and the size data of the power equipment. The spatial pose of the image acquisition device relative to the power equipment is calculated. The spatial pose includes a rotation matrix and a translation vector. The magnitude of the translation vector is the observation distance between the image acquisition device and the power equipment.

[0057] Step 4: Extract the medium parameters within the observation path, and calculate the medium transmittance of the observation path based on the observation distance and the medium parameters. The medium parameter is the atmospheric attenuation coefficient α. The atmospheric attenuation coefficient α = aS + b / V + c is determined based on the relative humidity S and visibility V in the current atmospheric environmental parameters. After converting the observation distance to kilometers, the medium transmittance δ = exp(-αd1) is calculated, where d1 is the observation distance, exp() is the natural exponential function, and a, b, and c are constants determined based on experimental or theoretical analysis.

[0058] Step 5: Based on the coordinate mapping relationship between the visible light image and the infrared image, map the current emissivity of each material region to each coordinate point in the infrared image. In this embodiment, the image acquisition device is a common-path optical system, and the visible light image and the infrared image have exactly the same imaging field of view, viewing angle, and spatial resolution. The coordinate points of pixels in the visible light image and the infrared image correspond one-to-one. For each material region segmented in the visible light image, the current emissivity ε of any coordinate point p is... p It can be directly assigned to the same coordinate point p in the infrared image.

[0059] Step 6: Correct the original radiance value of the corresponding coordinate point in the infrared image based on the medium transmittance and the current emissivity of each coordinate point to generate a blackbody radiance value. Calculate the temperature value of each coordinate point based on the blackbody radiance value. The temperature values ​​of multiple coordinate points constitute the corrected infrared image. The infrared image refers to the original thermal imaging data acquired by the image acquisition device. The pixel value of each coordinate point in the infrared image represents the original radiance value received at that coordinate point. The corrected infrared image is the true temperature distribution image obtained by physically inverting the blackbody radiance value after jointly correcting the original radiance value with the current emissivity and medium transmittance.

[0060] The blackbody radiance value of coordinate point p in an infrared image , of which F p F1 is the original radiance value of coordinate point p in the infrared image, F2 is the atmospheric radiance value, and ε is the ambient radiance value. p Let F' be the current emissivity at coordinate point p. Let F' be the blackbody radiance value. p Substitution Solve for the corresponding temperature value T1, where [λ1, λ2] are the working bands of the image acquisition device, C1 and C2 are the first and second radiation constants respectively, e is the base of the natural logarithm, Q(λ) is the spectral response function of the image acquisition device, and dλ is the derivative of the integral variable λ. Example 2

[0061] This embodiment further discloses a preferred method for constructing a roughness mapping model and an emissivity correction model.

[0062] A random two-dimensional rough surface is constructed. For each material type, parameter sampling is performed within the range of typical surface statistical parameters (including root mean square roughness, Hearst exponent, and skewness). Based on a fractal Brownian motion model, multiple sets of parameterized random two-dimensional rough surfaces are generated to cover the possible range of surface roughness variations for that material type. Specifically, parameter sampling is first performed within the range of typical surface statistical parameters to obtain the target root mean square roughness, target Hearst exponent, and target skewness. A square region with a side length of d2 is defined, and the number of grid points in this square region is determined according to the required geometric resolution. In the frequency domain space corresponding to the above grid points, a Gaussian random number following a standard normal distribution is generated for each frequency position, forming a Gaussian random spectrum. The frequency components in the Gaussian random spectrum are weighted according to the power spectral density corresponding to the target Hearst exponent, and an inverse fast Fourier transform is performed on the weighted Gaussian random number to obtain the spatial domain height field. The spatial domain height field is linearly scaled so that the root mean square roughness of the spatial domain height field is equal to the target root mean square roughness. The skewness of the spatial domain height field is adjusted to the target skewness through nonlinear height mapping, thereby forming a random two-dimensional rough surface with specified statistical properties.

[0063] Calculate the roughness factor. Divide the random two-dimensional rough surface into multiple tiny triangular units; that is, split each square unit enclosed by four adjacent grid points into two triangular units along the diagonal. (Refer to...) Figure 7 For any triangular element, let the spatial coordinates of its three grid points be P1(x1,y1,z1), P2(x2,y2,z2), and P3(x3,y3,z3), where z1, z2, and z3 are the height values ​​of the three grid points, and (x1,y1), (x2,y2), and (x3,y3) are the planar coordinates of the three grid points. Calculate the actual area of ​​this triangular element using the vector cross product method. The actual surface area A of the entire random two-dimensional rough surface is obtained by summing the actual surfaces of all triangular units. 10 Then the roughness factor R1=A of the random two-dimensional rough surface. 10 / d2 2 Among them, d2 2 Let be the projected area of ​​a random two-dimensional rough surface.

[0064] A roughness mapping model is constructed. Based on the random two-dimensional rough surface, a synthetic grayscale image corresponding to the random two-dimensional rough surface is rendered under standard lighting conditions. The gray-level co-occurrence matrix in the synthetic grayscale image is extracted, i.e., feature values ​​such as contrast, energy, homogeneity, and correlation are calculated in four directions (0°, 45°, 90°, 135°). At least two feature values ​​are selected to form the texture feature vector of the random two-dimensional rough surface. This texture feature vector is used as input, and the corresponding roughness factor is used as output to construct the roughness mapping model. In this embodiment, contrast E1, energy E2, and homogeneity E3 are extracted as components of the texture feature vector. The roughness mapping model is a multiple linear regression equation obtained by least squares fitting: R1=w1E1+w2E2+w3E3+g, where w1, w2, and w3 are the regression weight coefficients of the corresponding components, g is the bias term, and the values ​​of w1, w2, w3, and g are determined by minimizing the mean square error between the roughness factor predicted by the roughness mapping model and the actual calculated roughness factor on the training set.

[0065] An emissivity correction model is constructed. For each group of generated random two-dimensional rough surfaces, a corresponding reference emissivity is set according to the material type of the random two-dimensional rough surface. Based on the geometry of the random two-dimensional rough surface, the equivalent emissivity of the random two-dimensional rough surface in the working wavelength band (e.g., 8–14 μm) of the image acquisition device is calculated using electromagnetic scattering theory (such as Kirchhoff's approximation). Using the reference emissivity and roughness factor R1 of the random two-dimensional rough surface as inputs, and the equivalent emissivity of the random two-dimensional rough surface as output, an emissivity correction model is constructed. In this embodiment, the emissivity correction model is a multivariate nonlinear regression function or a neural network. The model parameters of the emissivity correction model are trained by minimizing the error between the current emissivity predicted by the emissivity correction model and the equivalent emissivity of the random two-dimensional rough surface. Example 3

[0066] like Figure 8 As the observation distance increases, the observed temperature measured by the infrared thermal imager shows a more significant downward trend compared to the actual surface temperature of the power equipment. This embodiment further illustrates the calculation method of the observation distance in step 3.

[0067] Define a coordinate system. For example... Figure 9 Image coordinate system (O-XY): A two-dimensional coordinate system with its origin O at the intersection of the optical axis of the image acquisition device and the imaging plane of the visible light image. Image acquisition device coordinate system (O c -X c Y c Z c ): The origin O is the optical center of the image acquisition device. c Z cA three-dimensional coordinate system in which the optical axis coincides with the optical axis of the image acquisition device. World coordinate system (O) w -X w Y w Z w ): The origin O is the geometric center of the electrical equipment. w The three-dimensional coordinate system. The three-dimensional coordinates of any feature point on the power equipment in this world coordinate system are P(x) w y w , z w ).

[0068] Construct the projection equation. Calibrate the image acquisition device using the camera calibration method and obtain its internal parameters, which are defined as an intrinsic parameter matrix. , where f x f y The focal length is expressed in pixels, and (u0, v0) are the coordinates of the principal point (the pixel coordinates of the origin O of the image coordinate system). Construct the projection equation: Through this projection equation, the three-dimensional coordinates (x, y, z) in the world coordinate system can be represented. w y w , z w The value is converted to two-dimensional pixel coordinates (x, y) in the image coordinate system, where s is an arbitrary scale factor, R is the rotation matrix, and t is the translation vector.

[0069] Calculate the observation distance. At least four non-collinear physical feature points (e.g., the two ends of an insulator, the corners of a clamp) are predefined on the power equipment. Based on the size data of the power equipment, the three-dimensional coordinates of these physical feature points in the world coordinate system are determined. Image recognition algorithms (e.g., corner detection, edge extraction) are used to locate the corresponding two-dimensional pixel coordinates of these physical feature points in the image coordinate system from the visible light image. The correspondence between the three-dimensional coordinates and two-dimensional pixel coordinates of multiple physical feature points, along with their internal parameters, are substituted into the aforementioned projection equation to obtain the translation vector t of the image acquisition device coordinate system relative to the world coordinate system. This translation vector t is the origin O of the world coordinate system. w (The geometric center of the power equipment) points to the optical center O of the image acquisition device. c The spatial vector. The final observation distance d1 between the image acquisition device and the power equipment is the magnitude of the translation vector t.

[0070] A laser rangefinder is used to obtain the actual distance between the power equipment and the image acquisition device. The method of this embodiment is used to obtain the observed distance between the power equipment and the image acquisition device, and the distance error between the actual distance and the observed distance is calculated. Figure 10The average distance error of the nine types of power equipment was 0.150 meters, and the maximum distance error was 0.466 meters. The temperature error of the image acquisition device within a distance error of 0.5 meters was less than 0.1℃, indicating that the method for calculating the observation distance in this embodiment can meet the actual needs. Example 4

[0071] Reference Figure 11 and Figure 12 A method for identifying defects in power equipment includes the following steps:

[0072] Step 100: The image acquisition device simultaneously acquires visible light and infrared images of the power equipment at the same sampling coordinates, and generates a corrected infrared image of the power equipment according to the correction method for the infrared image of the power equipment. The image acquisition device integrates a visible light sensor and an infrared sensor, and adopts a common-path optical system (e.g., through a beam splitter or wavelength-selective window) so that the visible light sensor and the infrared sensor share the same imaging field of view and optical axis, thereby ensuring that the visible light image and the infrared image are spatially aligned at the pixel level. Furthermore, the image acquisition device can also be equipped with a high-precision synchronization module to ensure that the visible light image and the infrared image of the power equipment are acquired at the same timestamp, so as to eliminate measurement errors caused by time differences.

[0073] Step 200: After inputting the visible light image and the corrected infrared image into the generator, a fused image is generated. The visible light image and the fused image are then input into the first discriminator, and the corrected infrared image and the fused image are input into the second discriminator. If neither the first nor the second discriminator can distinguish the corresponding two images, the fused image is output, and the process proceeds to step 300. Otherwise, the generator parameters are updated, and the fused image is regenerated. The discrimination results of the first and second discriminators are output through the output terminal. The generator determines the iteration direction of the fused image through a content loss function, which includes the infrared principal intensity loss generated from the fused image and the corrected infrared image, the infrared auxiliary gradient loss generated from the gradient of the fused image and the gradient of the corrected infrared image, the visible light principal intensity loss generated from the fused image and the visible light image, and the visible light auxiliary gradient loss generated from the gradient of the fused image and the gradient of the visible light image.

[0074] Step 300: Input the fused image into the defect diagnosis model to identify defects in the power equipment. The RT-DETRv3 model has end-to-end detection capabilities and can directly output defect results from the fused image, making it suitable for the real-time identification requirements of this invention. However, when processing minute defects in power equipment, its unoptimized attention mechanism is prone to overfitting and insufficient extraction of detailed features, affecting recognition accuracy. Therefore, this embodiment proposes an improved RT-DETRv3 defect diagnosis model, using the fused image as input, replacing the original attention module and hybrid encoder in RT-DETRv3 with a lightweight structure based on EfficientViT. The characteristics of EfficientViT are particularly suitable for pixel-level defect identification tasks in power equipment, accurately locating fine-grained abnormal areas such as minute cracks and localized overheating while maintaining high detection efficiency. Figure 13 In terms of average pixel accuracy (mPA), the defect diagnosis model built on the EfficientViT backbone network significantly outperforms defect diagnosis models built on CBVM-V7, MD-HOG, or CBM-V10. Example 7 further illustrates the model structure design of the defect diagnosis model. Example 5

[0075] Reference Figure 14 This embodiment further discloses the method for generating a fused image based on the DDcGAN network in step 200. For example... Figure 15 The average pixel accuracy of the fused image is significantly higher than that of the individual calibrated infrared image. The individual calibrated infrared image lacks obvious texture features, and the defect diagnosis model cannot accurately identify the device and its local structure through texture details, thus failing to accurately identify the device and structure and consequently, failing to accurately diagnose the fault. (Refer to...) Figure 16 In other embodiments, a DDcGAN network is used to generate a DDcGAN fused image, a cross-bilateral filtering network is used to generate a cross-bilateral filtering fused image, and a GANMcC network is used to generate a GANMcC fused image. According to evaluation metrics (information entropy, multi-scale structural similarity, or spatial frequency), the DDcGAN fused image significantly outperforms the cross-bilateral filtering fused image and the GANMcC fused image, indicating that the DDcGAN network can more effectively preserve the key information of the source image.

[0076] After the calibrated infrared and visible light images are input, the generator extracts features, which are then fed into the DDcGAN network for learning. Once learning is complete, the output is a fused image that retains important information from both the calibrated infrared and visible light images. The generator consists of three modules: an encoding module, a feature generation module, and a decoding module.

[0077] The encoding module is used for multi-scale feature extraction from visible light and corrected infrared images. The input visible light or corrected infrared image is passed in parallel through three different sizes of convolutional kernels: 3×3, 5×5, and 7×7. These kernels capture local detail, mid-range contextual information, and larger-scale structural information, respectively. Each kernel contains three convolutional modules, each consisting of a convolutional layer (n=16 channels, stride 1), a normalization layer, and a Corrected Linear Unit (ReLU) activation function. Subsequently, the feature maps extracted by the three kernels are concatenated along the channel dimension to achieve multi-scale information complementarity. A 1×1 convolutional layer (n=432 output channels) then performs feature interaction and compression on the concatenated feature maps, reducing redundancy while preserving rich semantics. This process generates visible light image features and corrected infrared image features that fuse information from multiple receptive fields.

[0078] The feature generation module is used to perform deep fusion of the visible light image features extracted by the encoding module and the corrected infrared image features. The feature generation module connects the visible light image features and the corrected infrared image features through convolution and skip connections in the residual block, and filters out redundant information to output a unified and robust fused feature as the input of the decoding module.

[0079] The decoding module receives the fused features output from the feature generation module and generates a fused image that retains key information from both. The decoding module first receives the fused features from the feature generation module and inputs them into the first convolutional module. This first convolutional module consists of a convolutional layer (1×1 kernel size) and a Rectified Linear Unit (ReLU) activation function. Subsequently, the output features pass through the second and third convolutional modules, whose structures are identical to those of the 3×3 kernel convolutional modules in the encoding module, both containing convolutional layers, normalization layers, and ReLU activation functions. The first three convolutional modules of the decoding module are connected to corresponding layers in the encoding module via skip connections. This skip connection design effectively compensates for information that may be lost during feature propagation in deeper layers and fully utilizes the detailed information extracted by the shallow layers of the encoding module, including texture features from the visible light image and temperature contrast features from the corrected infrared image. Finally, the fused features are processed by an output module consisting of a 1×1 convolutional layer and a hyperbolic tangent (Tanh) activation function to generate the fused image.

[0080] Controlling the Iteration Direction of the Fusion Image. While the aforementioned generator structure has advantages in multi-scale feature extraction and cross-modal fusion, without effective supervision signals, it remains difficult to ensure that the fused image simultaneously retains with high fidelity the temperature contrast information (mainly reflected in intensity distribution) of the corrected infrared image and the texture structure information (mainly reflected in gradient changes) of the visible light image. This embodiment introduces a content loss function during training to determine the iteration direction of the fused image. To obtain the temperature information (presented through contrast) of the corrected infrared image and the texture information (based on gradients) of the visible light image, the fused image needs to retain both temperature and texture information to the greatest extent possible. Considering that the corrected infrared image also contains texture information, and the visible light image contains contrast information, the generator design should ensure that the content loss function accurately captures and fuses these two characteristics. This guarantees that the fused image can accurately inherit the key attributes of its respective source images. Therefore, the content loss function L = w4L1 + w5L2 + w6L3 + w7L4, L1 = ||I3 - I1|| 2 L2 = ||ΔI3 - ΔI1|| 2 L3 = ||I3 - I2|| 2 L4 = ||ΔI3 - ΔI2|| 2 Let I1 represent the visible light image, I2 represent the corrected infrared image, I3 represent the fused image, ΔI1 represent the visible light image gradient, ΔI2 represent the corrected infrared image gradient, ΔI3 represent the fused image gradient, L1 represent the visible light principal intensity loss, L2 represent the visible light auxiliary gradient loss, L3 represent the infrared principal intensity loss, and L4 represent the infrared auxiliary gradient loss. 2 w4, w5, w6, and w7 represent the square of the Frobenius norm, and w4, w5, w6, and w7 are the weighting coefficients for balancing L1, L2, L3, and L4. Example 6

[0081] This embodiment further discloses the specific structures of the first discriminator and the second discriminator used in step 200. The first discriminator is used to determine the authenticity of the visible light image and the fused image, and the second discriminator is used to determine the authenticity of the corrected infrared image and the fused image. The first discriminator and the second discriminator have the same network architecture.

[0082] Each discriminator receives a pair of images as input (e.g., a visible light image and a fused image, or a corrected infrared image and a fused image), and concatenates the pair of images along the channel dimension before inputting them into the discriminator network. The discriminator network consists of four sequentially connected processing modules: the first module contains a convolutional layer with a kernel size of 3×3, 32 output channels, and a stride of 2, followed by a Corrected Linear Unit (ReLU) activation function; the second module contains a convolutional layer with a kernel size of 3×3, 64 output channels, and a stride of 2, followed by a normalization layer and a ReLU activation function; the third module has the same structure as the second module, containing a 3×3 convolutional layer (128 output channels, stride of 2), a normalization layer, and a ReLU activation function; the fourth module is a fully connected layer whose output is processed by a hyperbolic tangent (Tanh) activation function, ultimately outputting a scalar value representing the discrimination result for whether the input pair of images have the same characteristic information.

[0083] Furthermore, an auxiliary discrimination term for the authenticity of infrared images is introduced into the loss function of the first discriminator, and an auxiliary discrimination term for the authenticity of visible light images is introduced into the loss function of the second discriminator. This cross-supervision mechanism can effectively suppress overfitting and promote the generator to simultaneously retain the temperature contrast information of the corrected infrared image and the texture details of the visible light image during the fusion process. Example 7

[0084] Reference Figure 17 This embodiment further illustrates the model structure design of the defect diagnosis model built based on the improved EfficientViT module in step 300.

[0085] The attention module is improved. First, the fused image is processed through an overlapping block embedding layer to convert it into a feature map. Then, the feature map enters the first, second and third feature extraction stages in sequence. Three improved EfficientViT modules are stacked in each of the three stages. The feature map size is compressed by downsampling between feature extraction stages, gradually reducing the feature map size and increasing the number of feature channels. Finally, the defect judgment result is output after passing through an average pooling layer and a classifier.

[0086] The EfficientViT module employs a typical sandwich structure. Input features first pass through multiple feedforward neural networks, then through a bidirectional transformer attention mechanism, followed by multiple feedforward neural networks, and finally, residual connections yield the output features. By reducing the use of the attention mechanism, the memory access time caused by attention computation is effectively mitigated. A deep separable convolutional layer is embedded before the feedforward neural networks to enhance local perception capabilities through spatial dimension feature interactions. The core of the bidirectional transformer attention mechanism is the feature vector interaction module. Input features first generate a query vector, a global key vector, and a global value vector. After the attention score is normalized by an activation function, the global value vector is weighted and summed using matrix multiplication. Then, aggregation operations are used to collect corresponding information from the global key vector and global value vector. Finally, concatenation and projection processing yield the final feature map interaction result.

[0087] Haar wavelets are used for downsampling. (Refer to...) Figure 18 First, the input feature map is processed in the row dimension through row decomposition and reconstruction, transforming it into low-frequency core information and high-frequency detail features. Then, column decomposition and reconstruction are performed to complete the first decomposition of the two-dimensional Haar wavelet, outputting four sub-bands: an approximate component A1 containing the overall image contour, a vertical detail component A2, a diagonal detail component A3, and a horizontal detail component A4. A1 represents low-frequency basic information, while A2, A3, and A4 correspond to high-frequency detail signals in different directions. To achieve multi-scale feature extraction, the vertical detail component A2 obtained from the first decomposition can be further decomposed, repeating the row and column wavelet decomposition process to obtain the second-decomposition sub-bands A5, A6, A7, and A8. This process is iteratively repeated for each sub-band to complete the downsampling of the feature map. Example 8

[0088] Reference Figure 19 A defect identification device for implementing the defect identification method of the power equipment includes an image acquisition device, a data acquisition device, a material model library, an image correction device, and a defect identification device. The image acquisition device is configured to acquire visible light and infrared images of the power equipment. The data acquisition device is configured to acquire internal parameters of the image acquisition device and medium parameters within the observation path. The material model library is configured to store dimensional data of the power equipment and reference emissivity for different material types. The image correction device is configured to generate a corrected infrared image of the power equipment. The defect identification device is configured to combine the visible light image and the corrected infrared image to identify defects in the power equipment.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for correcting infrared images of power equipment, characterized in that, Includes the following steps: Step 1: Identify the outline area of ​​the power equipment from the visible light image, segment the outline area into at least one material area, extract the texture feature vector of the material area and generate the roughness factor of the corresponding material area; Step 2: Determine the reference emissivity based on the material type of each material region, and correct the reference emissivity based on the roughness factor to obtain the current emissivity of the corresponding material region; Step 3: Calculate the spatial pose of the image acquisition device relative to the power equipment based on the internal parameters of the image acquisition device and the contour area of ​​the power equipment, and generate the observation distance between the image acquisition device and the power equipment; Step 4: Extract the medium parameters within the observation path, and calculate the medium transmittance of the observation path based on the observation distance and medium parameters; Step 5: Based on the coordinate mapping relationship between the visible light image and the infrared image, map the current emissivity of each material region to each coordinate point in the infrared image; Step 6: Correct the original radiance value of the corresponding coordinate point in the infrared image according to the medium transmittance and the current emissivity of each coordinate point, generate the blackbody radiance value, calculate the temperature value of each coordinate point based on the blackbody radiance value, and the temperature values ​​of multiple coordinate points form the corrected infrared image.

2. The method for correcting infrared images of power equipment according to claim 1, characterized in that, In step 1, the gray-level co-occurrence matrix of the material region is calculated, and at least two of the following factors—contrast, correlation, energy, and homogeneity—are extracted from the gray-level co-occurrence matrix as components of the texture feature vector. The texture feature vector is then input into the roughness mapping model to obtain the roughness factor.

3. The method for correcting infrared images of power equipment according to claim 2, characterized in that, In step 2, the roughness factor and reference emissivity of the material region are input into the emissivity correction model to obtain the current emissivity of the material region. The emissivity correction model is constructed as follows: based on surface statistical parameters, multiple sets of random two-dimensional rough surfaces of different material types are constructed, and a composite grayscale image of the random two-dimensional rough surfaces is generated. The roughness factor of the composite grayscale image is calculated based on the roughness mapping model. The emissivity correction model is constructed by taking the roughness factor of the composite grayscale image and the reference emissivity of the random two-dimensional rough surfaces as inputs and the equivalent emissivity of the random two-dimensional rough surfaces as outputs.

4. The method for correcting infrared images of power equipment according to claim 1, characterized in that, In step 3, the contour region in the visible light image is registered with the size data of the power equipment, and the spatial pose of the image acquisition device relative to the power equipment is calculated. The spatial pose includes a rotation matrix and a translation vector, and the magnitude of the translation vector is the observation distance between the image acquisition device and the power equipment.

5. The method for correcting infrared images of power equipment according to claim 1, characterized in that, In step 4, the medium parameter is the atmospheric attenuation coefficient α, and the medium transmittance δ=exp(-αd1), where d1 is the observation distance and exp() is the natural exponential function.

6. The method for correcting infrared images of power equipment according to claim 1, characterized in that, In step 6, the original radiance value F of coordinate point p in the infrared image is calculated. p The corresponding blackbody radiance value F' p Then, based on the radiation temperature model, the temperature value T1 corresponding to the blackbody radiance value is calculated, and all temperature values ​​of the infrared image are mapped to the corresponding coordinate points to generate a corrected infrared image.

7. A method for defect identification of power equipment, characterized in that, Includes the following steps: Step 100: The image acquisition device simultaneously acquires visible light and infrared images of the power equipment at the same sampling coordinates, and generates a corrected infrared image of the power equipment according to the correction method for the infrared image of the power equipment as described in claim 1; Step 200: Input the visible light image and the corrected infrared image into the generator to generate a fused image. Input the visible light image and the fused image into the first discriminator. Input the corrected infrared image and the fused image into the second discriminator. If neither the first discriminator nor the second discriminator can distinguish the two corresponding images, output the fused image and proceed to step 300. Otherwise, update the parameters of the generator and regenerate the fused image. Step 300: Input the fused image into the defect diagnosis model to identify defects in the power equipment.

8. The defect identification method for power equipment according to claim 7, characterized in that, In step 200, the generator determines the iteration direction of the fused image through a content loss function, which includes the infrared light principal intensity loss generated by the fused image and the corrected infrared image, the infrared light auxiliary gradient loss generated by the gradient of the fused image and the gradient of the corrected infrared image, the visible light principal intensity loss generated by the fused image and the visible light image, and the visible light auxiliary gradient loss generated by the gradient of the fused image and the visible light image.

9. A defect identification device for implementing the defect identification method for power equipment according to claim 7, characterized in that, include: The image acquisition device is configured to acquire visible light and infrared images of power equipment; The data acquisition device is configured to acquire the internal parameters of the image acquisition device and the medium parameters within the observation path; The material model library is configured to store dimensional data of electrical equipment and reference emissivity for different material types; An image correction device is configured to generate a corrected infrared image of the power equipment; The defect identification device is configured to combine visible light images and corrected infrared images to identify defects in electrical equipment.

Images