Electric power hidden danger identification method based on AI vision

Abstract

The invention discloses an electric power hidden danger identification method based on AI vision, and the method comprises the following steps: collecting a visible light image and an infrared image, and carrying out the preprocessing, and obtaining a preprocessed dual-mode image pair; generating a candidate region set of the power equipment component; cFOG features are extracted from the preprocessed bimodal image pair, and a visible light CFOG feature map and an infrared CFOG feature map are output; performing frequency domain accelerated matching based on fast Fourier transform, and outputting a local similarity response diagram and a local alignment parameter; consistency constraint fusion is executed, coarse-to-fine pyramid alignment is executed, sub-pixel refinement processing is executed at the same time, and sub-pixel-level alignment parameters are output; performing alignment transformation on the preprocessed bimodal image pair, determining an infrared thermal anomaly region, and outputting a part to map the thermal anomaly region; and calculating an alignment confidence index, and extracting hidden danger judgment features to obtain a hidden danger recognition result. According to the method, the CFOG features and frequency domain accelerated matching are fused to realize electric power hidden danger identification.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for power systems, and in particular to a method for identifying potential power hazards based on AI vision. Background Technology

[0002] In the operation and maintenance of power systems, condition inspection of power equipment is a crucial link in ensuring the safe and stable operation of the power grid. With the continuous expansion of power equipment scale and the increasing complexity of the operating environment, traditional methods relying on manual inspection or single sensing methods are no longer sufficient to meet the demands for high-frequency, high-precision, and high-reliability inspections. In recent years, intelligent inspection technology based on visual perception has been gradually applied to the operation monitoring of transmission lines, substations, and distribution equipment. Visible light images can intuitively reflect the external structural condition of power equipment, while infrared images can reflect the thermal distribution characteristics of the equipment during operation. The two technologies are clearly complementary in identifying potential power hazards.

[0003] In existing technologies, some power inspection systems have begun to simultaneously acquire visible light and infrared images, and assist maintenance personnel in identifying potential hazards through simple overlay display or manual comparison. However, such methods typically rely on human experience to correlate thermal anomaly areas with visible light equipment structures, resulting in low automation and difficulty in adapting to large-scale, continuous inspection scenarios.

[0004] Furthermore, although some existing methods have introduced feature matching or frequency domain registration techniques, they mostly treat the entire image as the processing object and fail to perform regional-level modeling and alignment for specific components of power equipment. This leads to mismatches in complex backgrounds and requires a large amount of computation, which is not conducive to real-time deployment on edge devices such as drones or mobile inspection terminals.

[0005] Therefore, how to provide a method for identifying potential electrical hazards based on AI vision is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an AI-based vision-based method for identifying potential power hazards. This invention achieves efficient and stable alignment of visible light and infrared images by fusing pixel-wise directional gradient channel features with a frequency domain accelerated matching method based on fast Fourier transform within candidate regions of power equipment components. Furthermore, by combining consistency constraints and confidence assessment mechanisms, it improves the correspondence accuracy between infrared thermal anomalies and power equipment components, thereby enhancing the accuracy, reliability, and engineering applicability of the power hazard identification results.

[0007] A power hazard identification method based on AI vision according to an embodiment of the present invention includes the following steps:

[0008] Visible light images and infrared images are collected during the same power inspection period to form a dual-modal image pair. The dual-modal image pair is then preprocessed to obtain a preprocessed dual-modal image pair.

[0009] Generate a set of candidate regions for power equipment components based on preprocessed bimodal images;

[0010] For each candidate region in the candidate region set, CFOG features are extracted from the preprocessed bimodal image pair, and visible light CFOG feature map and infrared CFOG feature map are output.

[0011] Perform frequency domain accelerated matching based on fast Fourier transform on visible light CFOG feature maps and infrared CFOG feature maps, and output local similarity response maps and local alignment parameters;

[0012] Perform consistency constraint fusion on the local alignment parameters to output the global alignment parameters, and perform coarse-to-fine pyramid alignment based on the global alignment parameters to obtain fine alignment parameters. At the same time, perform sub-pixel refinement processing to output sub-pixel level alignment parameters.

[0013] Based on subpixel-level alignment parameters, an alignment transformation is performed on the preprocessed bimodal image pair to output an aligned bimodal image pair. The infrared thermal anomaly region is determined and mapped to the power equipment component region in the aligned visible light image. The component-mapped thermal anomaly region is then output.

[0014] Alignment confidence index is calculated based on local similarity response map, and hidden danger judgment features are extracted based on component mapping thermal anomaly area to obtain hidden danger identification results.

[0015] Optionally, obtaining the preprocessed bimodal image pair specifically includes:

[0016] Visible light images and infrared images are collected during the same power inspection period. The collected visible light images are recorded as the original visible light images, and the collected infrared images are recorded as the original infrared images, forming a dual-modal image pair.

[0017] Preprocessing of bimodal image pairs includes denoising, geometric scale unification, and grayscale normalization.

[0018] The denoising process includes filtering and denoising, and suppressing abnormal noise points. It outputs a visible light denoised image and an infrared denoised image. The visible light denoised image and the infrared denoised image are then subjected to geometric scale unification and grayscale normalization processing in sequence to obtain a preprocessed dual-modal image pair.

[0019] Optionally, the generation of the candidate region set for the power equipment components specifically includes:

[0020] Read the preprocessed dual-modal image pair, and use the visible light image and infrared image as the visible light input image and infrared input image respectively, while establishing a dual-modal joint input;

[0021] The dual-modal joint input is used to perform power equipment component localization processing. Based on the structural feature information of the visible light image and the brightness distribution information of the infrared image in the dual-modal joint input, feature extraction and determination are performed on the regions in the image that belong to power equipment components. Based on the determination results, the spatial location range of different power equipment components is determined, and a corresponding candidate box is generated for each power equipment component. At the same time, a corresponding component category label and confidence level are assigned to each candidate box, thereby obtaining a set of power equipment component detection results, which is used as the initial candidate box set.

[0022] Candidate region generation is performed on the initial candidate box set. Low-confidence candidate boxes are eliminated according to the confidence threshold. Overlapping box redundancy removal is performed on the remaining candidate boxes. At the same time, each candidate box after redundancy removal is extended by a preset extension pixel and cropped to the image boundary range. Each candidate box is converted into a candidate region, and all candidate regions are collected to obtain the candidate region set of power equipment components.

[0023] Optionally, the output of the visible light CFOG feature map and the infrared CFOG feature map specifically includes:

[0024] Each candidate region in the candidate region set is cropped from the visible light input image and the infrared input image respectively to obtain the visible light candidate region image and the infrared candidate region image, and the paired candidate region images are output.

[0025] For each visible light candidate region image, gradient calculation is performed pixel by pixel within the candidate region. For each pixel, the gray level difference between adjacent pixels in the horizontal direction and the gray level difference between adjacent pixels in the vertical direction are calculated, and the gradient magnitude and gradient direction of the current pixel are obtained. At the same time, the gradient direction of each pixel in the candidate region is quantized, the gradient direction is divided into a preset number of directional channels, and the gradient magnitude of the current pixel is accumulated into the channel value of the corresponding directional channel at the current pixel position. Thus, a pixel-by-pixel directional gradient channel feature representation is formed within the candidate region, and the visible light CFOG feature map corresponding to the visible light candidate region is output.

[0026] For each infrared candidate region image paired with the visible light candidate region image, the infrared CFOG feature map corresponding to the infrared candidate region is output using the same pixel-by-pixel gradient calculation method and directional quantization method.

[0027] Optionally, the output of the local similarity response map and local alignment parameters specifically includes:

[0028] Read the visible light CFOG feature map and the infrared CFOG feature map, and establish a local matching task using candidate regions as processing units;

[0029] For each local matching task, frequency domain matching preprocessing is performed on the visible light CFOG feature map and the infrared CFOG feature map, size unification processing is performed on the two-dimensional channel matrix of each directional channel, boundary filling processing is performed on the two-dimensional channel matrix of each directional channel to form a uniform transformation size, and windowing processing is performed on the filled two-dimensional channel matrix. The visible light directional channel matrix and the infrared directional channel matrix after frequency domain matching preprocessing are used as the frequency domain transformation input matrix.

[0030] The frequency domain transformation input matrix is ​​processed by performing a two-dimensional fast Fourier transform along the directional channel. A one-dimensional fast Fourier transform is performed row by row along the row direction of the two-dimensional channel matrix of each directional channel to obtain the row frequency domain result. At the same time, a one-dimensional fast Fourier transform is performed column by column along the column direction of the row frequency domain result to obtain the two-dimensional frequency domain result. The visible light CFOG feature map and the infrared CFOG feature map are obtained in the visible light frequency domain matrix and the infrared frequency domain matrix of the current directional channel, respectively.

[0031] Frequency domain correlation is calculated for each directional channel and channel fusion is performed. For each directional channel, the infrared frequency domain matrix is ​​conjugated element by element to obtain the conjugate infrared frequency domain matrix. The visible light frequency domain matrix and the conjugate infrared frequency domain matrix are multiplied element by element at the same frequency position to obtain the frequency domain correlation matrix of the current directional channel. The frequency domain correlation matrices of all directional channels are accumulated element by element at the corresponding frequency position to obtain the fused frequency domain correlation matrix. Channel weights are set for each directional channel and weighted accumulation is performed to obtain the weighted fused frequency domain correlation matrix.

[0032] A two-dimensional inverse fast Fourier transform is performed on the weighted fused frequency domain correlation matrix. The fused frequency domain correlation matrix is ​​used as a complex spectrum input while keeping the real and imaginary parts unchanged. A one-dimensional inverse fast Fourier transform is performed column by column along the column direction to restore the spectrum from the frequency domain to the intermediate domain. At the same time, a one-dimensional inverse fast Fourier transform is performed row by row along the row direction to restore the intermediate domain to the spatial domain, resulting in a spatial domain correlation response matrix. The magnitude of the correlation response matrix is ​​calculated to obtain a non-negative response value matrix, and normalization is performed according to the transform size. The normalized non-negative response value matrix is ​​determined as the local similarity response map of the current candidate region.

[0033] Peak localization processing is performed on the local similarity response map. All response values ​​are traversed, and the matrix position corresponding to the maximum response value is determined as the best matching displacement position. This position is then converted into the horizontal and vertical displacement of the current candidate region and used as the local alignment parameters of the current candidate region.

[0034] Optionally, the output of the sub-pixel alignment parameters specifically includes:

[0035] The local alignment parameters of all candidate regions are aggregated to form a local alignment parameter set. At the same time, the local similarity response map corresponding to each candidate region is read, and the maximum response value of each local similarity response map is used as the confidence weight input for the local alignment parameter of the current candidate region.

[0036] A consistency constraint fusion process is performed on the local alignment parameter set to obtain consistent local alignment parameters. All consistent local alignment parameters are aggregated to form a consistent local alignment parameter subset. Weighted fusion is performed on the consistent local alignment parameter subset based on confidence weights to output global alignment parameters.

[0037] Based on the global alignment parameters, the preprocessed bimodal image pairs are downsampled according to a preset downsampling ratio to obtain coarse preprocessed bimodal image pairs. On the coarse preprocessed bimodal image pairs, the global alignment parameters are used as the initial displacement to perform coarse alignment updates, and the coarse fine alignment parameters are output. The coarse fine alignment parameters are mapped to the fine layer according to the scale mapping relationship corresponding to the downsampling ratio to obtain the fine layer initial alignment parameters. At the same time, on the fine preprocessed bimodal image pairs at the original resolution, the fine layer initial alignment parameters are used as the center displacement and the fine layer alignment updates are performed within a preset local search range to output fine alignment parameters.

[0038] Subpixel refinement is performed on the fine alignment parameters. A preset response neighborhood is selected with the position of the maximum response value as the center in the local similarity response map. The subpixel-level peak position offset of the maximum response value is determined based on the response value distribution in the response neighborhood, and then superimposed on the fine alignment parameters to output the subpixel-level alignment parameters.

[0039] Optionally, the output of the component mapping thermal anomaly region specifically includes:

[0040] Using subpixel-level alignment parameters as input parameters for alignment transformation, subpixel-level geometric alignment transformation is performed on the infrared image in the preprocessed bimodal image, including subpixel-precision translation and resampling processing, outputting an aligned infrared image, which is then combined with the preprocessed visible light image to form an aligned bimodal image pair;

[0041] Infrared thermal anomaly region determination is performed on the aligned infrared image. Based on the distribution of infrared intensity values ​​of each pixel in the aligned infrared image, a set of pixels that meet the preset thermal anomaly judgment threshold is determined. Spatial connectivity analysis is performed on the pixel set, and high-intensity pixels that are spatially connected are merged into continuous regions. Region filtering processing is performed on the continuous regions, and a set of infrared thermal anomaly regions is output.

[0042] Read the candidate region set, and map each infrared thermal anomaly region in the infrared thermal anomaly region set to its corresponding spatial location in the visible light image coordinate system of the aligned dual-modal image pairs. Perform spatial association processing on the mapped infrared thermal anomaly regions and candidate regions, and determine the infrared thermal anomaly regions that overlap or contain the candidate regions of power equipment components as component-mapped thermal anomaly regions.

[0043] Optionally, obtaining the hazard identification result specifically includes:

[0044] Peak analysis is performed on the local similarity response maps corresponding to each candidate region to determine the maximum and second largest response values. Based on the degree of difference between the two, the alignment confidence index is calculated, and the alignment confidence indexes of each candidate region are aggregated to form an alignment confidence index set.

[0045] In the aligned infrared image, the maximum infrared intensity and area of ​​the mapped thermal anomaly region for each component are statistically analyzed. When there are consecutive different mapped thermal anomaly regions for different components, the temporal variation of the thermal anomaly intensity is statistically analyzed to form a set of features for hazard judgment.

[0046] The risk score corresponding to the thermal anomaly area mapped by each component is obtained by fusion calculation based on the aligned confidence index set and the hazard judgment feature set. The hazard risk level is determined according to the risk score and the preset threshold. The corresponding hazard location, hazard type and hazard risk level are output in the aligned visible light image, thereby obtaining the hazard identification result.

[0047] The beneficial effects of this invention are:

[0048] This invention avoids the background interference problem caused by registration of the entire image as the object in the prior art by jointly processing visible light images and infrared images in the candidate area of ​​power equipment components. This makes the cross-modal alignment process more focused on the main structure of the power equipment, thereby significantly improving the spatial correspondence accuracy between the infrared thermal anomaly area and the power equipment components and enhancing the accuracy of hazard location.

[0049] This invention constructs pixel-wise directional gradient channel features within candidate regions of power equipment components and combines them with a frequency-domain accelerated matching method based on fast Fourier transform to complete the structural alignment of visible light and infrared images. This transfers the computationally intensive spatial domain matching process to the frequency domain, effectively reducing computational complexity and improving overall processing efficiency while ensuring alignment accuracy and stability. It is suitable for deployment and application in resource-constrained scenarios such as UAV inspection and mobile inspection terminals.

[0050] This invention improves the robustness and precision of cross-modal alignment results in complex scenarios by performing consistency constraint fusion on local alignment parameters obtained from multiple candidate regions and combining coarse-to-fine pyramid alignment and sub-pixel refinement processing. It also reduces the impact of local mismatches on the overall alignment results, thereby ensuring the stability and continuity of infrared thermal anomaly mapping results.

[0051] Based on cross-modal alignment, this invention introduces a local similarity response map to calculate the alignment confidence index, and combines it with component mapping thermal anomaly regions to extract multi-dimensional hidden danger judgment features. This enables the credibility assessment and risk classification of power hidden danger identification results, effectively reducing the risk of false alarms and false alarms, and improving the reliability and application value of power hidden danger identification results in actual operation and maintenance decision-making. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart of an AI vision-based method for identifying potential electrical hazards proposed in this invention;

[0054] Figure 2 This diagram illustrates the frequency domain accelerated matching process of the Fast Fourier Transform (FFT) in the proposed AI vision-based power hazard identification method. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0056] refer to Figure 1-2 A method for identifying potential electrical hazards based on AI vision includes the following steps:

[0057] Visible light images and infrared images are collected during the same power inspection period to form a dual-modal image pair. The dual-modal image pair is then preprocessed to obtain a preprocessed dual-modal image pair.

[0058] Generate a set of candidate regions for power equipment components based on preprocessed bimodal images;

[0059] For each candidate region in the candidate region set, CFOG features are extracted from the preprocessed bimodal image pair, and visible light CFOG feature map and infrared CFOG feature map are output.

[0060] Perform frequency domain accelerated matching based on fast Fourier transform on visible light CFOG feature maps and infrared CFOG feature maps, and output local similarity response maps and local alignment parameters;

[0061] Perform consistency constraint fusion on the local alignment parameters to output the global alignment parameters, and perform coarse-to-fine pyramid alignment based on the global alignment parameters to obtain fine alignment parameters. At the same time, perform sub-pixel refinement processing to output sub-pixel level alignment parameters.

[0062] Based on subpixel-level alignment parameters, an alignment transformation is performed on the preprocessed bimodal image pair to output an aligned bimodal image pair. The infrared thermal anomaly region is determined and mapped to the power equipment component region in the aligned visible light image. The component-mapped thermal anomaly region is then output.

[0063] Alignment confidence index is calculated based on local similarity response map, and hidden danger judgment features are extracted based on component mapping thermal anomaly area to obtain hidden danger identification results.

[0064] In this embodiment, obtaining the preprocessed bimodal image pair specifically includes:

[0065] Visible light images and infrared images are collected during the same power inspection period. The collected visible light images are recorded as the original visible light images, and the collected infrared images are recorded as the original infrared images, forming a dual-modal image pair.

[0066] Preprocessing of bimodal image pairs includes denoising, geometric scale unification, and grayscale normalization.

[0067] The denoising process includes filtering and denoising, and suppressing abnormal noise points. It outputs a visible light denoised image and an infrared denoised image. Geometric scale unification and grayscale normalization are then performed sequentially on the visible light denoised image and the infrared denoised image to obtain a preprocessed bimodal image pair. The geometric scale unification process involves resampling the visible light denoised image and the infrared denoised image to the same image width and the same image height, and outputting a scale-unified visible light image and a scale-unified infrared image. The grayscale normalization process involves performing linear normalization on the scale-unified visible light image and the scale-unified infrared image respectively. This involves subtracting the minimum grayscale value of the current image from the grayscale value of each pixel in the corresponding image, and dividing the resulting difference by the difference between the maximum and minimum grayscale values ​​of the current image, thereby mapping the pixel grayscale values ​​to the range of zero to one, and finally outputting a normalized visible light image and a normalized infrared image.

[0068] In this embodiment, the generation of the candidate region set for the power equipment components specifically includes:

[0069] Read the preprocessed bimodal image pair, and use the visible light image and infrared image as the visible light input image and infrared input image respectively. At the same time, establish a bimodal joint input, which is composed of the visible light gray value and infrared gray value at each pixel position.

[0070] The dual-modal joint input is used to perform power equipment component localization processing. Based on the structural feature information of the visible light image and the brightness distribution information of the infrared image in the dual-modal joint input, feature extraction and determination are performed on the regions in the image that belong to power equipment components. According to the determination results, the spatial location range of different power equipment components is determined, and a corresponding candidate box is generated for each power equipment component. At the same time, a corresponding component category identifier and confidence level are assigned to each candidate box, thereby obtaining a set of power equipment component detection results. Each detection result in the set of power equipment component detection results includes a component category identifier, a component confidence level, and candidate box position information. The candidate box position information includes the coordinates of the upper left corner of the candidate box, the width of the candidate box, and the height of the candidate box, and is used as the initial candidate box set.

[0071] Candidate region generation is performed on the initial candidate box set. Low-confidence candidate boxes are eliminated according to the confidence threshold. Overlapping box redundancy removal is performed on the remaining candidate boxes. At the same time, each candidate box after redundancy removal is extended by a preset extension pixel and cropped to the image boundary range. Each candidate box is converted into a candidate region, and all candidate regions are collected to obtain the candidate region set of power equipment components.

[0072] In this embodiment, the output of the visible light CFOG feature map and the infrared CFOG feature map specifically includes:

[0073] Each candidate region in the candidate region set is cropped from the visible light input image and the infrared input image respectively to obtain the visible light candidate region image and the infrared candidate region image, and the paired candidate region images are output.

[0074] For each visible light candidate region image, gradient calculation is performed pixel by pixel within the candidate region. For each pixel, the gray level difference between adjacent pixels in the horizontal direction and the gray level difference between adjacent pixels in the vertical direction are calculated, and the gradient magnitude and gradient direction of the current pixel are obtained. At the same time, the gradient direction of each pixel in the candidate region is quantized, the gradient direction is divided into a preset number of directional channels, and the gradient magnitude of the current pixel is accumulated into the channel value of the corresponding directional channel at the current pixel position. Thus, a pixel-by-pixel directional gradient channel feature representation is formed within the candidate region, and the visible light CFOG feature map corresponding to the visible light candidate region is output.

[0075] For each infrared candidate region image paired with the visible light candidate region image, the infrared CFOG feature map corresponding to the infrared candidate region is output using the same pixel-by-pixel gradient calculation method and directional quantization method.

[0076] This invention constructs pixel-wise directional gradient channel features for visible light and infrared images respectively within candidate regions of power equipment components. This expresses the structural contour information of power equipment in a unified structural feature form, effectively reducing the impact of differences in imaging mechanisms on cross-modal analysis. This makes visible light and infrared images more consistent and comparable at the structural level, thereby improving the sensitivity and stability of subsequent cross-modal matching and alignment processes to the actual structure of power equipment. This provides a robust feature foundation for the accurate identification and reliable location of potential power hazards.

[0077] In this embodiment, the output of the local similarity response map and the local alignment parameters specifically includes:

[0078] Read the visible light CFOG feature map and the infrared CFOG feature map, and establish a local matching task with the candidate region as the processing unit. The input of each local matching task is the visible light CFOG feature map and the infrared CFOG feature map of the same candidate region, thereby limiting the frequency domain accelerated matching to the candidate region of the power equipment component.

[0079] For each local matching task, frequency domain matching preprocessing is performed on the visible light CFOG feature map and the infrared CFOG feature map, size unification processing is performed on the two-dimensional channel matrix of each directional channel, boundary filling processing is performed on the two-dimensional channel matrix of each directional channel to form a uniform transformation size, and windowing processing is performed on the filled two-dimensional channel matrix. The visible light directional channel matrix and the infrared directional channel matrix after frequency domain matching preprocessing are used as the frequency domain transformation input matrix.

[0080] The frequency domain transformation input matrix is ​​processed by performing a two-dimensional fast Fourier transform along the directional channel. A one-dimensional fast Fourier transform is performed row by row along the row direction of the two-dimensional channel matrix of each directional channel to obtain the row frequency domain result. At the same time, a one-dimensional fast Fourier transform is performed column by column along the column direction of the row frequency domain result to obtain the two-dimensional frequency domain result. The visible light CFOG feature map and the infrared CFOG feature map in the current directional channel are obtained respectively in the visible light frequency domain matrix and the infrared frequency domain matrix. The real part and imaginary part of the frequency domain matrix of each directional channel are recorded.

[0081] Frequency domain correlation is calculated and channel fusion is performed according to the directional channel. Under each directional channel, the infrared frequency domain matrix is ​​conjugated element by element to obtain the conjugate infrared frequency domain matrix. The visible light frequency domain matrix and the conjugate infrared frequency domain matrix are multiplied element by element at the same frequency position to obtain the frequency domain correlation matrix of the current directional channel. The frequency domain correlation matrices of all directional channels are accumulated element by element at the corresponding frequency position to obtain the fused frequency domain correlation matrix. Channel weights are set for each directional channel and weighted accumulation is performed to obtain the weighted fused frequency domain correlation matrix. The channel weights are determined by the gradient energy of the directional channels in the current candidate region, so that the directional channels with higher gradient energy contribute more to the fused frequency domain correlation matrix.

[0082] A two-dimensional inverse fast Fourier transform is performed on the weighted fused frequency domain correlation matrix. The fused frequency domain correlation matrix is ​​used as a complex spectrum input while keeping the real and imaginary parts unchanged. A one-dimensional inverse fast Fourier transform is performed column by column along the column direction to restore the spectrum from the frequency domain to the intermediate domain. At the same time, a one-dimensional inverse fast Fourier transform is performed row by row along the row direction to restore the intermediate domain to the spatial domain, resulting in a spatial domain correlation response matrix. The magnitude of the correlation response matrix is ​​calculated to obtain a non-negative response value matrix, and normalization is performed according to the transform size to make the response values ​​obtained by different candidate regions under different fill sizes comparable. The normalized non-negative response value matrix is ​​determined as the local similarity response map of the current candidate region.

[0083] Peak localization processing is performed on the local similarity response map. All response values ​​are traversed, and the matrix position corresponding to the maximum response value is determined as the best matching displacement position. This position is then converted into the horizontal and vertical displacement of the current candidate region and used as the local alignment parameters of the current candidate region.

[0084] This invention constructs pixel-wise directional gradient channel features within candidate regions of power equipment components and introduces a frequency-domain accelerated matching mechanism based on Fast Fourier Transform. This transfers the cross-modal structural matching process from high-complexity spatial domain computation to efficient frequency domain completion, significantly reducing computational overhead while enhancing the perception of structural similarity in power equipment. Furthermore, by constructing a stable and reliable local similarity response map through weighted fusion of directional channels and normalized similarity responses, the obtained local alignment parameters maintain good stability and discriminability even in complex backgrounds, noise interference, and scenarios with multiple structures coexisting. This provides an efficient and robust structural matching foundation for subsequent accurate cross-modal alignment, reliable mapping of thermal anomalies, and high-precision identification of power hazards.

[0085] In this embodiment, the output of the sub-pixel level alignment parameters specifically includes:

[0086] The local alignment parameters of all candidate regions are aggregated to form a local alignment parameter set. Each local alignment parameter includes the horizontal displacement and vertical displacement of the current candidate region. At the same time, the local similarity response map corresponding to each candidate region is read, and the maximum response value of each local similarity response map is used as the confidence weight input of the local alignment parameter of the current candidate region.

[0087] A consistency constraint fusion process is performed on the local alignment parameter set to obtain consistent local alignment parameters. All consistent local alignment parameters are aggregated to form a consistent local alignment parameter subset. Weighted fusion is performed on the consistent local alignment parameter subset based on confidence weights to output global alignment parameters. The global alignment parameters include global horizontal displacement and global vertical displacement. The consistency constraint fusion process includes calculating the horizontal displacement difference and vertical displacement difference for any two sets of local alignment parameters in the local alignment parameter set, and determining the local alignment parameters whose horizontal displacement difference and vertical displacement difference are both less than a preset consistency threshold as consistent local alignment parameters.

[0088] Based on global alignment parameters, the preprocessed bimodal image pairs are downsampled according to a preset downsampling ratio to obtain coarse preprocessed bimodal image pairs. On the coarse preprocessed bimodal image pairs, coarse alignment updates are performed with the global alignment parameters as the initial displacement, and coarse refined alignment parameters are output. The coarse refined alignment parameters are mapped to the fine layer according to the scale mapping relationship corresponding to the downsampling ratio to obtain the fine initial alignment parameters. At the same time, on the fine preprocessed bimodal image pairs at the original resolution, the fine initial alignment parameters are used as the center displacement and fine alignment updates are performed within a preset local search range to output fine alignment parameters. The fine alignment parameters include fine horizontal displacement and fine vertical displacement.

[0089] Subpixel refinement processing is performed on the fine alignment parameters. A preset response neighborhood is selected with the position of the maximum response value as the center in the local similarity response map. The subpixel-level peak position offset of the maximum response value is determined based on the response value distribution in the response neighborhood. These offsets are then superimposed on the fine alignment parameters to output the subpixel-level alignment parameters. The subpixel-level alignment parameters include subpixel-level horizontal displacement and subpixel-level vertical displacement.

[0090] This invention achieves stable and reliable global alignment results by constraining and weighting the local alignment parameters obtained from multiple candidate regions with consistency and confidence. Combined with a coarse-to-fine pyramid alignment mechanism and sub-pixel level refinement, it effectively suppresses the impact of local mismatches on the overall alignment accuracy. In complex power inspection scenarios, it significantly improves the accuracy, robustness, and continuity of cross-modal alignment results, providing a high-precision spatial foundation for accurate mapping of infrared thermal anomalies and reliable identification of power hazards.

[0091] In this embodiment, the output of the component mapping thermal anomaly region specifically includes:

[0092] Using subpixel-level alignment parameters as input parameters for alignment transformation, subpixel-level geometric alignment transformation is performed on the infrared image in the preprocessed bimodal image, including subpixel-precision translation and resampling processing, so that the preprocessed infrared image is spatially aligned with the preprocessed visible light image, outputting the aligned infrared image, and forming an aligned bimodal image pair with the preprocessed visible light image;

[0093] Infrared thermal anomaly region determination is performed on the aligned infrared image. Based on the distribution of infrared intensity values ​​of each pixel in the aligned infrared image, a set of pixels that meet the criteria of being higher than the preset thermal anomaly judgment threshold is determined. Spatial connectivity analysis is performed on the pixel set, and high-intensity pixels that are spatially connected are merged into continuous regions. Region filtering processing is performed on the continuous regions to remove regions with an area smaller than the preset area threshold, and a set of infrared thermal anomaly regions is output.

[0094] Read the candidate region set, and map each infrared thermal anomaly region in the infrared thermal anomaly region set to its corresponding spatial location in the visible light image coordinate system of the aligned dual-modal image pairs. Perform spatial association processing on the mapped infrared thermal anomaly regions and candidate regions, and determine the infrared thermal anomaly regions that overlap or contain the candidate regions of power equipment components as component-mapped thermal anomaly regions.

[0095] This invention achieves high-precision spatial alignment between infrared and visible light images based on sub-pixel level alignment parameters. On this basis, it reliably extracts and maps infrared thermal anomalies at the component level, enabling infrared thermal anomalies to be accurately mapped to the location of specific power equipment components. This effectively avoids the problem of inaccurate hazard location caused by registration errors and background interference in the prior art, thereby improving the overall performance of power hazard identification results in terms of spatial positioning accuracy, component attribution reliability, and engineering application stability.

[0096] In this embodiment, obtaining the hazard identification result specifically includes:

[0097] Peak analysis is performed on the local similarity response maps corresponding to each candidate region to determine the maximum and second largest response values. Based on the degree of difference between the two, the alignment confidence index is calculated, and the alignment confidence indexes of each candidate region are aggregated to form an alignment confidence index set.

[0098] In the aligned infrared image, the maximum infrared intensity and area of ​​the mapped thermal anomaly region for each component are statistically analyzed. When there are consecutive different mapped thermal anomaly regions for different components, the temporal variation of the thermal anomaly intensity is statistically analyzed to form a set of features for hazard judgment.

[0099] The risk score corresponding to the thermal anomaly area mapped by each component is obtained by fusion calculation based on the aligned confidence index set and the hazard judgment feature set. The hazard risk level is determined according to the risk score and the preset threshold. The corresponding hazard location, hazard type and hazard risk level are output in the aligned visible light image, thereby obtaining the hazard identification result.

[0100] Example 1:

[0101] To verify the feasibility of this invention in practice, it was applied to the task of identifying potential hazards in the inspection of electrical equipment in a factory. In this scenario, the inspection system needs to periodically inspect the operating electrical equipment to detect potential hazards of localized overheating caused by poor contact, aging, abnormal load, etc. Such hazards usually manifest as localized high-temperature areas in infrared images. However, relying solely on infrared images makes it difficult to accurately determine the specific electrical equipment component to which the thermal anomaly belongs, while relying solely on visible light images cannot reflect the thermal state of the equipment. Therefore, it is necessary to perform joint analysis of visible light and infrared images to achieve accurate identification and location of electrical hazards.

[0102] In this implementation scenario, the inspection system simultaneously collects visible light and infrared images within the same inspection period. The collected images contain various power equipment components and complex background information, such as wires, insulators, metal connectors, and non-equipment background areas. Due to the differences in imaging resolution, viewing angle, and imaging mechanism between visible light and infrared images, direct pixel-level alignment is easily affected by background interference and noise, resulting in thermal anomaly areas not being accurately mapped to the corresponding power equipment components.

[0103] To address the aforementioned issues, this embodiment first performs preprocessing operations on the acquired visible light and infrared images, including denoising, geometric scale unification, and grayscale normalization, thereby obtaining a preprocessed bimodal image pair. Based on this, by performing power equipment component localization processing on the preprocessed bimodal image pair, multiple candidate regions for power equipment components are automatically generated, so that subsequent processing focuses on the equipment body region and avoids interference from the background region on the matching results.

[0104] Subsequently, pixel-wise directional gradient channel features were extracted from both visible light and infrared images within each candidate region of a power equipment component, constructing visible light CFOG feature maps and infrared CFOG feature maps. This feature, centered on structural gradients, maintains a stable representation of the equipment's structural contour across different imaging modalities, effectively mitigating the impact of brightness differences on cross-modal analysis.

[0105] After obtaining the CFOG features, this embodiment employs a frequency-domain accelerated matching method based on Fast Fourier Transform to match the visible light CFOG feature map with the infrared CFOG feature map. The matching process uses candidate regions of power equipment components as processing units. After performing size uniformity, boundary filling, and windowing processing on the CFOG channels in each direction, the data is entered into the frequency domain. Local similarity response maps are obtained through frequency domain correlation calculations, and the local alignment parameters corresponding to each candidate region are determined accordingly. Compared to the traditional spatial domain sliding matching method, this method significantly reduces computational complexity, resulting in a marked reduction in single-frame image processing time, thus meeting the real-time requirements of inspection scenarios.

[0106] Based on this, the local alignment parameters obtained from all candidate regions are subjected to consistency constraint fusion processing. Alignment results with high consistency are then filtered and fused using confidence weights to obtain global alignment parameters. Subsequently, coarse-to-fine pyramid alignment is performed based on the global alignment parameters, combined with sub-pixel refinement processing, to further improve the alignment accuracy between visible light and infrared images. Through the above processing, thermal anomaly regions in infrared images can be accurately mapped to the locations of power equipment components in visible light images with sub-pixel precision.

[0107] After completing cross-modal alignment, this embodiment automatically extracts infrared thermal anomaly regions from the aligned infrared image and maps them to the aligned visible light image coordinate system. The relationship between the thermal anomaly regions and candidate regions of power equipment components is determined through spatial correlation, thereby identifying the component-mapped thermal anomaly regions. Furthermore, the alignment confidence index is calculated by analyzing the local similarity response map, and combined with hazard identification features such as thermal anomaly intensity, area, and temporal stability, the final power hazard identification result is output.

[0108] To verify the beneficial effects of the present invention, the method of this embodiment was compared with a comparative method that did not employ the technical solution of the present invention. Comparison method one only used spatial domain correlation of the entire image for visible light and infrared registration; comparison method two used traditional feature matching but did not introduce frequency domain acceleration and consistency fusion mechanism. Statistical analysis was performed on the three methods under the same dataset, and the specific comparison data is shown in Table 1:

[0109] Table 1. Performance Comparison of Different Methods in Power Hazard Identification

[0110] index Comparison Method 1 (Spatial Domain Whole Graph Matching) Comparison Method 2 (Traditional Feature Matching) Method of the present invention Average processing time (milliseconds / frame) 320 210 120 Average alignment error (pixels) 2.8 1.9 0.9 Accuracy rate of matching thermally abnormal components (%) 81.4 86.7 94.6 False alarm rate (%) 12.6 9.8 4.1 Missed report rate (%) 10.9 8.3 3.7

[0111] As shown in Table 1, the method of this invention outperforms traditional comparative methods in power equipment hazard identification. In terms of processing time, the method of this invention significantly reduces the processing time per frame to only 120 milliseconds, while comparative methods one and two require 320 milliseconds and 210 milliseconds, respectively. Secondly, the method of this invention demonstrates superior image alignment accuracy, with an average alignment error of 0.9 pixels, significantly better than comparative methods one (2.8 pixels) and two (1.9 pixels). In terms of the accuracy of matching thermal anomaly components, the method of this invention also shows a strong advantage, reaching 94.6%, far exceeding comparative methods one (81.4%) and two (86.7%). Furthermore, the method of this invention significantly reduces the false alarm rate (4.1%) and the false negative rate (3.7%), improving the accuracy and reliability of power equipment hazard identification. These data fully demonstrate the efficiency, accuracy, and stability of the method of this invention in power equipment hazard identification.

[0112] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying electrical hazards based on AI vision, characterized in that, Includes the following steps: Visible light images and infrared images are collected during the same power inspection period to form a dual-modal image pair. The dual-modal image pair is then preprocessed to obtain a preprocessed dual-modal image pair. Generate a set of candidate regions for power equipment components based on preprocessed bimodal images; For each candidate region in the candidate region set, CFOG features are extracted from the preprocessed bimodal image pair, and visible light CFOG feature map and infrared CFOG feature map are output. Perform frequency domain accelerated matching based on fast Fourier transform on visible light CFOG feature maps and infrared CFOG feature maps, and output local similarity response maps and local alignment parameters; Perform consistency constraint fusion on the local alignment parameters to output the global alignment parameters, and perform coarse-to-fine pyramid alignment based on the global alignment parameters to obtain fine alignment parameters. At the same time, perform sub-pixel refinement processing to output sub-pixel level alignment parameters. Based on subpixel-level alignment parameters, an alignment transformation is performed on the preprocessed bimodal image pair to output an aligned bimodal image pair. The infrared thermal anomaly region is determined and mapped to the power equipment component region in the aligned visible light image. The component-mapped thermal anomaly region is then output. Alignment confidence index is calculated based on local similarity response map, and hidden danger judgment features are extracted based on component mapping thermal anomaly area to obtain hidden danger identification results.

2. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The specific steps for obtaining the preprocessed bimodal image pair include: Visible light images and infrared images are collected during the same power inspection period. The collected visible light images are recorded as the original visible light images, and the collected infrared images are recorded as the original infrared images, forming a dual-modal image pair. Preprocessing of bimodal image pairs includes denoising, geometric scale unification, and grayscale normalization. The denoising process includes filtering and denoising, and suppressing abnormal noise points. It outputs a visible light denoised image and an infrared denoised image. The visible light denoised image and the infrared denoised image are then subjected to geometric scale unification and grayscale normalization processing in sequence to obtain a preprocessed dual-modal image pair.

3. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The generation of the candidate region set for the power equipment components specifically includes: Read the preprocessed dual-modal image pair, and use the visible light image and infrared image as the visible light input image and infrared input image respectively, while establishing a dual-modal joint input; The dual-modal joint input is used to perform power equipment component localization processing. Based on the structural feature information of the visible light image and the brightness distribution information of the infrared image in the dual-modal joint input, feature extraction and determination are performed on the regions in the image that belong to power equipment components. Based on the determination results, the spatial location range of different power equipment components is determined, and a corresponding candidate box is generated for each power equipment component. At the same time, a corresponding component category label and confidence level are assigned to each candidate box, thereby obtaining a set of power equipment component detection results, which is used as the initial candidate box set. Candidate region generation is performed on the initial candidate box set. Low-confidence candidate boxes are eliminated according to the confidence threshold. Overlapping box redundancy removal is performed on the remaining candidate boxes. At the same time, each candidate box after redundancy removal is extended by a preset extension pixel and cropped to the image boundary range. Each candidate box is converted into a candidate region, and all candidate regions are collected to obtain the candidate region set of power equipment components.

4. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The outputs of the visible light CFOG feature map and the infrared CFOG feature map specifically include: Each candidate region in the candidate region set is cropped from the visible light input image and the infrared input image respectively to obtain the visible light candidate region image and the infrared candidate region image, and the paired candidate region images are output. For each visible light candidate region image, gradient calculation is performed pixel by pixel within the candidate region. For each pixel, the gray level difference between adjacent pixels in the horizontal direction and the gray level difference between adjacent pixels in the vertical direction are calculated, and the gradient magnitude and gradient direction of the current pixel are obtained. At the same time, the gradient direction of each pixel in the candidate region is quantized, the gradient direction is divided into a preset number of directional channels, and the gradient magnitude of the current pixel is accumulated into the channel value of the corresponding directional channel at the current pixel position. Thus, a pixel-by-pixel directional gradient channel feature representation is formed within the candidate region, and the visible light CFOG feature map corresponding to the visible light candidate region is output. For each infrared candidate region image paired with the visible light candidate region image, the infrared CFOG feature map corresponding to the infrared candidate region is output using the same pixel-by-pixel gradient calculation method and directional quantization method.

5. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The output of the local similarity response map and local alignment parameters specifically includes: Read the visible light CFOG feature map and the infrared CFOG feature map, and establish a local matching task using candidate regions as processing units; For each local matching task, frequency domain matching preprocessing is performed on the visible light CFOG feature map and the infrared CFOG feature map, size unification processing is performed on the two-dimensional channel matrix of each directional channel, boundary filling processing is performed on the two-dimensional channel matrix of each directional channel to form a uniform transformation size, and windowing processing is performed on the filled two-dimensional channel matrix. The visible light directional channel matrix and the infrared directional channel matrix after frequency domain matching preprocessing are used as the frequency domain transformation input matrix. The frequency domain transformation input matrix is ​​processed by performing a two-dimensional fast Fourier transform along the directional channel. A one-dimensional fast Fourier transform is performed row by row along the row direction of the two-dimensional channel matrix of each directional channel to obtain the row frequency domain result. At the same time, a one-dimensional fast Fourier transform is performed column by column along the column direction of the row frequency domain result to obtain the two-dimensional frequency domain result. The visible light CFOG feature map and the infrared CFOG feature map are obtained in the visible light frequency domain matrix and the infrared frequency domain matrix of the current directional channel, respectively. Frequency domain correlation is calculated for each directional channel and channel fusion is performed. For each directional channel, the infrared frequency domain matrix is ​​conjugated element by element to obtain the conjugate infrared frequency domain matrix. The visible light frequency domain matrix and the conjugate infrared frequency domain matrix are multiplied element by element at the same frequency position to obtain the frequency domain correlation matrix of the current directional channel. The frequency domain correlation matrices of all directional channels are accumulated element by element at the corresponding frequency position to obtain the fused frequency domain correlation matrix. Channel weights are set for each directional channel and weighted accumulation is performed to obtain the weighted fused frequency domain correlation matrix. A two-dimensional inverse fast Fourier transform is performed on the weighted fused frequency domain correlation matrix. The fused frequency domain correlation matrix is ​​used as a complex spectrum input while keeping the real and imaginary parts unchanged. A one-dimensional inverse fast Fourier transform is performed column by column along the column direction to restore the spectrum from the frequency domain to the intermediate domain. At the same time, a one-dimensional inverse fast Fourier transform is performed row by row along the row direction to restore the intermediate domain to the spatial domain, resulting in a spatial domain correlation response matrix. The magnitude of the correlation response matrix is ​​calculated to obtain a non-negative response value matrix, and normalization is performed according to the transform size. The normalized non-negative response value matrix is ​​determined as the local similarity response map of the current candidate region. Peak localization processing is performed on the local similarity response map. All response values ​​are traversed, and the matrix position corresponding to the maximum response value is determined as the best matching displacement position. This position is then converted into the horizontal and vertical displacement of the current candidate region and used as the local alignment parameters of the current candidate region.

6. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The output of the sub-pixel level alignment parameters specifically includes: The local alignment parameters of all candidate regions are aggregated to form a local alignment parameter set. At the same time, the local similarity response map corresponding to each candidate region is read, and the maximum response value of each local similarity response map is used as the confidence weight input for the local alignment parameter of the current candidate region. A consistency constraint fusion process is performed on the local alignment parameter set to obtain consistent local alignment parameters. All consistent local alignment parameters are aggregated to form a consistent local alignment parameter subset. Weighted fusion is performed on the consistent local alignment parameter subset based on confidence weights to output global alignment parameters. Based on the global alignment parameters, the preprocessed bimodal image pairs are downsampled according to a preset downsampling ratio to obtain coarse preprocessed bimodal image pairs. On the coarse preprocessed bimodal image pairs, the global alignment parameters are used as the initial displacement to perform coarse alignment updates, and the coarse fine alignment parameters are output. The coarse fine alignment parameters are mapped to the fine layer according to the scale mapping relationship corresponding to the downsampling ratio to obtain the fine layer initial alignment parameters. At the same time, on the fine preprocessed bimodal image pairs at the original resolution, the fine layer initial alignment parameters are used as the center displacement and the fine layer alignment updates are performed within a preset local search range to output fine alignment parameters. Subpixel refinement is performed on the fine alignment parameters. A preset response neighborhood is selected with the position of the maximum response value as the center in the local similarity response map. The subpixel-level peak position offset of the maximum response value is determined based on the response value distribution in the response neighborhood, and then superimposed on the fine alignment parameters to output the subpixel-level alignment parameters.

7. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The output of the component mapping thermal anomaly region specifically includes: Using subpixel-level alignment parameters as input parameters for alignment transformation, subpixel-level geometric alignment transformation is performed on the infrared image in the preprocessed bimodal image, including subpixel-precision translation and resampling processing, outputting an aligned infrared image, which is then combined with the preprocessed visible light image to form an aligned bimodal image pair; Infrared thermal anomaly region determination is performed on the aligned infrared image. Based on the distribution of infrared intensity values ​​of each pixel in the aligned infrared image, a set of pixels that meet the preset thermal anomaly judgment threshold is determined. Spatial connectivity analysis is performed on the pixel set, and high-intensity pixels that are spatially connected are merged into continuous regions. Region filtering processing is performed on the continuous regions, and a set of infrared thermal anomaly regions is output. Read the candidate region set, and map each infrared thermal anomaly region in the infrared thermal anomaly region set to its corresponding spatial location in the visible light image coordinate system of the aligned dual-modal image pairs. Perform spatial association processing on the mapped infrared thermal anomaly regions and candidate regions, and determine the infrared thermal anomaly regions that overlap or contain the candidate regions of power equipment components as component-mapped thermal anomaly regions.

8. The method for identifying electrical hazards based on AI vision according to claim 1, characterized in that, The specific methods for obtaining the hazard identification results include: Peak analysis is performed on the local similarity response maps corresponding to each candidate region to determine the maximum and second largest response values. Based on the degree of difference between the two, the alignment confidence index is calculated, and the alignment confidence indexes of each candidate region are aggregated to form an alignment confidence index set. In the aligned infrared image, the maximum infrared intensity and area of ​​the mapped thermal anomaly region for each component are statistically analyzed. When there are consecutive different mapped thermal anomaly regions for different components, the temporal variation of the thermal anomaly intensity is statistically analyzed to form a set of features for hazard judgment. The risk score corresponding to the thermal anomaly area mapped by each component is obtained by fusion calculation based on the aligned confidence index set and the hazard judgment feature set. The hazard risk level is determined according to the risk score and the preset threshold. The corresponding hazard location, hazard type and hazard risk level are output in the aligned visible light image, thereby obtaining the hazard identification result.

Images