Infrared and visible light fusion detection method for hot spot defect of assembly

Abstract

The invention relates to the field of photovoltaic module detection, and discloses a module hot spot defect infrared and visible light fusion detection method, which comprises the following steps: multi-mode acquisition: multi-mode data of a photovoltaic module are synchronously acquired, and the multi-mode data comprise infrared temperature original data, visible light polarization images and environmental parameters; data preprocessing: preprocessing the multi-modal data based on a radiation transmission model and a polarization optical principle to obtain preprocessed data including an infrared temperature field and a visible light image; and dynamic feature extraction: extracting a space-time gradient tensor and a Riemannian curvature tensor of the infrared temperature field from the preprocessed data based on differential geometry and a computer vision algorithm. By synchronously acquiring multi-modal data, based on a radiation transmission model and a polarization optical principle, interference of atmospheric attenuation and environment temperature and humidity on infrared temperature measurement and influence of uneven illumination and fog effect on a visible light image are eliminated, and dynamic characteristics of an infrared temperature field and structural characteristics of visible light are mined.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic module testing technology, specifically to a method for detecting hot spot defects in modules using infrared and visible light fusion. Background Technology

[0002] With the rapid development of the photovoltaic industry, the installed capacity of photovoltaic systems continues to expand. Outdoor photovoltaic modules are subject to complex environmental factors such as sunlight, dust, and shadows for a long time. They are prone to hot spot effects caused by factors such as microcracks, broken grids, and foreign objects blocking the grid. This leads to abnormal increases in local temperature, which not only reduces power generation efficiency but also accelerates module aging and may even induce safety hazards. Therefore, accurate detection of hot spot defects has become a core technical requirement for the intelligent operation and maintenance of photovoltaic power plants.

[0003] Traditional hot spot defect detection for components relies on infrared images to filter abnormal areas by setting fixed temperature thresholds. While this method can quickly capture temperature differences, it does not consider atmospheric attenuation or the interference of ambient temperature and humidity on infrared thermometry. It also does not incorporate component structural information, often misjudging shadows, temporary heating, etc., as hot spots. This results in insufficient robustness and limited detection accuracy. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for detecting hot spot defects in components using a fusion of infrared and visible light, which solves the problem of insufficient robustness and limited detection accuracy in traditional hot spot defect detection methods.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting component hot spot defects using infrared and visible light fusion, comprising the following steps:

[0006] Multimodal acquisition: Simultaneously acquire multimodal data of photovoltaic modules, including raw infrared temperature data, visible light polarization images, and environmental parameters;

[0007] Data preprocessing: Based on the radiative transfer model and the principle of polarization optics, the multimodal data is preprocessed to obtain preprocessed data including infrared temperature field and visible light image;

[0008] Dynamic feature extraction: Based on differential geometry and computer vision algorithms, the spatiotemporal gradient tensor and Riemann curvature tensor of the infrared temperature field, as well as the polarization texture fusion features and surface normal vector field of the visible light image are extracted from the preprocessed data as dynamic features.

[0009] Coupling Alignment: Based on dynamic features, fiber bundle structures are constructed and coupling connections are calibrated through the coupling theory of Riemannian manifolds and fiber bundles. Spatiotemporal alignment and fusion of cross-modal features are then performed to obtain fused features.

[0010] Dynamic detection decision: Based on fused features, hot spot clustering and Bayesian determination are performed through mean-shift clustering and dynamic Bayesian inference to obtain hot spot candidate regions. The temperature characteristics of the hot spot candidate regions are compared with preset dynamic thresholds to obtain detection results.

[0011] By adopting the above technical solution, the system simultaneously acquires raw infrared temperature data, visible light polarization images, and environmental parameters. Based on the radiative transfer model and polarization optics principles, it eliminates the interference of atmospheric attenuation and ambient temperature and humidity on infrared temperature measurement, as well as the effects of uneven illumination and fog on visible light images. Utilizing differential geometry and computer vision algorithms, it mines the dynamic characteristics of the infrared temperature field, such as its spatiotemporal gradient and curvature, and the structural features of visible light, such as its polarization texture and surface normal vectors. Through Riemannian manifold and fiber bundle coupling theory, it constructs fiber bundle structures and calibrates coupling connections to achieve precise spatiotemporal alignment and fusion. Combining mean-shift clustering, dynamic Bayesian inference, and dynamic thresholds based on temperature change rate and environmental parameters, it adaptively identifies hot spot candidate regions. This enables multi-dimensional correlation analysis of hot spot defects, thermal anomalies, structural defects, and environmental adaptation, effectively distinguishing between real hot spots and interference from shadow occlusion and temporary heating, improving detection robustness, and solving the problem of insufficient robustness and limited detection accuracy in traditional component hot spot defect detection.

[0012] Preferably, in the multimodal acquisition step, the acquisition equipment includes an infrared camera, a visible light camera, an irradiance meter, and a weather station, and the environmental parameters include irradiance, wind speed, temperature, and humidity.

[0013] Preferably, in the data preprocessing step, the preprocessing includes radiometric calibration and atmospheric attenuation correction of the raw infrared temperature data in the multimodal data, and multi-scale enhancement and polarization dehazing of the visible light polarization image in the multimodal data. The radiometric calibration is based on blackbody furnace calibration data, and the mapping relationship between gray values ​​and temperature is fitted by a quadratic function. The multi-scale enhancement is to suppress illumination unevenness by convolving a multi-scale Gaussian kernel with the visible light polarization image. The polarization dehazing is based on the polarization optical characteristics to separate scene radiation and atmospheric scattered light.

[0014] Preferably, the quadratic function is T = a·DN 2 +b·DN+c, where T is the corrected temperature, DN is the original infrared grayscale value, and a, b, and c are calibration coefficients. The atmospheric attenuation correction is based on the atmospheric transmittance and ambient temperature to calculate the true temperature, and its correction formula is T. true =T meas +τ·(T atm -T meas ), where T true T represents the actual surface temperature of the component. meas To measure temperature, τ is atmospheric transmittance, and T is...atm The ambient temperature.

[0015] Preferably, in the dynamic feature extraction step, the differential geometry algorithm includes calculating the spatial gradient of the infrared temperature field using a gradient operator, calculating the temperature-time change rate using the time difference method, constructing a spatiotemporal gradient tensor by combining second-order partial derivatives, and calculating the Riemann curvature tensor based on the metric tensor of the spatiotemporal gradient tensor. The computer vision algorithm includes calculating Stokes parameters using light intensity in multiple polarization directions, extracting directional gradient histogram features, and estimating the surface normal vector field using a photometric stereo vision method.

[0016] Preferably, in the dynamic feature extraction step, the polarization texture fusion feature includes Stokes parameters and directional gradient histogram features. The Stokes parameters are calculated by the light intensity in the polarization direction, and the surface normal vector field is the distribution of unit normal vectors on the component surface estimated by a stereo vision algorithm based on the brightness changes under multiple illumination directions.

[0017] Preferably, in the coupling alignment step, the Riemannian manifold and fiber bundle coupling theory includes constructing a base manifold, fiber space, and fiber bundle structure with projection mapping, and realizing the spatiotemporal alignment of infrared feature manifold and visible light feature manifold through coupling connection equation. The base manifold is composed of infrared feature manifold and visible light feature manifold. The infrared feature manifold is constructed based on the spatiotemporal gradient tensor of the infrared temperature field and the Riemann curvature tensor. The visible light feature manifold is constructed based on polarization texture fusion features and surface normal vector field. The fiber space is the feature subspace corresponding to each point on the base manifold, containing the feature components of the Riemann curvature tensor, the feature energy of the orientation gradient histogram, and the components of the surface normal vector.

[0018] Preferably, in the coupling alignment step, the weight of the coupling connection is dynamically adjusted based on the hot spot temperature difference, where the hot spot temperature difference is the difference between the pixel temperature and the average temperature of the photovoltaic module in the infrared temperature field, and the weight calculation formula is as follows: Where ω1 is the weight of the infrared characteristic manifold, β is the temperature sensitivity coefficient, and ΔT is the hot spot temperature difference.

[0019] Preferably, in the dynamic detection decision step, the mean-shift clustering is performed by iteratively finding the density peak in the fused feature space to cluster hot spot candidate regions; the dynamic Bayesian inference is performed by calculating the posterior probability of hot spot existence based on the prior probability and likelihood function of the fused features to make a Bayesian determination; the prior probability is based on historical hot spot data statistics; the comparison is performed by numerically comparing the temperature characteristics of the hot spot candidate region with a dynamic threshold; when the temperature characteristics exceed the dynamic threshold, it is a hot spot defect; the dynamic threshold is updated in real time based on the temperature change rate of the hot spot and environmental parameters.

[0020] Preferably, the real-time update is Where τ(t) is the dynamic threshold at time t, and 65 is the reference temperature threshold. Let τ be the rate of temperature change, and α(τ) = 0.05·exp(-0.05·wind) be the wind speed correction factor, where wind is the wind speed.

[0021] This invention provides a method for detecting hot spot defects in components using a fusion of infrared and visible light. It offers the following advantages:

[0022] 1. This invention acquires multimodal data synchronously, and based on the radiative transfer model and polarization optics principle, eliminates the interference of atmospheric attenuation and ambient temperature and humidity on infrared thermometry, as well as the effects of uneven illumination and fog on visible light images. Utilizing differential geometry and computer vision algorithms, it mines the dynamic characteristics of the infrared temperature field and the structural features of visible light. Through Riemannian manifold and fiber bundle coupling theory, it constructs fiber bundle structures and calibrates coupling connections. Combining mean-shift clustering, dynamic Bayesian inference, and dynamic thresholds based on temperature change rate and environmental parameters, it adaptively identifies candidate hot spot regions, effectively distinguishing between real hot spots and interference from shadows, temporary heating, etc., thus improving detection robustness and solving the problem of insufficient robustness in traditional component hot spot defect detection, which limits detection accuracy.

[0023] 2. This invention achieves geometric correlation and alignment by combining dynamic features such as the spatiotemporal gradient tensor and Riemann curvature tensor of the infrared temperature field with structural features such as the polarization texture fusion feature and surface normal vector field of visible light through fiber bundle structure. This enables deep coupling between the thermal anomaly of the hot spot and the root cause of structural defects, improving the problem that a single mode or simple splicing cannot characterize the essential laws of the hot spot, and providing a feature expression paradigm for accurate detection.

[0024] 3. This invention eliminates the interference of atmospheric attenuation, ambient temperature and humidity on infrared thermometry, and the influence of uneven illumination on visible light images by using a radiative transfer model and polarization optics principles. This ensures the reliability of input features, explores the spatiotemporal evolution of the temperature field and the microscopic structural characteristics of visible light, enhances feature discrimination, and combines mean-shift clustering, dynamic Bayesian inference, and environmental adaptive thresholding to distinguish between real hot spots and interference such as shadow occlusion and temporary heating, reducing false alarms and false negatives. Thus, it can still output accurate results stably in complex environments. Attached Figure Description

[0025] Figure 1 This is a flowchart of the infrared and visible light fusion detection method for hot spot defects in components proposed in this invention. Detailed Implementation

[0026] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see the appendix Figure 1 This invention provides a method for detecting hot spot defects in components using infrared and visible light fusion, comprising the following steps:

[0028] Multimodal acquisition: Simultaneously acquire multimodal data of photovoltaic modules, including raw infrared temperature data, visible light polarization images, and environmental parameters;

[0029] Furthermore, in the multimodal acquisition step, the acquisition equipment includes infrared cameras, visible light cameras, radiometers, and weather stations, and the environmental parameters include irradiance, wind speed, temperature, and humidity.

[0030] Specifically, the multimodal acquisition step aims to provide comprehensive data support for subsequent hot spot defect detection by simultaneously acquiring multi-source information from photovoltaic modules. Generally, this step needs to ensure the consistency of different types of data in time and space to avoid fusion errors caused by data misalignment.

[0031] Specifically, the data acquisition equipment includes infrared cameras, visible light cameras, radiometers, and weather stations. Infrared cameras capture raw infrared temperature data of the photovoltaic modules, sensing temperature differences on the module surface caused by hot spot defects. Alternatively, a mid-wave infrared camera with a spectral range of 8-14 μm can be used, as this range effectively penetrates atmospheric interference to reflect the thermal radiation characteristics of the module surface, and the frame rate is set to 25 fps to meet the requirements of capturing the dynamic hot spot evolution process. Visible light cameras are used to acquire visible light polarization images of the modules. Equipped with switchable polarizers, they can acquire module texture information under different polarization directions. In one possible implementation, the polarizer's polarization directions include 0°, 90°, 45°, and 135°. By acquiring multi-directional polarization images, the influence of surface reflection on texture feature extraction can be effectively suppressed, highlighting structural defects related to hot spots, such as microcracks and broken grids.

[0032] Irradiance meters are used to measure environmental irradiance, and their measurement range typically covers 200-1200 W / m². 2 To adapt to changes in irradiance under different weather conditions, the meteorological station is responsible for collecting environmental parameters such as wind speed, ambient temperature, and humidity. These parameters are not only used for atmospheric attenuation correction in subsequent data preprocessing, but also participate in the calculation of dynamic thresholds in the dynamic detection decision-making process to adapt to the hot spot detection needs under different environments.

[0033] In some embodiments, to ensure data synchronization, the infrared camera and the visible light camera are controlled by the same trigger signal to ensure that temperature and texture information correspond at the same time. Simultaneously, all acquisition devices are installed on the same inspection platform, such as a drone or a fixed bracket, to ensure consistent spatial acquisition perspectives and avoid feature misalignment caused by differences in shooting angles.

[0034] In this embodiment, the multimodal data includes raw infrared temperature data, visible light polarization images, and environmental parameters. The raw infrared temperature data is stored in grayscale value format and will be converted into actual temperature values ​​through radiometric calibration. The visible light polarization images preserve the texture details and polarization characteristics of the component surface, providing a basis for identifying structural defects. The environmental parameters, including irradiance, wind speed, ambient temperature, and humidity, reflect the external conditions during detection from different perspectives, ensuring the reliability of the detection results.

[0035] Through the above multimodal acquisition steps, the thermal characteristics, structural characteristics, and environmental influencing factors of photovoltaic modules can be comprehensively captured, laying the foundation for subsequent data preprocessing and feature fusion, and helping to improve the accuracy and robustness of hot spot defect detection.

[0036] Data preprocessing: Based on the radiative transfer model and the principle of polarization optics, the multimodal data is preprocessed to obtain preprocessed data including infrared temperature field and visible light image;

[0037] Furthermore, in the data preprocessing steps, the preprocessing includes radiometric calibration and atmospheric attenuation correction of the raw infrared temperature data in the multimodal data, as well as multi-scale enhancement and polarization dehazing of the visible light polarization image in the multimodal data. The radiometric calibration is based on blackbody furnace calibration data, and the mapping relationship between gray values ​​and temperature is fitted by a quadratic function. The multi-scale enhancement is to suppress illumination unevenness by convolving the multi-scale Gaussian kernel with the visible light polarization image. The polarization dehazing is based on the polarization optical characteristics to separate scene radiation and atmospheric scattered light.

[0038] Furthermore, the quadratic function is T = a·DN 2 +b·DN+c, where T is the corrected temperature, DN is the original infrared grayscale value, a, b, and c are calibration coefficients, and atmospheric attenuation correction is based on calculating the true temperature using atmospheric transmittance and ambient temperature, with the correction formula being T. true =T meas +τ·(T atm -T meas ), where T true T represents the actual surface temperature of the component. meas To measure temperature, τ is atmospheric transmittance, and T is... atm The ambient temperature.

[0039] Specifically, the data preprocessing steps are based on the radiative transfer model and the principle of polarization optics. The raw infrared temperature data and visible light polarization images in the multimodal data are processed in a targeted manner to construct an accurate infrared temperature field and restore a clear visible light image.

[0040] For raw infrared temperature data, radiometric calibration is performed first. Generally, raw data acquired by infrared cameras is recorded in grayscale values, and radiometric calibration is needed to establish a mapping relationship between these grayscale values ​​and the actual temperature. Alternatively, based on blackbody furnace calibration data, a quadratic function T = a·DN is used. 2 The calibration process involves fitting the grayscale value DN to the corrected temperature T using the formula +b·DN+c. Specifically, during the calibration phase, multiple known temperature points are set using a blackbody furnace, and grayscale values ​​output from an infrared camera are simultaneously acquired. The calibration coefficients a, b, and c are then obtained using the least squares method. For example, when the blackbody furnace is set to 45℃, the infrared camera acquires a grayscale value of 190. This data is substituted into the fitted quadratic function to calculate the corrected temperature, achieving a quantitative conversion from grayscale value to temperature and providing accurate temperature data for constructing the infrared temperature field.

[0041] After radiometric calibration is completed, atmospheric attenuation correction is performed. Generally, infrared radiation is affected by absorption and scattering during atmospheric transmission, causing the measured temperature to deviate from the true value, which needs to be corrected through atmospheric attenuation correction. Specifically, this is based on atmospheric transmittance τ and ambient temperature T. atm Using formula T true =T meas +τ·(T atm -T meas Calculate the true surface temperature T of the component. true In one possible implementation, the atmospheric transmittance τ can be calculated using the ambient relative humidity RH, such as τ = 0.92 + 0.001 × (50 - RH). For example, when measuring temperature T... meas The temperature is 58℃, and the ambient temperature is T. atm At 28℃ and RH 42%, first calculate the atmospheric transmittance τ = 0.92 + 0.001 × (50 - 42) = 0.928, then... meas , τ, T atm By substituting the values ​​into the correction formula, the true surface temperature of the component is calculated, eliminating the attenuation interference of the atmosphere on infrared radiation and improving the reliability of the temperature data.

[0042] For visible light polarized images, multi-scale enhancement and polarization dehazing are required. In the multi-scale enhancement stage, visible light polarized images are generally susceptible to uneven illumination, causing blurred texture details in components. As an alternative, the multi-scale Retinex algorithm is employed, using convolution operations between multi-scale Gaussian kernels and the visible light polarized image to suppress uneven illumination. Specifically, Gaussian kernels with different standard deviations, such as 15, 30, and 60, are set, and corresponding weights, such as 0.5, 0.3, and 0.2, are sequentially used to convolve the visible light polarized image. For example, for component polarized images with lighting and shadows, after convolution with multi-scale Gaussian kernels, the images are weighted and fused to output an enhanced visible light polarized image that highlights the surface texture details of the components, laying a clear image foundation for subsequent polarization texture fusion feature extraction.

[0043] In the polarization dehazing process, due to the differences in polarization characteristics between scene radiation and atmospheric scattered light, the polarization properties of the light differ. Specifically, a switchable polarizer on a visible light camera is used to collect light intensities at polarization directions of 0°, 90°, 45°, and 135°. These polarization characteristics are then used to separate scene radiation and atmospheric scattered light. For example, based on the collected multi-directional polarized light intensities, Stokes parameters are calculated to distinguish between scene radiation and atmospheric scattered light, removing the interference of fog effects on the image, restoring the true texture of component surfaces, improving visible light image quality, and facilitating the subsequent extraction of features such as surface normal fields.

[0044] Through the above data preprocessing steps, the raw infrared temperature data is corrected and the visible light polarization image is enhanced. The resulting infrared temperature field accurately reflects the surface temperature distribution of the component, and the visible light image clearly presents the structural details of the component, providing reliable input data for the subsequent dynamic feature extraction steps.

[0045] Dynamic feature extraction: Based on differential geometry and computer vision algorithms, the spatiotemporal gradient tensor and Riemann curvature tensor of the infrared temperature field, as well as the polarization texture fusion features and surface normal vector field of the visible light image are extracted from the preprocessed data as dynamic features.

[0046] Furthermore, in the dynamic feature extraction step, the differential geometry algorithm includes calculating the spatial gradient of the infrared temperature field through gradient operators, calculating the temperature time change rate through the time difference method, constructing the spatiotemporal gradient tensor by combining the second-order partial derivatives, and calculating the Riemann curvature tensor based on the metric tensor of the spatiotemporal gradient tensor. The computer vision algorithm includes calculating the Stokes parameters through the light intensity of multiple polarization directions, extracting the directional gradient histogram features, and estimating the surface normal vector field through the photometric stereo vision method.

[0047] Furthermore, in the dynamic feature extraction step, the polarization texture fusion features include Stokes parameters and directional gradient histogram features. The Stokes parameters are calculated by the light intensity in the polarization direction, and the surface normal vector field is the distribution of unit normal vectors on the component surface estimated by a stereo vision algorithm based on the brightness changes under multiple illumination directions.

[0048] Specifically, the dynamic feature extraction step is based on differential geometry and computer vision algorithms to extract features from the preprocessed data in order to capture the dynamic evolution and structural characterization information of the hot spots in photovoltaic modules.

[0049] For extracting the spatiotemporal gradient tensor of an infrared temperature field, the spatial distribution differences and temporal evolution trends of the temperature field generally provide a direct reflection of the dynamic characteristics of the hot spot. Specifically, a gradient operator is used to calculate the spatial gradient pixel-by-pixel on the preprocessed infrared temperature field. and The rate of temperature change in a two-dimensional plane is obtained; simultaneously, the temporal rate of change of the infrared temperature field in consecutive frames is calculated using the time-difference method. To capture the temperature fluctuations of hot spots over time, one option is to construct a 3×3 dimensional spatiotemporal gradient tensor G using second-order partial derivatives. Tt Its elements integrate the changes in spatial and temporal dimensions. The input is a preprocessed infrared temperature field sequence, and the output is a tensor form that characterizes the dynamic changes of the temperature field, providing a basic metric for the subsequent calculation of the Riemann curvature tensor.

[0050] In the derivation of the Riemann curvature tensor, the metric tensor g based on the spatiotemporal gradient tensor is... ij Perform the operation. Generally, the metric tensor g... ij Constructed from elements of the spatiotemporal gradient tensor, it describes the geometric metric properties of the temperature field. Specifically, the Riemann curvature tensor R is extracted from the derivative relation of the metric tensor using the computational rules of the curvature tensor in differential geometry. ijkl Independent components, such as R 1212 R 1313 These components reflect the curvature characteristics of the infrared temperature field in the spatiotemporal domain. The input is the spatiotemporal gradient tensor, and the output is the components of the Riemann curvature tensor, which helps to distinguish the geometric differences between hot spot regions and normal regions.

[0051] For polarization texture fusion feature extraction of visible light images, the calculation of Stokes parameters relies on light intensity data in multiple polarization directions. Specifically, it utilizes the light intensities I0 and I2 in the preprocessed visible light polarization image at polarization directions of 0°, 90°, 45°, and 135°. 90 I 45 I 135 The formula for calculating the Stokes parameter is: I = I0 + I 90 Q = I0 - I90 U = I 45 -I 135 When the photovoltaic module is made of non-optically active material, V=0. The input is light intensity data in multiple polarization directions, and the output is four parameters: I, Q, U, and V, which characterize the polarization texture of the module surface.

[0052] In the extraction of Histogram of Oriented Gradients (HOG) features, one approach is to divide the visible light image into 8×8 pixel cells, calculate the gradient orientation histogram for each cell, and then form blocks of 2×2 cells. The histograms within these blocks are then normalized, resulting in 128 orientation intervals. Specifically, the input is an enhanced visible light polarized image. By calculating pixel gradients and statistically analyzing the gradient orientation distribution, the output is a HOG feature vector. This vector integrates the texture gradient information of the component surface and fuses it with Stokes parameters to form a polarization texture fusion feature.

[0053] The extraction of the surface normal vector field is based on photometric stereo vision methods. Generally, the component surface is assumed to be a Lambertian solid, and the normal vector is derived using brightness variations under multiple illumination directions. Specifically, visible light images under different illumination angles are acquired. Using variations in natural or active light sources, combined with the Lambertian reflection model, the brightness variations of pixels under different illuminations are calculated, deriving the unit normal vector distribution of the component surface. The input is the visible light image under multiple illumination conditions, and the output is the surface normal vector field, reflecting the geometric morphology of the component surface and providing a basis for judging structural anomalies in hot spot regions.

[0054] Through the above dynamic feature extraction steps, the spatiotemporal gradient tensor and Riemann curvature tensor are extracted from the infrared temperature field, and the polarization texture fusion features and surface normal vector field are extracted from the visible light image, providing multi-dimensional dynamic feature support for the subsequent coupling and alignment of cross-modal features.

[0055] Coupling Alignment: Based on dynamic features, fiber bundle structures are constructed and coupling connections are calibrated through the coupling theory of Riemannian manifolds and fiber bundles. Spatiotemporal alignment and fusion of cross-modal features are then performed to obtain fused features.

[0056] Furthermore, in the coupling alignment step, the Riemannian manifold and fiber bundle coupling theory includes constructing the fiber bundle structure of the base manifold, fiber space, and projection mapping, and realizing the spatiotemporal alignment of the infrared feature manifold and the visible light feature manifold through the coupling connection equation. The base manifold is composed of the infrared feature manifold and the visible light feature manifold. The infrared feature manifold is constructed based on the spatiotemporal gradient tensor of the infrared temperature field and the Riemann curvature tensor. The visible light feature manifold is constructed based on the polarization texture fusion feature and the surface normal vector field. The fiber space is the feature subspace corresponding to each point on the base manifold, which contains the feature components of the Riemann curvature tensor, the feature energy of the orientation gradient histogram, and the components of the surface normal vector.

[0057] Furthermore, in the coupling alignment step, the weights of the coupling connections are dynamically adjusted based on the hotspot temperature difference, which is the difference between the pixel temperature and the average temperature of the photovoltaic module in the infrared temperature field. The weight calculation formula is as follows: Where ω1 is the weight of the infrared characteristic manifold, β is the temperature sensitivity coefficient, and ΔT is the hot spot temperature difference.

[0058] Specifically, the coupling alignment step is based on dynamic features. It constructs a fiber bundle structure and calibrates the coupling connections through the coupling theory of Riemannian manifolds and fiber bundles to achieve spatiotemporal alignment and fusion of cross-modal features. Generally, the fiber bundle structure includes a bottom manifold, fiber space, and projection mapping to characterize the geometric relationship between infrared and visible light dynamic features.

[0059] Specifically, the base manifold is composed of an infrared characteristic manifold and a visible light characteristic manifold. The infrared characteristic manifold is constructed based on the spatiotemporal gradient tensor and the Riemann curvature tensor of the infrared temperature field. Independent components of the Riemann curvature tensor and the mean value of the infrared temperature field are selected to form a five-dimensional infrared characteristic manifold. In one embodiment, the independent components include R... 1212 R 1313 The visible light feature manifold is constructed based on polarization texture fusion features and surface normal vector field. Stokes parameters I, Q, U and the first component of the directional gradient histogram features are selected to form a five-dimensional visible light feature manifold. The two are then combined to form the base manifold M = M1 × M2, realizing the geometric space mapping of cross-modal features.

[0060] As an alternative, the fiber space is the feature subspace corresponding to each point on the base manifold. The maximum component of the Riemann curvature tensor, the energy of the histogram of directional gradients, and the z-component of the surface normal vector are input from the dynamic features to construct the feature dimensions of the fiber space. For example, the maximum component representing the bending characteristics of hot spots in the Riemann curvature tensor is extracted, the energy of the histogram of directional gradients, the normalized sum of the histograms within the block unit, and the vertical component of the surface normal vector are calculated. These three are used as primitives in the fiber space to achieve a hierarchical representation of dynamic features, providing local feature support for cross-modal fusion.

[0061] In the calibration of the coupling connection, the weights are dynamically adjusted based on the hotspot temperature difference. Specifically, the hotspot temperature difference is the difference between the pixel temperature and the average temperature of the photovoltaic module in the infrared temperature field, and the weight calculation formula is as follows: Where ω1 is the weight of the infrared characteristic manifold, β is the temperature sensitivity coefficient, and ΔT is the hot spot temperature difference. Inputting the hot spot temperature difference ΔT and a preset β value, the weight allocation of the infrared and visible light characteristic manifolds is calculated through experimental calibration: when the hot spot temperature difference is large, the weight of the infrared characteristic manifold is increased, enhancing the contribution of temperature information to the fusion; when the temperature difference is small, the weights of the two are balanced to adapt to the characteristic differences at different stages of hot spot evolution.

[0062] In one possible implementation, a connection equation is used to describe the spatiotemporal mapping relationship between infrared and visible light feature manifolds. Combined with weight allocation, cross-modal features are spatiotemporally aligned. For example, for dynamic features in consecutive frames, the connection equation is used to correct phase shifts in the temporal dimension, such as the temporal synchronization of hotspot temperature changes and texture changes. At the same time, pixel positions are matched in the spatial dimension, such as the spatial correspondence between infrared hotspot regions and visible light structural defects. This ensures the consistency of fused features in the spatiotemporal domain, providing accurate fused features for subsequent dynamic detection decisions.

[0063] Dynamic detection decision: Based on fused features, hot spot clustering and Bayesian determination are performed through mean-shift clustering and dynamic Bayesian inference to obtain hot spot candidate regions. The temperature characteristics of the hot spot candidate regions are compared with preset dynamic thresholds to obtain detection results.

[0064] Furthermore, in the dynamic detection decision-making step, mean-shift clustering is used to cluster hot spot candidate regions by iteratively finding density peaks in the fused feature space. Dynamic Bayesian inference is used to calculate the posterior probability of hot spot existence based on the prior probability and likelihood function of the fused features for Bayesian determination. The prior probability is based on historical hot spot data statistics. The comparison is to numerically compare the temperature characteristics of the hot spot candidate region with the dynamic threshold. When the temperature characteristics exceed the dynamic threshold, it is a hot spot defect. The dynamic threshold is updated in real time based on the temperature change rate of the hot spot and environmental parameters.

[0065] Furthermore, it is updated in real time. Where τ(t) is the dynamic threshold at time t, and 65 is the reference temperature threshold. Let τ be the rate of temperature change, and α(τ) = 0.05·exp(-0.05·wind) be the wind speed correction factor, where wind is the wind speed.

[0066] Specifically, the dynamic detection decision-making process is based on the fused features obtained from coupling alignment, and achieves hot spot defect determination through mean-shift clustering, dynamic Bayesian inference, and dynamic threshold comparison.

[0067] Generally, mean-shift clustering takes a fusion feature as input, which integrates multimodal information such as infrared Riemann curvature and visible light polarization texture. Specifically, iteratively calculating the drift vector of sample points in the fusion feature space converges to the density peak, and clustering yields candidate hotspot regions. For example, inputting a fusion feature containing infrared curvature components and HOG energy, the iterative output yields a set of spatially continuous and feature-homogeneous candidate regions, completing the initial screening of hotspots.

[0068] In dynamic Bayesian inference, the prior probability is derived from historical hotspot data statistics, and the likelihood function characterizes the probabilistic relationship between the fused features and the hotspot. In one possible implementation, the candidate region fusion features, pre-stored priors, and a likelihood model are input, and the posterior probability is calculated. For example, when there are infrared curvature components or HOG energy anomalies, the posterior probability approaches 1, quantifying the existence of the hotspot.

[0069] Dynamic threshold updates rely on the rate of temperature change and environmental parameters, using a formula. in, Extracted from the infrared temperature field, α(τ) = 0.05·exp(-0.05·wind), where wind is taken from the environmental parameters acquired through multimodal data collection. For example, when the wind speed is 4 m / s, α(τ) = 0.05·exp(-0.02) is calculated, and the threshold is updated by integrating the temperature change rate. The temperature characteristics of the candidate region are compared with the threshold; if the value exceeds the threshold, it is identified as a hot spot, and the detection result is output.

[0070] By using mean-shift clustering to locate regions, dynamic Bayesian inference to enhance robustness, and dynamic thresholding to adapt to dynamic changes, the three work together to achieve accurate hotspot detection.

[0071] By simultaneously acquiring raw infrared temperature data, visible light polarization images, and environmental parameters of photovoltaic modules, a multimodal data foundation is constructed. Preprocessing is performed using a radiative transfer model and polarization optics principles to eliminate interference from atmospheric attenuation, ambient temperature and humidity, and uneven illumination and fog effects on visible light images, ensuring data accuracy and reliability. Based on differential geometry and computer vision algorithms, dynamic features such as the spatiotemporal gradient tensor and Riemann curvature tensor of the infrared temperature field, as well as structural features such as polarization texture fusion features and surface normal fields of the visible light images, are extracted from the preprocessed data to achieve in-depth characterization of hot spot thermal evolution and structural defects. Finally, fiber bundle structures are constructed and coupling is calibrated using Riemannian manifold and fiber bundle coupling theory. This network addresses the cross-modal heterogeneity of infrared and visible light features, achieving precise spatiotemporal alignment and fusion to form fused features that take into account both thermal and structural information. Based on these fused features, mean-shift clustering, dynamic Bayesian inference, and dynamic thresholds combining temperature change rate and environmental parameters are used to adaptively identify hot spot candidate regions. This effectively distinguishes between real hot spots and interferences such as shadow occlusion and temporary heating, thereby establishing a multi-dimensional correlation analysis mechanism for hot spot defects, thermal anomalies, structural defects, and environmental adaptation. This enhances the adaptability to complex scenarios and enables the detection of hot spot defects. It solves the problems of insufficient robustness and limited accuracy in traditional component hot spot defect detection caused by single temperature judgment and failure to consider environmental interference and structural information.

[0072] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for detecting hot spot defects in components using infrared and visible light fusion, characterized in that, Includes the following steps: Multimodal acquisition: Simultaneously acquire multimodal data of photovoltaic modules, including raw infrared temperature data, visible light polarization images, and environmental parameters; Data preprocessing: Based on the radiative transfer model and the principle of polarization optics, the multimodal data is preprocessed to obtain preprocessed data including infrared temperature field and visible light image; Dynamic feature extraction: Based on differential geometry and computer vision algorithms, the spatiotemporal gradient tensor and Riemann curvature tensor of the infrared temperature field, as well as the polarization texture fusion features and surface normal vector field of the visible light image are extracted from the preprocessed data as dynamic features. Coupling Alignment: Based on dynamic features, fiber bundle structures are constructed and coupling connections are calibrated through the coupling theory of Riemannian manifolds and fiber bundles. Spatiotemporal alignment and fusion of cross-modal features are then performed to obtain fused features. Dynamic detection decision: Based on fused features, hot spot clustering and Bayesian determination are performed through mean-shift clustering and dynamic Bayesian inference to obtain hot spot candidate regions. The temperature characteristics of the hot spot candidate regions are compared with preset dynamic thresholds to obtain detection results.

2. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: In the multimodal acquisition step, the acquisition equipment includes an infrared camera, a visible light camera, an irradiance meter, and a weather station, and the environmental parameters include irradiance, wind speed, temperature, and humidity.

3. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: The data preprocessing steps include radiometric calibration and atmospheric attenuation correction of the raw infrared temperature data in the multimodal data, and multi-scale enhancement and polarization dehazing of the visible light polarization image in the multimodal data. The radiometric calibration is based on blackbody furnace calibration data and fits the mapping relationship between gray values ​​and temperature through a quadratic function. The multi-scale enhancement is to suppress illumination unevenness by convolving the multi-scale Gaussian kernel with the visible light polarization image. The polarization dehazing is based on the polarization optical characteristics to separate scene radiation and atmospheric scattered light.

4. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 3, characterized in that: The quadratic function is T = a·DN 2 +b·DN+c, where T is the corrected temperature, DN is the original infrared grayscale value, and a, b, and c are calibration coefficients. The atmospheric attenuation correction is based on the atmospheric transmittance and ambient temperature to calculate the true temperature, and its correction formula is T. true =T meas +τ·(T atm -T meas ), where T true T represents the actual surface temperature of the component. meas To measure temperature, τ is atmospheric transmittance, and T is... atm The ambient temperature.

5. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: In the dynamic feature extraction step, the differential geometry algorithm includes calculating the spatial gradient of the infrared temperature field using gradient operators, calculating the temperature time change rate using the time difference method, constructing a spatiotemporal gradient tensor by combining second-order partial derivatives, and calculating the Riemann curvature tensor based on the metric tensor of the spatiotemporal gradient tensor. The computer vision algorithm includes calculating Stokes parameters by multi-polarization direction light intensity, extracting directional gradient histogram features, and estimating the surface normal vector field using photometric stereo vision methods.

6. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: In the dynamic feature extraction step, the polarization texture fusion feature includes Stokes parameters and directional gradient histogram features. The Stokes parameters are calculated by the light intensity in the polarization direction, and the surface normal vector field is the distribution of unit normal vectors on the component surface estimated by a stereo vision algorithm based on the brightness changes under multiple illumination directions.

7. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: In the coupling alignment step, the Riemannian manifold and fiber bundle coupling theory includes constructing a fiber bundle structure of a base manifold, fiber space, and projection mapping, and realizing the spatiotemporal alignment of infrared feature manifolds and visible light feature manifolds through coupling connection equations. The base manifold is composed of infrared feature manifolds and visible light feature manifolds. The infrared feature manifold is constructed based on the spatiotemporal gradient tensor of the infrared temperature field and the Riemann curvature tensor. The visible light feature manifold is constructed based on polarization texture fusion features and surface normal vector field. The fiber space is the feature subspace corresponding to each point on the base manifold, containing the feature components of the Riemann curvature tensor, the feature energy of the orientation gradient histogram, and the components of the surface normal vector.

8. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: In the coupling alignment step, the weights of the coupling links are dynamically adjusted based on the hotspot temperature difference, which is the difference between the pixel temperature and the average temperature of the photovoltaic module in the infrared temperature field. The weight calculation formula is as follows: Where ω1 is the weight of the infrared characteristic manifold, β is the temperature sensitivity coefficient, and ΔT is the hot spot temperature difference.

9. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 1, characterized in that: In the dynamic detection decision-making step, the mean-shift clustering is to cluster hot spot candidate regions by iteratively finding the density peak in the fused feature space. The dynamic Bayesian inference is to calculate the posterior probability of hot spot existence based on the prior probability and likelihood function of the fused features for Bayesian determination. The prior probability is based on historical hot spot data statistics. The comparison is to numerically compare the temperature characteristics of the hot spot candidate region with a dynamic threshold. When the temperature characteristics exceed the dynamic threshold, it is a hot spot defect. The dynamic threshold is updated in real time based on the temperature change rate of the hot spot and environmental parameters.

10. The method for detecting hot spot defects in components using infrared and visible light fusion according to claim 9, characterized in that: The real-time update is Where τ(t) is the dynamic threshold at time t, and 65 is the reference temperature threshold. Let τ be the rate of temperature change, and α(τ) = 0.05·exp(-0.05·wind) be the wind speed correction factor, where wind is the wind speed.

Images