A multi-modal fusion mountain photovoltaic module thermal anomaly intelligent detection positioning method

Abstract

The present application relates to the technical field of photovoltaic power station operation and maintenance inspection, and discloses a multi-modal fusion mountain photovoltaic module thermal anomaly intelligent detection positioning method, which synchronously collects the unmanned aerial vehicle pose and double light images at the shutter trigger time, extracts thermal anomaly pixel features and actual temperature, extracts adjacent component corner features based on the support topology when the field of view is truncated, combines the elevation model and the building information model to perform spatial intersection and size optimization, fits the actual physical plane equation and normal vector of the target component, calculates the solar incident vector, calculates the effective irradiance coefficient to generate the terrain compensation dynamic temperature threshold, compares the actual temperature to output the abnormality determination result, reconstructs the thermal spot space ray according to the determination result, solves the ray and the actual physical plane equation, and obtains the three-dimensional absolute physical coordinates of the thermal anomaly area. The present application overcomes the size reference loss limitation caused by the field of view truncation, eliminates the thermal radiation interference caused by the terrain illumination, and improves the accuracy and integrity of the mountain thermal spot three-dimensional positioning.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power plant operation and maintenance inspection technology, specifically a multimodal fusion intelligent detection and location method for thermal anomalies in mountain photovoltaic modules. Background Technology

[0002] Due to the undulating terrain, manual inspection of mountain photovoltaic power stations is extremely difficult. Currently, automated inspections are mostly conducted using drones equipped with both infrared and visible light cameras to detect thermal anomalies such as hot spots on the modules. However, existing drone inspection and positioning methods still have many limitations in the complex mountainous environments of actual conditions.

[0003] In the defect spatial localization stage, the variable gusts of wind in mountainous environments can easily cause slight deformations in UAVs and gimbals, introducing transient attitude measurement errors. Simultaneously, due to limitations in flight paths and camera shooting angles, target photovoltaic modules located at the edge of the image often experience field-of-view truncation. Existing spatial mapping algorithms typically rely on the complete physical boundary features of the photovoltaic modules to establish geometric perspective relationships and impose dimensional constraints. Once field-of-view truncation occurs, the underlying algorithm loses the physical size scale reference, making it difficult to fit an accurate 3D physical plane of the module. This results in the subsequent construction of hot spot rays failing to achieve effective spatial mathematical intersection, leading to failure of 3D absolute coordinate mapping or significantly larger positioning errors.

[0004] In the thermal anomaly detection stage, traditional image recognition algorithms mainly rely on fixed absolute temperature thresholds or simple local temperature gradients as the basis for judgment. Mountainous photovoltaic modules are affected by complex terrain undulations, resulting in spatial differences in their actual installation tilt and azimuth angles, leading to varying solar radiation power received by different module surfaces. During low light periods in the early morning and late evening, or when modules are not directly facing the light source, the photothermal conversion effect weakens, and the temperature rise caused by actual hot spot defects is relatively low, often failing to reach the fixed temperature threshold, thus causing missed defect detection. Existing technologies lack a dynamic benchmark compensation mechanism that can multidimensionally couple the physical orientation of the terrain with transient illumination angles, making it difficult to meet the high-precision defect identification requirements under varying illumination conditions. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a multimodal fusion-based intelligent detection and location method for thermal anomalies in mountainous photovoltaic modules. This method aims to solve the problems inherent in existing technologies, such as the complex terrain and variable lighting conditions in some mountainous photovoltaic power stations, where drone inspections are easily affected by gusts and limited viewing angles. This can lead to field-of-view truncation of target photovoltaic modules in images, resulting in the loss of physical size references and causing failure to spatially locate hotspot defects or excessive absolute coordinate mapping errors. Furthermore, the complex terrain and lighting conditions make fixed temperature thresholds unsuitable for adapting to changes in actual light radiation power, easily leading to missed or false hotspot detections under weak light or non-direct lighting conditions.

[0006] To address the above problems, the present invention provides the following technical solution: The first aspect of this invention provides a multimodal fusion-based intelligent detection and positioning method for thermal anomalies in mountainous photovoltaic modules, comprising the following steps: simultaneously acquiring UAV pose data, infrared images, and visible light images at the shutter trigger moment; extracting initial thermal anomaly pixel features and their associated actual measured temperature values; and extracting pixel corner features of adjacent complete modules based on the topological constraints of the same-source support when the target photovoltaic module experiences field-of-view truncation; performing spatial mathematical intersection and size constraint optimization based on the pixel corner features of adjacent complete modules and the initial rotation matrix, combined with digital elevation model data and building information model data, and fitting and outputting the actual physical plane equation of the target photovoltaic module and its corresponding actual normal vector; solving the solar incident three-dimensional vector based on the acquisition moment, calculating the inner product cosine parameter of the actual normal vector and the solar incident three-dimensional vector to obtain the effective irradiance coefficient, thereby generating a dynamic temperature threshold with terrain compensation attributes, and comparing the dynamic temperature threshold with the actual measured temperature value to output an anomaly judgment result; reconstructing the hot spot spatial ray based on the anomaly judgment result by calling the optimized rotation matrix, and simultaneously solving the hot spot spatial ray with the actual physical plane equation to obtain the three-dimensional absolute physical coordinates of the thermal anomaly region of the target photovoltaic module. The solution addresses the limitation of physical scale loss caused by local field-of-view truncation by synchronously acquiring multimodal data and using topological constraints of the same support structure for feature compensation when field of view truncation occurs. Nonlinear optimization and plane fitting are performed by combining digital elevation model and building information model to output the actual physical plane equation and normal vector, providing a high-precision foundation for subsequent spatial calculations. By introducing terrain-light coupled thermal benchmark dynamic compensation logic, the interference of variable terrain and transient lighting angles on the thermal radiation characteristics of the components is eliminated. Utilizing the coplanar physical property of the photovoltaic array support, the equations of the hot spot spatial rays and the actual physical plane (equivalent to infinite extension) are mathematically combined to find their intersection, overcoming the blind spot of 3D projection positioning and achieving accurate analysis and output of 3D absolute physical coordinates.

[0007] Furthermore, at the shutter trigger moment, UAV pose data, infrared images, and visible light images are simultaneously acquired. Initial thermal anomaly pixel features and their associated actual measured temperature values ​​are extracted. This includes: calculating the difference between the hardware capture timestamp of the infrared image and the hardware capture timestamp of the visible light image, and determining multi-source parameter synchronization validity when the difference is less than or equal to the maximum allowable synchronization time error threshold; calculating the average temperature and standard deviation of all pixels in the infrared image, and constructing a temperature screening mathematical model based on the average temperature, standard deviation, and anomaly screening coefficient when the standard deviation is greater than or equal to a preset background uniformity threshold; and identifying all two-dimensional pixel coordinates conforming to the temperature screening mathematical model as initial thermal anomaly pixel features. The steps include setting timestamp constraints for multi-source parameters to ensure strict alignment of multimodal data in the time dimension to prevent motion blur distortion; utilizing thermodynamic statistical distribution characteristics, an adaptive screening model is constructed using the pixel temperature mean and standard deviation to dynamically extract valid initial thermal anomaly pixel locations based on real-time changes in the environmental background temperature distribution.

[0008] Furthermore, when the target photovoltaic module experiences field-of-view truncation, pixel corner features of adjacent complete modules are extracted based on the topological constraints of the same-origin support structure. This includes: counting the number of physical boundary pixels of the target photovoltaic module falling within the infrared image edge buffer, and calculating the ratio of this number to the total number of physical boundary pixels to construct a boundary truncation rate; when the boundary truncation rate is greater than a preset truncation threshold, based on the physical support topology connection tree pre-stored in the building information model data, neighboring photovoltaic modules belonging to the same physical support are retrieved in the infrared image along the boundaryless extension direction of the long side of the target photovoltaic module; when the boundary truncation rate of neighboring photovoltaic modules is equal to zero, the neighboring photovoltaic modules are determined as adjacent complete modules and pixel corner features of the adjacent complete modules are extracted. The steps implement topological constraint-based anti-truncation feature compensation. When the target photovoltaic module experiences image edge truncation due to limited shooting angle, corner features of coplanar adjacent complete modules are retrieved and extracted along the support topology direction, maintaining the reference geometric input required for plane fitting and preventing the positioning process from being interrupted due to missing reference.

[0009] Furthermore, spatial mathematical intersection and dimensional constraint optimization are performed by combining digital elevation model (DEM) data and building information model (BIM) data. This includes: converting the two-dimensional pixel coordinates corresponding to the corner features of adjacent complete components into three-dimensional direction vectors in the camera coordinate system; constructing a three-dimensional spatial ray with the camera optical center as the starting point and the three-dimensional direction vector as the direction; mathematically intersecting the three-dimensional spatial ray with the DEM data to obtain the initial altitude; and calling the design installation elevation parameters of the target photovoltaic module stored in the BIM data; performing linear superposition algebraic sum calculations on the initial altitude and the design installation elevation parameters to complete the elevation correction, outputting a primary three-dimensional spatial coordinate set with vertical height compensation. The steps utilize the DEM to perform basic terrain mapping intersection to obtain the macroscopic surface undulation state, superimposing the design elevation from the BIM to complete mechanical installation height difference compensation, achieving a real physical environment fusion correction of the spatial elevation coordinate parameters.

[0010] Furthermore, after outputting the initial 3D spatial coordinate set with vertical height compensation, the process includes: calculating the Euclidean spatial distance between adjacent pixel corner points in the 3D physical world using the initial 3D spatial coordinate set; constructing a nonlinear objective optimization function with the roll angle, pitch angle, and yaw angle of the initial rotation matrix as independent variables and the sum of squared differences between the Euclidean spatial distance and the preset standard physical dimensions in the building information model data as dependent variables; using the Levenburg-Marquardt algorithm to iteratively solve the nonlinear objective optimization function using gradient descent, and transforming the converged attitude parameter vector into the optimized rotation matrix after satisfying the iteration termination condition. This step applies mathematical constraints based on preset standard physical dimensions, iteratively calibrates the camera's transient pose using the Levenburg-Marquardt algorithm, eliminates transient mechanical measurement errors caused by the small deformation of the gimbal damper due to gusts of wind affecting the UAV, and outputs a high-precision rotation matrix converged under physical constraints.

[0011] Furthermore, after transforming the converged attitude parameter vector into an optimized rotation matrix, the process includes: re-performing spatial ray construction and elevation intersection calculations on the pixel corner features of all adjacent complete components using the optimized rotation matrix to obtain the target spatial coordinate set; using a random sampling consensus algorithm to perform interior point screening on the target spatial coordinate set, dividing coordinate points with orthogonal spatial distances less than the numerical judgment criteria into interior point sets, and retaining the optimal interior point set with the most interior points after multiple random iterations; using principal component analysis least squares to perform 3D surface approximation fitting on the optimal interior point set, outputting the actual physical plane equation and extracting the normal feature vector of the actual physical plane equation as the actual normal vector. This step utilizes a random sampling consensus algorithm for interior point spatial screening, eliminating outlier isolated coordinate data caused by local high-intensity reflection or optical blurring, ensuring that principal component analysis least squares can perform surface approximation based on the effective interior point set, and improving the structural robustness of the coplanar constraint mathematical model and the accuracy of normal vector calculation.

[0012] Furthermore, the solar incidence three-dimensional vector is calculated based on the acquisition time, including: calculating the solar altitude angle and solar azimuth angle at the corresponding time based on the shooting timestamp and longitude and latitude values; when the solar altitude angle value is greater than or equal to the preset lower limit of the effective illumination angle, the solar altitude angle and solar azimuth angle are used to construct the solar incidence three-dimensional vector with the target photovoltaic module as the origin, mapping the transient astronomical parameters of the spherical coordinate system to a standardized direction vector in the Cartesian three-dimensional rectangular coordinate system. This step utilizes spatial location and time parameters to perform high-precision solar trajectory extrapolation, transforming astronomical angles into standardized three-dimensional direction vectors in the Cartesian coordinate system, providing a pre-associative vector basis for performing geometric inner product operations with the normal plane of the photovoltaic module.

[0013] Furthermore, the cosine parameter of the inner product of the actual normal vector and the three-dimensional incident solar vector is calculated to obtain the effective irradiance coefficient. Based on this, a dynamic temperature threshold with terrain compensation attributes is generated. This includes: performing a vector inner product operation between the actual normal vector and the three-dimensional incident solar vector to obtain the effective irradiance coefficient; when the effective irradiance coefficient is greater than zero, integrating real-time ambient temperature, the inherent photothermal response coefficient of the target photovoltaic module, the theoretical irradiance, and the defect temperature rise tolerance to perform physical dimension parameter coupling calculations to generate a dynamic temperature threshold that includes terrain attitude compensation. This step extracts the light radiation power coefficient corresponding to the actual light-receiving attitude of the photovoltaic module based on Lambert's cosine law, and couples and extrapolates multi-dimensional parameters to generate a dynamic temperature benchmark that compensates for differences in terrain and light-receiving angle. This reduces the interference phenomenon of numerical attenuation of module thermal excitation under non-positive light-receiving conditions, significantly improving the accuracy of identifying hidden hot spot defects.

[0014] Furthermore, the dynamic temperature threshold is compared with the actual measured temperature value to output anomaly judgment results. This includes: when the actual measured temperature value is greater than the dynamic temperature threshold, the initial thermal anomaly pixel corresponding to the actual measured temperature value is initially determined to be a suspected defect point; the number of suspected defect points interconnected in the eight-neighbor space is counted as the connected region area; when the connected region area is greater than or equal to a preset area lower limit, the connected suspected defect points are confirmed as valid hot spot defects, and a final judgment result containing the two-dimensional coordinates of the valid hot spot defects is generated as the anomaly judgment result. After completing the temperature amplitude comparison, the step introduces spatial morphological constraint calculation, counts and verifies the connected area index of adjacent pixel sets, accurately isolates and eliminates random shot thermal noise generated by the infrared imaging sensor, and prevents false alarms caused by single-pixel thermodynamic disturbances.

[0015] Furthermore, based on the anomaly determination results, the optimized rotation matrix is ​​used to reconstruct the hot spot spatial ray. The hot spot spatial ray is then solved simultaneously with the actual physical plane equation to obtain the three-dimensional absolute physical coordinates of the target photovoltaic module's thermal anomaly region. This includes: converting the two-dimensional coordinates of the effective hot spot defect in the anomaly determination results into local three-dimensional direction vectors in the camera coordinate system using the camera intrinsic parameter matrix, and performing spatial rotation coordinate transformation on the local three-dimensional direction vectors using the optimized rotation matrix to obtain the absolute three-dimensional direction vectors in the world coordinate system; reconstructing the hot spot spatial ray equation starting from the absolute coordinates of the camera optical center and extending along the absolute three-dimensional direction vector; calculating the vector dot product of the actual normal vector and the absolute three-dimensional direction vector; when the absolute value of the vector dot product is greater than or equal to the zero-approach physical truncation threshold, solving the hot spot spatial ray equation and the actual physical plane equation, which is equivalent to an unbounded plane, to calculate the extension parameter solution; and substituting the extension parameter solution back into the hot spot spatial ray equation to calculate the three-dimensional absolute physical coordinates. This step relies on the physical characteristics of coplanar installation of photovoltaic supports from the same source, extending the actual physical plane infinitely in all directions as the a priori region of the unbounded plane. The hot spot rays that diverge outward from the edge of the image due to the field of view truncation can generate a unique mathematical intersection point with the actual physical plane equation of the virtual extension under this logic, avoiding matrix divergence by division by zero, and completing a high-precision, blind-zone-free positioning mapping from two-dimensional pixel space to three-dimensional absolute geographic space.

[0016] This invention provides a multimodal fusion-based intelligent detection and location method for thermal anomalies in mountainous photovoltaic modules. It offers the following advantages:

[0017] 1. Improved positioning integrity under field-of-view truncation conditions. When the target photovoltaic module experiences field-of-view truncation, this invention utilizes topological constraints of the same-origin support to extract pixel corner features of adjacent complete modules. Furthermore, leveraging the physical properties of the coplanar mounting of the photovoltaic array support, it simultaneously solves the equations of the reconstructed hotspot spatial ray and the equivalent outwardly extending actual physical plane. This method solves the problem of lost physical boundaries of components within the image due to the limited shooting angle of UAVs, ensuring that the hotspot ray at the imaging edge can produce a definite mathematical intersection with the virtually extended plane, avoiding interruptions in the three-dimensional absolute coordinate calculation process.

[0018] 2. Improved accuracy of hot spot defect identification in complex mountainous environments. This invention calculates the inner product of the actual normal vector and the three-dimensional incident solar vector to obtain the effective irradiance coefficient, and integrates ambient temperature and photothermal response parameters to generate a dynamic temperature threshold with terrain compensation attributes. This method, combined with the cosine law of light reception, quantitatively corrects the impact of different mountain slopes and solar eclipse angles on the actual light radiation power on the photovoltaic module surface, compensates for the thermal excitation attenuation characteristics of the module caused by non-direct light reception posture, and reduces the probability of missed or false alarms of defects caused by using a fixed temperature threshold under varying illumination conditions.

[0019] 3. Enhanced accuracy and noise resistance of 3D spatial mapping under transient disturbance conditions. This invention utilizes standard physical dimensions from the Building Information Model (BIM) to construct a nonlinear objective optimization function for iterative calibration of the initial rotation matrix. It employs a random sampling consensus algorithm to filter interior points in the spatial coordinates, and then uses principal component analysis and least squares to fit the actual physical plane equations. This method eliminates transient mechanical measurement errors introduced by UAV gusts and gimbal deformation, removes outlier coordinate points caused by local optical blurring, and provides a reliable 3D geometric plane reference for high-precision spatial ray intersection. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the system architecture of the present invention; Figure 2 This is a flowchart of the present invention. Detailed Implementation

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] See attached document Figures 1 to 2This method provides a multimodal fusion-based intelligent detection and location approach for thermal anomalies in mountainous photovoltaic modules. It relies on a drone platform equipped with a real-time dynamic differential positioning terminal and a dual-beam coaxial gimbal camera to perform front-end data acquisition, and utilizes computing equipment pre-loaded with digital elevation model (DEM) and building information model (BIM) data to perform data parsing and algorithm processing. The entire process is achieved through the collaborative operation of multiple data processing modules built in the background, enabling the transfer and calculation of multimodal parameters.

[0023] The overall system architecture of this method mainly includes a data synchronization module 100, a plane fitting module 200, a benchmark determination module 300, and a ray positioning module 400.

[0024] The data synchronization module 100 simultaneously acquires visible light images, infrared images, the three-dimensional spatial coordinates of the camera's optical center, and the three-axis attitude angles of the gimbal at the moment the UAV shutter is triggered. The data synchronization module 100 extracts the features of thermal anomaly pixels and their corresponding temperature parameters from the infrared images, and extracts multiple pixel corner features of adjacent complete modules based on the topological constraints of the homogeneous support when the target photovoltaic module is truncated due to the image edge.

[0025] The plane fitting module 200 receives pixel corner features and the initial rotation matrix corresponding to the three-axis attitude angles of the gimbal from the data synchronization module 100. The plane fitting module 200 combines the digital elevation model data to construct three-dimensional spatial rays and performs mathematical intersection to obtain the initial spatial coordinates. The plane fitting module 200 retrieves the standard physical dimensions of the components from the building information model data, performs nonlinear optimization of the initial rotation matrix with dimensional deviations as constraints, and performs elevation compensation and plane fitting operations on the initial spatial coordinates, solving and outputting the actual physical plane equation and corresponding actual normal vector of the target photovoltaic module.

[0026] The benchmark determination module 300 reads the actual normal vector output by the plane fitting module 200 and calculates the corresponding three-dimensional solar incidence vector based on the acquisition timestamp and location information. The benchmark determination module 300 calculates the cosine parameter of the inner product of the actual normal vector and the three-dimensional solar incidence vector to obtain the local effective irradiance coefficient. Based on the ambient temperature constant and the effective irradiance coefficient, the benchmark determination module 300 generates a dynamic temperature threshold with terrain compensation attributes, compares the dynamic temperature threshold with the temperature parameters extracted by the data synchronization module 100, and outputs a determination result indicating whether the target photovoltaic module exhibits any abnormalities.

[0027] After receiving the judgment result output by the benchmark judgment module 300, the ray positioning module 400 calls the optimized rotation matrix output by the plane fitting module 200 to construct a single-frame transient field of view direction vector pointing to the thermal anomaly pixel and the corresponding hot spot spatial ray. The ray positioning module 400 performs a simultaneous mathematical solution to the hot spot spatial ray and the actual physical plane equation output by the plane fitting module 200, and calculates the spatial intersection of the rays as the three-dimensional absolute physical coordinates of the thermal anomaly area of ​​the target photovoltaic module, and sends the three-dimensional absolute physical coordinates to an external system for maintenance identification.

[0028] This invention provides a multimodal fusion-based intelligent detection and location method for thermal anomalies in mountain photovoltaic modules, comprising the following steps: The S100 simultaneously acquires UAV pose data, infrared images, and visible light images at the shutter trigger moment, extracts the features of thermally abnormal pixels and their temperature parameters, and extracts the pixel corner features of adjacent complete components based on the topological constraints of the same-source bracket when the field of view of the target photovoltaic module is truncated. S200, based on pixel corner features and initial rotation matrix, combines digital elevation model data and building information model data to perform spatial mathematical intersection and size constraint optimization, and fits and outputs the actual physical plane equation of the target photovoltaic module and its corresponding actual normal vector. S300 calculates the solar incident three-dimensional vector at the acquisition time, calculates the inner product cosine parameter of the actual normal vector and the solar incident three-dimensional vector to obtain the effective irradiance coefficient, generates a dynamic temperature threshold based on this and compares it with the temperature parameter, and outputs the anomaly judgment result. S400, based on the anomaly determination result, calls the optimized rotation matrix to reconstruct the hot spot spatial ray, and solves the hot spot spatial ray and the actual physical plane equation simultaneously to obtain the three-dimensional absolute physical coordinates of the target photovoltaic module's thermal anomaly region.

[0029] The following will provide a detailed explanation of the specific technical implementation details and formula derivation process of each of the above methods and steps.

[0030] In the overall system architecture, the data synchronization module 100 performs multimodal transient data synchronous acquisition and topology feature extraction operations. The multimodal transient data synchronous acquisition and topology feature extraction operations provide a reliable physical data foundation and feature index for subsequent plane fitting, specifically including steps S101 and S102.

[0031] S101, performing multi-source parameter synchronous acquisition and basic feature extraction operations. The data synchronization module 100 receives hardware synchronization pulse signals sent by the UAV main control board to achieve synchronous triggering of multi-source parameters. To prevent hardware bus communication blockage from causing the program to hang indefinitely, the data synchronization module 100 is configured with a timeout watchdog mechanism. If the hardware synchronization pulse signal is not received within ten milliseconds beyond the preset timeout limit, the data synchronization module 100 automatically resets the bus communication interface and discards the current acquisition process. Under normal receiving conditions, the data synchronization module 100 synchronously acquires the infrared image, visible light image, spatial coordinates, and initial rotation matrix at the shutter trigger moment. To ensure effective alignment of multi-source parameters in the time dimension, the data synchronization module 100 calculates the difference between the hardware acquisition timestamp of the infrared image and the hardware acquisition timestamp of the visible light image. The mathematical model of the timestamp constraint satisfied by multi-source parameter synchronization is as follows. In the formula, For the hardware capture timestamp of the infrared image, For hardware capture timestamps of visible light images, The maximum permissible synchronization time error threshold is set to a range of one to five milliseconds. If the calculated difference exceeds this threshold, the data synchronization module 100 determines that the multi-source parameter synchronization for the current data acquisition cycle has failed and directly discards the infrared and visible light images to prevent motion blur distortion. After timestamp constraints, the data synchronization module 100 extracts thermally anomalous pixels and their corresponding temperature parameters from the infrared image. Based on the thermodynamic anomaly distribution hypothesis, the temperature distribution of normally operating photovoltaic modules typically conforms to the background characteristics under specific conditions. Hot spot defects, as local abnormal heat sources, will significantly increase the temperature statistics in local areas. Therefore, statistical distribution parameters are used for dynamic threshold calculation. To achieve adaptive extraction, the data synchronization module 100 calculates the average temperature and standard deviation of all pixels in the infrared image. The mathematical model for temperature screening of thermally anomalous pixels is as follows. In the formula, This represents the actual measured temperature value corresponding to the two-dimensional pixel coordinates in the infrared image. This represents the average temperature of all pixels in the infrared image. The standard deviation of temperature for all pixels in the infrared image. The anomaly screening coefficient is set to a value range of 2.5 to 3.5. When the calculated temperature standard deviation is less than the preset background uniformity threshold of 0.5 degrees Celsius, it indicates that the infrared image background is too hot or too cold. The data synchronization module 100 automatically switches to an edge segmentation algorithm based on a fixed absolute temperature slope. Specifically, it calculates the temperature difference gradient between adjacent pixels in the infrared image and defines the area where the temperature difference gradient is greater than 3 degrees Celsius per pixel as an anomaly boundary. The temperature difference gradient calculation operation avoids the extraction dead zone caused by the failure of statistical features. The data synchronization module 100 determines all two-dimensional pixel coordinates that meet the temperature screening mathematical model as the initial thermal anomaly pixel features and encapsulates and outputs them.

[0032] S102, perform anti-truncation feature compensation calculation based on topological constraints. Due to changes in the UAV's flight attitude or limitations in the shooting angle, the target photovoltaic module in the infrared or visible light image may have its field of view truncated. After extracting the initial thermal anomaly pixel features, the data synchronization module 100 initiates a boundary integrity self-check process for the target photovoltaic module. The data synchronization module 100 counts the number of physical boundary pixels of the target photovoltaic module that fall within the edge buffer of the infrared image. The edge buffer is set as a strip-shaped area ten to twenty pixels away from the outermost border of the infrared image. The data synchronization module 100 constructs the boundary truncation rate using the ratio of the number of physical boundary pixels falling within the edge buffer to the total number of physical boundary pixels. The mathematical model for calculating the boundary truncation rate is as follows. In the formula, The cutoff ratio of the target photovoltaic module. This represents the number of physical boundary pixels that fall within the edge buffer. The total number of physical boundary pixels of the target photovoltaic module is given. When the boundary truncation rate is greater than the preset truncation threshold, the data synchronization module 100 determines that the target photovoltaic module has truncated its field of view and triggers the anti-truncation feature compensation mechanism. The preset truncation threshold is set to a range of 0.15 to 0.30. If the boundary truncation rate is less than or equal to the preset truncation threshold, the data synchronization module 100 determines that the boundary of the target photovoltaic module is intact and directly outputs the initial thermal anomaly pixel features. After triggering the anti-truncation feature compensation mechanism, the data synchronization module 100 uses the coplanar physical properties of the same support components to search for and extract the pixel corner features of adjacent intact components in the infrared image. The specific search logic is based on the physical support pre-stored in the building information model data. The topology connection tree searches for neighboring photovoltaic modules belonging to the same physical support along the long side of the target photovoltaic module in the infrared image without any boundary extension. When the boundary truncation rate of the neighboring photovoltaic module is zero, the data synchronization module 100 determines that the neighboring photovoltaic module is an adjacent complete module and extracts the pixel corner features of the adjacent complete module. When no adjacent complete module that meets the coplanar topology features is found within the preset image search range, the data synchronization module 100 outputs a truncation anomaly mark and terminates the feature output of this data capture cycle to prevent the introduction of incorrect topology constraints. Finally, the data synchronization module 100 packages the pixel corner features of the adjacent complete module and the initial thermal anomaly pixel features and transmits them to the plane fitting module 200.

[0033] In the overall system architecture, the plane fitting module 200 receives the corner features of adjacent complete component pixels and the initial thermal anomaly pixel features transmitted by the data synchronization module 100 and performs a real three-dimensional plane robust fitting operation based on physical prior and random sampling consistency algorithm. The real three-dimensional plane robust fitting operation provides a high-precision normal vector reference and pose correction basis for subsequent terrain lighting coupled thermal benchmark dynamic compensation, specifically including steps S201, S202 and S203.

[0034] S201, performing spatial ray construction and initial elevation compensation calculations, the plane fitting module 200 retrieves the initial rotation matrix transmitted by the UAV gimbal and the pre-calibrated camera intrinsic parameter matrix, transforming the two-dimensional pixel coordinates corresponding to the corner features of adjacent complete components into three-dimensional direction vectors in the camera coordinate system. The transformation calculation from two-dimensional pixel coordinates to three-dimensional direction vectors can be performed based on the existing camera pinhole imaging model. The plane fitting module 200 constructs a three-dimensional spatial ray projected onto the ground, starting from the camera's optical center and using the three-dimensional direction vector as the direction. Before performing terrain intersection calculations, considering that the three-dimensional spatial ray may not point to the physical ground surface under extreme gust wind disturbances, the plane fitting module 200 extracts the vertical direction numerical components of the three-dimensional direction vector. When the vertical direction numerical component is greater than or equal to zero, the plane fitting module 200 determines that the three-dimensional spatial ray does not intersect with the ground and directly triggers an invalid ray discard instruction to avoid the underlying numerical intersection algorithm falling into an infinite search loop. Under normal mapping conditions where the quantity is less than zero, in order to obtain basic terrain undulation data, the plane fitting module 200 performs mathematical intersection calculations between the three-dimensional spatial ray and the pre-loaded digital elevation model to obtain the initial three-dimensional intersection point coordinates containing longitude and latitude values ​​as well as the initial altitude. When the three-dimensional spatial ray and the digital elevation model cannot successfully intersect due to data loss or field-of-view edge effects, the plane fitting module 200 automatically calls the preset reference surface elevation constant as a compensatory initial altitude to maintain the integrity of the solution process. Since the digital elevation model usually only reflects the macroscopic surface undulations and cannot accurately represent the mechanical installation height difference introduced by the photovoltaic support itself, the plane fitting module 200 calls the design installation elevation parameters of the target photovoltaic module stored in the building information model data, performs linear superposition algebraic sum calculations on the initial altitude and the design installation elevation parameters to complete the vertical elevation correction. The plane fitting module 200 finally encapsulates and outputs a primary three-dimensional spatial coordinate set with vertical height compensation.

[0035] S202, performing dynamic pose correction calculations incorporating prior physical dimensions. Due to the slight deformation of the gimbal damper caused by gusts during UAV flight, transient mechanical measurement errors may exist in the initial rotation matrix. The plane fitting module 200 uses a primary 3D spatial coordinate set to calculate the Euclidean distance between adjacent pixel corner points in the 3D physical world and performs an algebraic comparison between the Euclidean distance and the preset standard physical dimensions in the building information model data. When the number of side lengths extracted from the corner features of adjacent complete component pixels is less than a preset threshold of three, it is determined that the physical dimension constraint is insufficient. The plane fitting module 200 abandons iterative optimization and directly outputs the initial rotation matrix as the optimal rotation matrix. However, when the number of side lengths meets the constraint, the plane fitting module 200 constructs a nonlinear objective optimization function with the roll angle, pitch angle, and yaw angle of the initial rotation matrix as independent variables and the sum of squares of the differences between the Euclidean distance and the standard physical dimensions as the dependent variable. The mathematical model for calculating the nonlinear objective optimization function is as follows. In the formula, Let be the sum of distance residuals represented by the nonlinear objective function. The number of side lengths extracted from the corner features of adjacent complete component pixels. as well as This refers to the 3D spatial coordinate vectors of adjacent pixel corner points reconstructed from the pose parameter vector. Given the preset standard physical dimensions of the target photovoltaic module, in order to solve the relative optimal solution of the nonlinear objective optimization function, the plane fitting module 200 uses the Levenberg-Marquardt algorithm to perform gradient descent iterative solution on the attitude parameter vector. It also supplements the iterative intersection anti-fault logic: in each iteration of the Levenberg-Marquardt algorithm's residual evaluation, the plane fitting module 200 reconstructs the three-dimensional spatial ray using the currently updated attitude parameter vector, and performs mathematical intersection operations between the three-dimensional spatial ray and the digital elevation model to dynamically update the three-dimensional spatial coordinate vectors of adjacent pixel corner points. This achieves a physical topological closed loop for Euclidean distance calculation. The Levenberg-Marquardt algorithm combines the mathematical advantages of the Gauss-Newton method and the gradient descent method to overcome the iterative divergence problem caused by the ill-conditioned Hessian matrix under specific attitudes. The termination condition for iterative solution is set as the residual change between two consecutive iterations being less than 10 to the power of -5 or reaching the maximum number of iterations of 50. When the termination condition for iterative solution is met, the plane fitting module 200 transforms the converged attitude parameter vector into the optimal rotation matrix. In order to prevent the unconstrained mathematical optimization results from violating objective physical reality, the plane fitting module 200 performs mechanical activity limit verification on the converged attitude parameter vector. When the absolute value of the roll angle or pitch angle exceeds the preset physical motion limit of the gimbal of 60 degrees, the plane fitting module 200 determines that the optimization result is a non-physical mathematical artifact and reverts to using the initial rotation matrix. After successfully passing the mechanical activity limit verification, the plane fitting module 200 officially outputs the optimal rotation matrix to suppress the dynamic pose measurement error introduced by the transient deformation of the gimbal to the greatest extent.

[0036] S203, Perform plane fitting and actual normal vector extraction operations. The plane fitting module 200 uses the optimized rotation matrix to re-perform spatial ray construction and elevation intersection calculation on the corner features of all adjacent complete component pixels to obtain a high-precision target spatial coordinate set. Considering that the extracted pixel corners may be affected by local high-intensity reflection or optical blurring during imaging, resulting in isolated outlier noise points, the plane fitting module 200 uses the random sampling consensus algorithm to perform interior point screening logic on the target spatial coordinate set. From the perspective of physical statistics, the random sampling consensus algorithm can isolate effective data that conforms to the geometric constraints of the same plane in a large proportion of noise through iterative trial and error, thereby ensuring the robustness of the fitting model. Specifically, three spatial coordinate points are randomly selected from the target spatial coordinate set to calculate and generate an initial hypothetical physical plane. The plane fitting module 200 calculates the orthogonal spatial distance from all remaining coordinate points in the target spatial coordinate set to the initial hypothetical physical plane, and divides the coordinate points with an orthogonal spatial distance of less than 0.05 meters into the interior point set. The numerical judgment condition for the orthogonal spatial distance is set as follows. The 0.05-meter value is determined by combining the statistical physical upper limit of the manufacturing tolerance of photovoltaic modules and the on-site mechanical assembly error, thus avoiding the theoretical value setting from deviating from engineering reality. After multiple random iterations, the plane fitting module 200 retains the optimal set of interior points with the most interior points to eliminate obvious outlier coordinate data. When the number of interior points in the final retained optimal set is less than four (the preset lower limit of the number of interior points), the plane fitting confidence is deemed insufficient. The plane fitting module 200 triggers a fitting failure warning and forcibly assigns the actual normal vector of the target photovoltaic module to the theoretical benchmark normal vector of the design installation tilt angle and azimuth angle of the corresponding support array of the target photovoltaic module, which is pre-stored in the building information model data. When the number of interior points is sufficient, the plane fitting module 200 uses the principal component analysis least squares method to perform the final three-dimensional surface approximation fitting for the selected optimal set of interior points. The plane fitting module 200 finally outputs the actual physical plane equation representing the actual spatial light-receiving posture of the target photovoltaic module and extracts the normal feature vector of the actual physical plane equation as the actual normal vector of the target photovoltaic module, which is then passed to the benchmark judgment module 300.

[0037] In the overall system architecture, the benchmark determination module 300 receives the actual normal vector of the target photovoltaic module transmitted by the plane fitting module 200 and performs terrain-light coupled thermal benchmark dynamic compensation and anomaly determination operation. The terrain-light coupled thermal benchmark dynamic compensation and anomaly determination operation aims to eliminate the interference of variable terrain and transient illumination angle on the thermal radiation characteristics of photovoltaic modules, thereby improving the effectiveness and accuracy of hot spot defect identification. Specifically, it includes steps S301, S302 and S303.

[0038] S301, performing transient astronomical parameter and incident vector calculations, the reference determination module 300 retrieves the shooting timestamp, longitude, and latitude values ​​synchronously captured by the data synchronization module 100. To prevent invalid position parameters due to transient loss of the Global Positioning System signal, the reference determination module 300 performs data validity verification when acquiring longitude and latitude values. When the longitude and latitude values ​​are empty or exceed a reasonable geographical range, the reference determination module 300 calls the previously successfully captured valid position parameters combined with the aircraft's inertial navigation calculation data as compensatory input. After ensuring the validity of the position parameters, the reference determination module 300 calculates the solar altitude angle and solar azimuth angle extracted by the synchronous capture operation based on the shooting timestamp and longitude and latitude values. This calculation is used to solve the problem using the timestamp and position coordinates. The calculation of the solar altitude angle and solar azimuth angle can be performed using existing high-precision solar position calculation algorithms. Considering that photovoltaic modules cannot generate a significant thermodynamic temperature rise difference under low light conditions, the benchmark judgment module 300 extracts the calculated solar altitude angle value. When the solar altitude angle value is less than the preset effective illumination angle lower limit of 15 degrees, it indicates that the ambient light irradiance power cannot generate a sufficient thermal excitation effect. The benchmark judgment module 300 directly triggers an insufficient illumination condition interception command and terminates the defect judgment process of the target photovoltaic module. When the solar altitude angle value is greater than or equal to the preset effective illumination angle lower limit of 15 degrees, the benchmark judgment module 300 uses the solar altitude angle and solar azimuth angle to construct a three-dimensional solar incidence vector with the target photovoltaic module as the origin. The mathematical model for constructing the three-dimensional solar incidence vector is as follows. In the formula, Let the solar incident three-dimensional vector be... The solar altitude angle, The solar azimuth angle is measured clockwise from due north. The reference determination module 300 calculates and maps the transient astronomical parameters of the spherical coordinate system into a standardized direction vector in the Cartesian three-dimensional rectangular coordinate system for use in the multi-dimensional coupled dynamic temperature threshold generation calculation.

[0039] S302, performs multi-dimensional coupled dynamic temperature threshold generation calculation. To accurately reflect the differences in component light-receiving attitude caused by complex terrain undulations, the benchmark judgment module 300 performs a vector inner product operation between the actual normal vector of the target photovoltaic module extracted by the plane fitting module 200 and the three-dimensional vector of solar incident radiation to obtain the effective irradiance coefficient. From the physical mechanism of photothermal conversion, the photovoltaic module absorbs solar radiation energy, part of which is converted into electrical energy and the remainder into heat energy. Therefore, the actual received irradiance on the module surface directly determines the normal operating base temperature of the module. Based on Lambert's cosine law in optical physics, the actual light radiation power received by the target surface is directly proportional to the cosine value of the angle between the incident ray and the surface normal. The mathematical model for calculating the effective irradiance coefficient is as follows. In the formula, Here, n is the effective irradiance coefficient, and n is the actual normal vector of the target photovoltaic module. The solar incident three-dimensional vector is used. When the effective irradiance coefficient is less than or equal to zero, it indicates that the target photovoltaic module is on the back side or severely blocked. The benchmark judgment module 300 determines that the illumination does not meet the thermal excitation conditions and outputs a backlight anomaly mark. It then directly terminates the defect judgment process of the current target photovoltaic module to wait for the next effective data acquisition cycle, preventing subsequent parameter calls from causing null pointer exception deadlocks. Under normal light-receiving conditions where the effective irradiance coefficient is greater than zero, the benchmark judgment module 300 integrates the real-time ambient temperature returned by the ambient temperature sensor and the photothermal response coefficient inherently bound to the target photovoltaic module to generate a dynamic temperature threshold with terrain compensation attributes. The photothermal response coefficient inherently bound to the target photovoltaic module characterizes the temperature rise characteristics of the module under unit irradiance. The value of the photothermal response coefficient is obtained through standard environmental testing and calibration of the same batch of modules in the early stage, and is usually set to 0.025 to 0.035 degrees Celsius per watt-square meter. The theoretical basis irradiance is obtained by querying the sensor data of the synchronously accessed ground meteorological station. The mathematical model for calculating the dynamic temperature threshold is as follows. In the formula, For dynamic temperature threshold, For real-time ambient temperature, The photothermal response coefficient is inherently bound to the target photovoltaic module. The theoretical basis of irradiance is based on meteorological data. The effective irradiance coefficient, The defect temperature rise tolerance is set to a range of three to five degrees Celsius. The benchmark judgment module 300 eliminates the risk of false alarms or missed alarms that may be caused by a fixed temperature threshold under different mountain slopes through parameter coupling in the physical dimension.

[0040] S303, Execute benchmark comparison and anomaly result judgment calculation. The benchmark judgment module 300 extracts the initial thermal anomaly pixel features transmitted by the data synchronization module 100 and the actual measured temperature values ​​associated with the initial thermal anomaly pixel features. It compares the actual measured temperature values ​​with the dynamic temperature threshold. When all actual measured temperature values ​​are less than or equal to the dynamic temperature threshold, the benchmark judgment module 300 determines that the target photovoltaic module is operating normally and directly outputs a defect-free pass mark to end the current judgment process, avoiding unnecessary occupation of subsequent computing resources. However, in the case of abnormal temperature rise, when the actual measured temperature value is greater than the dynamic temperature threshold, the benchmark judgment module 300 initially determines that the initial thermal anomaly pixel features with actual measured temperature values ​​greater than the dynamic temperature threshold are suspected defect points. Considering infrared... In high-temperature environments, sensors may generate random, isolated thermal noise pixels. The benchmark judgment module 300 performs spatial connectivity area calculation on all suspected defect points. The benchmark judgment module 300 counts the number of suspected defect points that are interconnected in the eight-neighbor space as the connectivity area. When the connectivity area is less than a preset lower limit of three pixels, the benchmark judgment module 300 judges the suspected defect points that do not meet the lower limit as sensor shot noise and removes them. For connected suspected defect points that pass the noise removal logic, the benchmark judgment module 300 confirms that the connected suspected defect points are valid hot spot defects and generates a final judgment result containing the two-dimensional coordinates and temperature gradient of the valid hot spot defects. Finally, the benchmark judgment module 300 distributes the final judgment result to the ray positioning module 400 to initiate the calculation of spatial absolute coordinates.

[0041] In the overall system architecture, the ray positioning module 400 receives the final judgment result transmitted by the benchmark judgment module 300 and the optimal rotation matrix and actual physical plane equation transmitted by the plane fitting module 200, and performs a high-precision positioning operation based on unbounded plane prior vector ray projection. The high-precision positioning operation based on unbounded plane prior vector ray projection aims to solve the problem of hot spot spatial coordinate calculation failure caused by field of view truncation by utilizing the coplanar infinite extension property of the same photovoltaic modules, thereby completing the accurate mapping, extraction and communication distribution of defect physical coordinates, specifically including steps S401, S402 and S403.

[0042] S401, perform transient field-of-view hotspot spatial ray reconstruction operation. The ray positioning module 400 extracts the final judgment result output by the benchmark judgment module 300 and the optimal rotation matrix output by the plane fitting module 200. The ray positioning module 400 parses the effective hotspot defect two-dimensional coordinates from the final judgment result, and combines them with the pre-calibrated camera intrinsic parameter matrix to transform the effective hotspot defect two-dimensional coordinates into a local three-dimensional direction vector in the camera coordinate system. From the reverse tracing principle of optical imaging, a two-dimensional pixel is essentially the perspective projection of the line connecting a three-dimensional physical space point and the camera optical center onto the two-dimensional imaging plane. Therefore, the ray positioning module 400 can restore the effective hotspot defect two-dimensional coordinates to the original projection direction vector by performing the inverse matrix multiplication calculation of the camera intrinsic parameter matrix. In order to eliminate the projection offset caused by the transient deformation of the gimbal, the ray positioning module 400 performs the following steps: The position module 400 uses the optimal rotation matrix to perform spatial rotation coordinate transformation on the local three-dimensional direction vector to obtain the absolute three-dimensional direction vector in the world coordinate system. To prevent the loss of the spatial mapping starting point due to transient failure of the position sensor, the ray positioning module 400 performs numerical validity verification on the absolute coordinates of the camera optical center synchronously captured by the data synchronization module 100. When the absolute coordinates of the camera optical center are null, the ray positioning module 400 retrieves the real-time Kalman filter-estimated position coordinates of the UAV flight control system as a compensatory input starting point to avoid the solution dead zone caused by positioning signal obstruction. After confirming the validity of the spatial mapping starting point, the ray positioning module 400 reconstructs the hotspot spatial ray equation with the absolute coordinates of the camera optical center as the starting point and the absolute three-dimensional direction vector as the extension direction. The mathematical model for constructing the hotspot spatial ray equation is as follows. In the formula, Let be the three-dimensional absolute coordinate vector of any point on the hot spot space ray. The absolute coordinates of the camera's optical center For the extension parameters of the ray, As an absolute three-dimensional direction vector, the ray positioning module 400 establishes the linear projection path of the effective hot spot defect in the three-dimensional physical world through the hot spot space ray equation.

[0043] S402 performs unbounded plane mathematical intersection and coordinate output operations. The ray positioning module 400 retrieves the actual physical plane equation extracted by the plane fitting module 200. The general mathematical expression model of the actual physical plane equation is as follows. In the formula, Let be the actual normal vector of the target photovoltaic module. This is the set of three-dimensional coordinate vectors contained in the actual physical plane equations. The orthogonal distance constant between the actual physical plane equation and the origin of the three-dimensional coordinate system is used. The ray positioning module 400 solves the hot spot spatial ray equation and the actual physical plane equation mathematically to calculate the spatial intersection point. From the perspective of the physical space mechanism of three-dimensional geometric projection, since the photovoltaic module support of the same array has the mechanical installation property of strict coplanarity, the actual physical plane equation is equivalent to an unbounded plane that extends infinitely in all directions at the pure mathematical expression level. The infinite extension characteristic of the homogeneous physical plane equation means that even if the extracted initial thermal anomaly pixel features are located at the edge of the imaging plane and the field of view is truncated due to the viewing angle, the outwardly diverging hot spot spatial ray can still detach from the original component boundary and virtually extend outward. The actual physical plane equations produce definite mathematical intersections. The infinite extension of the same physical plane equations constitutes the underlying mathematical logic for accepting hot spot rays caused by field-of-view truncation. To avoid the matrix division-to-zero collapse dead zone caused by the non-intersection of rays and planes, the ray positioning module 400 calculates the vector dot product of the actual normal vector and the absolute three-dimensional direction vector of the target photovoltaic module. When the absolute value of the vector dot product is less than 10 to the power of -6, it is determined that the hot spot space ray is parallel to the actual physical plane equations. 10 to the power of -6 is a typical zero-to-zero physical truncation threshold used in computer double-precision floating-point arithmetic to avoid underlying rounding errors. The ray positioning module 400 triggers a ray intersection failure warning and terminates the three-dimensional absolute direction vector division-to-zero threshold calculation. For the physical coordinate calculation process, under normal spatial intersection conditions where the absolute value of the vector inner product is greater than or equal to 10 to the power of negative 6, the ray positioning module 400 calculates a uniquely determined extension parameter solution by simultaneously solving the hotspot spatial ray equation and the actual physical plane equation. To prevent the analyzed spatial intersection point from violating the laws of optical physics, the ray positioning module 400 performs a physical rationality check on the positive and negative polarities of the extension parameter solution. When the calculated extension parameter solution is less than zero, it indicates that the spatial intersection point is located in a non-physical imaging area behind the camera's optical center. The ray positioning module 400 determines the spatial intersection point as an invalid back projection and discards it. Conversely, when the extension parameter solution is greater than or equal to zero, the ray positioning module 400 will... The extended parameter solution is substituted back into the hot spot space ray equation to calculate the three-dimensional absolute physical coordinates, supplementing the unbounded extension over-boundary interception logic: In order to prevent the intersection point from physical over-boundary due to extreme deviation of the ray direction, the ray positioning module 400 calculates the three-dimensional Euclidean space distance between the three-dimensional absolute physical coordinates and the absolute coordinates of the camera optical center. When the three-dimensional Euclidean space distance is greater than the preset maximum effective projection radius of 100 meters, it is determined that the spatial analytical solution point has deviated from the actual physical array area where the target photovoltaic module is located and a forward projection over-boundary discard instruction is triggered. When the three-dimensional Euclidean space distance is valid, the three-dimensional absolute physical coordinates are determined to be valid. The three-dimensional absolute physical coordinates represent the precise location of the effective hot spot defect in the real physical world on Earth.

[0044] S403, the three-dimensional absolute physical coordinates are distributed to the external operation and maintenance terminal. The ray positioning module 400 encapsulates and merges the calculated three-dimensional absolute physical coordinates with the temperature gradient value and data capture timestamp contained in the final judgment result. The ray positioning module 400 constructs a comprehensive positioning output message according to the data message structure requirements of the IoT message queue telemetry transmission protocol. As a lightweight publish-subscribe network communication protocol, the IoT message queue telemetry transmission protocol can effectively adapt to the lossless data transmission requirements in low bandwidth and unstable mobile network environments. For the underlying communication connection mechanism of sending comprehensive positioning output messages using wireless network channels, handshake and transmission can be performed according to the existing IoT message queue telemetry transmission protocol specifications. The ray positioning module 400 pushes and distributes the comprehensive positioning output message to the external operation and maintenance terminal through the wireless communication gateway. After receiving the comprehensive positioning output message, the external operation and maintenance terminal uses the three-dimensional absolute physical coordinates to drive the ground intelligent inspection robot navigation guidance system to directly reach the real physical space location of the effective hot spot defect to complete the automated business closed loop of inspection and defect elimination.

[0045] Example

[0046] In this application embodiment, a drone equipped with a dual-beam coaxial gimbal camera performs automated inspection tasks on a mountain photovoltaic power station. The data synchronization module 100 receives the hardware synchronization pulse signal sent by the drone's main control board and acquires the infrared image, visible light image, spatial coordinates, and initial rotation matrix at the shutter trigger moment. The data synchronization module 100 calculates the difference between the hardware capture timestamp of the infrared image and the hardware capture timestamp of the visible light image to be 2 milliseconds. The data synchronization module 100 extracts all pixels in the infrared image, calculates the average temperature to be 34.7 degrees Celsius, and the temperature standard deviation to be 2.3 degrees Celsius. The data synchronization module 100 calculates the target photovoltaic module's boundary truncation rate to be 0.26. Since the boundary truncation rate of 0.26 is greater than the preset truncation threshold of 0.20, the data synchronization module 100 activates the anti-truncation feature compensation mechanism. The data synchronization module 100 searches for neighboring photovoltaic modules within the infrared image along the unbounded extension direction of the target photovoltaic module's long side and extracts the pixel corner features of adjacent complete modules.

[0047] The plane fitting module 200 performs matrix transformation using the two-dimensional pixel coordinates corresponding to the corner features of adjacent complete component pixels to obtain a three-dimensional direction vector. The plane fitting module 200 constructs a three-dimensional spatial ray and performs mathematical intersection with the digital elevation model to obtain the initial three-dimensional intersection point coordinates. The plane fitting module 200 calls the design installation elevation parameters from the building information model data to perform elevation algebra and correction, outputting a primary three-dimensional spatial coordinate set. The plane fitting module 200 calculates the Euclidean spatial distance using the primary three-dimensional spatial coordinate set and substitutes the Euclidean spatial distance and standard physical dimensions into a nonlinear objective optimization function. The plane fitting module 200 uses the Levenburg-Marquardt algorithm, and after 18 gradient descent iterations, satisfies the termination condition of continuous residual change, outputting the optimal rotation matrix. The plane fitting module 200 uses the random sampling consensus algorithm to perform orthogonal spatial distance filtering in the target spatial coordinate set, retaining the optimal set of 16 interior points. The plane fitting module 200 performs principal component analysis and least squares method on the optimal interior point set to perform surface approximation fitting, outputting the actual physical plane equation and the actual normal vector of the target photovoltaic module.

[0048] The benchmark determination module 300 uses longitude, latitude, and shooting timestamps to calculate the solar altitude angle as 43.5 degrees and the solar azimuth angle as 109.2 degrees. The benchmark determination module 300 performs a vector inner product operation on the actual normal vector of the target photovoltaic module and the three-dimensional incident solar vector to calculate the effective irradiance coefficient as 0.782. The benchmark determination module 300 integrates the real-time ambient temperature of 24.1 degrees Celsius and the effective irradiance coefficient to generate a dynamic temperature threshold of 42.3 degrees Celsius with terrain compensation attributes. The benchmark determination module 300 extracts the actual measured temperature value of 44.8 degrees Celsius associated with the initial thermal anomaly pixel features. Since the actual measured temperature value of 44.8 degrees Celsius is greater than the dynamic temperature threshold of 42.3 degrees Celsius, the benchmark determination module 300 calculates the connected region area as 14 pixels. The benchmark determination module 300 determines the initial thermal anomaly pixel features as effective hot spot defects and generates the final determination result.

[0049] The ray positioning module 400 analyzes the two-dimensional coordinates of the valid hotspot defects in the final judgment result and uses the optimal rotation matrix to perform a spatial rotation coordinate transformation to obtain the absolute three-dimensional direction vector. The ray positioning module 400 reconstructs the hotspot spatial ray equation. The absolute value of the vector dot product between the actual normal vector of the target photovoltaic module and the absolute three-dimensional direction vector is calculated to be 0.841. The ray positioning module 400 solves the hotspot spatial ray equation and the actual physical plane equation simultaneously to obtain the extension parameter solution, which is 37.9. The ray positioning module 400 substitutes the extension parameter solution back into the hotspot spatial ray equation to calculate the three-dimensional absolute physical coordinates. The three-dimensional Euclidean distance between the three-dimensional absolute physical coordinates and the absolute coordinates of the camera's optical center is 37.9 meters. The ray positioning module 400 encapsulates the three-dimensional absolute physical coordinates and temperature parameters using a protocol, constructs a comprehensive positioning output message, and distributes it to the external operation and maintenance terminal.

[0050] Table 1. Performance comparison test dataset of different detection and localization algorithms in a mountainous photovoltaic power station scenario. According to the data in Table 1, the method of this invention can maintain extremely low spatial 3D positioning error in imaging environments with increased boundary truncation rates. Traditional 2D judgment and monocular positioning algorithms face a boundary truncation rate of 0.28 in test sample S-109, resulting in a spatial 3D positioning error of 2.153 meters. Traditional point cloud spatial mapping algorithms face a boundary truncation rate of 0.24 in test sample S-411, resulting in a spatial 3D positioning error of 0.749 meters. The method of this invention, when inputting an image with a boundary truncation rate of 0.28 in test sample S-582, compresses the spatial 3D positioning error to 0.043 meters. This method uses the data synchronization module 100 to extract the corner features of adjacent complete component pixels to perform anti-truncation feature compensation, and combines this with the ray positioning module 400 to mathematically solve the hotspot spatial ray equation and the actual physical plane equation, which is equivalent to infinitely extended properties. This overcomes the limitation of physical scale loss caused by local field-of-view boundary truncation, ensuring the analytical completeness of the reverse spatial mapping projection.

[0051] According to the data in Table 1, the method of this invention can identify hidden hot spot defects in the low effective irradiance coefficient range. The measured anomaly temperatures of test samples S-582 and S-851 are 42.11 degrees Celsius and 39.75 degrees Celsius, respectively. In the traditional algorithm using 45.00 degrees Celsius as a fixed calculation reference temperature threshold, neither test sample S-582 nor test sample S-851 reached the anomaly judgment line, thus outputting a false negative judgment result. The reference judgment module 300 calculates the effective irradiance coefficient using the actual normal vector of the target photovoltaic module and the three-dimensional vector of solar incident radiation. The effective irradiance coefficients of test samples S-582 and S-851 exhibit numerical characteristics of 0.613 and 0.428, respectively. The reference judgment module 300 performs algebraic calculations of the photothermal response using the effective irradiance coefficient to generate calculation reference temperature thresholds of 40.57 degrees Celsius and 38.04 degrees Celsius. The benchmark judgment module 300 performs a numerical comparison between the measured abnormal point temperature and the calculated benchmark temperature threshold to identify and extract the effective hot spot defects associated with test samples S-582 and S-851. The terrain and illumination coupled thermal benchmark dynamic compensation logic reduces the numerical attenuation interference of component thermal excitation caused by low illumination conditions and non-direct lighting posture, thereby improving the defect identification rate under complex mountainous conditions.

[0052] According to the data in Table 1, there is a direct computational correlation between the plane fitting confidence score and the pose constraint algorithm. The plane fitting module 200 uses the Levenberg-Marquardt algorithm to construct a nonlinear objective optimization function using the difference parameter between the Euclidean spatial distance and the standard physical size. The method of this invention extracts 18 and 25 in-plane fitting points in test samples S-637 and S-794, respectively. Sufficient spatial coordinate points provide the computational matrix basis for principal component analysis least squares, eliminating interference from outlier noise coordinate systems. The traditional point cloud spatial mapping algorithm without prior optimization of physical size extracts only 2 in-plane fitting points in test sample S-305, causing the internal mathematical model to fail to converge. The single-frame parsing time of the algorithm of this invention is maintained within the range of 96.5 ms to 121.8 ms. The multimodal computing architecture uses three-dimensional unbounded plane mathematical intersection to replace high-density three-dimensional point cloud iterative reconstruction, compressing the spatial three-dimensional positioning error while maintaining the single-frame parsing time of the algorithm, supporting the high-frequency concurrent communication service requirements of data processing equipment and UAV flight control module.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multimodal fusion intelligent detection and location method for thermal anomalies in mountain photovoltaic modules, characterized in that, Includes the following steps: Simultaneously acquire UAV pose data, infrared images and visible light images at the shutter trigger moment, extract the initial thermal anomaly pixel features and their associated actual measured temperature values, and extract the pixel corner features of adjacent complete modules based on the topological constraints of the same source bracket when the field of view of the target photovoltaic module is truncated. Based on the pixel corner features and initial rotation matrix of the adjacent complete components, spatial mathematical intersection and size constraint optimization are performed by combining digital elevation model data and building information model data, and the actual physical plane equation of the target photovoltaic module and its corresponding actual normal vector are fitted and output. Based on the acquisition time, the solar incidence three-dimensional vector is calculated, and the inner product cosine parameter of the actual normal vector and the solar incidence three-dimensional vector is calculated to obtain the effective irradiance coefficient. Based on this, a dynamic temperature threshold with terrain compensation attributes is generated, and the dynamic temperature threshold is compared with the actual measured temperature value to output the anomaly judgment result. Based on the anomaly determination result, the optimized rotation matrix is ​​called to reconstruct the hot spot spatial ray, and the hot spot spatial ray is solved simultaneously with the actual physical plane equation to obtain the three-dimensional absolute physical coordinates of the thermal anomaly region of the target photovoltaic module.

2. The method as described in claim 1, characterized in that, The process of simultaneously acquiring UAV pose data, infrared images, and visible light images at the shutter trigger moment, and extracting initial thermal anomaly pixel features and their associated actual measured temperature values ​​includes: Calculate the difference between the hardware capture timestamp of the infrared image and the hardware capture timestamp of the visible light image, and determine that the multi-source parameter synchronization is effective when the difference is less than or equal to the maximum allowable synchronization time error threshold; Calculate the average temperature and standard deviation of all pixels in the infrared image. When the standard deviation of temperature is greater than or equal to a preset background uniformity threshold, construct a temperature screening mathematical model based on the average temperature, the standard deviation of temperature, and the anomaly screening coefficient. All two-dimensional pixel coordinates that conform to the temperature screening mathematical model are determined as the initial thermal anomaly pixel features.

3. The method as described in claim 1, characterized in that, The step of extracting pixel corner features of adjacent complete modules based on the topological constraints of the same support structure when the field of view of the target photovoltaic module is truncated includes: The number of physical boundary pixels of the target photovoltaic module falling into the edge buffer of the infrared image is counted, and the ratio of the number to the total number of physical boundary pixels is calculated to construct the boundary truncation rate. When the boundary truncation rate is greater than the preset truncation threshold, based on the physical support topology connection tree pre-stored in the building information model data, neighboring photovoltaic modules belonging to the same physical support are retrieved in the infrared image along the long side of the target photovoltaic module without boundary extension. When the boundary truncation rate of the adjacent photovoltaic module is equal to zero, the adjacent photovoltaic module is determined as the adjacent complete module and the pixel corner features of the adjacent complete module are extracted.

4. The method as described in claim 1, characterized in that, The process of combining digital elevation model data and building information model data to perform spatial mathematical intersection and dimensional constraint optimization includes: The two-dimensional pixel coordinates corresponding to the pixel corner features of the adjacent complete components are transformed into three-dimensional direction vectors in the camera coordinate system. A three-dimensional spatial ray is constructed with the camera optical center as the starting point and the three-dimensional direction vector as the direction. The initial altitude is obtained by mathematically intersecting the three-dimensional spatial ray with the digital elevation model data, and the design installation elevation parameters of the target photovoltaic module stored in the building information model data are called. The initial altitude and the design installation elevation parameters are linearly superimposed and algebraically summed to complete the elevation correction, outputting a primary three-dimensional spatial coordinate set with vertical height compensation.

5. The method as described in claim 4, characterized in that, After outputting the primary 3D spatial coordinate set with vertical height compensation, it also includes: The Euclidean distance between adjacent pixel corner points in the three-dimensional physical world is calculated using the aforementioned primary three-dimensional spatial coordinate set. A nonlinear objective optimization function is constructed using the roll angle, pitch angle, and yaw angle of the initial rotation matrix as independent variables, and the sum of squared differences between the Euclidean spatial distance and the preset standard physical dimensions in the building information model data as dependent variables. The nonlinear objective function is solved iteratively by gradient descent using the Levenburg-Marquardt algorithm, and the converged attitude parameter vector is transformed into the optimized rotation matrix after the termination condition of the iteration is met.

6. The method as described in claim 5, characterized in that, After converting the converged attitude parameter vector into the optimized rotation matrix, the process also includes: Using the optimized rotation matrix, spatial ray construction and elevation intersection calculation are re-performed on the pixel corner features of all adjacent complete components to obtain the target spatial coordinate set; The random sampling consensus algorithm is used to filter the interior points of the target space coordinate set. The coordinate points whose orthogonal spatial distance is less than the numerical judgment condition are divided into interior point sets. After multiple random iterations, the optimal interior point set with the most interior points is retained. The optimal set of interior points is approximated and fitted to a three-dimensional surface using principal component analysis and least squares method. The equation of the actual physical plane is output, and the normal feature vector of the equation of the actual physical plane is extracted as the actual normal vector.

7. The method as described in claim 1, characterized in that, The calculation of the three-dimensional vector of solar incidence based on the acquisition time includes: The solar altitude angle and solar azimuth angle at the corresponding time are calculated based on the shooting timestamp, longitude and latitude values; When the solar elevation angle is greater than or equal to the preset lower limit of the effective illumination angle, the solar elevation angle and the solar azimuth angle are used to construct the solar incident three-dimensional vector with the target photovoltaic module as the origin, and the transient astronomical parameters of the spherical coordinate system are mapped to the standardized direction vector in the Cartesian three-dimensional rectangular coordinate system.

8. The method as described in claim 7, characterized in that, The calculation of the inner product cosine parameter of the actual normal vector and the solar incident three-dimensional vector to obtain the effective irradiance coefficient, and the generation of a dynamic temperature threshold with terrain compensation attributes, includes: The effective irradiance coefficient is obtained by performing a vector dot product operation between the actual normal vector and the solar incident three-dimensional vector. When the effective irradiance coefficient is greater than zero, the physical dimension parameter coupling calculation is performed by integrating the real-time ambient temperature, the inherent photothermal response coefficient of the target photovoltaic module, the theoretical irradiance, and the defect temperature rise tolerance to generate the dynamic temperature threshold that includes terrain attitude compensation.

9. The method as described in claim 1, characterized in that, The step of comparing the dynamic temperature threshold with the actual measured temperature value to output an anomaly determination result includes: When the actual measured temperature value is greater than the dynamic temperature threshold, the initial thermal anomaly pixel feature corresponding to the actual measured temperature value is initially determined to be a suspected defect point. The number of suspected defect points that are interconnected in the eight neighboring domains is counted as the area of ​​the connected domain. When the area of ​​the connected domain is greater than or equal to a preset lower limit, the connected suspected defect points are confirmed as valid hot spot defects and a final judgment result containing the two-dimensional coordinates of the valid hot spot defects is generated as the anomaly judgment result.

10. The method as described in claim 1, characterized in that, The process involves reconstructing the hot spot spatial ray using the optimized rotation matrix based on the anomaly determination result, solving the hot spot spatial ray and the actual physical plane equations simultaneously to obtain the three-dimensional absolute physical coordinates of the target photovoltaic module's thermal anomaly region, including: The effective hot spot defect two-dimensional coordinates in the anomaly determination result are transformed into local three-dimensional direction vectors in the camera coordinate system by combining the camera intrinsic parameter matrix, and the local three-dimensional direction vectors are transformed by spatial rotation coordinates using the optimized rotation matrix to obtain the absolute three-dimensional direction vectors in the world coordinate system. The hot spot spatial ray equation is reconstructed using the absolute coordinates of the camera's optical center as the starting point and the absolute three-dimensional direction vector as the extension direction. Calculate the vector dot product of the actual normal vector and the absolute three-dimensional direction vector. When the absolute value of the vector dot product is greater than or equal to the zero-approach physical cutoff threshold, solve the hot spot space ray equation and the actual physical plane equation, which is equivalent to an unbounded plane, in a mathematical simultaneous solution to calculate the extension parameter solution. Substitute the extended parameter solution back into the hot spot space ray equation to calculate the three-dimensional absolute physical coordinates.

Images