Photovoltaic hot spot classification and diagnosis method based on double model cooperation and time sequence space alignment

Abstract

This invention provides a photovoltaic hotspot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment, relating to the field of photovoltaic technology. The method includes the following steps: acquiring infrared radiation images and image metadata of a photovoltaic module; inputting the infrared radiation images into a photovoltaic panel semantic segmentation model and a hotspot target detection model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module, as well as the hotspot detection box, the coordinates of the center pixel point, and the highest pixel temperature; calculating a homography mapping matrix to map the coordinates of the hotspot center point to the local normalized coordinate system of the module through the homography mapping matrix, and removing false hotspots; based on the pixel-level mask and the hotspot detection box, extracting the dynamic background reference temperature within the photovoltaic module, constructing a normalized hotspot index, and constructing a temporal feature vector of the same hotspot at different time points, calculating the statistical characteristics of the temporal feature vector, and comparing it with a preset threshold to determine the hotspot type.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic technology, and in particular to a photovoltaic hotspot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment. Background Technology

[0002] As photovoltaic power plants rapidly develop towards large-scale and clustered operations, traditional operation and maintenance (O&M) inspection methods are no longer sufficient to meet practical needs due to their low efficiency, high cost, and safety risks. Therefore, automated inspection using drones equipped with thermal infrared cameras has become the mainstream approach in the industry, capable of efficiently detecting abnormal temperature areas on the surface of photovoltaic modules, i.e., hot spots. However, the causes of hot spots are complex. They can be permanent hot spots caused by defects in the module itself, such as microcracks or diode damage, or they can be transient hot spots caused by transient disturbances such as fallen leaves, bird droppings, or cloud shadows. The two types require fundamentally different O&M handling; the former requires immediate repair, while the latter does not require intervention. Therefore, accurate identification of hot spot types is crucial for improving the level of intelligent O&M and reducing ineffective costs.

[0003] Currently, mainstream hotspot detection technologies mainly rely on the analysis of single-frame infrared images, such as segmentation by setting fixed temperature thresholds or identification using deep learning target detection models. However, these technologies can only detect and locate hotspots, but cannot effectively distinguish between transient and permanent hotspots. This is because image information at a single moment cannot reflect the characteristics of hotspot changes over time, and the shooting angles and positional deviations of drones during different inspections make it difficult to accurately track and compare the same hotspot across time sequences. This causes the maintenance system to report all detected hotspots as faults, forcing maintenance personnel to conduct on-site verification of numerous false faults that are actually transient interferences, resulting in significant costs associated with ineffective inspections and false alarms. Summary of the Invention

[0004] The purpose of this invention is to provide a photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal-spatial alignment, in order to solve the problem mentioned in the background art that the existing hot spot detection technology cannot effectively distinguish between transient hot spots and permanent hot spots, which leads to the maintenance personnel having to conduct on-site verification of a large number of false faults that are actually transient interferences, resulting in huge costs of ineffective inspections and false alarms.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a photovoltaic hotspot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment, comprising the following steps: acquiring infrared radiation images of photovoltaic modules collected by a UAV and simultaneously extracting image metadata; inputting the infrared radiation images into a photovoltaic panel semantic segmentation model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module; inputting the infrared radiation images into a hotspot target detection model to determine the position of the hotspot detection box in the infrared radiation images, calculating the coordinates of the center pixel of the hotspot detection box, and extracting the highest temperature of the pixel within the hotspot detection box as the highest pixel temperature; and calculating the homography mapping matrix based on the boundary corner point set and the preset standard rectangular normalized space vertices to construct... A perspective transformation relationship is established from the image coordinate system to the component's local normalized coordinate system. The coordinates of the center pixel are mapped to the component's local normalized coordinate system through the homography mapping matrix to obtain the absolute position coordinates of the hot spot on the photovoltaic module. The spatial validity of the hot spot is verified, and false hot spots that do not meet the internal spatial constraints of the photovoltaic module are eliminated. Based on the pixel-level mask and hot spot detection box, the dynamic background reference temperature inside the photovoltaic module is extracted. Combined with the image metadata, a normalized hot spot index is constructed. Using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot on the photovoltaic module as indexes, a temporal feature vector of the same hot spot at different time points is constructed. The statistical characteristics of the temporal feature vector are calculated and compared with a preset threshold to determine the hot spot type.

[0006] Optionally, the step of inputting the infrared radiation image into the photovoltaic panel semantic segmentation model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module specifically includes: the photovoltaic panel semantic segmentation model adopts an instance segmentation network based on YOLOv11-seg, its backbone network adopts the CSPDarknet structure, the neck adopts PANet for multi-scale feature fusion, and the head contains parallel detection branches and mask branches; after scaling the infrared radiation image to a uniform size, it is input into the photovoltaic panel semantic segmentation model, outputting the pixel-level segmentation mask of the photovoltaic module, and extracting the four vertices of the minimum bounding polygon based on the mask as the boundary corner point set.

[0007] Optionally, the step of inputting the infrared radiation image into the hot spot target detection model to determine the position of the hot spot detection box in the infrared radiation image specifically includes: the hot spot target detection model adopts a target detection network based on YOLOv11-det, whose backbone network and neck structure are shared with the photovoltaic panel semantic segmentation model, and the head is the detection branch; the infrared radiation image is scaled to a uniform size and then input into the hot spot target detection model, outputting the rectangular detection box of the hot spot and its confidence score; the coordinates of the center pixel of the hot spot detection box are calculated, the coverage area of ​​the hot spot detection box is located in the infrared radiation image, the temperature values ​​of all pixels in the coverage area are traversed, and the maximum value is taken as the highest pixel temperature.

[0008] Optionally, the step of calculating the homography mapping matrix based on the set of boundary corner points and the preset standard rectangular normalized space vertices specifically includes: using the set of boundary corner points as the four boundary corner points of the photovoltaic module, and using the preset standard rectangular normalized space vertices as the normalized local coordinate system point set; using the direct linear transformation algorithm, based on the four pairs of matching points between the set of boundary corner points and the normalized local coordinate system point set, constructing a system of linear equations and solving them through singular value decomposition to obtain the homography mapping matrix.

[0009] Optionally, the step of mapping the coordinates of the center pixel point to the local normalized coordinate system of the component through the homography mapping matrix to obtain the absolute position coordinates of the hot spot on the photovoltaic module specifically includes: substituting the coordinates of the center pixel point of the hot spot detection frame into the transformation equation defined by the homography mapping matrix to calculate the intermediate variable; and normalizing the intermediate variable to obtain the absolute position coordinates of the hot spot on the photovoltaic module.

[0010] Optionally, the step of verifying the spatial validity of hot spots and removing false hot spots that do not meet the internal spatial constraints of the photovoltaic module specifically includes: performing morphological etching on the pixel-level mask of the photovoltaic module to obtain an effective module region mask; calculating the pixel coverage ratio of the hot spot detection frame within the effective module region mask as the coverage degree; calculating the temperature centroid coordinates of the hot spot detection frame; when the coverage degree is greater than a preset coverage threshold and the temperature centroid coordinates are located within the effective module region mask, the hot spot is determined to be a valid hot spot; otherwise, the hot spot is removed as a false hot spot.

[0011] Optionally, the step of extracting the dynamic background reference temperature within the photovoltaic module based on the pixel-level mask and hot spot detection frame specifically includes: performing a Boolean operation to remove the pixel area covered by the hot spot detection frame within the area covered by the pixel-level mask; performing morphological erosion processing on the remaining area to remove edge interference pixels; and calculating the average radiation temperature of all pixels in the remaining area after erosion as the dynamic background reference temperature of the module.

[0012] Optionally, the step of constructing the normalized hot spot index by combining the image metadata specifically includes: extracting real-time irradiance, gimbal pitch angle and fuselage attitude angle from the image metadata; calculating the thermal radiation incident angle based on the gimbal pitch angle and fuselage attitude angle; constructing an irradiance compensation operator, and calculating the normalized hot spot index based on the irradiance compensation operator.

[0013] Optionally, the step of constructing a time-series feature vector of the same hot spot at different time points using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot on the photovoltaic module as indexes specifically includes: using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot in the module's local normalized coordinate system as a joint index; retrieving hot spot records with the same joint index key from the historical data of multiple inspections, and extracting the corresponding normalized hot spot index from each record; arranging the extracted normalized hot spot index values ​​in chronological order to construct the time-series feature vector.

[0014] Optionally, the step of calculating the statistical characteristics of the time-series feature vector and comparing them with a preset threshold to determine the hotspot type specifically includes: calculating the mean and the coefficient of variation of the time-series feature vector; comparing the mean and the coefficient of variation with preset mean threshold, stability threshold, and fluctuation threshold respectively; if the mean is greater than the mean threshold and the coefficient of variation is less than the stability threshold, it is determined to be a permanent hotspot; if the coefficient of variation is greater than the fluctuation threshold or there is a normalized hotspot index that is approximately zero in the time-series feature vector, it is determined to be a transient hotspot; if the mean is greater than the mean threshold and the coefficient of variation is between the stability threshold and the fluctuation threshold, it is determined to be a suspected hotspot.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This application employs a dual-model collaborative strategy to simultaneously acquire precise boundary information and hotspot location information of photovoltaic modules during a single inspection, laying a data foundation for subsequent spatial alignment and physical modeling. Based on the boundary corner points extracted through semantic segmentation and a preset standard vertex, a homography mapping matrix is ​​calculated to map the hotspot center point to the module's local normalized coordinate system. This eliminates the spatial position shift of hotspots caused by differences in shooting position and angle during multiple UAV inspections, achieving accurate association of the same hotspot in images from different time points. By verifying the spatial validity of the hotspots and eliminating false alarms located in the module border or background areas, the application ensures the purity and reliability of the data entering subsequent analysis. Furthermore, by combining environmental parameters from image metadata to construct a normalized hot spot index, the interference of factors such as irradiance and shooting angle at different times on temperature values ​​is eliminated, making hot spot data acquired at different times comparable. Finally, a time-series feature vector is constructed using component identification and normalized location as indexes. Through statistical feature analysis, the temporal evolution pattern of hot spots is distinguished, thereby accurately identifying transient hot spots that do not require maintenance intervention and permanent hot spots that require timely treatment. This fundamentally solves the problem of high false alarm rate and ineffective maintenance caused by the inability of existing technologies to distinguish the nature of hot spots, significantly reducing the manual verification cost of photovoltaic power plants and improving maintenance efficiency and intelligence. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method steps of the present invention.

[0017] Figure 2 This is a flowchart of the method steps of the present invention. Detailed Implementation

[0018] The present invention will now be clearly and completely described in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] Those skilled in the art will understand that, unless explicitly stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in the specification of this application means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0021] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0022] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Please refer to Figures 1-2This invention discloses a photovoltaic hotspot classification and diagnosis method based on dual-model collaboration and temporal-spatial alignment. The steps include: acquiring infrared radiation images of photovoltaic modules collected by a drone and simultaneously extracting image metadata; inputting the infrared radiation images into a photovoltaic panel semantic segmentation model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module; inputting the infrared radiation images into a hotspot target detection model to determine the position of the hotspot detection box in the infrared radiation images, calculating the coordinates of the center pixel of the hotspot detection box, and extracting the highest temperature of the pixel within the hotspot detection box as the highest pixel temperature; and calculating a homography mapping matrix based on the boundary corner point set and a preset standard rectangular normalized space vertex to establish a mapping from the image coordinate system to... The perspective transformation relationship of the component's local normalized coordinate system is used to map the coordinates of the center pixel point to the component's local normalized coordinate system through the homography mapping matrix, thereby obtaining the absolute position coordinates of the hot spot on the photovoltaic module. The spatial validity of the hot spot is verified, and false hot spots that do not meet the internal spatial constraints of the photovoltaic module are eliminated. Based on the pixel-level mask and hot spot detection box, the dynamic background reference temperature inside the photovoltaic module is extracted, and combined with the image metadata, a normalized hot spot index is constructed. Using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot on the photovoltaic module as indexes, a temporal feature vector of the same hot spot at different time points is constructed. The statistical characteristics of the temporal feature vector are calculated and compared with a preset threshold to determine the hot spot type.

[0025] Specifically, infrared radiation images of photovoltaic modules are collected by thermal infrared sensors mounted on the drone, and image metadata is extracted simultaneously from the flight control system and mounted equipment. The metadata includes at least real-time irradiance, gimbal pitch angle, fuselage attitude angle, and unique identifier ID of photovoltaic modules obtained based on RTK-GPS high-precision positioning and GIS map mapping.

[0026] The acquired infrared radiation images are input into two pre-trained deep learning models: a photovoltaic panel semantic segmentation model and a hot spot target detection model. The semantic segmentation model performs pixel-level classification of the images, outputs a precise mask for each photovoltaic module, and extracts the four boundary corners of the module based on the mask. The target detection model outputs a rectangular detection box for the hot spot. Subsequently, the coordinates of the center pixel are calculated based on the coordinates of the detection box, and the temperature values ​​of all pixels within the box are iterated, with the maximum value being taken as the highest pixel temperature of the hot spot.

[0027] To achieve accurate matching of hot spots at different inspection time points, a homography mapping matrix is ​​calculated using the four boundary corner points output by the semantic segmentation model and the preset standard rectangular normalized space vertices. This matrix describes the perspective transformation relationship from the image coordinate system to the component's local normalized coordinate system.

[0028] Substituting the coordinates of the hot spot center point into this transformation relationship, we obtain its absolute position coordinates in the normalized coordinate system. This coordinate, together with the component ID, constitutes a unique spatiotemporal identifier for the hotspot. Simultaneously, spatial validity verification is performed on each detected hotspot, eliminating false alarms caused by edge reflections, ground debris, etc., by determining whether the hotspot is truly located inside the component.

[0029] After obtaining clean hotspot information, the area covered by the hotspot detection frame is subtracted from the component mask area using a component mask and a hotspot detection frame. Morphological processing is then performed on the remaining area, and the average temperature is calculated as the dynamic background reference temperature for the component. Combined with real-time irradiance, attitude angle, and other environmental parameters from the metadata, a normalized hotspot index is constructed. This index eliminates the influence of external factors such as illumination and angle on temperature values ​​at different inspection times, making hot spot data acquired at different times comparable. Finally, using the component's unique identifier and the absolute location coordinates of the hot spot as a joint index, the normalized hot spot index of the same hot spot at different time points is calculated. The values ​​are arranged in chronological order to form a time-series feature vector. The statistical characteristics of this vector, such as the mean and coefficient of variation, are calculated and compared with preset thresholds to determine whether the hot spot is a permanent hot spot, a transient hot spot, or a suspected hot spot, providing a scientific basis for operation and maintenance decisions.

[0030] This application employs a dual-model collaborative strategy to simultaneously acquire precise boundary information and hotspot location information of photovoltaic modules during a single inspection, laying a data foundation for subsequent spatial alignment and physical modeling. Based on the boundary corner points extracted through semantic segmentation and a preset standard vertex, a homography mapping matrix is ​​calculated to map the hotspot center point to the module's local normalized coordinate system. This eliminates the spatial position shift of hotspots caused by differences in shooting position and angle during multiple UAV inspections, achieving accurate association of the same hotspot in images from different time points. By verifying the spatial validity of the hotspots and eliminating false alarms located in the module border or background areas, the application ensures the purity and reliability of the data entering subsequent analysis. Furthermore, by combining environmental parameters from image metadata to construct a normalized hot spot index, the interference of factors such as irradiance and shooting angle at different times on temperature values ​​is eliminated, making hot spot data acquired at different times comparable. Finally, a time-series feature vector is constructed using component identification and normalized location as indexes. Through statistical feature analysis, the temporal evolution pattern of hot spots is distinguished, thereby accurately identifying transient hot spots that do not require maintenance intervention and permanent hot spots that require timely treatment. This fundamentally solves the problem of high false alarm rate and ineffective maintenance caused by the inability of existing technologies to distinguish the nature of hot spots, significantly reducing the manual verification cost of photovoltaic power plants and improving maintenance efficiency and intelligence.

[0031] In some embodiments, the step of inputting the infrared radiation image into the photovoltaic panel semantic segmentation model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module specifically includes: the photovoltaic panel semantic segmentation model adopts an instance segmentation network based on YOLOv11-seg, its backbone network adopts the CSPDarknet structure, the neck adopts PANet for multi-scale feature fusion, and the head contains parallel detection branches and mask branches; after scaling the infrared radiation image to a uniform size, it is input into the photovoltaic panel semantic segmentation model, outputting the pixel-level segmentation mask of the photovoltaic module, and extracting the four vertices of the minimum bounding polygon based on the mask as the boundary corner point set.

[0032] Specifically, the photovoltaic panel semantic segmentation model adopts an instance segmentation network based on YOLOv11-seg, with its backbone network using the CSPDarknet structure. This structure enhances gradient flow through cross-stage local connections, effectively extracting multi-scale features. The neck uses PANet for top-down and bottom-up feature fusion, combining high-level semantic information with low-level detail information to output feature maps at multiple scales. The head contains parallel detection and mask branches. The detection branch is used to output bounding boxes and categories, while the mask branch learns mask coefficients and combines them with the prototype mask to generate a pixel-level segmentation mask for each photovoltaic module.

[0033] In practice, the acquired infrared radiation images are scaled to a uniform size, such as 640×640 pixels, and then input into the photovoltaic panel semantic segmentation model to output a segmentation mask for each photovoltaic module. For each mask, the four vertices of its minimum bounding polygon are extracted using a contour detection algorithm, which serve as the set of boundary corner points for that module.

[0034] This application employs an instance segmentation network based on YOLOv11-seg as the semantic segmentation model for photovoltaic panels. It utilizes the CSPDarknet backbone network to extract multi-scale features and combines them with PANet for feature fusion, enabling accurate identification of photovoltaic module regions in complex backgrounds. The parallel detection branch and mask branch design allows the model to generate a pixel-level segmentation mask while outputting the module bounding box. Based on the mask, the four vertices of the smallest bounding polygon are extracted as boundary corner points, ensuring the accuracy of corner point extraction. This provides a high-precision geometric benchmark for the subsequent calculation of the homography mapping matrix, which is the fundamental guarantee for the realization of spatial alignment.

[0035] In some embodiments, the step of inputting the infrared radiation image into the hot spot target detection model and determining the position of the hot spot detection box in the infrared radiation image specifically includes: the hot spot target detection model adopts a target detection network based on YOLOv11-det, whose backbone network and neck structure are shared with the photovoltaic panel semantic segmentation model, and the head is the detection branch; the infrared radiation image is scaled to a uniform size and then input into the hot spot target detection model, outputting the rectangular detection box of the hot spot and its confidence score; the coordinates of the center pixel of the hot spot detection box are calculated, the coverage area of ​​the hot spot detection box is located in the infrared radiation image, the temperature values ​​of all pixels in the coverage area are traversed, and the maximum value is taken as the highest pixel temperature.

[0036] Specifically, the hot spot target detection model employs a target detection network based on YOLOv11-det. Its backbone and neck structure are shared with the aforementioned photovoltaic panel semantic segmentation model, using a CSPDarknet+PANet structure. The head is a dedicated detection branch used to output the target's bounding box and confidence score. Infrared radiation images are scaled to a uniform size and input into the network, which outputs the rectangular detection box of the hot spot and its confidence score. Subsequently, the coordinates of the center pixel are calculated based on the coordinates of the top-left and bottom-right corners of the detection box. ; Simultaneously, the area covered by the detection frame is located in the original infrared radiation image, and the temperature values ​​of all pixels within the area are traversed. The maximum value is taken as the highest pixel temperature of the hot spot, so as to accurately locate the hot spot position and quantify its temperature intensity.

[0037] This application uses a target detection network based on YOLOv11-det as the hot spot target detection model. Its backbone network and neck structure are shared with the segmentation model, which reduces the model complexity and computational resource consumption. After the model outputs the rectangular detection box of the hot spot and its confidence score, it further calculates the coordinates of the center point of the detection box and extracts the highest temperature by traversing the pixels in the box. This achieves accurate localization of the hot spot position and accurate quantification of the hot spot intensity, providing key input parameters for subsequent spatial mapping and normalized hot spot index calculation, and ensuring the complete acquisition of hot spot information.

[0038] In some embodiments, the step of calculating the homography mapping matrix based on the set of boundary corner points and the preset standard rectangular normalized space vertices specifically includes: using the set of boundary corner points as the four boundary corner points of the photovoltaic module, and using the preset standard rectangular normalized space vertices as the normalized local coordinate system point set; using the direct linear transformation algorithm, based on the four pairs of matching points between the set of boundary corner points and the normalized local coordinate system point set, constructing a system of linear equations and solving them through singular value decomposition to obtain the homography mapping matrix.

[0039] Specifically, the four boundary corner points output by the semantic segmentation model are used as the point set of the image coordinate system. ; Normalize the spatial vertices using a pre-defined standard rectangle. , usually take , which represents the aspect ratio of the component mapped to the [0,1] range.

[0040] Using the direct linear transformation algorithm, a system of linear equations is established based on four pairs of matching points.

[0041] For each pair of matching points Two equations can be obtained: ; Substitute the four pairs of points to construct the matrix equation. ,in .

[0042] The singular value decomposition method is used to solve for the least squares solution, thereby obtaining the optimal transformation matrix H. This matrix H accurately describes the perspective transformation relationship from the image plane to the component's locally normalized plane.

[0043] This application establishes a precise perspective transformation relationship from distorted image space to standard normalized space by using the set of boundary corner points as the image coordinate system point set and the vertices of a preset standard rectangle as the normalized local coordinate system point set. It constructs a system of linear equations using the direct linear transformation algorithm and solves the homography mapping matrix through singular value decomposition. This effectively corrects component image distortion caused by changes in the shooting angle and altitude of the UAV, laying a mathematical foundation for accurate cross-time matching of hot spots.

[0044] In some embodiments, the step of mapping the coordinates of the center pixel point to the component local normalized coordinate system through the homography mapping matrix to obtain the absolute position coordinates of the hot spot on the photovoltaic module specifically includes: substituting the coordinates of the center pixel point of the hot spot detection frame into the transformation equation defined by the homography mapping matrix to calculate the intermediate variable; and normalizing the intermediate variable to obtain the absolute position coordinates of the hot spot on the photovoltaic module.

[0045] Specifically, the coordinates of the center pixel of the hot spot detection frame Substituting the transformation equation defined by the homography mapping matrix H, we first calculate the intermediate variables: ; ; ; Then, normalization is performed to obtain the absolute position coordinates of the hot spot in the local normalized coordinate system of the photovoltaic module: absolute position coordinates The value of is between [0,1], representing the position of the hot spot on the component, and has stability across time and angle.

[0046] In some embodiments, the step of verifying the spatial validity of hot spots and removing false hot spots that do not meet the internal spatial constraints of the photovoltaic module specifically includes: performing morphological etching on the pixel-level mask of the photovoltaic module to obtain an effective module region mask; calculating the pixel coverage ratio of the hot spot detection frame within the effective module region mask as the coverage degree; calculating the temperature centroid coordinates of the hot spot detection frame; when the coverage degree is greater than a preset coverage threshold and the temperature centroid coordinates are located within the effective module region mask, the hot spot is determined to be a valid hot spot; otherwise, the hot spot is removed as a false hot spot.

[0047] Specifically, firstly, pixel-level masks for photovoltaic modules. Morphological etching is performed using a structuring element S of size n*n (e.g., 5*5) to etch the mask, resulting in a mask for the effective component region. ; This operation removes interfering pixels at component edges, avoiding misjudgments caused by edge thermal stress. Next, the hotspot detection bounding box is calculated. Post-corrosion masking The coverage ratio, or inclusion degree: ; Among them, the area of ​​the detection frame is calculated as follows: ; Calculate the total number of pixels within the detection bounding box that belong to the effective mask area: .

[0048] Calculate the temperature centroid coordinates of the hot spot detection frame. This coordinate is determined by the temperature value of each pixel within the bounding box. The weighted average yields: ; .

[0049] Determine if the coverage is greater than a preset coverage threshold, such as 0.95, and whether the temperature centroid coordinates are located within the effective mask area. Within the database, a hotspot is considered a valid hotspot if both conditions are met simultaneously; otherwise, it is removed as a false hotspot and not included in the subsequent time-series analysis database. For high-frequency false alarm areas, their location information can be collected and added to a whitelist to optimize false alarm filtering during subsequent inspections.

[0050] This application obtains an effective module area mask by performing morphological etching on the pixel-level mask of the photovoltaic module, effectively eliminating interference caused by thermal stress at the module edge. By calculating the pixel coverage ratio (i.e., the coverage) of the hot spot detection frame within the effective area, as well as the temperature centroid coordinates of the hot spot, and setting dual judgment conditions, it can accurately identify and eliminate false hot spots caused by edge reflection, ground debris, lens flare, etc., significantly improving the purity of the detection data, preventing false alarms from entering the subsequent analysis process, and effectively reducing the false alarm rate.

[0051] In some embodiments, the step of extracting the dynamic background reference temperature within the photovoltaic module based on the pixel-level mask and the hot spot detection frame specifically includes: performing a Boolean operation to remove the pixel area covered by the hot spot detection frame within the area covered by the pixel-level mask; performing morphological erosion processing on the remaining area to remove edge interference pixels; and calculating the average radiation temperature of all pixels in the remaining area after erosion as the dynamic background reference temperature of the module.

[0052] Specifically, in the pixel-level mask of photovoltaic modules Within the covered area, hot spot detection frames are removed using Boolean operations. All pixels are covered, resulting in a pixel set containing only the clean area of ​​the component. Then, morphological erosion is performed on this remaining area to remove any potentially interfering edge pixels. Finally, the average radiation temperature of all pixels in the remaining area after erosion is calculated as the dynamic background reference temperature for the component. The background reference temperature This reflects the typical temperature level of the module in the hot spot-free region under current operating conditions, providing an accurate benchmark for subsequent normalized hot spot index.

[0053] This application achieves accurate extraction of dynamic background reference temperature by performing Boolean operations within a pixel-level mask area to remove the area covered by the hot spot detection frame, and then calculating the average radiation temperature after morphological erosion processing of the remaining area. This background temperature reflects the true temperature level of the component in the hot spot-free area under the current operating conditions, and more accurately reflects the operating status of the component compared to a fixed reference point or the average value of the entire image.

[0054] In some embodiments, the step of constructing a normalized hotspot index by combining the image metadata specifically includes: extracting real-time irradiance, gimbal pitch angle, and fuselage attitude angle from the image metadata; calculating the thermal radiation incident angle based on the gimbal pitch angle and fuselage attitude angle; and constructing an irradiance compensation operator. And calculate the normalized hot spot index. In the formula: Here is a compensation operator based on real-time irradiance, where G is the real-time irradiance. For reference irradiance, The highest pixel temperature, For background reference temperature, The incident angle for comprehensive thermal radiation.

[0055] Specifically, real-time irradiance G, gimbal pitch angle, and fuselage attitude angle are extracted from image metadata, and the comprehensive thermal radiation incident angle is calculated based on the gimbal pitch angle and fuselage attitude angle. This angle reflects the incident relationship of sunlight relative to the component plane. Construct an irradiance compensation operator: And calculate the normalized hot spot index. In the formula: Here is a compensation operator based on real-time irradiance, where G is the real-time irradiance. For reference irradiance, The highest pixel temperature, For background reference temperature, The normalized hot spot index is calculated based on the incident angle of the combined thermal radiation. The influence of irradiance variations and shooting angle differences on temperature values ​​has been eliminated, making hot spot data acquired at different times and under different environmental conditions comparable.

[0056] This application extracts real-time irradiance, gimbal pitch angle, and fuselage attitude angle from image metadata, calculates the thermal radiation incident angle, and constructs an irradiance compensation operator to ultimately form a normalized hotspot index. By introducing an environmental compensation factor, it eliminates temperature fluctuations caused by changes in solar irradiance and differences in shooting angles during different inspection periods, enabling hotspot data acquired at different times and under different environmental conditions to have a unified and comparable benchmark.

[0057] In some embodiments, the step of constructing a time-series feature vector of the same hot spot at different time points using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot on the photovoltaic module as indexes specifically includes: using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot in the module's local normalized coordinate system as a joint index; retrieving hot spot records with the same joint index key from historical data of multiple inspections, and extracting the corresponding normalized hot spot index from each record; arranging the extracted normalized hot spot index values ​​in chronological order to construct the time-series feature vector.

[0058] Specifically, the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot in the module's local normalized coordinate system are used as the joint index key. This key can uniquely identify a specific physical hot spot. In the system's time-series database, all hot spot records with the same joint index key are retrieved. Each record includes the normalized hot spot index at the corresponding inspection time point. The retrieved NHSI values ​​are arranged in chronological order according to the inspection time to construct a temporal feature vector of the hot spot. Where n≥2. Temporal feature vector This reflects the evolution of hot spot intensity over time. This application uses the unique identifier of the photovoltaic module and the normalized absolute position coordinates of the hot spot as a joint index to retrieve hot spot records with the same index from historical data from multiple inspections. It then extracts the normalized hot spot index in chronological order to construct a time-series feature vector, thus achieving effective organization and storage of individual hot spot lifecycle data.

[0059] In some embodiments, the step of calculating the statistical characteristics of the time-series feature vector and comparing them with preset thresholds to determine the hotspot type specifically includes: calculating the mean and the coefficient of variation of the time-series feature vector; comparing the mean and the coefficient of variation with preset mean thresholds, stability thresholds, and fluctuation thresholds, respectively; if the mean is greater than the mean threshold and the coefficient of variation is less than the stability threshold, it is determined to be a permanent hotspot; if the coefficient of variation is greater than the fluctuation threshold or there is a normalized hotspot index that is approximately zero in the time-series feature vector, it is determined to be a transient hotspot; if the mean is greater than the mean threshold and the coefficient of variation is between the stability threshold and the fluctuation threshold, it is determined to be a suspected hotspot.

[0060] Specifically, time series mean : Reflects the average temperature rise energy consumption level of hot spots The coefficient of variation (CV) reflects the degree of fluctuation in the hot spot over time. .

[0061] Permanent hotspot determination: The average value exceeds the preset threshold, and the fluctuation is small. and This indicates that the hot spot energy is significant and the anomaly does not disappear with time or environmental changes.

[0062] Transient hotspot determination: Hotspot values ​​fluctuate drastically over time, or disappear during subsequent inspections. or .

[0063] Suspected hot spot identification: Hot spots persist but the values ​​are unstable. and .

[0064] in, It is the time series mean. It is a preset threshold. These are the discrete coefficients. It is the stability threshold. It is the fluctuation threshold.

[0065] This application establishes a clear set of hotspot classification rules by calculating the mean and coefficient of variation of the time-series feature vector and comparing them with preset mean threshold, stability threshold, and fluctuation threshold. The mean reflects the average energy level of the hotspot, and the coefficient of variation reflects the temporal stability of the hotspot. When the mean is high and the coefficient of variation is low, it is judged as a permanent hotspot, indicating that maintenance intervention is required; when the coefficient of variation is high or the hotspot disappears, it is judged as a transient hotspot, indicating that no action is required; when it falls between the two, it is judged as a suspected hotspot, indicating that enhanced monitoring is required. This achieves automatic and accurate classification of hotspot properties, fundamentally solving the problem that existing technologies cannot distinguish hotspot types and significantly reducing ineffective maintenance costs.

[0066] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0067] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0068] The above are merely embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention's specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment, characterized in that the steps are as follows: include: Acquire infrared radiation images of photovoltaic modules collected by drones and simultaneously extract image metadata; The infrared radiation image is input into the photovoltaic panel semantic segmentation model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module; The infrared radiation image is input into the hot spot target detection model to determine the position of the hot spot detection box in the infrared radiation image, calculate the coordinates of the center pixel of the hot spot detection box, and extract the highest temperature of the pixel in the hot spot detection box as the highest pixel temperature. Based on the set of boundary corner points and the preset standard rectangular normalized space vertices, a homography mapping matrix is ​​calculated to establish a perspective transformation relationship from the image coordinate system to the component local normalized coordinate system. The coordinates of the center pixel point are mapped to the component local normalized coordinate system through the homography mapping matrix to obtain the absolute position coordinates of the hot spot on the photovoltaic module. The spatial validity of the hot spot is verified, and false hot spots that do not meet the internal spatial constraints of the photovoltaic module are eliminated. Based on the pixel-level mask and hot spot detection frame, the dynamic background reference temperature inside the photovoltaic module is extracted, and combined with the image metadata, a normalized hot spot index is constructed. Using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot on the photovoltaic module as indexes, a temporal feature vector of the same hot spot at different time points is constructed. The statistical features of the temporal feature vector are calculated and compared with a preset threshold to determine the hot spot type.

2. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 1, characterized in that, The step of inputting the infrared radiation image into the photovoltaic panel semantic segmentation model to extract the pixel-level mask and its boundary corner point set of the photovoltaic module specifically includes: The semantic segmentation model for photovoltaic panels adopts an instance segmentation network based on YOLOv11-seg. Its backbone network adopts the CSPDarknet structure, the neck network adopts PANet for multi-scale feature fusion, and the head network contains parallel detection branches and mask branches. After scaling the infrared radiation image to a uniform size, it is input into the semantic segmentation model of the photovoltaic panel, and the pixel-level segmentation mask of the photovoltaic module is output. Based on the mask, the four vertices of the minimum bounding polygon are extracted as the set of boundary corner points.

3. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 1, characterized in that, The step of inputting the infrared radiation image into the hot spot target detection model and determining the position of the hot spot detection box in the infrared radiation image specifically includes: The hot spot target detection model adopts a target detection network based on YOLOv11-det, and its backbone network and neck structure are shared with the photovoltaic panel semantic segmentation model, with the head being the detection branch. After scaling the infrared radiation image to a uniform size, it is input into the hot spot target detection model, and the rectangular detection box of the hot spot and its confidence level are output. Calculate the coordinates of the center pixel of the hot spot detection frame, locate the coverage area of ​​the hot spot detection frame in the infrared radiation image, traverse the temperature values ​​of all pixels within the coverage area, and take the maximum value as the highest pixel temperature.

4. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 1, characterized in that, The step of calculating the homography mapping matrix based on the set of boundary corner points and the preset standard rectangular normalized space vertices specifically includes: The set of boundary corner points is used as the four boundary corner points of the photovoltaic module, and the preset standard rectangular normalized space vertices are used as the normalized local coordinate system point set. Using the direct linear transformation algorithm, a system of linear equations is constructed based on the four pairs of matching points between the boundary corner point set and the normalized local coordinate system point set, and solved by singular value decomposition to obtain the homography mapping matrix.

5. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 4, characterized in that, The step of mapping the coordinates of the center pixel to the local normalized coordinate system of the module through the homography mapping matrix to obtain the absolute position coordinates of the hot spot on the photovoltaic module specifically includes: Substitute the coordinates of the center pixel of the hot spot detection frame into the transformation equation defined by the homography mapping matrix to calculate the intermediate variable; The intermediate variables are normalized to obtain the absolute position coordinates of the hot spot on the photovoltaic module.

6. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 5, characterized in that, The specific steps for verifying the spatial validity of hot spots and eliminating false hot spots that do not meet the internal space constraints of photovoltaic modules include: The pixel-level mask of the photovoltaic module is subjected to morphological etching to obtain an effective module area mask; The pixel coverage ratio of the hot spot detection frame within the effective component region mask is calculated as the inclusion degree; Calculate the temperature centroid coordinates of the hot spot detection frame; When the inclusion degree is greater than the preset inclusion degree threshold and the temperature centroid coordinate is located within the mask of the effective component area, the hot spot is determined to be a valid hot spot; otherwise, the hot spot is discarded as a false hot spot.

7. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 1, characterized in that, The step of extracting the dynamic background reference temperature within the photovoltaic module based on the pixel-level mask and hot spot detection frame specifically includes: Within the area covered by the pixel-level mask, Boolean operations are performed to remove the pixel area covered by the hot spot detection box; Morphological erosion is performed on the remaining area to remove edge interference pixels; Calculate the average radiation temperature of all pixels in the remaining area after erosion, and use it as the dynamic background reference temperature for this component.

8. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 7, characterized in that, The step of constructing the normalized hotspot index by combining the image metadata specifically includes: Real-time irradiance, gimbal pitch angle, and fuselage attitude angle are extracted from the image metadata. Calculate the incident angle of thermal radiation based on the gimbal pitch angle and the fuselage attitude angle; An irradiance compensation operator is constructed, and a normalized hot spot index is calculated based on the irradiance compensation operator.

9. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 8, characterized in that, The step of constructing a temporal feature vector of the same hot spot at different time points using the unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot on the photovoltaic module as indexes specifically includes: The unique identifier of the photovoltaic module and the absolute position coordinates of the hot spot in the module's local normalized coordinate system are used as the joint index; In the historical data of multiple inspections, hot spot records with the same composite index key are retrieved, and the normalized hot spot index corresponding to each record is extracted. The extracted normalized hot spot index values ​​are arranged in chronological order to construct the temporal feature vector.

10. The photovoltaic hot spot classification and diagnosis method based on dual-model collaboration and temporal spatial alignment according to claim 9, characterized in that, The step of calculating the statistical features of the time-series feature vector and comparing them with a preset threshold to determine the hotspot type specifically includes: Calculate the mean and coefficients of variation of the time-series feature vector; The mean and the coefficient of variation are compared with preset mean threshold, stability threshold and fluctuation threshold, respectively; If the mean is greater than the mean threshold and the coefficient of variation is less than the stability threshold, it is determined to be a permanent hot spot; If the discrete coefficient is greater than the fluctuation threshold or there is a normalized hot spot index that is approximately zero in the time-series feature vector, it is determined to be a transient hot spot. If the mean is greater than the mean threshold and the coefficient of variation is between the stability threshold and the fluctuation threshold, it is judged as a suspected hot spot.

Images