An image recognition-based intelligent photovoltaic module defect inspection method

Abstract

This invention discloses an intelligent inspection method for photovoltaic modules based on image recognition, comprising the following steps: acquiring visible light and infrared images of the photovoltaic modules and recording flight control positioning information, storing them as raw image data; preprocessing the raw image data to obtain preprocessed images; extracting features from the preprocessed images and organizing them into a modal feature set; mapping the modal feature set uniformly to a manifold space to generate a manifold feature representation; applying robust sparse subspace clustering to the manifold feature representation to output a comprehensive feature representation; using the comprehensive feature representation as input to identify and classify surface defects, outputting defect category information; combining the defect category information with the flight control positioning information to form a positioning result; generating a defect distribution map and an inspection report based on the defect category information and the positioning result, and outputting maintenance suggestions. This invention realizes intelligent inspection of photovoltaic module defects.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic power plant operation and maintenance technology, and in particular to an intelligent inspection method for photovoltaic module defects based on image recognition. Background Technology

[0002] Existing photovoltaic module inspection technologies primarily rely on manual inspection and drone-based image acquisition. Manual inspection, due to the large scale of power plants and the sheer number of modules, often requires significant manpower and time, and is easily affected by differences in personnel experience, leading to inconsistent results. Furthermore, existing technologies generally suffer from problems when processing multimodal data; most methods simply stitch visible light and infrared images together or analyze them independently, lacking an effective multimodal feature fusion mechanism. This approach easily causes feature distortion or alignment errors between different modes, resulting in insufficient accuracy in defect identification, especially in environments with strong light variations, dust covering the module surface, or interference from reflected noise, leading to frequent misjudgments and missed detections.

[0003] Most existing image processing and analysis methods use traditional feature extraction and classification algorithms, which have limited ability to model complex component surface feature changes and infrared thermal distribution information. They are also unable to cope with the relationships between high-dimensional and nonlinear features. This leads to insufficient stability of detection results in the process of defect identification and localization. At the same time, existing methods often only output defect categories without deep integration with flight control and positioning information, making it difficult to accurately map to specific components of the photovoltaic array, which affects subsequent operation and maintenance efficiency and decision support.

[0004] Therefore, how to provide an intelligent inspection method for defects in photovoltaic modules based on image recognition is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose an intelligent inspection method for photovoltaic module defects based on image recognition. This invention uses a drone to collect visible light and infrared images and combines them with flight control positioning information. After preprocessing and feature extraction, multimodal features are uniformly mapped to a manifold space, and robust sparse subspace clustering is used to achieve feature fusion and generate a comprehensive feature representation. This method can automatically identify and classify surface defects of photovoltaic modules, obtain the defect location by combining positioning information, further generate a defect distribution map and inspection report, and output maintenance suggestions, thereby achieving efficient, accurate and intelligent inspection of photovoltaic module defects.

[0006] An intelligent inspection method for photovoltaic modules based on image recognition according to an embodiment of the present invention includes the following steps:

[0007] The visible light camera and infrared thermal imager on the drone are used to simultaneously collect visible light and infrared images of the photovoltaic modules, and the drone's flight control and positioning information is recorded in real time. The collected results are stored as raw image data.

[0008] The original image data is preprocessed, including using denoising algorithms to eliminate image noise, using geometric correction methods to eliminate distortion, using brightness equalization methods to balance illumination differences, and performing image enhancement processing to obtain a preprocessed image with optimized quality.

[0009] Feature extraction is performed on the preprocessed image. Texture and edge information are extracted from the visible light image, and temperature distribution and anomaly information are extracted from the infrared image. These are then organized into a modal feature set.

[0010] Different modal features of the modal feature set are uniformly mapped to the Grassmann manifold space to generate manifold feature representations that maintain geometric consistency;

[0011] Robust sparse subspace clustering is applied to the manifold feature representation within the Grassmann manifold space. The features are fused and clustered using sparse representation and robust constraints, and a comprehensive feature representation is output.

[0012] Using comprehensive feature representation as input, the system automatically identifies and classifies surface defects of photovoltaic modules, and outputs defect category information including cracks, hot spots, delamination, ablation, and shading.

[0013] By combining the defect category information with the flight control positioning information, historical detection data is constructed, and the actual position of the defect in the photovoltaic module is calculated through coordinate mapping and position correction methods to form a positioning result.

[0014] Based on the defect category information and location results, a defect distribution map and detection report are generated. Trend analysis is performed in conjunction with historical detection data. When the analysis results exceed a set threshold, an early warning mechanism is triggered and maintenance suggestions are output.

[0015] Optionally, the storage process of the original image data specifically includes:

[0016] The drone is controlled to fly over the photovoltaic power station at a recorded speed and altitude according to the preset route. It uses a visible light camera to collect images of the photovoltaic module surface and assigns a unique identifier to each frame of visible light image, records the shooting time, records the imaging resolution and lens focal length, records the internal and external parameters of the camera and exposure information, and writes the frame sequence number, timestamp and camera identifier of the same batch into the image metadata to form the first image data.

[0017] The infrared thermal imager on the drone is used to perform thermal imaging acquisition on the same flight path coverage area, and the frame number, timestamp and sensor identifier are written into the image metadata to form the second image data. The timestamp and frame number of the second image data are associated with the first image data in a one-to-one correspondence.

[0018] Record the UAV flight control positioning information, which includes geographical coordinates, attitude angle information, flight speed and heading angle, as well as a timestamp and unique identifier consistent with the imaging frame. The first image data, the second image data, and the flight control positioning information are associated and stored with the same timestamp and unique identifier to form raw image data. The raw image data includes visible light images, infrared images, corresponding imaging and calibration metadata, and corresponding flight control positioning information.

[0019] Optionally, obtaining the quality-optimized preprocessed image specifically includes:

[0020] The system receives raw image data, pairs and sorts visible light image sequences and infrared image sequences according to timestamps, and performs consistency checks on resolution, imaging parameters and unique identifiers. After removing frames with missing timestamps and inconsistent parameters, a set of paired frames is generated.

[0021] The grayscale histogram of the paired frame set is statistically analyzed frame by frame, and brightness equalization processing is performed on each frame while keeping the original spatial resolution, timestamp and unique identifier unchanged, to obtain a brightness equalized image sequence.

[0022] The brightness-equalized image sequence is sequentially enhanced with edge enhancement, detail sharpening, and contrast improvement. A small-scale dynamic range extension is applied to the infrared image to highlight areas of temperature abrupt change. The enhanced image is then re-associated and verified with the corresponding flight control positioning information according to the timestamp and unique identifier, and the pre-processed image data with optimized quality is output.

[0023] Optionally, the organization of the modal feature set specifically includes:

[0024] Receive the pre-processed image data with optimized quality, pair the visible light image sequence and the infrared image sequence one by one according to the timestamp and unique identifier consistent with the acquisition time, and renumber the frames that pass the verification according to the time sequence to form an aligned set of pre-processed frames.

[0025] Feature extraction is performed frame by frame on the visible light images in the aligned preprocessed frame set. The texture is statistically analyzed to form texture description information. Edge direction and edge intensity are extracted to form edge description information. The texture description information and edge description information of the same frame are merged according to the predetermined field order, and the visible light feature record sequence is output in chronological order.

[0026] Feature extraction is performed frame by frame on the infrared images in the aligned preprocessed frame set. Temperature distribution description information is statistically generated, and the location, range and intensity of anomalies are marked to form anomaly description information. The temperature distribution description information and anomaly description information of the same frame are merged according to the predetermined field order, and the infrared feature record sequence is output.

[0027] The visible light feature recording sequence and the infrared feature recording sequence are paired frame by frame according to the timestamp and unique identifier. For each pair of paired records, dimensional unification and numerical standardization are performed while keeping the original timestamp and unique identifier unchanged. The processed paired records are summarized and stored in chronological order as a modal feature set.

[0028] Optionally, the process of generating the manifold feature representation specifically includes:

[0029] Receive a set of modal features, perform dimensional analysis on the visible light and infrared feature records in the set, determine the dimensional size and numerical range of each feature record, expand or compress inconsistent dimensions to a uniform dimensional length, perform linear normalization on the numerical range, and output modal feature pairs with uniform dimensions.

[0030] The visible light feature records in the modal feature pairs are represented in a subspace. The texture description information and edge description information of each frame are combined into a vector representation in sequence and orthogonalized to obtain a subspace basis that maintains linear independence. The visible light subspace basis sequence is then output.

[0031] The infrared feature records in the modal feature pairs are represented in a subspace. The temperature distribution description information and anomaly point description information of each frame are combined into a vector representation in sequence and orthogonalized to obtain a subspace basis that maintains linear independence. The infrared subspace basis sequence is then output.

[0032] The visible light subspace basis sequence and the infrared subspace basis sequence are paired frame by frame according to the same timestamp and unique identifier, uniformly mapped to the Grassmann manifold space, and transformed into point representation in the manifold space. They are then stored in chronological order as manifold feature representations.

[0033] Optionally, the output process of the comprehensive feature representation specifically includes:

[0034] The manifold feature representations are sorted sequentially according to timestamp and unique identifier. The point descriptions of each frame in the manifold space are grouped and bound with the corresponding visible light subspace basis descriptions and infrared subspace basis descriptions to obtain an index mapping table and the corresponding manifold point sequence.

[0035] The similarity weights of the manifold point sequences are calculated pairwise. Based on the orthogonal subspace basis of each frame, the protagonist set between them is calculated. The protagonist set consists of a set of angles from zero to a semicircle, reflecting the relative angle between the subspaces of the two frames. Based on the current angle, a geodesic length is obtained to measure the geometric distance between the two frames on the Grassmann manifold. Substituting this length into an exponential decay rule, the first part of the similarity score is obtained. At the same time, based on the volume consistency between the subspace basis of the two frames, a volume coherence factor between zero and one is calculated and corrected by a preset positive exponent to obtain the second part of the correction factor. The first part of the similarity score is multiplied by the second part of the correction factor to obtain the similarity weight between the two frames. The similarity weights calculated pairwise for all frames are filled into a symmetric similarity weight matrix according to their row and column positions, maintaining a one-to-one correspondence with the timestamp and unique identifier.

[0036] Within the tangent space of the Grassmann manifold, taking the manifold points of each frame as references, other points in the current neighborhood are transformed into tangent space tensors through logarithmic mapping to obtain the corresponding set of tangent vectors. A weighted robust sparse subspace optimization model is established using the similarity weight matrix as the weighting coefficients for the reconstruction error. The model minimizes the sum of three parts: the weighted reconstruction error of the tangent space data, the row direction sparsity penalty of the sparse coefficient matrix, and the column direction robust penalty of the residual matrix. An augmented Lagrangian strategy is used to alternately update the sparse coefficient matrix and the residual matrix until the convergence condition is met. The convergence condition is that the decrease in reconstruction error and the change in coefficients are both lower than the convergence threshold. The output is the reconstruction coefficient matrix and residual matrix corresponding to the timestamp and unique identifier frame by frame.

[0037] The sparse coefficient matrix is ​​symmetricized to generate a symmetric correlation matrix. A graph structure is constructed by combining the index information corresponding to the index mapping table. Then, the clustering index of each frame is obtained through spectral decomposition. The clustering index is merged with the timestamp, unique identifier and manifold point description to form a comprehensive feature representation for photovoltaic module defect patterns.

[0038] Optionally, the process of outputting the defect category information specifically includes:

[0039] Using the comprehensive feature representation as input, the clustering index of each frame is bound together with the visible light and infrared features corresponding to the current frame to obtain a single feature record. For the surface defects of photovoltaic modules, all feature records are stored in a unified input dataset in chronological order.

[0040] The standardized input dataset is processed frame by frame. Each feature is traversed, and the minimum and maximum values ​​of the current feature in the current batch are calculated. Then, each feature is scaled frame by frame to be mapped to the range between zero and one. Missing values ​​and outliers that appear during the scaling process are removed, truncated or imputed. Each feature record is re-attached with a corresponding timestamp and unique identifier, and the standardized feature dataset is output.

[0041] The standardized feature dataset is input into a classification method trained on historical defect samples. This classification method is established through supervised learning on a large number of labeled photovoltaic module defect images. It can identify the defect categories corresponding to different feature patterns, read each feature record frame by frame, and generate the defect category prediction result and corresponding confidence score for the current frame. The historical defect samples are defect samples collected during the daily inspection of photovoltaic power stations and manually labeled. The input features are judged, and the defect category prediction result and corresponding confidence score for the current frame are output. These are bound to timestamps and unique identifiers to form defect category information.

[0042] Optionally, the process of forming the positioning result specifically includes:

[0043] Receive defect category information and match it frame by frame with the flight control positioning information to form historical detection data. At the same time, establish a joint index table of timestamp-unique identifier-defect category-flight control positioning. Each frame defect category is bound to the corresponding UAV three-dimensional positioning coordinates and attitude angle information to generate a positioning input dataset.

[0044] Coordinate calculation is performed on the positioning input dataset. The longitude, latitude, and altitude information of the UAV during flight are combined with the flight attitude angle to deduce the spatial position and orientation of the camera coordinate system in the geographic coordinate system during image acquisition. The defect pixel position in the image coordinate system is mapped to the physical position in the geographic coordinate system through geometric projection relationship to obtain the preliminary positioning result of the defect in the actual space.

[0045] Based on the preliminary location results, the defect category of each frame is attached with the corresponding geographic coordinates, and then corrected in combination with the arrangement rules of the photovoltaic module array. The geographic coordinates are then matched with the specific photovoltaic module number and the module row and column index to form the location result.

[0046] Optionally, the output of the maintenance recommendations specifically includes:

[0047] Defect category labels are bound to corresponding component numbers, row and column indices, and geographic coordinates, arranged according to the spatial distribution order of the component array, and a defect distribution map is drawn on a two-dimensional plane. The distribution map uses the component matrix as the background and different symbols or colors to identify various defects, and the defect distribution map is output as a visual display.

[0048] Based on the generated defect distribution map, the number and distribution location of each type of defect are statistically analyzed, and a detection report is generated by combining the timestamp information. The detection report includes the defect category, the number of defects, the specific location of the defects in the component array, and the corresponding historical occurrence count.

[0049] The historical detection data is used to perform trend analysis on the changes in the number and spatial distribution of defects. The frequency of occurrence of the same defect category in different detection cycles is statistically analyzed on the time axis, and the concentration of defect locations is analyzed in the spatial dimension to obtain trend curves and spatial distribution indices. When the growth rate of the trend curve or the spatial distribution index exceeds a preset threshold, an early warning mechanism is triggered, and maintenance suggestions are output. The maintenance suggestions include the component numbers to be prioritized, the possible fault range, and recommended maintenance measures.

[0050] The beneficial effects of this invention are:

[0051] 1. This invention achieves a unified mapping of the modal features of visible light and infrared images in the Grassmann manifold space, avoiding the feature distortion and modal misalignment problems caused by simple splicing or independent analysis in traditional methods, thereby effectively improving the accuracy and robustness of defect identification.

[0052] 2. Based on manifold feature representation, this invention introduces a robust sparse subspace clustering method, which realizes efficient fusion and clustering of complex high-dimensional features. It can resist external interference such as illumination changes, dust occlusion and reflection noise, and improve the stability and reliability of defect detection results.

[0053] 3. This invention combines defect category information with UAV flight control positioning information to accurately map to the specific component number and location in the photovoltaic array, achieving precise defect location and greatly reducing the workload of subsequent re-inspection and maintenance.

[0054] 4. This invention also constructs historical detection data and combines it with a trend analysis mechanism, which can not only output defect distribution maps and detection reports, but also trigger early warnings and provide maintenance suggestions when the defect development trend exceeds a set threshold. This realizes the transformation from single static detection to long-term intelligent prediction and auxiliary decision-making, which significantly improves the operation and maintenance efficiency and safety of photovoltaic power plants. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of an intelligent inspection method for photovoltaic modules based on image recognition proposed in this invention;

[0057] Figure 2 This is a schematic diagram of the algorithm structure of an intelligent inspection method for photovoltaic modules based on image recognition proposed in this invention. Detailed Implementation

[0058] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0059] refer to Figure 1-2 A method for intelligent inspection of defects in photovoltaic modules based on image recognition includes the following steps:

[0060] The visible light camera and infrared thermal imager on the drone are used to simultaneously collect visible light and infrared images of the photovoltaic modules, and the drone's flight control and positioning information is recorded in real time. The collected results are stored as raw image data.

[0061] The original image data is preprocessed, including using denoising algorithms to eliminate image noise, using geometric correction methods to eliminate distortion, using brightness equalization methods to balance illumination differences, and performing image enhancement processing to obtain a preprocessed image with optimized quality.

[0062] Feature extraction is performed on the preprocessed image. Texture and edge information are extracted from the visible light image, and temperature distribution and anomaly information are extracted from the infrared image. These are then organized into a modal feature set.

[0063] Different modal features of the modal feature set are uniformly mapped to the Grassmann manifold space to generate manifold feature representations that maintain geometric consistency;

[0064] Robust sparse subspace clustering is applied to the manifold feature representation within the Grassmann manifold space. The features are fused and clustered using sparse representation and robust constraints, and a comprehensive feature representation is output.

[0065] Using comprehensive feature representation as input, the system automatically identifies and classifies surface defects of photovoltaic modules, and outputs defect category information including cracks, hot spots, delamination, ablation, and shading.

[0066] By combining the defect category information with the flight control positioning information, historical detection data is constructed, and the actual position of the defect in the photovoltaic module is calculated through coordinate mapping and position correction methods to form a positioning result.

[0067] Based on the defect category information and location results, a defect distribution map and detection report are generated. Trend analysis is performed in conjunction with historical detection data. When the analysis results exceed a set threshold, an early warning mechanism is triggered and maintenance suggestions are output.

[0068] This invention combines flight control positioning information to achieve precise location of defects in photovoltaic arrays, reducing the difficulty of re-inspection and maintenance. At the same time, it constructs historical detection data and conducts trend analysis, which can trigger early warnings and output maintenance suggestions in a timely manner, realizing the expansion from single detection to intelligent prediction and operation and maintenance decision-making, and significantly improving the operational safety and maintenance efficiency of photovoltaic power plants.

[0069] In this embodiment, the storage process of the original image data specifically includes:

[0070] The drone is controlled to fly over the photovoltaic power station at a recorded speed and altitude according to the preset route. It uses a visible light camera to collect images of the photovoltaic module surface and assigns a unique identifier to each frame of visible light image, records the shooting time, records the imaging resolution and lens focal length, records the internal and external parameters of the camera and exposure information, and writes the frame sequence number, timestamp and camera identifier of the same batch into the image metadata to form the first image data.

[0071] Within the time window of visible light imaging synchronization, the infrared thermal imager on the UAV is used to perform thermal imaging acquisition on the same flight path coverage area. Each frame of infrared image is assigned a unique identifier, the shooting time is recorded, the infrared imaging resolution and band information are recorded, the internal and external parameters and radiometric calibration information of the thermal imager are recorded, and the frame number, timestamp and sensor identifier are written into the image metadata to form the second image data. The timestamp and frame number of the second image data are associated with the first image data in a one-to-one correspondence.

[0072] Record the UAV flight control positioning information, which includes geographical coordinates, attitude angle information, flight speed and heading angle, as well as a timestamp and unique identifier consistent with the imaging frame. The first image data, the second image data, and the flight control positioning information are associated and stored with the same timestamp and unique identifier to form raw image data. The raw image data includes visible light images, infrared images, corresponding imaging and calibration metadata, and corresponding flight control positioning information.

[0073] This invention achieves one-to-one correspondence and precise association of multi-source data by simultaneously acquiring visible light images, infrared images, and flight control positioning information along a preset flight path of an unmanned aerial vehicle (UAV), and fully recording imaging parameters, calibration information, and timestamps in the image metadata. This ensures a high degree of consistency between multimodal images and positioning data in time and space, avoids the accuracy problems caused by data misalignment and parameter loss in traditional methods, and thus significantly improves the reliability and traceability of inspections.

[0074] In this embodiment, obtaining the quality-optimized preprocessed image specifically includes:

[0075] The system receives raw image data, pairs and sorts visible light image sequences and infrared image sequences according to timestamps, and performs consistency checks on resolution, imaging parameters and unique identifiers. After removing frames with missing timestamps and inconsistent parameters, a set of paired frames for brightness processing is generated.

[0076] The grayscale histogram of the paired frame set is statistically analyzed frame by frame to obtain the minimum grayscale value, maximum grayscale value, average grayscale value and contrast index of each frame. Then, brightness equalization processing is performed on each frame to standardize the pixel brightness to a uniform range from zero to 255, while keeping the original spatial resolution, timestamp and unique identifier unchanged, to obtain a brightness equalization image sequence.

[0077] The brightness-equalized image sequence is sequentially enhanced with edge enhancement, detail sharpening, and contrast improvement. A small-scale dynamic range extension is applied to the infrared image to highlight areas of temperature abrupt change. The enhanced image is then re-associated and verified with the corresponding flight control positioning information according to the timestamp and unique identifier, and the pre-processed image data with optimized quality is output.

[0078] This invention ensures the integrity and reliability of input data by performing timestamp pairing and parameter consistency verification on visible light and infrared image sequences. Based on this, it implements processes such as brightness equalization, edge enhancement, detail sharpening, and infrared dynamic range expansion to effectively improve image clarity and contrast. This method enhances the visibility of defective areas while preserving the original timestamps and unique identifiers.

[0079] In this embodiment, the organization of the modal feature set specifically includes:

[0080] Receive the pre-processed image data with optimized quality, pair the visible light image sequence and the infrared image sequence one by one according to the timestamp and unique identifier consistent with the acquisition time, and renumber the frames that pass the verification according to the time sequence to form an aligned set of pre-processed frames.

[0081] Feature extraction is performed frame by frame on the visible light images in the aligned preprocessed frame set. Local neighborhoods of fixed size are used to statistically analyze the texture and form texture description information. A gradient-based boundary detection method is used to extract edge direction and edge intensity to form edge description information. The texture description information and edge description information of the same frame are merged according to a predetermined field order and the timestamp and unique identifier of the current frame are attached. The output is a visible light feature record sequence arranged in chronological order.

[0082] Feature extraction is performed frame by frame on the infrared images in the aligned preprocessed frame set. Temperature distribution description information is generated by statistical analysis of pixel brightness after radiometric calibration. The location, range and intensity of outliers are marked by a method based on local extrema and connectivity to form outlier description information. The temperature distribution description information and outlier description information of the same frame are merged according to a predetermined field order and the timestamp and unique identifier of the current frame are attached. The infrared feature record sequence is output in chronological order.

[0083] The visible light feature recording sequence and the infrared feature recording sequence are paired frame by frame according to the timestamp and unique identifier. For each pair of paired records, dimensional unification and numerical standardization are performed while keeping the original timestamp and unique identifier unchanged. The processed paired records are summarized and stored in chronological order as a modal feature set.

[0084] This invention extracts multi-dimensional features such as texture, edges, temperature distribution, and anomalies, and constructs a modal feature set through unified dimensionality and standardization, ensuring the comparability of different modal features at the same scale. This method significantly enhances the expressive power of defect information and the effectiveness of data fusion, laying a solid foundation for subsequent manifold mapping and cluster analysis, and improving the accuracy and robustness of defect identification.

[0085] In this embodiment, the process of generating the manifold feature representation specifically includes:

[0086] Receive a set of modal features, perform dimensional analysis on the visible light and infrared feature records in the set, determine the dimensional size and numerical range of each feature record, expand or compress inconsistent dimensions to a uniform dimensional length, perform linear normalization on the numerical range, and output modal feature pairs with uniform dimensions.

[0087] The visible light feature records in the modal feature pairs are represented in a subspace. The texture description information and edge description information of each frame are combined into a vector representation and orthogonalized to obtain a subspace basis that maintains linear independence. At the same time, the relationship between the current subspace basis and the corresponding timestamp and unique identifier is recorded, and the visible light subspace basis sequence is output.

[0088] The infrared feature records in the modal feature pairs are represented in a subspace. The temperature distribution description information and anomaly point description information of each frame are combined into a vector representation and orthogonalized to obtain a subspace basis that maintains linear independence. At the same time, the relationship between the subspace basis and the corresponding timestamp and unique identifier is recorded, and the infrared subspace basis sequence is output.

[0089] The visible light subspace basis sequence and the infrared subspace basis sequence are paired frame by frame according to the same timestamp and unique identifier, uniformly mapped to the Grassmann manifold space, and transformed into point representation in the manifold space. They are then stored in chronological order as manifold feature representations.

[0090] This invention represents visible light and infrared features separately using subspace orthogonalization and establishes a correspondence between timestamps and unique identifiers, thus maintaining the linear independence and traceability of the features. Finally, the multimodal subspace basis is uniformly mapped to the Grassmann manifold space to form point representations. This not only preserves the geometric structural relationships between modes but also improves the stability and discriminative power of feature fusion, providing high-quality input for subsequent robust clustering and defect identification.

[0091] In this embodiment, the output process of the comprehensive feature representation specifically includes:

[0092] The manifold feature representations are sorted sequentially according to timestamp and unique identifier. The point descriptions of each frame in the manifold space are grouped and bound with the corresponding visible light subspace basis descriptions and infrared subspace basis descriptions to obtain an index mapping table and the corresponding manifold point sequence.

[0093] The similarity weights of the manifold point sequences are calculated pairwise. Based on the orthogonal subspace basis of each frame, the protagonist set between them is calculated. The protagonist set consists of a set of angles from zero to a semicircle, reflecting the relative angle between the subspaces of the two frames. Based on the current angle, a geodesic length is obtained to measure the geometric distance between the two frames on the Grassmann manifold. Substituting this length into an exponential decay rule, the first part of the similarity score is obtained. At the same time, based on the volume consistency between the subspace basis of the two frames, a volume coherence factor between zero and one is calculated and corrected by a preset positive exponent to obtain the second part of the correction factor. The first part of the similarity score is multiplied by the second part of the correction factor to obtain the similarity weight between the two frames. The similarity weights calculated pairwise for all frames are filled into a symmetric similarity weight matrix according to their row and column positions, maintaining a one-to-one correspondence with the timestamp and unique identifier.

[0094] Within the tangent space of the Grassmann manifold, taking the manifold points of each frame as references, other points in the current neighborhood are transformed into tangent space tensors through a logarithmic mapping, resulting in the corresponding set of tangent vectors. A weighted robust sparse subspace optimization model is then established, using the similarity weight matrix as the weighting coefficients for the reconstruction error.

[0095]

[0096] Among them, Ξ i Let Z represent the tangent space representation of the i-th frame, and let Z denote the sparse coefficient matrix. iLet E represent the sparse coefficient column vector corresponding to the i-th frame, and let E represent the residual matrix. i Let ||·|| represent the residual vector corresponding to the i-th frame. F,W Let ∥Z∥ be the weighted F-norm that incorporates similarity weights. 1,2 To represent a mixed row-column sparsity constraint, first take the l2 norm of each row of the matrix, then take the l1 norm of all rows, ∥E∥ 2,1 This indicates that a robust constraint is applied to the residual matrix. First, the l2 norm of each column of the matrix is ​​taken, and then the l1 norm of all columns is taken. λ and μ represent the sparse regularization and robust regularization parameters, respectively. stdiag(Z) = 0 indicates that a self-representation constraint with zero diagonal elements is applied, so that the representation of each point can only depend on other points and not on itself. The sum of three parts, including the weighted reconstruction error of the tangent space data, the row direction sparse penalty of the sparse coefficient matrix, and the column direction robust penalty of the residual matrix, is minimized. An augmented Lagrangian strategy is used to alternately update the sparse coefficient matrix and the residual matrix until the convergence condition is met. The convergence condition is that the decrease in reconstruction error and the change in coefficients are both lower than the convergence threshold. The output is the reconstruction coefficient matrix and residual matrix corresponding to the timestamp and unique identifier frame by frame.

[0097] The sparse coefficient matrix is ​​symmetricized to generate a symmetric correlation matrix. A graph structure is constructed by combining the index information corresponding to the index mapping table. Then, the clustering index of each frame is obtained through spectral decomposition. The clustering index is merged with the timestamp, unique identifier and manifold point description to form a comprehensive feature representation for photovoltaic module defect patterns.

[0098] This invention introduces a similarity metric of protagonist set and volume coherence factor on the Grassmann manifold, and combines exponential decay and power correction mechanisms to construct a similarity weight matrix that preserves geometric structure and is robust. Furthermore, a weighted robust sparse subspace optimization model is established in the tangent space, achieving robust fusion of multimodal features. This ensures the independence between features while effectively suppressing the influence of noise and outliers, improving the accuracy and stability of defect pattern clustering, and enhancing the intelligence and reliability of the inspection system.

[0099] In this embodiment, the process of outputting the defect category information specifically includes:

[0100] Using the comprehensive feature representation as input, the clustering index of each frame is bound together with the visible light and infrared features corresponding to the current frame to obtain a single feature record. For the surface defects of photovoltaic modules, all feature records are stored in a unified input dataset in chronological order.

[0101] The standardized input dataset is processed frame by frame. Each feature is traversed, and the minimum and maximum values ​​of the current feature in the current batch are calculated. Then, each feature is scaled frame by frame to be mapped to the range between zero and one. Missing values ​​and outliers that appear during the scaling process are removed, truncated or imputed. Each feature record is re-attached with a corresponding timestamp and unique identifier, and the standardized feature dataset is output.

[0102] A standardized feature dataset is input into a classification method trained on historical defect samples. This classification method is established through supervised learning on a large number of labeled photovoltaic module defect images. It can identify the defect categories corresponding to different feature patterns. Each feature record is read frame by frame, and the defect category prediction result and corresponding confidence score of the current frame are generated. The historical defect samples are defect samples collected during the daily inspection of photovoltaic power stations and manually labeled. The input features are judged, and the defect category prediction result and corresponding confidence score of the current frame are output. They are bound to timestamps and unique identifiers to form defect category information including cracks, hot spots, delamination, ablation, and shading.

[0103] This invention constructs a unified input dataset and performs standardization processing to ensure consistency of features from different sources at the same scale, effectively eliminating interference from dimensional differences and outliers. Combined with a classification method trained under supervision based on historical defect samples, it achieves accurate discrimination of multimodal features and defect category identification, outputting various defect information including cracks, hot spots, delamination, ablation, and shading. This significantly improves the accuracy and robustness of defect identification and ensures the correspondence between the results and timestamps and unique identifiers, providing reliable support for the intelligent diagnosis of photovoltaic modules.

[0104] In this embodiment, the process of forming the positioning result specifically includes:

[0105] Receive defect category information and match it frame by frame with the flight control positioning information to form historical detection data. At the same time, establish a joint index table of timestamp-unique identifier-defect category-flight control positioning. Each frame defect category is bound to the corresponding UAV three-dimensional positioning coordinates and attitude angle information to generate a positioning input dataset.

[0106] Coordinate calculation is performed on the positioning input dataset. The longitude, latitude, and altitude information of the UAV during flight are combined with the flight attitude angle to deduce the spatial position and orientation of the camera coordinate system in the geographic coordinate system during image acquisition. The defect pixel position in the image coordinate system is mapped to the physical position in the geographic coordinate system through geometric projection relationship to obtain the preliminary positioning result of the defect in the actual space.

[0107] Based on the preliminary location results, the defect category of each frame is attached with the corresponding geographic coordinates, and then corrected in combination with the arrangement rules of the photovoltaic module array. The geographic coordinates are then matched with the specific photovoltaic module number and the module row and column index to form the location result.

[0108] This invention combines the arrangement rules of photovoltaic module arrays to complete position correction, so that defects can be matched one-to-one with specific module numbers and row and column indices. This effectively solves the problems of ambiguous defect location and difficulty in tracing in existing inspections, significantly improves the accuracy and reliability of positioning, provides intuitive and operable data support for subsequent maintenance, and enhances the practicality of the inspection system.

[0109] In this embodiment, the output of the maintenance suggestion specifically includes:

[0110] Defect category labels are bound to corresponding component numbers, row and column indices, and geographic coordinates, arranged according to the spatial distribution order of the component array, and a defect distribution map is drawn on a two-dimensional plane. The distribution map uses the component matrix as the background and different symbols or colors to identify various defects, and the defect distribution map is output as a visual display.

[0111] Based on the generated defect distribution map, the number and distribution location of each type of defect are statistically analyzed, and a detection report is generated by combining the timestamp information. The detection report includes the defect category, the number of defects, the specific location of the defects in the component array, and the corresponding historical occurrence count.

[0112] The historical detection data is used to perform trend analysis on the changes in the number and spatial distribution of defects. The frequency of occurrence of the same defect category in different detection cycles is statistically analyzed on the time axis, and the concentration of defect locations is analyzed in the spatial dimension to obtain trend curves and spatial distribution indices. When the growth rate of the trend curve or the spatial distribution index exceeds a preset threshold, an early warning mechanism is triggered, and maintenance suggestions are output. The maintenance suggestions include the component numbers to be prioritized, the possible fault range, and recommended maintenance measures.

[0113] This invention constructs a two-dimensional distribution map to visually display the defect situation by combining defect categories with component numbers, row and column indexes, and geographical coordinates. It also generates a test report that includes quantity statistics and historical comparisons, realizing the visualization and traceability of defect information. Furthermore, by combining historical test data to conduct trend analysis, it can not only reveal the evolution of defects in time and space, but also trigger early warnings when there is abnormal growth or concentrated distribution, and output specific maintenance suggestions.

[0114] Example 1:

[0115] To verify the feasibility of this invention in practice, it was applied to the module inspection scenario of a photovoltaic power station. In practical applications, photovoltaic power stations typically contain tens of thousands of photovoltaic modules, densely arranged and distributed over a large area. Traditional manual inspection methods often suffer from low efficiency, high labor intensity, and high missed detection rates. Especially under conditions of high temperatures in summer or low sunlight in winter, inspection results are easily affected by environmental factors, leading to insufficient accuracy in defect identification. This invention utilizes a drone equipped with a visible light camera and an infrared thermal imager to achieve simultaneous imaging, and combines flight control positioning information to uniformly manage inspection data. This effectively solves the technical problems of difficulty in aligning multimodal information, low defect identification accuracy, and inaccurate positioning results in traditional inspections.

[0116] In this embodiment, the UAV flew over the photovoltaic power station along a preset route, collecting over 30,000 frames of visible light and infrared image data, which were then matched one-to-one with flight control positioning information. After preprocessing operations such as denoising, brightness equalization, and image enhancement, the clarity of the raw data was improved by approximately 25% compared to the unprocessed data. Subsequently, texture and edge features were extracted from the visible light images, and temperature distribution and anomaly features were extracted from the infrared images. These were then uniformly organized into a modal feature set. The Grassmann manifold embedding method was used to map the modal features, preserving the geometric structural relationships between features and avoiding the problem of modal inconsistency in traditional stitching methods. Next, a robust sparse subspace clustering method was used to cluster the manifold feature representation, demonstrating strong robustness against changes in illumination and noise interference. Finally, the comprehensive features were input into a classification method trained based on historical defect samples to achieve automatic identification of defects such as cracks, hot spots, delamination, ablation, and occlusion.

[0117] Table 1 Comparison data between the method of the present invention and the traditional manual inspection method

[0118]

[0119]

[0120] As shown in Table 1, under traditional manual inspection conditions, it takes approximately 12 hours to complete the inspection of 1000 modules on average, with a defect identification accuracy of around 78%. However, using the method of this invention, the same number of modules can be inspected in only about 1.5 hours, with an identification accuracy exceeding 94%. In data comparison, especially in areas with high reflection interference, the false positive rate of the traditional method reaches 15%, while the false positive rate of the method of this invention is controlled below 5%. Furthermore, in terms of defect location, this invention utilizes flight control positioning information and geometric projection relationships to accurately map defects to specific module numbers and array positions, with a positioning error of less than 15 cm, reducing the error by nearly 60% compared to traditional methods. In addition, this invention can draw defect distribution maps and generate inspection reports based on historical inspection data, analyzing defect development trends. When the growth rate of the same type of defect exceeds 20% in three consecutive inspections, the system can automatically trigger an early warning and output maintenance suggestions. This function significantly improves the foresight of photovoltaic power plant operation and maintenance, reducing the decline in power generation efficiency caused by defect accumulation.

[0121] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent inspection of defects in photovoltaic modules based on image recognition, characterized in that, Includes the following steps: The drone collects visible light and infrared images of photovoltaic modules and records the drone's flight control and positioning information in real time, storing it as raw image data. The original image data is preprocessed, the illumination difference is balanced using a brightness equalization method, and image enhancement processing is performed to obtain a preprocessed image with optimized quality. Feature extraction is performed on the preprocessed image. Texture and edge information are extracted from the visible light image, and temperature distribution and anomaly information are extracted from the infrared image. These are then organized into a modal feature set. Different modal features of the modal feature set are uniformly mapped to the Grassmann manifold space to generate manifold feature representations; Robust sparse subspace clustering is applied to the manifold feature representation within the Grassmann manifold space. The features are fused and clustered using sparse representation and robust constraints, and a comprehensive feature representation is output. By taking the comprehensive feature representation as input, the surface defects of photovoltaic modules are automatically identified and classified, and defect category information is output. By combining the defect category information with the flight control positioning information, historical detection data is constructed, and the actual location of the defect in the photovoltaic module is calculated to form a positioning result; Based on the defect category information and location results, a defect distribution map and detection report are generated. Trend analysis is performed in conjunction with historical detection data. When the analysis results exceed a set threshold, an early warning mechanism is triggered and maintenance suggestions are output.

2. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The storage process for the original image data specifically includes: The drone is controlled to fly over the photovoltaic power station at a recorded speed and altitude according to the preset route. The visible light camera is used to collect images of the surface of the photovoltaic modules. The frame number, timestamp and camera identifier of the same batch are written into the image metadata to form the first image data. The infrared thermal imager on the drone is used to perform thermal imaging acquisition on the same flight path coverage area, and the frame number, timestamp and sensor identifier are written into the image metadata to form the second image data; Record the UAV flight control positioning information, and associate and store the first image data, the second image data, and the flight control positioning information with the same timestamp and unique identifier to form the original image data.

3. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The process of obtaining the quality-optimized preprocessed image specifically includes: The system receives raw image data, pairs and sorts visible light image sequences and infrared image sequences according to timestamps, performs consistency checks, removes frames with missing timestamps and inconsistent parameters, and generates a set of paired frames. The grayscale histogram of the paired frame set is statistically analyzed frame by frame, and brightness equalization processing is performed on each frame while keeping the original spatial resolution, timestamp and unique identifier unchanged, to obtain a brightness equalized image sequence. The brightness-equalized image sequence is sequentially enhanced with edge enhancement, detail sharpening, and contrast improvement. The enhanced image is then re-associated and verified with the corresponding flight control positioning information according to the timestamp and unique identifier, and the pre-processed image data with optimized quality is output.

4. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The organization of the modal feature set specifically includes: Receive the pre-processed image data with optimized quality, pair the visible light image sequence and the infrared image sequence one by one according to the timestamp and unique identifier consistent with the acquisition time, and renumber the frames that pass the verification according to the time sequence to form an aligned set of pre-processed frames. Feature extraction is performed frame by frame on the visible light images in the aligned preprocessed frame set. The texture is statistically analyzed to form texture description information. Edge direction and edge intensity are extracted to form edge description information. The texture description information and edge description information of the same frame are merged according to the predetermined field order, and the visible light feature record sequence is output in chronological order. Feature extraction is performed frame by frame on the infrared images in the aligned preprocessed frame set. Temperature distribution description information is statistically generated, and the location, range and intensity of anomalies are marked to form anomaly description information. The temperature distribution description information and anomaly description information of the same frame are merged according to the predetermined field order, and the infrared feature record sequence is output. The visible light feature recording sequence and the infrared feature recording sequence are paired frame by frame according to the timestamp and unique identifier. Dimensional unification and numerical standardization are performed on each pair of paired records. The processed paired records are summarized and stored in chronological order as a modal feature set.

5. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The generation process of the manifold feature representation specifically includes: Receive a set of modal features, perform dimensional analysis on the visible light and infrared feature records in the set, determine the dimensional size and numerical range of each feature record, and linearly normalize the numerical range with a unified dimensional length, and output the modal feature pairs with unified dimensions. The visible light feature records in the modal feature pairs are represented in a subspace. The texture description information and edge description information of each frame are combined into a vector representation in sequence and orthogonalized to output the visible light subspace basis sequence. The infrared feature records in the modal feature pairs are represented in a subspace. The temperature distribution description information and anomaly point description information of each frame are combined into a vector representation in sequence and orthogonalized to output the infrared subspace basis sequence. The visible light subspace basis sequence and the infrared subspace basis sequence are paired frame by frame according to the same timestamp and unique identifier, uniformly mapped to the Grassmann manifold space, and transformed into point representation in the manifold space. They are then stored in chronological order as manifold feature representations.

6. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The output process of the comprehensive feature representation specifically includes: The manifold feature representations are sorted sequentially according to timestamp and unique identifier. The point descriptions of each frame in the manifold space are grouped and bound with the corresponding visible light subspace basis descriptions and infrared subspace basis descriptions to obtain an index mapping table and the corresponding manifold point sequence. The similarity weights are calculated pairwise for each manifold point sequence. Based on the orthogonal subspace basis of each frame, the protagonist set between them is calculated. A geodesic length is obtained based on the current angle to measure the geometric distance between the two frames on the Grassmann manifold. An exponential decay rule is applied to obtain the similarity score. At the same time, based on the volume consistency between the subspace basis of the two frames, the volume coherence factor is calculated and corrected by a preset positive exponent to obtain the correction factor. The similarity score is multiplied by the correction factor to obtain the similarity weight. The similarity weights calculated pairwise for all frames are filled into a symmetric similarity weight matrix according to their row and column positions. Within the tangent space of the Grassmann manifold, taking the manifold points of each frame as references, other points in the current neighborhood are transformed into tangent space tensors through logarithmic mapping to obtain the corresponding set of tangent vectors. A weighted robust sparse subspace optimization model is established using the similarity weight matrix as the weighting coefficients for the reconstruction error. The model minimizes the sum of three parts: the weighted reconstruction error of the tangent space data, the row direction sparsity penalty of the sparse coefficient matrix, and the column direction robust penalty of the residual matrix. An augmented Lagrangian strategy is used to alternately update the sparse coefficient matrix and the residual matrix until the convergence condition is met. The convergence condition is that the decrease in reconstruction error and the change in coefficients are both lower than the convergence threshold. The reconstruction coefficient matrix and the residual matrix are then output. The sparse coefficient matrix is ​​symmetricized to generate a symmetric correlation matrix. A graph structure is constructed by combining the index information corresponding to the index mapping table. Then, the clustering index of each frame is obtained through spectral decomposition. The clustering index is merged with the timestamp, unique identifier and manifold point description to form a comprehensive feature representation.

7. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The process of outputting the defect category information specifically includes: Using the comprehensive feature representation as input, the clustering index of each frame is bound together with the visible light and infrared features corresponding to the current frame to obtain a single feature record. For the surface defects of photovoltaic modules, all feature records are stored in a unified input dataset in chronological order. Perform standardization processing on the unified input dataset frame by frame, re-attach the corresponding timestamp and unique identifier to each feature record, and output the standardized feature dataset. The standardized feature dataset is input into the classification method trained based on historical defect samples. The input features are judged, and the defect category prediction result and corresponding confidence score of the current frame are output. These are then bound to the timestamp and unique identifier to form defect category information.

8. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The process of forming the positioning result specifically includes: Receive defect category information and match it frame by frame with the flight control positioning information to form historical detection data. At the same time, establish a joint index table. Each frame defect category is bound to the corresponding UAV three-dimensional positioning coordinates and attitude angle information to generate a positioning input dataset. Coordinate calculation is performed on the localization input dataset to deduce the spatial position and orientation of the camera coordinate system in the geographic coordinate system during image acquisition. Then, the defect pixel position in the image coordinate system is mapped to the physical position in the geographic coordinate system through geometric projection relationship to obtain the preliminary localization result of the defect in the actual space. Based on the preliminary location results, the defect category of each frame is attached with the corresponding geographic coordinates, and then corrected in combination with the arrangement rules of the photovoltaic module array. The geographic coordinates are then matched with the specific photovoltaic module number and the module row and column index to form the location result.

9. The intelligent inspection method for photovoltaic module defects based on image recognition according to claim 1, characterized in that, The output of the maintenance recommendations specifically includes: The defect category labels are bound to the corresponding component numbers, row and column indices, and geographic coordinates. They are arranged according to the spatial distribution order of the component array, and a defect distribution map is drawn on a two-dimensional plane. The defect distribution map is then output as a visual representation. Based on the generated defect distribution map, the number and distribution location of each type of defect are statistically analyzed, and a detection report is generated by combining the timestamp information; The historical detection data is used to perform trend analysis on the changes in the number and spatial distribution of defects, resulting in a trend curve and a spatial distribution index. When the growth rate of the trend curve or the spatial distribution index exceeds a preset threshold, an early warning mechanism is triggered, and maintenance suggestions are output.

Images