Solar cell panel defect detection method and device based on Fourier transform infrared spectral imaging

Abstract

The invention provides a solar cell panel defect detection method and device based on Fourier transform infrared spectrum imaging, and relates to the technical field of defect detection, and the method comprises the steps: obtaining an infrared spectrum data cube and a super-resolution RGB image of the outer surface of a solar cell panel; wherein the infrared spectrum data cube comprises all positions on the outer surface and spectrum information of the corresponding positions under different wavelengths; performing super-resolution reconstruction on the infrared spectrum data cube by taking the spatial resolution of the super-resolution RGB image as a reference to obtain a super-resolution infrared spectrum data cube; and inputting the super-resolution RGB image and the super-resolution infrared spectrum data cube into a pre-trained solar cell panel defect detection model to obtain a defect detection result. According to the invention, accurate detection and classification identification of defects of the solar cell panel can be realized, and the detection precision and reliability are improved.

Description

Technical Field

[0001] This application relates to the technical field of defect detection, and more particularly, to a method and device for detecting defects in a solar panel based on Fourier transform infrared spectroscopy imaging. Background Art

[0002] Silicon crystal solar panel modules are key devices in photovoltaic power generation systems and are widely used in various photovoltaic power generation scenarios. However, during production, transportation, installation, and operation, the panel modules often have problems such as local heating, thermal patches, and hot spots due to external stress or internal defects. These problems not only seriously affect the photoelectric conversion efficiency and the service life of the panels, but also may pose a threat to the stable operation of the entire solar power generation system. Therefore, efficient detection of these defects has become an important research topic in the photovoltaic industry. Since these defects are difficult to be directly observed by the naked eye, various imaging detection methods have been adopted in the prior art, including infrared thermal imaging, electroluminescence imaging, and photoluminescence imaging.

[0003] Infrared thermal imaging technology, as a mature non-contact detection means, plays a dominant role in the detection of local heating and hot spots in solar panel modules. This technology captures the thermal distribution information on the surface of the panel through an infrared focal plane camera and can quickly locate the abnormally heated areas.

[0004] However, current infrared thermal imaging technology usually uses imaging devices with a single wavelength band (such as infrared focal plane cameras in the 3-5μm or 8-14μm wavelength bands). The imaging information is limited to the thermal radiation signals in a certain fixed wavelength band, making it difficult to obtain the comprehensive spectral information of the defects, unable to capture the spectral detail features of the defects, and restricting the defect detection accuracy and applicable range. At the same time, current infrared imaging devices are difficult to simultaneously balance the spatial resolution and spectral resolution, resulting in the generated infrared data being unable to provide high-resolution spatial and spectral information, affecting the accurate detection and analysis of subtle defects. Moreover, the computational complexity is high, and real-time defect detection cannot be achieved. Under changes in illumination or external environmental interference, the detection performance may drop significantly.

[0005] In addition, the existing technologies usually output two-dimensional thermal imaging images instead of infrared spectral data cubes containing rich spectral dimension information. Insufficient spectral information makes it difficult for the detection results to reflect the physical and chemical characteristics of defects, limiting the in-depth analysis of the causes and properties of defects. For example, in US Patent US11003940B2, Chinese Patent CN206114555U, WO2017172611A1, etc., although various automated detection methods for solar panel defects have been proposed, none of them involve the content related to infrared spectral imaging. Other technologies, such as "Research on Defect Detection Technology of Photovoltaic Panels Based on High Frame Rate InGaAs Cameras" and "Application of Photoluminescence Technology in Defect Detection of Si-based Solar Cells", also mainly focus on single-band imaging and detection technologies, and do not provide a solution for obtaining detailed information using infrared spectral cubes.

[0006] Meanwhile, due to the lack of spectral information support, the existing technologies have insufficient accuracy in distinguishing defect types (such as the distinction between local heating and hot spots), making it difficult to meet the application requirements in complex scenarios. Summary of the Invention

[0007] The purpose of the embodiments of this application is to provide a method and device for detecting solar panel defects based on Fourier transform infrared spectral imaging, which solves the problems of single band, insufficient spectral information, limited classification ability, and low data resolution existing in the prior art, and can achieve precise detection and classification recognition of solar panel defects, improving the detection accuracy and reliability.

[0008] In the first aspect, the present invention provides a method for detecting solar panel defects based on Fourier transform infrared spectral imaging, the method comprising:

[0009] Obtain an infrared spectral data cube and a super-resolution RGB image of the outer surface of the solar panel; wherein, the infrared spectral data cube is collected by a Fourier transform infrared spectrometer; the infrared spectral data cube includes: each position on the outer surface and the spectral information at corresponding positions at different wavelengths;

[0010] Based on the spatial resolution of the super-resolution RGB image, perform super-resolution reconstruction on the infrared spectral data cube to obtain a super-resolution infrared spectral data cube;

[0011] Input the super-resolution RGB image and the super-resolution infrared spectral data cube into a pre-trained solar panel defect detection model to obtain a defect detection result.

[0012] In an optional implementation manner, the method for obtaining the infrared spectral data cube includes:

[0013] Obtain the sequence of phase-plane interference images of the outer surface collected by the Fourier transform infrared spectroscopy imager;

[0014] Correct the sequence of phase-plane interference images to obtain a pixel interference cube;

[0015] Extract infrared spectral data from the pixel interference cube using Fourier transform;

[0016] Perform spectral normalization and filtering on the infrared spectral data to obtain processed infrared spectral data;

[0017] Perform three-dimensional reconstruction on the processed infrared spectral data to obtain an infrared spectral data cube.

[0018] In an alternative embodiment, the wavelength range corresponding to the spectral information is 700 - 1500 nm; the wavelength interval is 100 nm;

[0019] The spatial resolution of the super-resolution RGB image is greater than 1024×1024 pixels.

[0020] In an alternative embodiment, based on the spatial resolution of the super-resolution RGB image, perform super-resolution reconstruction on the infrared spectral data cube, including:

[0021] Perform spatial alignment on the infrared spectral data cube and the super-resolution RGB image;

[0022] Extract the spectral features of the spatially aligned infrared spectral data cube and the spatial features of the spatially aligned super-resolution RGB image;

[0023] Match the feature dimensions of the spectral features and the spatial features;

[0024] Based on the spatial resolution of the super-resolution RGB image, adjust the spatial resolution of the matched spectral features to obtain a super-resolution infrared spectral data cube; wherein, the spatial resolution of the super-resolution infrared spectral data cube is the same as the spatial resolution of the super-resolution RGB image.

[0025] In an alternative embodiment, the solar panel defect detection model includes:

[0026] An input layer for inputting the super-resolution RGB image and the super-resolution infrared spectral data cube;

[0027] A first feature extraction layer for extracting features from the input super-resolution infrared spectral data cube to obtain spectral features;

[0028] A second feature extraction layer for extracting features from the input super-resolution RGB image to obtain texture features;

[0029] A feature fusion layer for fusing the spectral features and the texture features to obtain fused features;

[0030] A defect recognition layer for recognizing defects in the fused features to obtain defect detection results.

[0031] In an optional embodiment, the feature fusion layer includes:

[0032] A first fusion layer for respectively performing feature weighting on the spectral features and the texture features; performing coupling processing on the spectrally weighted features and the texture features; and splicing the coupled spectral features and texture features to obtain a joint feature representation;

[0033] A second fusion layer for performing feature compression, non-linear mapping, and adaptive weight adjustment on the joint feature representation output by the first fusion layer to obtain fused features.

[0034] In an optional embodiment, the defect recognition layer includes:

[0035] A two-dimensional convolutional layer for extracting features from the fused features to obtain a fused feature extraction result;

[0036] A Sigmoid activation function for performing non-linear transformation on the fused feature extraction result to obtain defect detection results.

[0037] In a second aspect, the present invention provides a solar panel defect detection device based on Fourier transform infrared spectroscopy imaging, the device comprising:

[0038] An acquisition unit for acquiring an infrared spectral data cube and a super-resolution RGB image of the outer surface of a solar panel; wherein, the infrared spectral data cube is collected by a Fourier transform infrared spectrometer; the infrared spectral data cube includes: spectral information at each position on the outer surface and corresponding positions at different wavelengths;

[0039] A reconstruction unit for performing super-resolution reconstruction on the infrared spectral data cube based on the spatial resolution of the super-resolution RGB image to obtain a super-resolution infrared spectral data cube;

[0040] A detection unit for inputting the super-resolution RGB image and the super-resolution infrared spectral data cube into a pre-trained solar panel defect detection model to obtain defect detection results.

[0041] In a third aspect, the present invention provides an electronic device, which includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0042] The memory is used to store a computer program;

[0043] The processor is used to implement the method described in any one of the foregoing embodiments when executing the program stored on the memory.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method described in any one of the foregoing embodiments is implemented.

[0045] The infrared spectral data cube of this application captures the full-band spectral information on the outer surface of the solar panel during operation, and can effectively distinguish the nature and causes of different defects; by fusing the super-resolution RGB image and the infrared spectral data cube, a super-resolution infrared spectral data cube is obtained, which can more accurately present the spatial details of the defects, improve the accuracy and comprehensiveness of defect localization and analysis, realize the precise detection and classification recognition of solar panel defects, effectively improve the defects of the existing technology, and improve the detection accuracy and reliability; at the same time, through the complementary characteristics of spectral and texture features, it can maintain high detection performance under complex environmental conditions and has stronger robustness;

[0046] This application combines infrared spectral features with RGB texture features through the joint processing of multi-modal data of super-resolution RGB images and infrared spectral data cubes, and through feature interaction (for example, the infrared band may be more sensitive to surface cracks, while the visible light band may have a better response to pollution or oxidation), significantly improving the accuracy and robustness of defect classification.

[0047] This application improves the existing solar panel defect detection model, combines deep learning technology, multi-scale feature fusion and lightweight, reduces the calculation cost, realizes real-time defect detection with higher accuracy, and can meet the needs of large-scale online detection of solar panels in industrial sites; this application significantly improves the spectral dimension, spatial resolution and classification ability of solar panel defect detection, and provides a more comprehensive and reliable solution for photovoltaic module defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0049] Figure 1 Flowchart of a method for detecting defects in a solar panel based on Fourier transform infrared spectroscopy imaging provided by an embodiment of the present application;

[0050] Figure 2 Schematic diagram of a device for collecting an infrared spectral data cube and a super-resolution RGB image provided by an embodiment of the present application;

[0051] Figure 3 Schematic diagram of the reconstruction of an infrared spectral data cube provided by an embodiment of the present application;

[0052] Figure 4 Schematic diagram of a defect detection model architecture for a solar panel provided by an embodiment of the present application;

[0053] Figure 5 Schematic diagram of a method for detecting defects in a solar panel provided by an embodiment of the present application;

[0054] Figure 6 Schematic diagram of the structure of a device for detecting defects in a solar panel based on Fourier transform infrared spectroscopy imaging provided by an embodiment of the present application;

[0055] Figure 7 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of them. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by the present application.

[0057] The method for detecting defects in a solar panel based on Fourier transform infrared spectroscopy imaging provided by an embodiment of the present application can be applied to a server or a terminal with strong computing capabilities. The server can be a physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms. The terminal can be a user equipment (UE) such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a personal digital assistant (PDA), a tablet computer (PAD), a handheld device, a vehicle-mounted device, a wearable device, a computing device or other processing devices connected to a wireless modem, a mobile station (MS), a mobile terminal, etc. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, which are not limited in this application.

[0058] The following describes the preferred embodiments of the present application with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. And without conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0059] Figure 1 It is a schematic flowchart of a method for detecting defects in a solar panel based on Fourier transform infrared spectroscopy imaging provided by an embodiment of the present application. As Figure 1 shown, the method may include:

[0060] Step S110, obtain an infrared spectral data cube and a super-resolution RGB image of the outer surface of the solar panel.

[0061] In the embodiment of the present application, the infrared spectral data cube is collected by a Fourier transform infrared spectroscopy imager; the infrared spectral data cube includes: spectral information of each position on the outer surface during the operation of the solar panel and the corresponding position at different wavelengths; the super-resolution RGB image includes each position on the outer surface during the operation of the solar panel and the spatial texture features and color information of the corresponding position.

[0062] In the embodiment of the present application, the wavelength range corresponding to the spectral information is 700 - 1500 nm; the wavelength interval is 100 nm; that is, the infrared spectral data cube consists of the spectral information at each position and the corresponding positions at 700 nm, 800 nm, 900 nm, 1000 nm, 1100 nm, 1200 nm, 1300 nm, 1400 nm, and 1500 nm. The wavelength range of the spectral information obtained in the present application covers multiple band ranges from short-wave infrared to long-wave infrared, thereby comprehensively recording the spectral characteristics of defects and solving the problem of insufficient defect discrimination ability caused by a single band in the prior art.

[0063] In the embodiment of the present application, the spatial resolution of the super-resolution RGB image is greater than 1024 * 1024 pixels; preferably, the spatial resolution of the super-resolution RGB image can be 2048 * 2048 pixels or 4096 * 4096 pixels, and the total resolution of the corresponding super-resolution RGB image is 2048 * 2048 * 3 or 4096 * 4096 * 3. The resolution of the super-resolution RGB image is significantly higher than that of the spectral information, which can make up for the deficiency of the infrared imaging system in terms of spatial resolution.

[0064] In the embodiment of the present application, the method for obtaining the infrared spectral data cube includes:

[0065] Obtaining a sequence of phase-plane interferograms of the outer surface collected by a Fourier transform infrared spectroscopic imager; correcting the sequence of phase-plane interferograms to obtain a pixel interferogram cube; extracting infrared spectral data from the pixel interferogram cube by using Fourier transform; performing spectral normalization and filtering on the infrared spectral data to obtain processed infrared spectral data; and performing three-dimensional reconstruction on the processed infrared spectral data to obtain an infrared spectral data cube.

[0066] In the embodiment of the present application, the acquisition device for the infrared spectral data cube and the super-resolution RGB image is as Figure 2 shown; it includes: a scanning mirror, a front objective lens, a static interference structure, a focal plane detector, a beam splitter, an RGB camera, and a relay lens.

[0067] Step S120: Based on the spatial resolution of the super-resolution RGB image, perform super-resolution reconstruction on the infrared spectral data cube to obtain a super-resolution infrared spectral data cube.

[0068] In the embodiment of the present application, the spatial resolution of the super-resolution infrared spectral data cube is the same as that of the super-resolution RGB image; the resolution of the super-resolution infrared spectral data cube is 4096 * 4096 * 9 or 2048 * 2048 * 9 or 1024 * 1024 * 9; and its pixels are consistent with those of the super-resolution RGB image.

[0069] In the embodiments of the present application, based on the spatial resolution of the super-resolution RGB image, super-resolution reconstruction of the infrared spectral data cube is performed, including:

[0070] Spatially align the infrared spectral data cube and the super-resolution RGB image; extract the spectral features of the spatially aligned infrared spectral data cube and the spatial features of the spatially aligned super-resolution RGB image; match the feature dimensions of the spectral features and the spatial features; based on the spatial resolution of the super-resolution RGB image, adjust the spatial resolution of the matched spectral features to obtain a super-resolution infrared spectral data cube.

[0071] In the embodiments of the present application, the coordinate systems of the infrared spectral data cube and the super-resolution RGB image are the same, which can be the geodetic coordinate system or a coordinate system established based on the outer surface of the solar panel; before spatially aligning the infrared spectral data cube and the super-resolution RGB image, it is also necessary to: perform normalization processing on the infrared spectral data cube and the super-resolution RGB image, such as normalization and denoising, etc., to effectively improve the data quality of the infrared spectral data cube and the super-resolution RGB image.

[0072] In the embodiments of the present application, the following methods can be used to spatially align the infrared spectral data cube and the super-resolution RGB image:

[0073] Perform spatial dimension alignment on the infrared spectral data cube and the super-resolution RGB image through blurred upsampling;

[0074] Or, use a deep learning enhanced feature extraction method to extract key points and descriptors from the super-resolution RGB image and the infrared spectral data cube; screen out reliable key point matching pairs from the extracted key points and descriptors through the RANSAC algorithm; calculate the affine transformation or projection transformation matrix according to the key point matching pairs, and perform global alignment on the infrared spectral data cube and the super-resolution RGB image; use the Iterative Closest Point (ICP) algorithm to optimize the alignment accuracy of the globally aligned infrared spectral data cube and the super-resolution RGB image to obtain the spatially aligned infrared spectral data cube and the spatially aligned super-resolution RGB image;

[0075] Or, pre-construct an objective function to maximize the mutual information between the RGB image and the infrared spectral data cube; adjust the geometric transformation parameters (such as translation, rotation, scaling, etc.) of the infrared spectral data cube and the super-resolution RGB image through an optimization algorithm to make the mutual information reach the maximum value; adopt a multi-resolution strategy, first perform rough registration at a low resolution, and then gradually increase the resolution for fine adjustment to obtain the spatially aligned infrared spectral data cube and the spatially aligned super-resolution RGB image.

[0076] In an embodiment of the present application, extracting spectral features of an infrared spectral data cube after spatial alignment includes:

[0077] Analyzing 9 spectral bands at each position in the spatially aligned infrared spectral data cube to obtain spectral response curves at each position; using PCA or independent component analysis (ICA) to analyze the spectral response curves at each position to obtain first spectral features at each position, where the first spectral features are the main spectral features, and through this step, the spectral dimension is reduced; using a pre-trained convolutional neural network (CNN) or multi-layer perceptron (MLP) to learn high-level feature representations from the infrared spectral data cube to obtain second spectral features at each position; fusing the first spectral features and the second spectral features at each position to obtain spectral features at each position.

[0078] In an embodiment of the present application, extracting spatial features of a super-resolution RGB image after spatial alignment includes:

[0079] Using operators such as Sobel and Canny to detect the edges of the spatially aligned super-resolution RGB image, and combining the gray-level co-occurrence matrix (GLCM) to analyze the texture features of the super-resolution RGB image; extracting local binary pattern features of the super-resolution RGB image to describe the local structure of the super-resolution RGB image; using pre-trained architectures such as U-Net and ResNet to extract multi-level spatial features from the super-resolution RGB image to obtain the spatial features of the spatially aligned super-resolution RGB image.

[0080] In an embodiment of the present application, matching the feature dimensions of the spectral features and the spatial features includes:

[0081] Fusing the spectral features and the spatial features at the feature level through weighted average or concatenation operations; or comprehensively integrating the information of the spectral features and the spatial features through an ensemble learning method (such as random forest, AdaBoost) for decision-level fusion.

[0082] In an embodiment of the present application, taking the spatial resolution of the super-resolution RGB image as a reference, adjusting the spatial resolution of the matched spectral features includes:

[0083] Reconstructing the super-resolution of the matched spectral features based on deep learning by pre-training a resolution reconstruction model;

[0084] Alternatively, pre-training includes a multi-modal fusion network for multi-modal resolution reconstruction; the multi-modal fusion network includes: a multi-modal fusion network with a shared encoder and two decoders, which are respectively used to process RGB images and infrared spectral data cubes; feature fusion is performed at the intermediate feature layer, for example, through concatenation, weighted average, or attention mechanism; at the same time, the parameters of the two decoders are optimized so that the reconstructed RGB images and infrared spectral data cubes can both reach higher resolutions; a joint loss function is defined, including pixel-level reconstruction loss (such as L1 or L2 loss), perceptual loss, and adversarial loss, to ensure that the generated images perform excellently in multiple evaluation metrics; end-to-end training is performed using optimization algorithms such as Adam, and the model parameters are continuously adjusted to minimize the joint loss.

[0085] In an embodiment of the present application, a super-resolution reconstruction model can be pre-constructed, and the infrared spectral data cube and the super-resolution RGB image are input into the super-resolution reconstruction model to obtain a super-resolution infrared spectral data cube; the super-resolution reconstruction model includes:

[0086] An alignment network, which is used to align the infrared spectral data cube and the super-resolution RGB image in the spatial dimension by using blurry upsampling;

[0087] A cross-modal feature fusion network, including:

[0088] An extraction layer, which is used to extract the spatial features of the super-resolution RGB image and the spectral features of the infrared spectral data cube respectively through a dual-branch encoder;

[0089] An alignment layer, which is used to use deformable convolution and a spectral attention module to achieve feature dimension matching and spatial alignment, and obtain the aligned spatial features and enhanced spectral features;

[0090] A shallow feature fusion layer, which is used to dynamically fuse the shallow features of the aligned spatial features and the enhanced spectral features by using a channel attention mechanism to obtain a fused shallow feature map;

[0091] A deep feature fusion layer, which is used to introduce a non-local attention mechanism to model cross-modal long-range dependencies, and construct a cascaded feature fusion network to perform deep feature fusion on the fused shallow feature map to obtain a fused deep feature map;

[0092] A dual-path reconstruction network, including:

[0093] A path reconstruction layer, which is used to perform dual-path reconstruction on the fused deep feature map to obtain an enhanced spatial feature map and an enhanced spectral feature map; among them, the spatial enhancement path realizes resolution improvement and edge sharpness constraint through sub-pixel convolution; the spectral fidelity path uses a 3D convolution block combined with a spectral angle loss to maintain spectral continuity;

[0094] An adaptive weighting layer, which is used to adjust the weight parameters in real time according to the spectral correlation coefficient, dynamically fuse the enhanced spatial feature map and the enhanced spectral feature map, and obtain a fused feature map;

[0095] A two-stage amplification network, including:

[0096] A primary amplification network, which integrates a cyclic residual group structure and enhances the feature expression ability through channel shuffling operations; it is used to perform 2-fold spatial amplification on the fused feature map by combining separable convolution with the cyclic residual group structure and generate an intermediate result feature map;

[0097] A secondary amplification network, which is used to optimize the intermediate result feature map by combining the integrated cyclic residual group structure imaging degradation model constraint, regularize the reconstruction process by combining the physical model, and obtain a super-resolution infrared spectral data cube.

[0098] In the embodiment of the present application, after adjusting the spatial resolution of the matched spectral features, the method further includes:

[0099] Using a pre-constructed atmospheric correction model to correct the super-resolution infrared spectral data cube to prevent unreasonable spectral mutations during the reconstruction process, improve the realism of the image; ensure that the reconstructed spectral data cube is highly consistent with the original data; the atmospheric correction model can simulate the reflectivity changes in different bands to ensure that the reconstructed spectral data conforms to the actual physical process; use the Total Variation (TV) regularization term or the Laplacian smoothing term to reduce spectral fluctuations and maintain the smoothness of the spectral curve;

[0100] Perform radiometric calibration on the super-resolution infrared spectral data cube after atmospheric correction to ensure that the radiation values of each band are accurate;

[0101] Use a pre-trained generative adversarial perceptual network to optimize the perceptual quality of the super-resolution infrared spectral data cube after radiometric calibration;

[0102] Use a pre-trained super-resolution convolutional neural network (SR-CNN) to perform secondary enhancement on the super-resolution infrared spectral data cube after perceptual quality optimization to obtain the final super-resolution infrared spectral data cube, which is input into the solar panel defect detection model.

[0103] In the embodiment of the present application, as Figure 3As shown in the figure, a reconstruction algorithm based on multimodal fusion is adopted. Through texture-guided super-resolution technology, the high-resolution texture features of the super-resolution RGB image are mapped into the infrared spectral data cube, enhancing the spatial resolution of the infrared spectral image, thereby achieving detail enhancement and defect saliency. The super-resolution infrared spectral data cube has both high spatial resolution and high spectral resolution, and can accurately capture the thermal characteristics, chemical composition, and physical structure information of defects, significantly superior to the detection effect of existing single-band imaging.

[0104] Step S130: Input the super-resolution RGB image and the super-resolution infrared spectral data cube into a pre-trained solar panel defect detection model to obtain a defect detection result.

[0105] In the embodiment of the present application, as Figure 4 shown, the solar panel defect detection model includes:

[0106] An input layer for inputting the super-resolution RGB image and the super-resolution infrared spectral data cube;

[0107] A first feature extraction layer for extracting features from the input super-resolution infrared spectral data cube to obtain spectral features;

[0108] A second feature extraction layer for extracting features from the input super-resolution RGB image to obtain texture features;

[0109] A feature fusion layer for fusing the spectral features and the texture features to obtain fused features;

[0110] A defect recognition layer for recognizing defects from the fused features to obtain a defect detection result.

[0111] In the embodiment of the present application, the feature fusion layer includes:

[0112] A first fusion layer for respectively performing feature weighting on the spectral features and the texture features; performing coupling processing on the spectrally and texture-feature weighted features; and splicing the coupled spectral and texture features to obtain a joint feature representation;

[0113] A second fusion layer for performing feature compression, non-linear mapping, and adaptive weight adjustment on the joint feature representation output by the first fusion layer to obtain fused features.

[0114] In the embodiment of the present application, the dimension of the spectral features input to the first fusion layer is H*W*128, and the dimension of the texture features is H*W*128; first, the spectral features and texture features are each weighted by channel attention, then a feature interaction matrix is constructed, and a pixel-level spectral-texture coupling relationship is established through element-wise dot product operation to obtain the coupled spectral features and texture features; subsequently, the coupled spectral features and texture features are concatenated along the channel dimension to generate a joint feature representation (H×W×256).

[0115] In the embodiment of the present application, the input of the second fusion layer uses convolution to perform feature compression and non-linear mapping on the joint feature representation (H×W×256), and the multi-modal feature contribution degree is adaptively adjusted through a cross-channel feature recalibration module to obtain a fusion feature with a dimension of H×W×64; this fusion strategy guides local interaction and global feature compression through spectral attention, while retaining the advantages of the original multi-modal features, constructs a discriminative joint representation space to improve the defect recognition accuracy.

[0116] In the embodiment of the present application, the defect recognition layer includes:

[0117] A two-dimensional convolutional layer for extracting features from the fusion features to obtain a fusion feature extraction result;

[0118] A Sigmoid activation function for performing non-linear transformation on the fusion feature extraction result to obtain a defect detection result.

[0119] In the embodiment of the present application, spectral features in the infrared spectral data cube are extracted through a deep convolutional network, and jointly analyzed with texture features in a super-resolution RGB image to construct a multi-modal feature space, solving the problem of low classification accuracy caused by the lack of spectral information in the prior art; an improved convolutional neural network (CNN) is used to quickly detect the location of defects through multi-scale fusion of local features and global features; by optimizing the data flow and algorithm efficiency, the system can process and analyze large-scale solar panel data in real time in a complex environment to meet the requirements of large-scale industrial applications.

[0120] In the embodiment of the present application, as Figure 5 shown, the solar panel defect detection based on Fourier transform infrared spectroscopy imaging includes:

[0121] The first step is that under natural daylight illumination, the interior and surface of the solar panel assembly reflect visible light and infrared light. The visible light is imaged by a high-resolution RGB camera to generate a high-resolution RGB image. At the same time, a Fourier transform infrared spectrometer is used to capture the interference signal of the infrared light to generate a sequence of original phase-plane interference diagrams, recording the infrared spectral information of the solar panel.

[0122] In the second step, the original fringe pattern interference image sequence is corrected to generate a pixel interference cube. Infrared spectral data cubes covering the range of 700 - 1500 nm are extracted through Fourier transform, with a wavelength interval of 100 nm. Spectral normalization and filtering are performed on the infrared spectral data cubes to reduce the influence of noise and environmental interference. At the same time, brightness correction and contrast enhancement are performed on the RGB images to improve the clarity of texture features.

[0123] In the third step, the infrared spectral data and the RGB images are aligned in terms of spatial resolution and coordinate system, enabling the analysis of data from both modalities within the same spatial range. A reconstruction algorithm based on multimodal fusion is adopted, and through texture-guided super-resolution technology, the high-resolution texture features of the RGB images are mapped into the infrared spectral data cubes to enhance the spatial resolution of the infrared spectral images.

[0124] In the fourth step, an improved deep convolutional network is used to extract spectral features from the infrared spectral data cubes, capturing the subtle changes in spectral signals at different bands to reflect the optical characteristics of potential defects. At the same time, texture features are extracted from the RGB images to identify the defect morphology and distribution on the surface of the solar panel components.

[0125] In the fifth step, a multimodal feature fusion module is used to integrate the extracted spectral features and texture features into a joint feature space. Combining the complementary characteristics of spectral and texture information, through cross-attention mechanisms and feature weighting operations, a highly discriminative feature representation is generated, providing more comprehensive feature support for defect detection.

[0126] In the sixth step, a convolutional decoding network is used to gradually restore the resolution of the feature maps. A classification prediction head composed of two-dimensional convolutional layers and Sigmoid activation functions is adopted to classify the fused features, generating the final defect detection result map.

[0127] The technical effects of this application are further described below in combination with specific experiments.

[0128] This application compares the defect recognition effects of existing defect recognition models based on infrared images, the defect recognition model of this application with ordinary infrared images or infrared spectral data cubes as inputs, and the defect recognition model of this application with super-resolution infrared spectral data cubes as inputs in the same scenario. The results are as follows:

[0129]

[0130] The results show that, compared with the current state-of-the-art method (SOTA), the defect recognition model of the present application that takes ordinary infrared images or infrared spectral data cubes as inputs has its accuracy in the defect recognition task improved from 95.1% to 96.6%. After introducing the infrared hyperspectral super-resolution reconstruction based on physical constraints, by enhancing spatial details and spectral fidelity, the accuracy of the defect recognition model of the present application that takes the super-resolution infrared spectral data cube as the input is further improved to 97.3%.

[0131] Corresponding to the above method, an embodiment of the present application further provides a solar panel defect detection device based on Fourier transform infrared spectroscopy imaging, as Figure 6 shown. The solar panel defect detection device includes:

[0132] An acquisition unit 610, configured to acquire an infrared spectral data cube and a super-resolution RGB image of the outer surface of a solar panel; wherein, the infrared spectral data cube is collected by a Fourier transform infrared spectrometer; the infrared spectral data cube includes: spectral information of each position on the outer surface and the corresponding position at different wavelengths;

[0133] A reconstruction unit 620, configured to perform super-resolution reconstruction on the infrared spectral data cube based on the spatial resolution of the super-resolution RGB image to obtain a super-resolution infrared spectral data cube;

[0134] A detection unit 630, configured to input the super-resolution RGB image and the super-resolution infrared spectral data cube into a pre-trained solar panel defect detection model to obtain a defect detection result.

[0135] The functions of the functional units of the solar panel defect detection device based on Fourier transform infrared spectroscopy imaging provided in the above embodiments of the present application can be implemented by the above method steps. Therefore, the specific working processes and beneficial effects of each unit in the solar panel defect detection device based on Fourier transform infrared spectroscopy imaging provided in the embodiments of the present application will not be repeated here.

[0136] An embodiment of the present application further provides an electronic device, as Figure 7 shown, including a processor 710, a communication interface 720, a memory 730, and a communication bus 740. Among them, the processor 710, the communication interface 720, and the memory 730 complete communication with each other through the communication bus 740.

[0137] The memory 730 is used to store a computer program;

[0138] The processor 710, when executing the program stored on the memory 730, implements the following steps:

[0139] Obtain the infrared spectral data cube and the super-resolution RGB image of the outer surface of the solar panel; wherein, the infrared spectral data cube includes: each position on the outer surface and the spectral information at the corresponding position at different wavelengths;

[0140] Based on the spatial resolution of the super-resolution RGB image, perform super-resolution reconstruction on the infrared spectral data cube to obtain a super-resolution infrared spectral data cube;

[0141] Input the super-resolution RGB image and the super-resolution infrared spectral data cube into a pre-trained solar panel defect detection model to obtain a defect detection result.

[0142] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0143] The communication interface is used for communication between the above-mentioned electronic device and other devices.

[0144] The memory can include a Random Access Memory (RAM), and can also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located far from the aforementioned processor.

[0145] The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processing (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0146] Since the implementation manners and beneficial effects of each device of the electronic device in the above embodiments for solving problems can be referred to Figure 1It is implemented by the steps in the illustrated embodiments. Therefore, the specific working process and beneficial effects of the electronic device provided in the embodiments of this application will not be elaborated here.

[0147] In another embodiment provided by this application, a computer-readable storage medium is also provided. Instructions are stored in the computer-readable storage medium. When it runs on a computer, it causes the computer to execute the solar panel defect detection based on Fourier transform infrared spectroscopy imaging in any of the above embodiments.

[0148] In another embodiment provided by this application, a computer program product containing instructions is also provided. When it runs on a computer, it causes the computer to execute the solar panel defect detection based on Fourier transform infrared spectroscopy imaging in any of the above embodiments.

[0149] Those skilled in the art should understand that the embodiments in the embodiments of this application can be provided as a method, a system, or a computer program product. Therefore, the embodiments in this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments in this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The embodiments in the embodiments of this application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products in the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 a process or multiple processes and / or blocks Figure 1 a device for the functions specified in one or more blocks.

[0151] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements in the process Figure 1 a process or multiple processes and / or blocks Figure 1 a device for the functions specified in one or more blocks.

[0152] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the steps specified in one process or multiple processes and / or blocks Figure 1 one process or multiple processes and / or blocks Figure 1 steps for implementing the functions specified in one block or multiple blocks.

[0153] Although the preferred embodiments in the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0154] Obviously, those skilled in the art can make various changes and modifications to the embodiments in the embodiments of the present application without departing from the spirit and scope of the embodiments in the embodiments of the present application. Thus, if these modifications and variations of the embodiments in the embodiments of the present application fall within the scope of the claims of the embodiments of the present application and their equivalent technologies, the embodiments of the present application are also intended to include these changes and modifications.

Claims

1. A method for detecting defects in solar panels based on Fourier transform infrared spectroscopy imaging, characterized in that: The method comprises: Acquire an infrared spectrum data cube and a super-resolution RGB image of the outer surface of the solar panel; wherein the infrared spectrum data cube is collected by a Fourier transform infrared spectrum imager; the infrared spectrum data cube includes: spectrum information of each position on the outer surface and corresponding positions at different wavelengths; Based on the spatial resolution of the super-resolution RGB image, super-resolution reconstruction is performed on the infrared spectrum data cube to obtain a super-resolution infrared spectrum data cube; The super-resolution RGB image and the super-resolution infrared spectrum data cube are input into a pre-trained solar panel defect detection model to obtain a defect detection result.

2. The method according to claim 1, characterized in that The method for obtaining the infrared spectrum data cube comprises: Acquire a phase plane interference pattern sequence of the outer surface collected by a Fourier transform infrared spectroscopic imager; Correcting the phase plane interference pattern sequence to obtain a pixel interference cube; Extracting infrared spectrum data from the pixel interference cube using Fourier transform; Performing spectral normalization and filtering processing on the infrared spectrum data to obtain processed infrared spectrum data; The processed infrared spectrum data is reconstructed in three dimensions to obtain an infrared spectrum data cube.

3. The method according to claim 1, characterized in that The wavelength range corresponding to the spectral information is 700-1500nm; the wavelength interval is 100nm; The spatial resolution of the super-resolution RGB image is greater than 1024×1024 pixels.

4. The method according to claim 1, characterized in that The infrared spectrum data cube is reconstructed with super resolution based on the spatial resolution of the super resolution RGB image, including: spatially aligning the infrared spectrum data cube and the super-resolution RGB image; Extract the spectral features of the spatially aligned infrared spectral data cube and the spatial features of the spatially aligned super-resolution RGB image; Matching the spectral features and the spatial features in terms of feature dimensions; Taking the spatial resolution of the super-resolution RGB image as a reference, the spatial resolution of the matched spectral features is adjusted to obtain a super-resolution infrared spectrum data cube; wherein the spatial resolution of the super-resolution infrared spectrum data cube is the same as the spatial resolution of the super-resolution RGB image.

5. The method according to claim 1, characterized in that The solar panel defect detection model comprises: An input layer, used for inputting the super-resolution RGB image and the super-resolution infrared spectrum data cube; The first feature extraction layer is used to extract features from the input super-resolution infrared spectrum data cube to obtain spectral features; The second feature extraction layer is used to extract features from the input super-resolution RGB image to obtain texture features; A feature fusion layer, used for fusing the spectral feature and the texture feature to obtain a fusion feature; The defect recognition layer is used to perform defect recognition on the fused features to obtain defect detection results.

6. The method according to claim 5, characterized in that The feature fusion layer includes: The first fusion layer is used to perform feature weighting on the spectral feature and the texture feature respectively; couple the weighted spectral feature and the texture feature; and splice the coupled spectral feature and the texture feature to obtain a joint feature representation; The second fusion layer is used to perform feature compression, nonlinear mapping and adaptive weight adjustment on the joint feature representation output by the first fusion layer to obtain fusion features.

7. The method according to claim 5, characterized in that The defect identification layer comprises: A two-dimensional convolutional layer, used to extract features from the fused features to obtain fused feature extraction results; Sigmoid activation function is used to perform nonlinear transformation on the fusion feature extraction results to obtain defect detection results.

8. A solar panel defect detection device based on Fourier transform infrared spectroscopy imaging, characterized in that: The device comprises: An acquisition unit is used to acquire an infrared spectrum data cube and a super-resolution RGB image of the outer surface of the solar cell panel; wherein the infrared spectrum data cube is acquired by a Fourier transform infrared spectrum imager; the infrared spectrum data cube includes: spectrum information of each position on the outer surface and corresponding positions at different wavelengths; A reconstruction unit, used to perform super-resolution reconstruction on the infrared spectrum data cube based on the spatial resolution of the super-resolution RGB image to obtain a super-resolution infrared spectrum data cube; The detection unit is used to input the super-resolution RGB image and the super-resolution infrared spectrum data cube into a pre-trained solar panel defect detection model to obtain a defect detection result.

9. An electronic device, characterized in that: The electronic device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images