Defect detection method for perovskite thin film solar cells based on improved U-Net++

Abstract

The application relates to a perovskite thin film solar cell defect detection method based on an improved U-Net++, and relates to perovskite thin film defect detection processing. An image acquisition device is built, images are collected under suitable environments, and a data set is enhanced in pertinence; an original U-Net++ model is improved, a fine-grained feature mining module is added, a skip connection structure is adjusted, and an EfficientNet is replaced as a backbone network to obtain an improved U-Net++ model; a Focal_Loss and Dice_Loss weighted sum is used as a loss function to train the model; and finally, an intuitive display of defect detection results is realized through a visual interface developed based on PyQt5. The application solves the problems of low efficiency and poor precision of traditional detection methods, and has obvious advantages in small target defect detection, and can be effectively applied to perovskite solar cell production quality control and promote the intelligent development of industry detection technology.

Description

Technical Field

[0001] This invention relates to the field of perovskite thin film defect detection and processing, specifically to a defect detection method for perovskite thin film solar cells based on an improved U-Net++. Background Technology

[0002] Solar energy, as an ideal solution to meet energy demands, occupies an important position in the energy sector. Among them, perovskite solar cells stand out from many other types of solar cells due to their high efficiency and low cost. However, during the production process, perovskite solar cells are prone to defects such as voids, incomplete solution coverage, and cell rupture due to factors such as uneven annealing and improper operation. These defects can cause partial power loss in the solar cell, severely reducing power generation efficiency. Therefore, timely and accurate detection of these defects is crucial for improving the durability and reliability of solar cells.

[0003] Traditional methods for detecting surface defects in solar cells mainly rely on manual visual inspection and machine vision inspection based on industrial cameras. However, manual visual inspection is labor-intensive, inefficient, and susceptible to subjective factors by the inspectors, leading to missed or false detections. Machine vision inspection also has limitations and cannot meet the requirements for high-precision inspection.

[0004] In recent years, some progress has been made in defect detection of silicon-based solar cells, but research on novel perovskite solar cells is relatively limited. Compared with silicon-based cells, perovskite solar cells have significant differences in detection imaging methods and defect morphologies. Furthermore, perovskite cells are more sensitive to minute defects, and even tiny defects can have a significant impact on their stability, which places higher demands on the detection accuracy of small-area defects. Summary of the Invention

[0005] The purpose of this invention is to address the problems of existing technologies, such as the large workload, low efficiency, and susceptibility to subjective factors leading to missed or false detections in manual visual inspection, the difficulty of machine vision inspection in meeting high-precision requirements, and the lack of research on defect detection in perovskite solar cells and insufficient accuracy in detecting small-area defects. This invention provides a highly automated, efficient, and accurate method for detecting defects in perovskite thin films based on an improved U-Net++, which can achieve high-precision detection of tiny defects in perovskite thin films.

[0006] To achieve the above-mentioned objectives, the present invention provides the following technical solutions.

[0007] This invention provides a defect detection method for perovskite thin-film solar cells based on an improved U-Net++, comprising the following steps:

[0008] 1) Image acquisition equipment setup: Set up a perovskite thin film image collection device, create light-proof acquisition conditions, and complete image acquisition in a low humidity and low temperature environment.

[0009] 2) Data Augmentation: Data augmentation is performed on the acquired images. Operations such as small defect sample oversampling, rotation, and horizontal flipping are used to divide the labeled dataset to obtain training and testing datasets.

[0010] 3) Improved U-Net++ model design: The original U-Net++ model is improved by adding a fine-grained feature mining module, adjusting the skip connection structure, and changing the backbone network to EfficientNet, resulting in the improved U-Net++ model.

[0011] 4) Model training: Input the enhanced dataset into the improved U-Net++ model, adjust the loss function to a weighted sum of Focal_Loss and Dice_Loss, increase the loss weight of small target defects, and train the improved U-Net++ model.

[0012] 5) Visualization Interface Development: Using the pre-trained U-Net++ model, defects are detected on the surface of perovskite solar cell films. The precise location of the defects and their area percentage are output through a visual interface.

[0013] In step 1), the specific method for setting up the image acquisition device can be as follows: based on the light transmission characteristics of the perovskite film, an image acquisition environment with bottom lighting and isolation from ambient light is set up, a sample is prepared in a vacuum chamber in a cleanroom, and the sample is transferred to the acquisition device through a vacuum transfer box, and data acquisition is completed using an industrial high-speed CCD camera.

[0014] In step 2), the specific method of data augmentation can be as follows: Oversampling of small target defect images. Specifically, the method is to perform connected component detection on the image label image, use the findContours function of OpenCV to find closed contour lines in the image, and use the contourArea function to calculate the area of ​​each contour line. Samples containing points with an area of ​​less than 5 pixels are selected and oversampled when constructing the dataset to increase the proportion of small defect samples in the dataset.

[0015] Image rotation processing includes 90-degree and 270-degree rotations; this rotation operation significantly changes the image context information while maintaining the basic structure of the image; through this processing method, the model can learn the features of the image in different directions, thereby improving the model's ability to recognize images at various angles.

[0016] In step 3), the specific method for designing the improved U-Net++ model can be as follows: In the data input layer, a fine-grained feature mining module is introduced, and its main steps are as follows:

[0017] First, the input image is divided into four equal-sized sub-images. Then, bilinear interpolation is used to smoothly scale these four sub-images to the same size as the original image. Next, the original image and the four scaled sub-images are concatenated along the channel dimension to form a new tensor. The first dimension of this tensor corresponds to the original image, and the following four dimensions correspond to the four scaled sub-images.

[0018] Next, the generated tensor is passed as input data to the multi-scale feature fusion network in the module. In this network, the tensor undergoes feature extraction operations through two convolutional layers. To ensure the stability of network training and accelerate convergence, a batch normalization layer is introduced after the convolutional layers.

[0019] Finally, the attention weights, after feature fusion from the Sigmoid layer output, are multiplied element-wise with the original input image to obtain the enhanced Tensor for small defects, which serves as the input to the improved U-Net++ network. This module effectively enhances the weight of small defects in the final image result, thereby improving the network's ability to identify and locate small defects.

[0020] Replacing the backbone network with EfficientNet improves training and inference speed. EfficientNet is a convolutional neural network with a composite scaling strategy. Its core consists of a series of specific convolutional layers designed to efficiently extract image features and reduce the spatial dimensionality of images. Specifically, the network first applies a 3×3 convolutional layer for feature extraction, followed by a 1×1 convolutional layer to expand the feature dimension, then a k×k depthwise separable convolutional layer, a channel compression and activation network module, and finally another 1×1 convolutional layer to reduce the feature dimension. Through this structural design, EfficientNet can improve the computational efficiency of the model while maintaining accuracy.

[0021] For the skip connection structure, based on the original four skip connections of the U-Net++ feature fusion network, more skip connections and feature sampling networks are integrated. This structure enables U-Net++ to capture features at different levels and integrate them through feature stacking, reducing semantic gaps and improving segmentation accuracy. At the same time, a deep supervised convolutional network is added after each module, which can significantly reduce the number of parameters within an acceptable accuracy range and improve the efficiency of the model.

[0022] In step 4), the loss function method is specifically as follows:

[0023] Set the loss function to a weighted sum of Focal_Loss and Dice_Loss, with a weight ratio of 6:4.

[0024] Focal_Loss is an improved loss function, based on the binary cross-entropy loss function (CE), designed to address class imbalance, particularly in tasks like object detection and image segmentation. Focal Loss reduces the weights of easily classified samples, allowing the model to focus more on difficult-to-classify samples during training, thereby improving the detection accuracy of small object defects. It also enhances the capture of the bulk morphology of perovskite thin films. The specific calculation formula is as follows:

[0025]

[0026] in, Represents the Focal loss function. It is the model's predicted probability for positive class samples. It is a balance factor between the positive and negative classes, and can be set to 0.5; is a modulating factor used to control the weight of hard-to-distinguish samples, and can be set to 4; y represents the true label of the sample.

[0027] The Dice loss function is used to improve the model's ability to detect defects. The specific calculation formula is as follows:

[0028]

[0029] in, Represents the Dice loss function. This represents the total number of positive sample pixels that were correctly detected as defects. This represents the total number of pixels that were correctly detected as negative samples. This represents the total number of pixels that were incorrectly detected as positive samples when they were negative samples.

[0030] In step 5), the perovskite thin film defect detection visualization method is specifically as follows:

[0031] A graphical user interface based on the PyQt5 framework was designed and integrated with a defect detection network. Image files were uploaded through a visual interface, and then the images were analyzed and processed using a pre-trained improved U-Net++ model. After the model outputs the detection results, the system calculates and displays the proportion of defective regions and presents the defect detection results visually in image form.

[0032] This invention provides a defect detection system for perovskite thin-film solar cells based on an improved U-Net++, comprising: a data acquisition module, a data processing and enhancement module, a model building and training module, a defect detection inference module, and a visualization and interaction module;

[0033] The data acquisition module is used to acquire raw images of perovskite thin films in a controlled environment;

[0034] The data processing and enhancement module is used to clean, enhance, and divide the dataset of the original image.

[0035] The model building and training module is used to build and train the improved U-Net++ defect detection model;

[0036] The defect detection inference module is used to load the pre-trained model and perform defect detection and statistics on the input image;

[0037] The visualization and interaction module is used to provide a user interface and display the detection results.

[0038] Furthermore, the data acquisition module includes:

[0039] Industrial camera and environmental control equipment; the industrial camera is a HIKROBOT 4K CCD camera, supporting a white light source with a wavelength of 400-800nm, which illuminates the defect areas of the perovskite thin film to make them appear white; the environmental control equipment includes a glove box, a vacuum transfer box, and a constant temperature and humidity chamber, used to maintain a low temperature and low humidity shooting environment to prevent film deformation.

[0040] Furthermore, the data processing and enhancement module includes:

[0041] Small target oversampling unit is used to detect and filter defect samples with an area of ​​less than 5 pixels by connecting components and perform oversampling;

[0042] Multi-angle enhancement unit for rotating and mirroring images by 90° and 270°;

[0043] Dataset partitioning unit, used to divide the data into training set, validation set and test set in a 7:2:1 ratio.

[0044] Furthermore, the model building and training module includes:

[0045] The fine-grained feature mining module is used to divide the input image into 4 sub-images and perform bilinear interpolation to stitch them together, and then enhance the ability to extract small defect features through a two-layer feature fusion network.

[0046] The encoding module uses EfficientNet-B7 as the backbone network, which includes an MBConv structure and an SE module for efficient extraction of deep semantic features.

[0047] A multi-scale feature fusion module is used to fuse encoder and decoder features from different levels through dense skip connections;

[0048] The loss function is a weighted sum of Focal Loss and Dice Loss, with a weight ratio of 6:4.

[0049] Furthermore, the SE module includes:

[0050] Global average pooling layer is used to compress spatial information of feature maps;

[0051] Two fully connected layers are used to generate channel attention weights;

[0052] The feature recalibration unit is used to multiply the attention weights with the original feature map channel by channel to enhance key features.

[0053] Furthermore, the defect detection inference module includes:

[0054] The preprocessing unit is used to adjust the input image to a 512×512 resolution and normalize it to [0,1].

[0055] The model inference unit is used to load the pre-trained improved U-Net++ model and output a defect probability map.

[0056] The post-processing unit is used to generate a binary mask through threshold segmentation and to calculate the percentage and number of defect areas, as well as the maximum / minimum pixel values.

[0057] Furthermore, the visualization interaction module includes:

[0058] The image upload and display unit supports users in uploading JPG / PNG format images and previewing them in real time.

[0059] The detection result visualization unit is used to compare the defect mask with the original image and display the defect area, number, and pixel statistics.

[0060] The operation log unit is used to record user operations and system status information.

[0061] Compared with the prior art, the present invention has the following outstanding advantages and technical effects:

[0062] This invention discloses a high-precision perovskite thin film defect detection system. First, a low-temperature, low-humidity, light-protected image acquisition device is built based on the characteristics of perovskite to acquire original images and construct a dataset. Then, based on the improved U-Net++ network structure, a fine-grained feature mining mechanism is proposed. The input image undergoes scale transformation and feature fusion, enabling the model to better focus on minute features in the image. This model is deployed on the image segmentation model U-Net++, and the backbone network is replaced with EfficientNet. For the training loss function, a hybrid loss is proposed to address imbalanced samples, allowing the network to better select difficult categories. After obtaining the trained improved U-Net++ model parameters, a graphical visualization interface is developed based on PyQt5. Users can upload images to be detected through the interface and intuitively view the defect detection results and defect area ratios. The user-friendly graphical interface makes this invention widely applicable and easy to use, automating and intelligentizing the defect detection process, greatly reducing the need for professional inspection personnel, and improving the efficiency and accuracy of defect detection.

[0063] This invention effectively addresses the problems of low efficiency and poor accuracy in detecting minute defects in traditional detection methods through innovative equipment setup, data augmentation strategies, network structure improvements, and the development of a visual interface. Experimental results show that the method of this invention achieves a recall rate of 99.22% and a precision of 97.74% for small defect detection on a self-made perovskite thin film dataset, representing improvements of 2.38% and 0.53% respectively compared to the U-Net model. This invention demonstrates significant advantages in detecting small target defects and can provide reliable assurance for the quality control of perovskite solar cell production. This invention will be widely applied in the field of perovskite solar cell production, promoting the intelligent development of industry detection technology. Attached Figure Description

[0064] Figure 1 A schematic diagram of the high-precision perovskite thin film defect detection method provided for the implementation of this invention;

[0065] Figure 2 Schematic diagram of a perovskite thin film image acquisition device;

[0066] Figure 3 A schematic diagram of the fine-grained feature mining module provided for the implementation of this invention;

[0067] Figure 4 The overall architecture of the visual automatic perovskite defect detection interface provided for the implementation of this invention;

[0068] Figure 5 The implementation process of the visual automatic perovskite defect detection interface provided for the implementation of this invention;

[0069] Figure 6A schematic diagram of a visual automatic perovskite defect detection interface provided for the implementation of this invention;

[0070] Figure 7 This is a visualization diagram of the attention module for implementing the fine-grained feature mining of this invention. Detailed Implementation

[0071] The technical solution of the present invention will be described in detail below with reference to the embodiments and accompanying drawings. It should be noted that the described embodiments are only some examples of the present invention and do not represent all implementations of the present invention. All other implementations that can be made by those skilled in the art based on the teachings of the present invention without inventive effort are within the protection scope of the present invention.

[0072] Example 1: Defect Detection Method for Perovskite Thin-Film Solar Cells

[0073] Figure 1 A schematic diagram of the high-precision perovskite thin film defect detection method provided for the implementation of this invention. Figure 1 As shown, the system flow includes dataset expansion → optimization of fine-grained feature mining module → backbone network iteration to EfficientNet → construction of improved U-Net++ model → modification of loss function → model training → defect detection and result analysis; specifically, the perovskite thin film defect detection method in this embodiment of the invention includes the following steps:

[0074] S1: Obtain target detection data, classify and process the data to obtain a dataset.

[0075] S11, such as Figure 2 As shown, a four-sided light-shielded image acquisition device is constructed, mainly including a robotic arm module and a light source module. The robotic arm module ensures the stability of the camera position, while the adjustable power stable light source module ensures the consistency of image acquisition, so as to meet the high requirements of imaging accuracy for perovskite thin film defect segmentation.

[0076] The robotic arm module utilizes a Hikvision industrial-grade three-axis slide to achieve precise camera positioning in three-dimensional space along the XYZ axes, ensuring the focal plane is parallel to the surface of the thin film sample. The end effector of the robotic arm is equipped with a Hikvision industrial CCD color camera with a focal length of 35 mm and an imaging distortion rate of <0.1%. This camera is capable of capturing high-resolution, high-contrast sub-micron color images at 13 μm / pixel, providing ample pixel information for subsequent algorithm model training.

[0077] The bottom light source of the light source module uses a custom-designed 400-800 nm white LED array, which distributes the light evenly through an acrylic diffuser plate to ensure consistent brightness across all areas of the perovskite thin film during data acquisition. Simultaneously, the light source power is controllable, supporting illumination intensity adjustment from 0-1000 lux. A built-in light intensity sensor monitors the stability of the light source intensity to avoid overexposure or underexposure. After the bottom light source module is activated, the normal areas of the perovskite thin film appear orange-red, while defective areas appear white, exhibiting high contrast.

[0078] S12. Due to the unstable nature of perovskite, after sample preparation is completed in a glove box, the thin film is transferred to a low-temperature and low-humidity environment using a vacuum transfer box to complete image acquisition, so as to avoid drastic changes in the surface morphology of perovskite and affect the image acquisition quality.

[0079] S13, after collecting the original images, the dataset is augmented and divided into training set, validation set and test set in a ratio of 7:2:1.

[0080] Data augmentation methods mainly include:

[0081] The specific implementation method for oversampling images of small defects is as follows: Connectivity detection is performed on the image label image, the findContours function of OpenCV is used to find closed contour lines in the image, and the area of ​​each contour line is calculated using the contourArea function. Samples containing points with an area of ​​less than 5 pixels are selected and oversampled during the construction of the dataset to increase the proportion of small defect samples in the dataset.

[0082] The image is rotated, specifically by 90 degrees and 270 degrees. This rotation significantly alters the image's contextual information while preserving its basic structure. This approach allows the model to learn features from different directions, thereby improving its ability to recognize images from various angles.

[0083] S2: Improve the original U-Net++ model to obtain the improved U-Net++ model.

[0084] S21, a fine-grained feature mining module is added to the detection head to more effectively extract minute defects in the image. Specifically, the input image is first divided into four equal-sized sub-images. Bilinear interpolation is used to smoothly scale these four sub-images to the same size as the original image. Then, the original image and the scaled sub-images are concatenated along the channel dimension to form a new tensor. This tensor is then input into a multi-scale feature fusion network, where features are extracted sequentially through two convolutional layers. A batch normalization layer is introduced after each convolutional layer. Finally, the new feature-fused tensor is output as input to the improved U-Net++ network, enhancing the weight of small defects in the image and improving the network's ability to identify and locate small defects.

[0085] like Figure 3 As shown, the specific implementation method of the fine-grained feature mining module is as follows:

[0086] First, the input image is divided into four equal-sized sub-images. Then, bilinear interpolation is used to smoothly scale these four sub-images to the same size as the original image. Next, the original image and the four scaled sub-images are concatenated along the channel dimension to form a new tensor. The first dimension of this tensor corresponds to the original image, and the following four dimensions correspond to the four scaled sub-images.

[0087] Next, the generated tensor is passed as input data to the multi-scale feature fusion network in the module. In this network, the tensor will be processed for feature extraction through two convolutional layers. To ensure the stability of network training and accelerate convergence, a batch normalization (BN) layer is introduced after the convolutional layers.

[0088] Finally, the attention weights, after feature fusion from the Sigmoid layer output, are multiplied element-wise with the original data to obtain the enhanced Tensor for small defects, which serves as the input to the improved U-Net++ network. This module effectively enhances the weight of small object defects in the final image result, thereby improving the network's ability to identify and locate small object defects.

[0089] S22, the backbone network is replaced with Efficient-B7 to accelerate model extrapolation capabilities. The core architecture of EfficientNet is based on the MBConv structure, enhancing network efficiency and performance by integrating depthwise separable convolutions and Squeeze-and-Excitation (SE) modules. The encoder of the original U-Net++ is replaced with EfficientNet-B7, while retaining its pre-trained weights.

[0090] The MBConv structure first expands the number of channels in the input feature map using 1×1 convolutional layers, then performs depthwise convolutions to extract spatial features. Next, the SE module learns the relationships between channels to generate attention weights, enhancing key features and suppressing non-key features. Finally, another 1×1 convolutional layer adjusts the number of channels in the feature map. EfficientNet is divided into nine stages. The first stage uses standard convolutional layers with 3×3 kernels and a stride of 2 (integrating batch normalization and Swish activation functions). Stages two through eight repeatedly stack the MBConv structure. The ninth stage consists of 1×1 convolutional layers (containing BN and Swish activation functions), average pooling layers, and fully connected layers. Multiple stages are sequentially split into different positions in the encoding module, gradually expanding the channel dimension to extract deep image features.

[0091] The specific implementation method is as follows:

[0092] The core architecture of EfficientNet is based on the MBConv structure, which integrates depthwise separable convolutions and a Squeeze-and-Excitation (SE) module to enhance network efficiency and performance. The initial step of the MBConv structure involves a 1×1 convolutional layer that expands the number of channels in the input feature map, followed by depthwise convolution to extract spatial features independently on each channel. Next, the SE module uses global average pooling and two fully connected layers to compress and then expand the channel-dimensional features, learning the relationships between channels and generating attention weights to enhance key features and suppress non-key features. Finally, another 1×1 convolutional layer adjusts the number of channels in the feature map to the desired number of output channels. With the input and output channel numbers consistent and a stride of 1, the MBConv structure also utilizes skip connections to maintain the continuity of feature information. This allows MBConv to effectively extract and enhance features while maintaining computational efficiency.

[0093] EfficientNet is divided into nine stages. Stage 1 consists of a standard convolutional layer with a 3×3 kernel and a stride of 2, which integrates batch normalization (BN) and the Swish activation function. From Stage 2 to Stage 8, the network repeatedly stacks the MBConv structure. Stage 9 consists of a 1×1 convolutional layer (also containing BN and the Swish activation function), an average pooling layer, and a fully connected layer.

[0094] In EfficientNet, multiple stages are sequentially split into different positions within the encoding module, gradually increasing the channel dimension to extract deep features from the image.

[0095] S23 replaces the single-hop connection in the feature connection network with multiple hops, thereby improving the network's feature fusion capability.

[0096] The improvement of the jump connection structure described in S23 is specifically implemented as follows:

[0097] Building upon the traditional single-hop connection between the U-Net++ encoder and decoder, the concept of dense skip connections is introduced, allowing feature maps at different levels to be directly connected. During decoding, the model can extract information from feature maps at multiple scales, thereby better recovering image details, enhancing the model's ability to capture detailed information, and improving its robustness and generalization ability. Simultaneously, through dense skip connections, U-Net++ can better preserve fine structures and edge information in the image.

[0098] S3: Input the dataset into the improved U-Net++ model and train the improved U-Net++ model.

[0099] First, a training dataset containing multiple perovskite thin film sample images and corresponding defect labels is obtained.

[0100] Then, after constructing the improved U-Net++ network, the improved U-Net++ network is trained using the gradient descent method with the perovskite thin film image as input and the defect image label as output, until an effective perovskite thin film defect detection model is obtained.

[0101] The loss function used is a weighted sum of Focal_Loss and Dice_Loss.

[0102] Focal_Loss is an improved loss function, based on the binary cross-entropy loss function (CE), designed to address class imbalance, particularly in tasks like object detection and image segmentation. Focal Loss reduces the weights of easily classified samples, allowing the model to focus more on difficult-to-classify samples during training, thereby improving the detection accuracy of small object defects. It also enhances the capture of the bulk morphology of perovskite thin films. The specific calculation formula is as follows:

[0103]

[0104] in, Represents the Focal loss function. It is the model's predicted probability for positive class samples. This is the balance factor between the positive and negative classes, set to 0.5; is a modulating factor used to control the weight of hard-to-distinguish samples, and is set to 4; y represents the true label of the sample.

[0105] The Dice loss function is used to improve the model's ability to detect defects. The specific calculation formula is as follows:

[0106]

[0107] in, Represents the Dice loss function. This represents the total number of positive sample pixels that were correctly detected as defects. This represents the total number of pixels that were correctly detected as negative samples. This represents the total number of pixels that were incorrectly detected as positive samples when they were negative samples.

[0108] The optimizer used is the Adam optimizer, and the specific calculation formula is as follows:

[0109]

[0110]

[0111]

[0112] in, It is the first-moment estimate of the gradient at the current step. It is the first-moment estimate from the previous step. It is the exponential decay factor of the first moment, which is set to 0.9 here. This is the gradient value at the current step. It is the second moment estimate of the gradient of the current step. It is the second-order moment estimate from the previous step. This is the exponentially decaying number of the second moment, which is set to 0.999 here. These are the current parameters. For the updated parameters, The learning rate is set to 0.001 here. It is a very small constant, set to 10 here. -8 To prevent division by zero errors.

[0113] The specific training was conducted using Python 3.11 as the programming language and PyTorch 1.12 as the deep learning framework. The runtime environment was Windows 11, and the computing device was an NVIDIA 3080 GPU. The hyperparameters for the training process were set as follows: batch size of 2 and number of iterations of 100 epochs.

[0114] S4: Detect defects on the surface of the perovskite solar cell thin film using a pre-trained U-Net++ model. This step, based on the model trained in S3, performs defect detection on the input perovskite solar cell thin film image.

[0115] S5: Outputs images showing the precise location and area percentage of defects through a visual interface.

[0116] like Figure 4 As shown, the overall architecture of the visualized automatic perovskite defect detection interface includes a user interaction layer, a core algorithm layer, and a data processing layer. The user interaction layer is based on the PyQT5 development framework and contains various visualization and human-computer interaction components; the core algorithm layer realizes the system's underlying signal communication and deep model loading and calling through the signal-slot mechanism; the data processing layer performs quantitative analysis of the model inference results through matrix operations and connected component detection algorithms, and intuitively displays the thin film quality assessment results in the form of statistical indicators.

[0117] like Figure 5 As shown, the implementation process of the visualized automatic perovskite defect detection interface is as follows:

[0118] S51, Model Initialization: The user opens the defect detection system interface through the start command. At the same time, the system preloads parameters with perovskite as the default model. Then, the user decides whether to change the model based on the model identifier.

[0119] S52, Image Import: Users can select and import images via the "Upload Image" button on the interface, and the images will then be displayed on the interface.

[0120] S53, Defect Detection: After uploading the image, it is automatically preprocessed. The preprocessing steps include image resizing and normalization. The preloaded depth defect segmentation model quickly detects the image to be inspected and outputs a binary image of the defect.

[0121] S54, Quantitative Index Statistics: After obtaining the prediction results, the program calculates the percentage of the defect area in the entire image, the number of defects, and the maximum / minimum defect pixels, and displays them on the interface. The specific implementation method is as follows:

[0122] The defect area percentage is calculated by statistically analyzing the ratio of defective pixels to the total number of pixels in the prediction results. Pixels with a value of 1 represent defective areas, and pixels with a value of 0 represent normal areas. Next, the number of defective pixels in the binary image is counted, and combined with the total number of pixels in the image, the percentage of the defective area in the entire image is calculated. Finally, this percentage is displayed at a designated location on the visualization interface.

[0123] The number of defects and the calculation of the maximum / minimum defect pixels are accomplished by the connectedComponentsWithStats function of the OpenCV library. The defect region is divided into non-overlapping connected components and the number of connected components is counted. The area of ​​all connected components is calculated and the maximum and minimum defect areas are selected.

[0124] S55, the results show that after thresholding, the model's output generates a binarized image representing the detected defect areas. The binarized image is then visualized and displayed on the interface for user viewing, providing a basis for subsequent processes.

[0125] The entire process is now complete.

[0126] Example 2: Perovskite Thin Film Defect Detection System

[0127] The high-precision perovskite thin film defect detection system of this invention consists of five core modules: a data acquisition module, a data processing and enhancement module, a model building and training module, a defect detection inference module, and a visualization and interaction module. These modules work together to achieve full automation from data acquisition to defect detection.

[0128] The data acquisition module is used to acquire raw images of the perovskite thin film in a controlled environment using an industrial camera. The module includes an industrial camera and environmental control equipment. The industrial camera supports a 400-800nm ​​wavelength white light source, with bottom lighting to make defect areas appear white. In this embodiment, a HIKROBOT 4K CCD is used. The environmental control equipment maintains a low-temperature, low-humidity shooting environment to prevent deformation of the perovskite solar cell. The environmental control equipment includes a glove box (for sample preparation), a vacuum transfer box (for sample transfer), and a constant temperature and humidity chamber (for the shooting environment). The data acquisition module outputs raw perovskite thin film images (high-resolution RGB format).

[0129] The data processing and enhancement module is used to clean and enhance the data (e.g., oversampling small defects, rotation) to generate a standardized training set. The module receives the raw image output from the data acquisition module, processes it through data cleaning, data enhancement, oversampling small defects, and multi-scale enhancement, and outputs the enhanced image. Further, the module includes a small target oversampling unit, a multi-angle enhancement unit, and a dataset partitioning unit. The small target oversampling unit increases the proportion of small defect samples in the dataset. The multi-angle enhancement unit enhances the model's robustness to different shooting angles. The dataset partitioning unit proportionally divides the dataset into training, validation, and test sets.

[0130] The model building and training module is used to design and improve the U-Net++ model, and train a high-precision defect detection network using augmented data. Further, the model building and training module includes a fine-grained feature mining module, an encoding module, and a multi-scale feature fusion module. The fine-grained feature mining module divides the input image into four sub-images, performs bilinear interpolation, concatenates multi-scale features, and then passes them through a two-layer feature fusion network to enhance the model's sensitivity to small defects (<5 pixels) and improve small target detection capabilities. The encoding module replaces the original U-Net++ encoder, using EfficientNet-B7 to efficiently extract deep semantic features. The multi-scale feature fusion module is used to fuse features from different levels through dense skip connections, improving the model's adaptability to defects of various sizes. The loss function uses Focal Loss (to address class imbalance) and Dice Loss (to optimize segmentation continuity) with a weight ratio of 6:4.

[0131] The defect detection inference module loads a pre-trained model, performs defect segmentation and statistical calculations on the input image, and performs image preprocessing and model inference on the user-uploaded image to be detected (from the visualization interaction module); it outputs a defect mask map and numerical statistical results. Further, the defect detection inference module includes a preprocessing unit, a model inference unit, and a post-processing unit; the preprocessing unit standardizes the input image to adapt to model requirements. The model inference unit calls the improved U-Net++ model to generate a defect probability map. The post-processing unit parses the model output and generates visualization results and statistical reports.

[0132] The visual interaction module, such as Figure 6 As shown, this module provides a user interface that supports image uploading, detection triggering, and result visualization. Further, the visualization interaction module includes an image uploading and display unit, a detection result visualization unit, and an operation log unit; the image uploading and display unit allows users to upload local images and preview them in real time. The detection result visualization unit highlights defect areas and displays statistical values. The operation log unit records the system's operating status and user operation history.

[0133] Example 3: Model Performance Validation

[0134] To further verify the effectiveness and generalization ability of this method, the detection capabilities of mainstream defect segmentation models were validated on a self-made perovskite solar cell defect dataset. The proposed method achieved a recall of 99.22% and a precision of 97.74% for small defect detection, representing improvements of 2.38% and 0.53% respectively compared to the traditional U-Net model, fully demonstrating the sensitivity of the proposed method for detecting minute defects. A comparison of the performance of each model is shown in Table 1.

[0135] Table 1

[0136]

[0137] To further verify the effectiveness of the improved method proposed in this invention, ablation experiments were conducted on each module. The specific experimental methods are as follows:

[0138] Experiment 1: The original U-Net model;

[0139] Experiment 2: Add a dense skip connection structure to U-Net to transform it into a U-Net++ model;

[0140] Experiment 3: Based on the improvements made in Experiment 2, the encoder network was replaced with EfficientNet-B7;

[0141] Experiment 4: Based on the improvements in Experiment 3, the loss function was changed to a hybrid loss function of Focal Loss and Dice Loss;

[0142] Experiment 5: Based on the improvements made in Experiment 4, a fine-grained feature mining module was added;

[0143] Experimental results demonstrate that the proposed improved methods can effectively enhance the model's ability to detect perovskite defects and improve the F1-score, especially the recall rate. This proves that the proposed methods have a stronger ability to mine small defect features and achieve the best balance between precision and recall.

[0144] The results of the ablation experiment of the improved module are shown in Table 2.

[0145] Table 2

[0146]

[0147] like Figure 7 As shown, to further verify the feature enhancement effect of the fine-grained feature mining module on defects, its attention map was visualized. The results show that darker red indicates higher weight values. The attention map reveals that the fine-grained feature mining module successfully extracted defect features; defect areas were marked with a darker red, while the weight of background areas was suppressed. Experimental results demonstrate that this module alleviates the class imbalance problem of minor defects in the original input through early feature enhancement, and combined with the multi-scale decomposition strategy and attention weight mechanism, provides higher-quality initial features for subsequent networks.

[0148] Specific examples are used in this invention to illustrate the principles and implementation methods of the invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the invention.

Claims

1. A defect detection method for perovskite thin-film solar cells based on an improved U-Net++, characterized in that... Includes the following steps: 1) Image acquisition equipment setup: Set up a perovskite thin film image collection device, create light-proof acquisition conditions, and complete image acquisition in a low humidity and low temperature environment; 2) Data Augmentation: Data augmentation is performed on the acquired images. Operations such as small defect sample oversampling, rotation, and horizontal flipping are used to divide the labeled dataset to obtain training and testing datasets. 3) Improved U-Net++ model design: The original U-Net++ model is improved by introducing a fine-grained feature mining module, adjusting the skip connection structure, and changing the backbone network to EfficientNet, resulting in the improved U-Net++ model. 4) Model training: Input the enhanced dataset into the improved U-Net++ model, adjust the loss function to a weighted sum of Focal_Loss and Dice_Loss, increase the loss weight of small target defects, and train the improved U-Net++ model. 5) Visualization Interface Development: Using the pre-trained U-Net++ model, defects are detected on the surface of perovskite solar cell films. The precise location of the defects and their area percentage are output through a visual interface.

2. The defect detection method for perovskite thin-film solar cells based on improved U-Net++ as described in claim 1, characterized in that... In step 1), the specific method for setting up the image acquisition device can be as follows: based on the light transmission characteristics of the perovskite film, an image acquisition environment with bottom lighting and isolation from ambient light is set up, a sample is prepared in a vacuum chamber in a cleanroom, and the sample is transferred to the acquisition device through a vacuum transfer box, and data acquisition is completed using an industrial high-speed CCD camera.

3. The defect detection method for perovskite thin-film solar cells based on improved U-Net++ as described in claim 1, characterized in that... In step 2), the specific method of data augmentation can be as follows: Oversmapling of small target defect images. Specifically, the method is to perform connected component detection on the image label image, use the findContours function of OpenCV to find closed contour lines in the image, and use the contourArea function to calculate the area of ​​each contour line. Samples containing points with an area of ​​less than 5 pixels are selected and oversampled when constructing the dataset to increase the proportion of small defect samples in the dataset.

4. The defect detection method for perovskite thin-film solar cells based on improved U-Net++ as described in claim 1, characterized in that... In step 2), the image rotation process includes rotations of 90 degrees and 270 degrees; this rotation operation significantly changes the image context information while keeping the basic structure of the image unchanged; it allows the model to learn the features of the image in different directions, thereby improving the model's ability to recognize the image at various angles.

5. The defect detection method for perovskite thin-film solar cells based on improved U-Net++ as described in claim 1, characterized in that... In step 3), the fine-grained feature mining module is introduced, and the steps are as follows: First, the input image is divided into four sub-images of equal size. Then, bilinear interpolation is used to smoothly scale these four sub-images to make their size the same as that of the original image. The original image and the four scaled sub-images are then concatenated along the channel dimension to form a new tensor. The first dimension of this tensor corresponds to the original image, and the next four dimensions correspond to the four scaled sub-images respectively. Next, the generated tensor is passed as input data to the multi-scale feature fusion network in the module; in this network, the tensor is subjected to feature extraction operation through two convolutional layers in sequence; to ensure the stability of network training and accelerate the convergence speed, a batch normalization layer is introduced after the convolutional layer. Finally, the attention weights after feature fusion from the Sigmoid layer are multiplied element-wise with the original data to obtain the enhanced Tensor for small defects, which serves as the input to the improved U-Net++ network. This module enhances the weight of small target defects in the final image result, thereby improving the network's ability to identify and locate small target defects. Replacing the backbone network with EfficientNet improves training and inference speed. EfficientNet is a convolutional neural network with a composite scaling strategy. Its core consists of a series of specific convolutional layers designed to efficiently extract image features and reduce the spatial dimensionality of images. Specifically, the network first applies a 3×3 convolutional layer for feature extraction, followed by a 1×1 convolutional layer to expand the feature dimension, then a k×k depthwise separable convolutional layer, and a channel compression and activation network module. Finally, another 1×1 convolutional layer reduces the feature dimension. Through this structural design, EfficientNet can improve the computational efficiency of the model while maintaining accuracy. For the skip connection structure, based on the original four skip connections of the U-Net++ feature fusion network, more skip connections and feature sampling networks are integrated. This structure enables U-Net++ to capture features at different levels and integrate them through feature stacking, reducing semantic gaps and improving segmentation accuracy. At the same time, a deep supervised convolutional network is added after each module, which can significantly reduce the number of parameters within an acceptable accuracy range and improve the efficiency of the model.

6. The defect detection method for perovskite thin-film solar cells based on improved U-Net++ as described in claim 1, characterized in that... In step 4), the loss function method is specifically as follows: Set the loss function to a weighted sum of Focal_Loss and Dice_Loss; Focal_Loss is an improved loss function that is based on the binary cross-entropy loss function (CE) and aims to solve the class imbalance problem, especially in tasks such as object detection and image segmentation. Focal Loss improves the detection accuracy of small-target defects by reducing the weights of easily classified samples, allowing the model to focus more on difficult-to-classify samples during training. It also enhances the capture of the bulk morphology of perovskite thin films. The specific calculation formula is as follows: in, Represents the Focal loss function. It is the model's predicted probability for positive class samples. It is a balancing factor between positive and negative classes; y is a modulating factor used to control the weight of hard-to-classify samples; y represents the true label of the sample. The Dice loss function is used to improve the model's ability to detect defects; the specific calculation formula is as follows: in, Represents the Dice loss function. This represents the total number of positive sample pixels that were correctly detected as defects. This represents the total number of pixels that were correctly detected as negative samples. This represents the total number of pixels that were incorrectly detected as positive samples when they were negative samples.

7. The defect detection method for perovskite thin-film solar cells based on improved U-Net++ as described in claim 1, characterized in that... In step 5), the perovskite thin film defect detection visualization method is specifically as follows: A graphical user interface was designed based on the PyQt5 framework and a defect detection network was integrated. Image files were uploaded through a visual interface, and then the images were analyzed and processed using a pre-trained improved U-Net++ model. After the model outputs the detection results, the system calculates and displays the proportion of the defect area, and presents the defect detection results in an intuitive image format.

8. A defect detection system for perovskite thin-film solar cells based on an improved U-Net++, characterized in that... It includes a data acquisition module, a data processing and enhancement module, a model building and training module, a defect detection and inference module, and a visualization and interaction module; The data acquisition module is used to acquire raw images of perovskite thin films in a controlled environment; The data processing and enhancement module is used to clean, enhance, and divide the dataset of the original image. The model building and training module is used to build and train the improved U-Net++ defect detection model; The defect detection inference module is used to load the pre-trained model and perform defect detection and statistics on the input image; The visualization and interaction module is used to provide a user interface and display the detection results.

Images