Fullerene film defect detection method based on machine learning and acquisition system

Abstract

The invention discloses a fullerene film defect detection method based on machine learning and an acquisition system, and belongs to machine vision. The problems of low efficiency and low precision of traditional manual detection are solved. Obtaining an original image of the fullerene film to be detected and constructing an enhanced data set; and after training the strong robustness network model by using the enhanced data set, inputting a to-be-detected fullerene film picture into the model, and identifying the accurate defect position of the film. According to the method, a fullerene film image data set is collected and marked, data enhancement processing is carried out, an improved UNet network model is constructed, and optimization is carried out in combination with an OfficientNet backbone network, a channel compression excitation network module and a BCE and Dice loss function. And performing defect detection on the to-be-detected fullerene film image by using the trained model, and outputting a defect detection result picture. The method has the advantages of high detection precision, high speed, strong generalization ability and the like, and is suitable for fullerene film defect detection tasks in a complex industrial production environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and specifically relates to a fullerene film defect detection method and acquisition system based on machine learning. Background Art

[0002] Fullerene, as an allotrope of carbon, is often used as an electron transport layer in perovskite thin-film solar cells due to its unique physical and chemical properties, especially its excellent thermal stability and electrical conductivity. In the production process of thin films, defects such as pinholes and cracks often appear due to the complexity of the production process. These defects will affect the uniformity and electrical conductivity of the film, and then affect the efficiency and stability of the final product. Therefore, it is particularly important to detect and identify defects in fullerene films in a timely manner during the production process.

[0003] Traditional detection methods rely on manual visual inspection, which is not only inefficient but also easily affected by subjective factors, resulting in low detection accuracy. With the advancement of automation and machine vision technology, the use of intelligent means to quickly and accurately detect fullerene film defects has become a key means to improve production efficiency and product quality.

[0004] At present, defect detection convolutional neural networks have achieved good results in some laboratory tasks. However, due to the complex industrial production conditions and the changing equipment detection environment, the imaging quality will be affected by factors such as light and shooting angle; and the fullerene film types vary, and different formula solutions present different color morphology, which requires the defect detection model to have stronger robustness and generalization ability for film samples of different colors and shapes. Summary of the invention

[0005] The purpose of the present invention is to address the problems of low accuracy, low efficiency, and insufficient robustness and generalization of fullerene film defect detection in the prior art, and to provide a fullerene film defect detection method and acquisition system based on machine learning that can quickly and accurately detect fullerene film defects. By optimizing the detection algorithm and improving data acquisition and processing capabilities, the present invention aims to improve the production quality and efficiency of fullerene films and reduce the cost and subjective impact of manual detection.

[0006] In order to achieve the above-mentioned object of the invention, the present invention adopts the following technical solutions.

[0007] The present invention provides a fullerene film defect detection method based on machine learning, comprising the following steps:

[0008] 1) Image collection: Collect fullerene film images and annotate defects in the images; the annotated information includes the location and size of the defects;

[0009] 2) Data enhancement: The data set is enhanced by increasing the number of training images through rotation, HSV color gamut transformation, etc., so that the model has stronger generalization ability. The enhanced data set is divided into training set, test set and validation set;

[0010] 3) Network model design: Use the improved UNet network structure as the detection model, add a channel compression excitation network module between the encoder and the decoder, so that the model can better capture the color gradient changes between defects and films; replace the backbone network of UNet with EfficientNet to improve the training and reasoning speed; combine the cross entropy loss function and the Dice loss function to balance the model's detection capabilities for defects and the film itself; establish an improved UNet fullerene film defect detection network;

[0011] 4) Model training: Train the improved UNet model on the enhanced dataset, optimize the network parameters by gradient descent, obtain the optimal network parameters and save them; during the training process, use the validation set to evaluate the performance of the model, and adjust the hyperparameters according to the validation set effect after training to prevent overfitting;

[0012] 5) Defect detection: The fullerene film image to be detected is input into the trained improved UNet. The model outputs the defect detection result image and performs post-processing on the detection result, such as binarization and morphological operations, to extract the precise location and shape information of the defect.

[0013] In step 2), the data enhancement method is specifically:

[0014] Image rotation, specifically 90 degrees and 270 degrees, can significantly change the context of the image while maintaining the basic structure of the image content. In this way, the model can learn image features in different directions and enhance its recognition ability at different angles.

[0015] Color space transformation converts the image from RGB color space to HLS (hue, brightness, saturation) color space; HLS color space is closer to the way humans perceive color; specifically, adjust the hue H to increase and decrease by 70, the brightness L to increase and decrease by 32.5%, and the saturation to decrease by 20%.

[0016] In step 3), the channel compression excitation network module performs global average pooling on the feature map, then performs nonlinear transformation through two fully connected layers, and finally uses the Sigmoid function to generate channel attention weights to strengthen or suppress the feature response of a specific channel; the specific formula is as follows:

[0017]

[0018]

[0019]

[0020] in, is the original feature map after the convolution operation, z is the result after global average pooling (dimension is 1×1×C), represents a fully connected layer (dimensionality reduction), a ReLu activation function, and a fully connected layer (dimensionality increase), σ is the Sigmoid activation function, s is the output channel weight (dimension is 1×1×C), ⊙ represents element-by-element multiplication (Hadamard product); is the output feature map.

[0021] In step 3), the EfficientNet is a convolutional neural network using a compound scaling strategy. The core structure is a 3×3 convolutional layer for preliminarily extracting features and reducing the spatial dimension of the image, followed by a 1×1 convolutional layer for dimensionality increase, a k×k depthwise separable convolution and a channel compression excitation network module, and finally a 1×1 convolutional layer for dimensionality reduction.

[0022] In step 3), the cross entropy loss function and the Dice loss function are combined, and the two loss functions are directly weighted and summed as the total loss function, wherein the Dice loss function is used to promote the model to detect defects, and the cross entropy loss function is used to strengthen the detection of the model film body; the specific formula of the cross entropy loss function is as follows:

[0023]

[0024]

[0025] in, represents the cross entropy loss function, Indicates the label of the pixel sample data, the positive sample is 1, and the negative sample is 0. It indicates the probability of predicting that the pixel is a positive sample. represents the Dice loss function, Represents the total number of positive sample pixels that are correctly detected as defects, Indicates the total number of pixels whose negative sample pixels are correctly detected as negative samples, Represents the total number of pixels where negative samples are falsely detected as positive samples.

[0026] In step 3), the improved UNet network structure is used as the detection model, which mainly consists of two symmetrical paths: an encoder (downsampling path) and a decoder (upsampling path);

[0027] In the encoder, UNet extracts features through a series of efficient convolution operations, applying the ReLU activation function after each convolution, followed by a 2×2 maximum pooling layer to halve the feature map size, and double the number of channels after each downsampling to extract richer features layer by layer.

[0028] In the decoder, the feature map is restored to its original size layer by layer through 3×3 convolution operations and 2×2 upsampling layers, while reducing the number of channels. In each upsampling step, the feature map of the corresponding layer in the encoder is concatenated with the current upsampled feature map through a jump connection to combine high-level and low-level features and preserve spatial resolution and detail information.

[0029] After all upsampling steps, UNet adjusts the number of channels to the number of target categories through a 1×1 convolutional layer.

[0030] The present invention provides a fullerene film defect detection and acquisition system, which comprises an image acquisition module, a data enhancement module, a defect detection module, and a post-processing module in sequence;

[0031] The image acquisition module is used to collect image data of the fullerene film during the production process;

[0032] The data enhancement module is used to preprocess and enhance the collected images; the number of training images is increased by rotation and HSV color gamut transformation, so that the model has stronger generalization ability;

[0033] The defect detection module is the core part of the system. After using the image algorithm to perform size transformation and normalization on the original fullerene image, the pre-processed image is input into the machine learning model to extract image-related features and determine whether there are defects. The machine learning model adopts an improved UNet network structure and has powerful feature extraction and classification capabilities.

[0034] The post-processing module is used to further process the output results of the detection module to extract the precise position and shape information of the defect.

[0035] Furthermore, the improved UNet network structure achieves more efficient and accurate feature extraction and defect detection by introducing the EfficientNet architecture, SE module, and combining the cross entropy loss function and Dice loss function. At the same time, the segmentation accuracy and robustness of the model are further improved through skip connections and sophisticated network design.

[0036] The improved UNet network structure is mainly composed of two major paths: encoder (responsible for downsampling) and decoder (responsible for upsampling). The two present a symmetrical layout. At the same time, a channel compression excitation network module (SE module) is added between the encoder and decoder to further enhance the network's sensitivity to feature channels.

[0037] The encoder is used to extract the features of the input image layer by layer. The encoder is composed of multiple EfficientNet architecture modules connected in series, and each module contains the following components:

[0038] EfficientNet architecture: includes MBConv modules, convolutional layers, fully connected layers and other modules for feature extraction and image spatial dimensionality reduction.

[0039] ReLU activation function: Introduces nonlinear characteristics into the network, which helps the model learn more complex features.

[0040] Max pooling layer: A 2×2 pooling window is used to downsample the feature map to reduce its spatial resolution while increasing the number of channels to capture more feature information.

[0041] In the encoder, as the image is continuously downsampled, more abstract and rich feature representations are extracted layer by layer.

[0042] The channel compression excitation network module (SE module) is arranged between the encoder and the decoder, and the SE module is introduced to enhance the network's attention to the feature channel.

[0043] The decoder is used to restore the feature map extracted by the encoder to the size of the original image layer by layer and output the final defect detection result. Its structure includes the following components:

[0044] Upsampling layer: A 2×2 upsampling window is used to upsample the feature map to restore its spatial resolution.

[0045] Convolutional layer: used to further extract and refine features.

[0046] Skip connection: concatenates the feature map of the corresponding layer in the encoder with the current upsampled feature map to combine high-level and low-level features and retain more spatial resolution and detail information.

[0047] ReLU activation function: increases the nonlinear characteristics of the network.

[0048] After all upsampling steps, the decoder adjusts the number of channels to the number of target categories (i.e., the number of defect types + background class) through a 1×1 convolution layer, thereby outputting the final defect detection result.

[0049] The present invention discloses a fullerene film defect detection method and system based on machine learning. Firstly, the original image of the fullerene film to be inspected is obtained and an enhanced data set is constructed. After a strong robustness network model is trained using the enhanced data set, the fullerene film image to be inspected is input into the model to identify the precise location of the film defect.

[0050] Compared with the prior art, the present invention has the following outstanding technical effects and advantages:

[0051] 1. The present invention introduces a channel compression excitation network module, which generates channel attention weights through global average pooling, nonlinear transformation of the fully connected layer and Sigmoid function to strengthen or suppress the characteristic response of a specific channel. This mechanism enables the model to better capture the color gradient changes between defects and films, thereby improving the accuracy and robustness of detection.

[0052] 2. The present invention replaces the UNet backbone network with EfficientNet, optimizes the network structure using a composite scaling strategy, and improves the efficiency of feature extraction. This improvement not only reduces the computational complexity of the model, but also significantly increases the training and reasoning speed, enabling the system to complete defect detection tasks for a large number of images in a short period of time.

[0053] 3. The present invention adopts a loss function that combines the cross entropy loss function (BCE) and the Dice loss function to balance the model's ability to detect defects and the film body. The BCE loss function helps the model to accurately identify the boundaries of the film body, while the Dice loss function promotes the model's accurate detection of defect areas. This combination enables the model to maintain good detection performance under complex backgrounds.

[0054] 4. The UNet network model of the present invention uses two symmetrical paths, the encoder (downsampling path) and the decoder (upsampling path), combined with jump connections, to effectively fuse high-level and low-level features and retain spatial resolution and detail information. The optimization enables the model to more accurately identify tiny defects in fullerene films and improve the accuracy and reliability of detection.

[0055] 5. The present invention significantly improves the accuracy and efficiency of detection by improving the network structure, optimizing the loss function and constructing an enhanced data set, and can achieve fast and accurate fullerene film defect detection in a complex production environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic flow chart of a method for detecting defects in fullerene films provided in Example 1 of the present invention;

[0057] Figure 2 Schematic diagram of the training set enhancement process;

[0058] Figure 3This is a schematic diagram of EfficientNet;

[0059] Figure 4 Schematic diagram of SE attention mechanism;

[0060] Figure 5 This is the effect diagram of fullerene film defect detection. DETAILED DESCRIPTION

[0061] The technical scheme of the present invention is described in detail below in conjunction with the embodiments and drawings of the present invention. It should be noted that the described embodiments are only partial examples of the present invention and do not represent all implementation methods of the present invention. All other implementation methods that can be made by those skilled in the art based on the enlightenment of the present invention without creative work belong to the protection scope of the present invention.

[0062] The present invention aims to provide a fullerene film defect detection method based on machine learning, aiming to improve the defect detection accuracy of fullerene films in the production process of solar cells. In order to make the above-mentioned objects, features and advantages of the present invention clearer, the present invention is described in more detail below in conjunction with specific embodiments and drawings.

[0063] Example 1

[0064] Figure 1 The flowchart of the fullerene film defect detection method provided in Example 1 of the present invention is shown in FIG. Specifically, the fullerene film defect detection method in the embodiment of the present invention comprises the following steps:

[0065] Step 101: Acquire an image of the fullerene film to be detected.

[0066] Step 102: Input the image into a pre-trained fullerene film defect detection model, and the model will output the precise location of the defect of the fullerene film.

[0067] Among them, the fullerene film defect detection model is obtained by training an improved UNet network through a training data set. The training data set contains multiple fullerene film sample images and their corresponding defect coordinates. The improved UNet network includes an encoding network, a channel attention network and a decoding network. The encoding network structure adopts the EfficientNet structure.

[0068] like Figure 2As shown in the figure, the training dataset enhancement process includes the following methods. Rotate the image to let the model learn spatial features at different angles. Change the color space and adjust the hue to help the model learn to maintain color sensitivity under different lighting conditions; adjust the brightness to simulate different light intensities from shadows to direct light, which is crucial to improving the performance of the model under different lighting conditions; adjust the saturation to help the model learn to recognize image features under different color saturations. By simulating variable imaging in complex environments, the generalization ability of the model after training is enhanced.

[0069] like Figure 3 As shown in the figure, the core of the EfficientNet structure is the MBConv structure, which uses depth-separable convolution and Squeeze-and-Excitation (SE) modules to improve the efficiency and performance of the network. The MBConv module first expands the number of channels of the input feature map through 1×1 point-wise expansion, and then applies depth-wise convolution to extract spatial features independently on each channel. Next, the Squeeze-and-Excitation (SE) module learns the correlation between channels through global average pooling and two fully connected layers, generates attention weights to strengthen important features and suppress unimportant features. Finally, another 1×1 point convolution (Point-wise LinearProjection) adjusts the number of channels of the feature map to the required number of output channels. When the number of input and output channels is the same and the stride is 1, MBConv also uses jump connections to keep the flow of feature information, so that MBConv can effectively extract and strengthen features while maintaining computational efficiency.

[0070] like Figure 4 As shown in Figure 1, the channel compression network module enhances the network's ability to detect defects in different color domains by combining a channel attention mechanism. This mechanism mimics the attention mechanism of the human visual system, allowing the model to extract key areas in different color channels, thereby focusing more accurately on the defective areas of the fullerene film.

[0071] During the inspection process, the encoding network first extracts features from the input fullerene film image and passes the encoding results to the SENet network to generate a weighted feature map. After that, the decoding network processes these weighted features and finally outputs the defect type of the film.

[0072] The encoding network consists of four encoding modules, each of which includes a Stage in the EfficientNet architecture, a normalization layer, and a ReLU activation function.

[0073] EfficientNet is divided into 9 stages. Stage 1 is a common convolution layer with a convolution kernel size of 3×3 and a stride of 2 (including BN and activation function Swish); Stage 2 to Stage 8 repeatedly stack the MBConv structure; Stage 9 consists of a common 1×1 convolution layer (including BN and activation function Swish), an average pooling layer, and a fully connected layer. Multiple stages in EfficientNet are sequentially split into different positions in the encoding module, gradually expanding the channel dimension to extract deep features in the image.

[0074] The decoding network consists of four decoding modules, each of which contains an upsampling operation, two convolutional layers, two ReLU activation functions, and two BatchNorm layers. Upsampling uses a bilinear interpolation method to restore the spatial feature information in the image.

[0075] Through skip connections, the features of the encoding stage are fused with the features of the decoding stage, combining the underlying spatial information with the high-level semantic information, thereby improving the segmentation accuracy of the model. The skip connection method of the decoding network can effectively improve the model's ability to identify defect areas.

[0076] As an optional implementation, the training process of the fullerene film defect detection model is as follows:

[0077] First, a training dataset containing multiple fullerene film sample images and corresponding defect types is obtained;

[0078] Then, data augmentation is performed based on the original dataset.

[0079] Secondly, we build an improved UNet network, change the backbone network and add SENet at the jump connection; the specific calculation formula is as follows:

[0080]

[0081]

[0082]

[0083] in, is the original feature map after the convolution operation, z is the result after global average pooling (dimension is 1×1×C), It means passing through a fully connected layer (dimensionality reduction), a ReLu activation function, and a fully connected layer (dimensionality increase). σ is the Sigmoid activation function, s is the output channel weight (dimension is 1×1×C), and ⊙ represents element-by-element multiplication (Hadamard product). is the output feature map.

[0084] Finally, the improved UNet network is trained using the training data set, with the film image as input and the corresponding precise defect position as output, until an effective fullerene film defect detection model is obtained.

[0085] The loss function used is the weighted sum of BCE loss and Dice loss. The specific formula is as follows:

[0086]

[0087]

[0088] in, represents the cross entropy loss function, Indicates the label of the pixel sample data, the positive sample is 1, and the negative sample is 0. It indicates the probability of predicting that the pixel is a positive sample. represents the Dice loss function, Represents the total number of positive sample pixels that are correctly detected as defects, Indicates the total number of pixels whose negative sample pixels are correctly detected as negative samples, Represents the total number of pixels where negative samples are falsely detected as positive samples.

[0089] During the specific training, the programming language used was Python 3.9 and the deep learning framework was Pytorch 1.12. The operating environment used the Windows 11 operating system and the computing device was an NVIDIA 3080 GPU. The hyperparameters of the training process were set as follows: the batch size was 5, the optimizer used the Adam optimizer, the initial value of the learning rate was 0.001, and the number of iterations was set to 200 rounds.

[0090] The present invention completes the training on the self-made data set training set and verification set, and finally uses the test set to evaluate the network detection effect. Table 1 shows the final effect.

[0091] Table 1 Model performance comparison

[0092]

[0093] Figure 5 The effect diagram of fullerene film defect detection is given. Figure 5In the figure, the left side is the original fullerene film image, which contains some subtle defects that are not easy to detect, such as pinholes and cracks. These defects may be difficult to accurately identify in manual visual inspection, and the efficiency is low. The right side is the defect detection result image after being processed by the method of the present invention. Figure 5 In the detection result diagram, it can be clearly seen that the model of the present invention can accurately identify defects in the film.

[0094] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A fullerene film defect detection method based on machine learning, characterized in that The following steps are involved: 1) Image collection: Collect fullerene film images and annotate defects in the images; the annotated information includes the location and size of the defects; 2) Data enhancement: The data set is enhanced by increasing the number of training images through rotation and HSV color gamut transformation to make the model more generalizable. The enhanced data set is divided into training set, test set and validation set. 3) Network model design: Use the improved UNet network structure as the detection model, add a channel compression excitation network module between the encoder and the decoder, so that the model can better capture the color gradient changes between defects and films; replace the backbone network of UNet with EfficientNet to improve the training and reasoning speed; combine the cross entropy loss function and the Dice loss function to balance the model's detection capabilities for defects and the film itself; establish an improved UNet fullerene film defect detection network; 4) Model training: Train the improved UNet model on the enhanced data set, optimize the network parameters by gradient descent method, obtain the optimal network parameters and save them; During the training process, the validation set is used to evaluate the performance of the model. After the training is completed, the hyperparameters are adjusted based on the validation set results to prevent overfitting. 5) Defect detection: The fullerene film image to be detected is input into the trained improved UNet, and the model outputs the defect detection result image.

2. A method for detecting defects in fullerene films based on machine learning as claimed in claim 1, characterized in that In step 2), the data enhancement method is specifically: Image rotation, specifically 90 degrees and 270 degrees, can significantly change the context of the image while maintaining the basic structure of the image content. In this way, the model can learn image features in different directions and enhance its recognition ability at different angles. Color space transformation converts the image from RGB color space to HLS (hue, brightness, saturation) color space; HLS color space is closer to the way humans perceive color; specifically, adjust the hue H to increase and decrease by 70, the brightness L to increase and decrease by 32.5%, and the saturation to decrease by 20%.

3. A method for detecting defects in fullerene films based on machine learning as claimed in claim 1, characterized in that In step 3), the channel compression excitation network module performs global average pooling on the feature map, then performs nonlinear transformation through two fully connected layers, and finally uses the Sigmoid function to generate channel attention weights to strengthen or suppress the feature response of a specific channel; the specific formula is as follows: in, is the original feature map after the convolution operation, z is the result after global average pooling (dimension is 1×1×C), represents a fully connected layer (dimensionality reduction), a ReLu activation function, and a fully connected layer (dimensionality increase), σ is the Sigmoid activation function, s is the output channel weight (dimension is 1×1×C), ⊙ represents element-by-element multiplication (Hadamard product); is the output feature map.

4. A method for detecting defects in fullerene films based on machine learning as claimed in claim 1, characterized in that In step 3), the EfficientNet is a convolutional neural network using a compound scaling strategy. The core structure is a 3×3 convolutional layer for preliminarily extracting features and reducing the spatial dimension of the image, followed by a 1×1 convolutional layer for dimensionality increase, a k×k depthwise separable convolution and a channel compression excitation network module, and finally a 1×1 convolutional layer for dimensionality reduction.

5. A method for detecting defects in fullerene films based on machine learning as claimed in claim 1, characterized in that In step 3), the cross entropy loss function and the Dice loss function are combined, and the two loss functions are directly weighted and summed as the total loss function, wherein the Dice loss function is used to promote the model to detect defects, and the cross entropy loss function is used to strengthen the detection of the model film body; the specific formula of the cross entropy loss function is as follows: in, represents the cross entropy loss function, Indicates the label of the pixel sample data, the positive sample is 1, and the negative sample is 0. It indicates the probability of predicting that the pixel is a positive sample. represents the Dice loss function, Represents the total number of positive sample pixels that are correctly detected as defects, Indicates the total number of pixels whose negative sample pixels are correctly detected as negative samples, Represents the total number of pixels where negative samples are falsely detected as positive samples.

6. A method for detecting defects in fullerene films based on machine learning as claimed in claim 1, characterized in that In step 3), the improved UNet network structure is used as the detection model, which mainly consists of two symmetrical paths: an encoder (downsampling path) and a decoder (upsampling path); In the encoder, UNet extracts features through a series of efficient convolution operations, applying the ReLU activation function after each convolution, followed by a 2×2 maximum pooling layer to halve the feature map size, and double the number of channels after each downsampling to extract richer features layer by layer; In the decoder, the feature map is restored to its original size layer by layer through 3×3 convolution operations and 2×2 upsampling layers, while reducing the number of channels. In each upsampling step, the feature map of the corresponding layer in the encoder is concatenated with the current upsampled feature map through skip connections to combine high-level and low-level features and preserve spatial resolution and detail information. After all upsampling steps, UNet adjusts the number of channels to the number of target categories through a 1×1 convolutional layer.

7. A fullerene film defect detection and acquisition system, characterized in that It includes an image acquisition module, a data enhancement module, a defect detection module, and a post-processing module in sequence; The image acquisition module is used to collect image data of the fullerene film during the production process; The data enhancement module is used to perform preprocessing and data enhancement on the collected images; Increase the number of training images by rotating and transforming the HSV color space, so that the model has stronger generalization ability; The defect detection module is the core part of the system. After using the image algorithm to perform size transformation and normalization on the original fullerene image, the pre-processed image is input into the machine learning model to extract image-related features and determine whether there are defects. The machine learning model adopts an improved UNet network structure and has powerful feature extraction and classification capabilities. The post-processing module is used to further process the output results of the detection module to extract the precise position and shape information of the defect.

Images