Defect detection method and device and storage medium

Abstract

The invention discloses a defect detection method and device and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: collecting a plurality of surface images of a to-be-detected OLED film; extracting features of each surface image under different scales through a pre-trained deep learning model to obtain a feature map of the surface of the to-be-detected OLED film; and predicting defects of the feature map based on a pre-trained deep learning model to obtain a defect detection result of the surface of the to-be-detected OLED film. According to the method, the multi-scale features of the OLED film surface image are extracted, and the feature map containing different scale features of the OLED film surface image is constructed based on the multi-scale features, so that the problem that a traditional deep learning model is difficult to capture multi-scale defect features is avoided, and the method can adapt to the scale diversity of OLED film surface defects, and is high in robustness. Therefore, the effect of improving the defect detection accuracy and the model generalization ability is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a defect detection method, device and storage medium. Background Art

[0002] OLED (Organic Light-Emitting Diode) film surface defect detection is extremely critical to ensure product quality. It can quickly identify defects in the production process, prevent substandard products from entering the market, and reduce economic losses. With the advancement of deep learning technology, especially the application of convolutional neural networks, the automation and accuracy of OLED film surface defect detection have been significantly improved. These deep learning models can automatically learn complex features from large amounts of image data without manual intervention in the feature extraction process.

[0003] However, although deep learning technology has shown great potential in OLED film defect detection, it still faces some challenges in practical applications, among which the scale diversity of defects is the main problem currently. The defects on the surface of OLED films vary in shape, from tiny scratches to larger dents, which requires the detection system to be able to adapt to defects of different scales. Traditional deep learning models are sensitive to scale changes and it is difficult to capture defect features of all scales at the same time, thus limiting the adaptability and generalization ability of the model.

[0004] Therefore, how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films is a problem that needs to be solved urgently. Summary of the invention

[0005] The main purpose of this application is to provide a defect detection method, device and storage medium, aiming to solve the technical problem of how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films.

[0006] To achieve the above objectives, the present application proposes a defect detection method, which includes: Collect multiple surface images of the OLED film to be inspected; Extracting features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected; The defects of the characteristic graph are predicted based on the pre-trained deep learning model to obtain defect detection results of the surface of the OLED film to be detected.

[0007] In one embodiment, the step of acquiring multiple surface images of the OLED film to be inspected comprises: Collecting multi-illumination images and multi-viewing angle images of the OLED film to be detected, wherein the multi-illumination images are multiple images of the OLED film to be detected collected under multiple color lights, and the multi-viewing angle images are multiple images of the OLED film to be detected collected under different angles of illumination based on preset color lights; Based on the multi-illumination images and the multi-viewing angle images, a plurality of surface images of the OLED film to be inspected are composed.

[0008] In one embodiment, the pre-trained deep learning model is configured with a plurality of scalable convolutional layers with different dilation rates, and the step of extracting features of each surface image at different scales through the pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected comprises: For any scalable convolutional layer in the pre-trained deep learning model, extracting target scale features of each surface image at a current dilation rate through the scalable convolutional layer; After traversing each scalable convolutional layer, a feature map of the surface of the OLED film to be detected is constructed based on each target scale feature.

[0009] In one embodiment, the step of constructing a feature map of the surface of the OLED film to be inspected based on each target scale feature comprises: Performing residual connection on each target scale feature at the output end to obtain a candidate feature map of the surface of the OLED film to be detected; Detecting the image size of the candidate feature map, and when the size of the candidate feature map is inconsistent with the size of the surface image, adjusting the shape of the candidate feature map by a preset convolution to obtain a feature map of the surface of the OLED film to be detected; In the case where the size of the candidate feature map is consistent with the size of the surface image, the candidate feature map is used as the feature map of the surface of the OLED film to be detected.

[0010] In one embodiment, after the step of extracting the features of each surface image at different scales by a pre-trained deep learning model to obtain the feature map of the surface of the OLED film to be detected, the step further includes: Processing the feature map based on a channel attention mechanism and a spatial attention mechanism respectively, and constructing a new feature map based on an output of the channel attention mechanism after processing the feature map and an output of the spatial attention mechanism after processing the feature map; The characteristic map of the surface of the OLED film to be detected is updated based on the new characteristic map.

[0011] In one embodiment, the step of predicting the defects of the feature map based on the pre-trained deep learning model to obtain the defect detection result of the surface of the OLED film to be detected includes: Mapping the feature map based on the fully connected layer in the pre-trained deep learning model to obtain scores for each preset defect category; Convert each score into a probability distribution, and determine a target defect category among the preset defect categories based on the converted probability distribution, wherein the score corresponding to the target defect category has the highest probability in the probability distribution; The target defect category is used as the defect detection result of the surface of the OLED film to be detected.

[0012] In one embodiment, the defect has multiple defect categories, and any defect category corresponds to a defect detection training task. The defect detection method further includes: For any defect detection training task among the defect detection training tasks, determine a historical deep learning model obtained after the previous defect detection training task of the defect detection training task is completed, and obtain the Fisher information matrix and loss function corresponding to the historical deep learning model, the historical weight corresponding to the previous defect detection training task, and the current weight corresponding to the defect detection training task; Optimizing the loss function according to the Fisher information matrix, the historical weights and the current weights, and training the historical deep learning model based on the optimized loss function; After each defect detection training task is completed, the pre-trained deep learning model is obtained.

[0013] In one embodiment, the step of training the historical deep learning model based on the optimized loss function includes: A sample data set of multi-view and multi-illumination images is obtained, and the sample data set is input into the historical deep learning model to train the historical deep learning model, wherein during the training process of the historical deep learning model, the model parameters in the historical deep learning model are adjusted according to the optimized loss function.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a defect detection device, which includes: An image acquisition module, used for acquiring multiple surface images of the OLED film to be inspected; A feature extraction module, used to extract features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected; A defect detection module is used to predict the defects of the feature map based on the pre-trained deep learning model to obtain defect detection results on the surface of the OLED film to be detected.

[0015] In addition, to achieve the above objectives, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the defect detection method described above.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the defect detection method described above are implemented.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the defect detection method described above are implemented.

[0018] One or more technical solutions proposed in this application have at least the following technical effects: The present application first collects multiple surface images of the OLED film to be inspected to obtain the necessary data source for subsequent defect detection; extracts the features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be inspected, so as to more comprehensively capture the defect features of the surface of the OLED film by extracting the multi-scale features of the surface image, thereby providing a rich defect information basis at different scales for subsequent defect detection; predicts the defects of the feature map based on the pre-trained deep learning model, and obtains the defect detection result of the surface of the OLED film to be inspected, so as to accurately identify the defect type of the surface of the OLED film through the pre-trained deep learning model, thereby providing effective detection results for product quality control.

[0019] In summary, the present application extracts multi-scale features of the OLED film surface image and constructs a feature map containing different scale features of the OLED film surface image based on the multi-scale features. This avoids the problem that traditional deep learning models are sensitive to scale changes and can only identify defects at a single scale or fewer scales, making it difficult to capture multi-scale defect features. This achieves the ability to adapt to the diversity of defect scales on the OLED film surface, thereby achieving the effect of improving defect detection accuracy and model generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the defect detection method of the present application; Figure 2 A schematic diagram of a collection scenario of a defect detection method provided in Example 1 of the present application; Figure 3 A schematic diagram of a process flow provided for the second embodiment of the defect detection method of the present application; Figure 4 A schematic diagram of a process flow provided for the third embodiment of the defect detection method of the present application; Figure 5 A brief flowchart of the defect detection method provided in Example 3 of the present application; Figure 6 A schematic diagram of an optimized process of a defect detection method provided in Embodiment 3 of the present application; Figure 7 This is a schematic diagram of the module structure of the defect detection device according to an embodiment of the present application; Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the defect detection method in the embodiment of the present application.

[0023] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0025] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0026] The main solution of the embodiment of the present application is: collect multiple surface images of the OLED film to be inspected; extract the features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be inspected; predict the defects of the feature map based on the pre-trained deep learning model to obtain the defect detection result of the surface of the OLED film to be inspected.

[0027] Although deep learning technology has shown great potential in OLED film defect detection, it still faces some challenges in practical applications. Among them, the scale diversity of defects is the main problem currently. The defects on the surface of OLED films vary in shape, from tiny scratches to larger dents, which requires the detection system to be able to adapt to defects of different scales. However, traditional deep learning models are sensitive to scale changes and it is difficult to capture defect features of all scales at the same time, which limits the adaptability and generalization ability of the model. Therefore, how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films is a problem that needs to be solved urgently.

[0028] The present application provides a solution, by extracting multi-scale features of the OLED film surface image, and constructing a feature map containing different scale features of the OLED film surface image based on the multi-scale features, thereby avoiding the problem that the traditional deep learning model is sensitive to scale changes and can only identify defects at a single scale or fewer scales, resulting in difficulty in capturing multi-scale defect features, and achieving the ability to adapt to the diversity of scales of OLED film surface defects, thereby achieving the effect of improving defect detection accuracy and model generalization ability.

[0029] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device that can realize the above functions. The following takes an electronic device as an example to illustrate this embodiment and the following embodiments.

[0030] Based on this, the present application embodiment provides a defect detection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the defect detection method of the present application.

[0031] In this embodiment, the defect detection method includes steps S10 to S30: Step S10, collecting multiple surface images of the OLED film to be tested; It should be noted that the surface image refers to a detailed visual image of the surface of the OLED film captured using a high-resolution camera or other imaging equipment.

[0032] Exemplarily, a high-resolution industrial camera is used to photograph the OLED film to be inspected at multiple angles and focal lengths under controlled lighting conditions to obtain a series of surface images of the OLED film to be inspected.

[0033] In a feasible implementation manner, the step of collecting multiple surface images of the OLED film to be inspected in step S10 may include steps S100 to S200: Step S100, collecting multi-illumination images and multi-viewing angle images of the OLED film to be detected, wherein the multi-illumination images are multiple images collected under the illumination of multiple color lights, and the multi-viewing angle images are multiple images collected under the illumination of different angles of the OLED film to be detected based on preset color lights; It should be noted that the multi-light image refers to the image obtained by photographing the OLED film to be tested under different color light conditions, and these color lights may include red light, green light, blue light, etc.; the multi-viewing angle image refers to the image obtained by photographing the OLED film to be tested from different angles under the same color light.

[0034] It is understandable that, since OLED films usually have different sensitivities to different colors of light, there are often subtle defect features under single illumination, and there are also often subtle defect features under a single illumination angle. Therefore, step S100 is performed to obtain the surface details of the OLED film under different illumination and the surface information of the OLED film at different viewing angles. This can avoid surface defects that may be missed under single illumination conditions and the viewing blind spot problem caused by a single viewing angle, thereby achieving all-round and multi-dimensional detection of surface defects of the OLED film to be detected, ensuring accurate identification and classification of defects, and improving the accuracy and reliability of detection.

[0035] For example, please refer to Figure 2 First, the OLED film to be tested is illuminated in sequence with different colored lights (such as red, green, and blue) through the lighting device, and an industrial camera is used to capture images from a fixed position under each colored light to form a multi-lighting image set. Next, the color light is kept as white light, and the camera positioning device is used to adjust the camera angle, and a white light strip light source is used to shoot the details and texture of the highlight surface of the OLED film from multiple different perspectives (such as horizontal and vertical edge sides, etc.), thereby obtaining a multi-perspective image set.

[0036] Step S200: compose a plurality of surface images of the OLED film to be inspected based on the multi-illumination images and the multi-viewing angle images.

[0037] It is understandable that since it is necessary to integrate information under different lighting and viewing angles to form a comprehensive surface image set so as to more accurately analyze the surface state of the OLED film, performing step S200 can avoid the deviation of the defect detection results caused by the singleness of the image information. By combining multi-lighting and multi-viewing angle images, a more comprehensive understanding of the surface defects of the OLED film to be detected can be achieved, thereby improving the accuracy and robustness of the detection system.

[0038] In this embodiment, by using multiple color lights to illuminate and capture OLED film surface images at different angles, the problem of incomplete recognition of OLED film surface defects due to single illumination and single viewing angle is avoided, and the technical effect of all-round and multi-dimensional detection of OLED film surface defects to be detected is achieved. This method combines multi-illumination images and multi-viewing angle images to form a comprehensive surface image set, thereby improving the accuracy and robustness of defect detection and ensuring quality control in the OLED film production process.

[0039] Step S20, extracting features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected; It should be noted that the pre-trained deep learning model refers to a pre-designed and partially trained neural network, which is specifically used for defect detection on the surface image of OLED film. In this embodiment, the model can be implemented using Python and PyTorch framework; the feature map refers to the image processed by the deep learning model, which contains abstract feature information for identifying defects.

[0040] It is understandable that since the defects on the surface of the OLED film vary in shape, from tiny scratches to larger dents, and the traditional deep learning model is sensitive to scale changes and it is difficult to capture defect features of all scales at the same time, thereby limiting the adaptability and generalization ability of the model. Therefore, step S20 is performed to extract features of the surface image at different scales, which can avoid the problem that the traditional deep learning model is difficult to capture defect features of all scales at the same time, thereby achieving more accurate feature representation and providing reliable basic data support for subsequent defect detection.

[0041] For example, the collected images are input into a designed convolutional neural network, which contains multiple convolutional layers and pooling layers, and can extract image features at different scales, such as edges, textures, and shapes. Through the processing of these layers, a series of feature maps are finally obtained, which concentrate on the defect information that may exist on the surface of the OLED film.

[0042] In a feasible implementation, the pre-trained deep learning model is configured with a plurality of scalable convolutional layers with different dilation rates, and step S20 may include steps S21-S22: Step S21, for any scalable convolutional layer in the pre-trained deep learning model, extracting target scale features of each surface image at the current expansion rate through the scalable convolutional layer; It should be noted that a scalable convolutional layer refers to a convolutional layer in a deep learning model whose receptive field size can be adjusted, which is usually achieved by changing the dilation rate of the convolution kernel in the convolutional layer; the current dilation rate refers to the size of the interval between the kernel elements when the convolution kernel in the current convolutional layer performs a convolution operation, which determines the size of the receptive field of the convolutional layer on the image; the target scale feature refers to the image features that the convolutional layer can extract at a specific dilation rate (i.e., the current dilation rate), which reflect the detailed information of the image at a specific scale.

[0043] It is understandable that since the surface defects of the OLED film may exist at different scales, performing step S21 can avoid the problem of being unable to capture multi-scale defects due to the fixed-scale convolution layer. By adjusting the expansion rate, multi-scale feature extraction of the OLED film surface image to be inspected can be achieved, thereby ensuring the comprehensiveness of defect detection.

[0044] Exemplarily, in the pre-trained deep learning model, different expansion rates are set for each scalable convolution layer, such as 1, 2, 4, etc. For the input surface image, the basic features of the image are extracted by scalable convolution layers with different expansion rates, for example, when the expansion rates are 1, 2, and 4, respectively, so as to obtain feature information at different scales.

[0045] Step S22, after traversing each scalable convolutional layer, constructing a feature map of the surface of the OLED film to be detected based on each target scale feature.

[0046] It can be understood that in order to integrate the features extracted under different expansion rates to form a comprehensive feature representation, step S22 is performed to avoid the problem of inaccurate defect detection caused by a single scale feature. By constructing a feature map, the surface defects of the OLED film to be detected can be accurately identified and located, thereby improving the accuracy and efficiency of the defect detection system.

[0047] For example, after traversing all scalable convolutional layers and extracting the target scale features of each layer, these features are fused in the feature space. The specific operation can be to upsample or downsample the feature maps output by each convolutional layer so that they have the same size, and then merge these feature maps into a comprehensive feature map by element addition or splicing. This comprehensive feature map contains detailed information on the surface of the OLED film at different scales, providing rich feature representation for subsequent defect detection.

[0048] In this embodiment, by using multiple scalable convolutional layers with different expansion rates to extract features of surface images from different scales, the technical problem caused by the inability of fixed-scale convolutional layers to adapt to the multi-scale characteristics of OLED film surface defects is avoided, and the extraction of detailed features of the OLED film surface image to be detected at different scales and the construction of comprehensive features are achieved, thereby ensuring the comprehensiveness and accuracy of defect detection and improving the performance of the defect detection system.

[0049] Step S30: predicting defects in the feature map based on the pre-trained deep learning model to obtain defect detection results on the surface of the OLED film to be detected.

[0050] It should be noted that defect categories refer to different types of defects that may appear on the surface of the OLED film, such as Mura defects, foreign points, bright lines, light leakage, dark lines, scratches, bubbles, etc.

[0051] It is understandable that due to the different types of defects that may appear on the surface of the OLED film, such as Mura defects, foreign points, bright lines, etc., in order to successfully identify various defect conditions, step S30 is performed. By using the trained target model to classify the feature map and predict the specific category of the defect, the subjectivity and inefficiency of manual inspection can be avoided, and fast and accurate automatic defect classification can be achieved, thereby improving production efficiency and product quality control.

[0052] Exemplarily, the newly captured surface image of the OLED film to be inspected is subjected to the same preprocessing and feature extraction steps as in the model training to obtain feature maps. These feature maps are then input into the trained target model. The model analyzes the feature maps using its learned weights and structures, and outputs the probability distribution of the defect category corresponding to each feature map. Based on this probability distribution, the category with the highest probability is selected as the defect detection result, thereby achieving automatic recognition and classification of surface defects of the OLED film.

[0053] In a feasible implementation, step S30 may include steps S31 to S33: Step S31, mapping the feature map based on the fully connected layer in the pre-trained deep learning model to obtain scores for each preset defect category; It should be noted that the fully connected layer refers to a layer in the deep learning model, in which each neuron is connected to all neurons in the previous layer, and is used to map the learned features to the label space of the sample; the score under the preset defect category refers to a value output by the model for each preset defect category after processing by the fully connected layer, which indicates the possibility that the input feature map belongs to this category.

[0054] It is understandable that in order to convert the feature map into a specific defect category score, step S31 is performed, which can avoid the problem that a clear classification result cannot be obtained directly from the feature map, and achieves the effect of converting the feature information into a classification score.

[0055] For example, in a deep learning model, the feature map processed by the convolution layer and the pooling layer is input into the fully connected layer. Each neuron in the fully connected layer is connected to each pixel of the feature map. Through the linear combination of weights and biases, the feature map is mapped to a one-dimensional vector. Each element of this vector corresponds to a score of a preset defect category: , where z is the output of the fully connected layer, and are the weights and biases of the fully connected layer respectively.

[0056] Step S32, converting each score into a probability distribution, and determining a target defect category among the preset defect categories based on the converted probability distribution, wherein the score corresponding to the target defect category has the highest probability in the probability distribution; It should be noted that the converted probability distribution refers to converting the scores into probability form through the softmax function, so that the sum of the probabilities of all categories is 1, and the probability of each category represents the probability that the input feature map belongs to that category; the target defect category refers to the defect category with the highest probability in the probability distribution, that is, the most likely defect type predicted by the model.

[0057] It can be understood that in order to convert the score into an interpretable probability form, step S32 is performed, which can avoid the problem of incomparability between scores, thereby achieving the conversion from score to probability and determining the most likely defect category.

[0058] Exemplarily, the score vector output by the fully connected layer is transformed using the softmax function:

[0059]

[0060] in, is the score of the i-th defect category, is the converted probability distribution. The softmax function converts the score vector into a probability distribution, where the probability of each category is the ratio of the indexed score of that category to the sum of the indexed scores of all categories. Then, the category with the highest probability is selected as the target defect category. .

[0061] Step S33: taking the target defect category as the defect detection result of the surface of the OLED film to be detected.

[0062] In this embodiment, the feature map is converted into scores under each preset defect category through a fully connected layer, and then these scores are converted into probability distributions through, for example, a softmax function, so that the target defect category with the highest probability can be clearly identified, and this is used as the defect detection result of the surface of the OLED film to be inspected, thereby avoiding the problem of incomparability between clear classification results and scores that cannot be obtained directly from the feature map, and achieving the effect of accurately mapping the feature map to the defect category and determining the most likely defect type, thereby improving the accuracy and reliability of defect detection.

[0063] The present embodiment provides a defect detection method, which extracts multi-scale features of the OLED film surface image and constructs a feature map including different scale features of the OLED film surface image based on the multi-scale features. This avoids the problem that the traditional deep learning model is sensitive to scale changes and can only identify defects at a single scale or fewer scales, making it difficult to capture multi-scale defect features. This method can adapt to the diversity of scales of OLED film surface defects, thereby achieving the effect of improving defect detection accuracy and model generalization ability.

[0064] In a feasible implementation manner, the step of constructing the feature map of the surface of the OLED film to be detected based on each target scale feature in step S22 may include steps S221 to S223: Step S221, performing residual connection on each target scale feature at the output end to obtain a candidate feature map of the surface of the OLED film to be detected; It should be noted that residual connection refers to a connection method introduced in the network, which allows the network input to directly skip certain layers and connect to the subsequent layers, which is used to alleviate the gradient vanishing problem in deep network training; candidate feature maps refer to feature maps that have been preliminarily processed in the network but have not yet been finalized as input to subsequent classifiers or detectors.

[0065] It is understandable that, since there is usually a problem of gradient disappearance in deep learning networks, step S221 is performed to add the input directly to the output through the residual connection, thereby maintaining the flow of feature information and avoiding the problem of feature information loss due to increased network depth. The residual connection can achieve feature retention and enhancement, thereby improving the network's ability to characterize the surface details of the OLED film.

[0066] For example, in the network architecture, for each feature output by a scalable convolutional layer, the input of the layer is directly added to the output by adding a skip connection (i.e., residual connection). In this way, the output of each convolutional layer is the sum of its input and the result of the convolution operation, and these outputs are then aggregated to form a candidate feature map containing information at multiple scales.

[0067] Step S222, detecting the image size of the candidate feature map, and when the size of the candidate feature map is inconsistent with the size of the surface image, adjusting the shape of the candidate feature map by a preset convolution to obtain a feature map of the surface of the OLED film to be detected; It should be noted that the image size refers to the number of channels as well as the height and width of the candidate feature map.

[0068] It is understandable that since the feature map size may change during the feature extraction process, performing step S222 can avoid the problem of subsequent processing difficulties caused by size mismatch. By adjusting the size, the size of the feature map can be ensured to be consistent with the original surface image, thereby ensuring the accurate transmission of information and facilitating accurate defect location and detection.

[0069] Exemplarily, first, detect the candidate feature map If the size is found to be the same as the original surface image size Inconsistency, among which is the number of channels, is the height, If is the width, the preset convolution layer (such as 1x1 convolution) is used to resize the candidate feature map:

[0070]

[0071]

[0072] Among them, ReLU represents the activation function, Represents an expanded convolution operation with a specified kernel size and expansion rate, and Conv1x1 represents a 1x1 convolution operation, so that the image size of the candidate feature map matches the input original image, thereby obtaining the final feature map.

[0073] Step S223, when the size of the candidate feature map is consistent with the size of the surface image, the candidate feature map is used as the feature map of the surface of the OLED film to be detected.

[0074] It can be understood that since no additional adjustment steps are required when the sizes have already matched, performing step S223 can avoid unnecessary computational overhead and directly use the candidate feature map as the final feature map, which can achieve efficient feature extraction and provide accurate feature representation for subsequent defect detection.

[0075] For example, after confirming that the size of the candidate feature map has matched the size of the original surface image, no additional resizing operation is performed, and the candidate feature map is directly used as the feature map for defect detection. This means that the candidate feature map already contains enough information and can be directly used for subsequent classification or detection tasks.

[0076] In this embodiment, residual connection and selective shape adjustment are performed, wherein the residual connection helps to deepen the network depth without losing information and improve the feature expression capability, while the shape adjustment ensures the size consistency of the feature map, which facilitates subsequent defect detection and positioning, avoids the loss of feature information due to increased network depth and the inaccurate information transmission caused by feature map size mismatch, and achieves the technical effect of ensuring that the feature map size is consistent with the original surface image while maintaining feature integrity, thereby improving the accuracy and robustness of the entire OLED film surface defect detection system.

[0077] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 After step S20, the defect detection method may further include steps S201-S202: Step S201, processing the feature map based on the channel attention mechanism and the spatial attention mechanism respectively, and constructing a new feature map based on the output of the channel attention mechanism after processing the feature map and the output of the spatial attention mechanism after processing the feature map; It should be noted that the channel attention mechanism refers to a mechanism that can automatically learn the importance of each channel (or feature dimension) in the feature map, calculate the weight of each channel through adaptive average pooling and convolution operations, emphasize important feature channels and suppress unimportant channels, thereby improving the efficiency of feature representation; the spatial attention mechanism refers to a mechanism that focuses on the spatial position relationship in the feature map, calculates the weight of each spatial position through convolution and sigmoid activation function, identifies the key areas in the image, and enhances these areas to make the network pay more attention to the important parts of the image; the new feature map refers to a feature map that is constructed by combining the outputs of the channel attention mechanism and the spatial attention mechanism, which pays more attention to key features and areas and optimizes the channel importance.

[0078] It can be understood that in order to improve the network's sensitivity to important channels and spatial positions in the feature map, step S201 is performed, which can avoid the problem of decreased detection performance caused by the network ignoring certain key features. The effect achieved is to improve the characterization ability of the feature map, making subsequent defect detection more accurate.

[0079] For example, first, the channel attention module is used to calculate the weight of each channel of the feature map to highlight the important feature channels, where in_channels is set to 64 and reduction is set to 4 to reduce the number of parameters and increase the training speed; secondly, the spatial position of the feature map is calculated by the spatial attention module to emphasize the key area; finally, the output results of the two modules are multiplied to form a new feature map :

[0080] in, represents the fully connected layer, is adaptive average pooling, represents the convolution operation, represents the sigmoid activation function, represents element-wise multiplication, Represents the broadcast operation of the feature map dimension. The map contains attention information in both channel and spatial dimensions.

[0081] Step S202: updating the characteristic map of the surface of the OLED film to be inspected based on the new characteristic map.

[0082] For example, the newly constructed feature map is added or concatenated with the original feature map at the element level to enhance and refine the original feature map. This process can be achieved by adding additional connections to the corresponding layers of the network, so that the updated feature map can better capture the subtle defect features on the surface of the OLED film, providing more accurate data support for subsequent defect classification or positioning.

[0083] In this embodiment, the model's performance in capturing various defect features is improved by utilizing multiple cross-attention mechanisms, that is, the combination of channel attention mechanism and spatial attention mechanism, so that the network can pay more attention to features related to defect detection, and by constructing a new feature map and updating the original feature map, the problems of insufficient feature representation and loss of key information caused by a single attention mechanism are avoided, and the important channels and spatial positions in the feature map are emphasized at the same time, thereby improving the expression ability of the feature map and the defect detection accuracy, and enhancing the network's ability to recognize surface defects of OLED films.

[0084] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the above-mentioned embodiments 1 and 2 can be referred to the above introduction, and will not be repeated in the following. Figure 4, the defect has multiple defect categories, and any defect category corresponds to a defect detection training task. The defect detection method also includes steps S01 to S03: Step S01, for any one of the defect detection training tasks, determine a historical deep learning model obtained after the previous defect detection training task of the defect detection training task is completed, and obtain the Fisher information matrix and loss function corresponding to the historical deep learning model, the historical weight corresponding to the previous defect detection training task, and the current weight corresponding to the defect detection training task; It should be noted that the defect detection training task refers to the deep learning model training process for specific types of defects (such as scratches, dents, etc.); the previous defect detection training task refers to the model training task for different types of defects that has been completed before the current task; the historical deep learning model refers to the model trained in the previous defect detection training task; the Fisher information matrix refers to a statistic that measures the contribution of data to the accuracy of model parameter estimation and is used to describe the uncertainty of parameter estimation. In machine learning, it can be used to guide weight adjustments during model training to improve the generalization ability of the model; the loss function refers to a mathematical function that quantifies the model prediction error and is used to guide the model optimization training process; the historical weight refers to the weight value obtained after the deep learning model parameters are trained in the past defect detection tasks (historical defect detection tasks), which reflects the performance and preference of the model on historical data; the current weight refers to the initial weight of the model parameter or the weight value that has been preliminarily trained in the current defect detection task, which is the immediate performance of the model on the current task.

[0085] It is understandable that since it is difficult for existing deep learning models to learn new tasks without forgetting the knowledge of old tasks, step S01 is performed to avoid ignoring useful information in historical data by using the information of historical tasks to guide the model training of the current task, thereby improving the efficiency and accuracy of model training.

[0086] Exemplarily, first, identify the previous task of the current defect detection training task, that is, the previous defect detection training task, which has been completed and the trained deep learning model is saved. Then, extract the Fisher information matrix and loss function from the model, where the matrix is ​​obtained by calculating the second derivative of the log-likelihood function of the model parameters. Next, extract the historical weights from the historical model training records corresponding to the previous defect detection task. Then, for the current defect detection task, obtain the current weights from the initialization or preliminary training results of the current model.

[0087] Step S02, optimizing the loss function according to the Fisher information matrix, the historical weights and the current weights, and training the historical deep learning model based on the optimized loss function; It can be understood that in order to adjust the model parameters by optimizing the loss function to adapt to the requirements of the new defect detection task, step S02 is performed, which can avoid the problem of overfitting or underfitting of the model during the training process, thereby improving the performance of the model in the current defect detection task and ensuring that the model can be better generalized to new data.

[0088] For example, using the Fisher information matrix To adjust the loss function to make it pay more attention to the performance of the model on historical tasks. and the current weight Combined with gradient descent or other optimization algorithms, the loss function L Optimize to find a new set of model parameters. These parameters can better balance the needs of historical and current tasks. Finally, the deep learning model is trained using the optimized loss function and feature map, and the model parameters are updated through multiple iterations until the predetermined training goal or convergence condition is reached. For example, the EWC penalty term is calculated by the following formula:

[0089] in, is the contrast loss of the current defect detection task, is the balancing factor (also called the regularization strength coefficient, with a default value of 1, which can be adjusted according to the specific defect detection task and deep learning model), i is the index representing the model parameter, which is reflected by traversing all the parameters of the model and calculating a penalty term for each parameter. In addition, the loss function L Including contrast loss and noise contrast estimation loss , the loss function is calculated by the EWC penalty term based on the Fisher information matrix, historical weights and current weights L To optimize:

[0090] in, and The corresponding weight factors for the contrast loss function and the noise contrast estimation loss are as follows: c = 0.3 and f=0.7 can provide a better performance balance. Then through this loss function L The historical deep learning model is trained to obtain a pre-trained deep learning model.

[0091] In a feasible implementation, the step of training the historical deep learning model based on the optimized loss function in step S02 may include step S021: Step S021, obtain a sample data set of multi-view and multi-illumination images, and input the sample data set into the historical deep learning model to train the historical deep learning model, wherein, during the training process of the historical deep learning model, the model parameters in the historical deep learning model are adjusted according to the optimized loss function.

[0092] It should be noted that multi-view and multi-illumination images refer to a collection of OLED film surface images acquired from different illumination angles and under different color light conditions. These images can reflect the details and defects of the OLED film surface under different viewing angles and illumination conditions, thereby providing more comprehensive and rich training data for deep learning models.

[0093] It is understandable that in order to enable the historical deep learning model to learn the characteristics of OLED film surface defects under different viewing angles and lighting conditions and enhance the generalization and robustness of the model, step S021 is performed. This can avoid the problem that the model is overly dependent on specific conditions due to the single viewing angle and lighting conditions, and is unable to adapt to the complex and changeable environment in actual production. For example, the model overfits to images under specific angles or lighting conditions, resulting in a decrease in detection accuracy in actual applications. At the same time, it can also improve the model's ability to recognize defects in different production environments and reduce false detections and missed detections.

[0094] Exemplarily, first, an automated image acquisition system is designed and built, which includes multiple high-resolution cameras mounted on adjustable brackets and controllable multi-color light sources. Next, the system is operated to shoot the OLED film to be inspected from multiple angles to ensure that each angle covers different areas of the film surface. At the same time, by adjusting the brightness and color of the light source, different lighting conditions are simulated to obtain a series of multi-view and multi-lighting images. These images are then annotated according to defect types and angular lighting conditions to construct a diverse sample data set. Subsequently, this sample data set is input into the designed historical deep learning model to start the training process. During the training process, one or more optimization algorithms (such as gradient descent, Adam, etc.) are used. When using the Adam optimizer, the initial learning rate can be configured to be 2×10 5, weight decay is 0.01, and the training cycle is 200 epochs) to minimize the loss function, which is a quantitative indicator of the difference between the model prediction and the actual label. The model training is completed by updating the parameters such as weights and biases in the model through multiple iterations until the performance of the model on the validation set reaches the preset threshold or no longer improves significantly. This process includes not only the calculation of forward propagation and backpropagation, but also strategies such as learning rate adjustment and regularization to prevent overfitting and ensure that the model has good generalization ability.

[0095] Step S03: After each defect detection training task is completed, the pre-trained deep learning model is obtained.

[0096] It can be understood that in order to obtain a general model that has been trained on multiple related tasks and can adapt to a variety of new types of defect detection, step S03 is performed to avoid the problem that the model cannot adapt to a variety of defect types, thereby obtaining a multifunctional defect detection model that can identify a variety of different types of defects, thereby improving the adaptability and practicality of the detection system.

[0097] For example, after all the training tasks of the predetermined types of defect detection are completed, the model parameters obtained from the last task training are used to construct a comprehensive pre-trained deep learning model, which will contain the encoding of the features learned in all training tasks, so that it can be generalized to the detection of multiple defect types. Ultimately, this pre-trained model can be used for actual OLED film surface defect detection.

[0098] In this embodiment, the loss function corresponding to the historical deep learning model is optimized based on the Fisher information matrix of the historical defect detection model, the historical weights of the historical defect detection tasks, and the current weight of the current defect detection task, thereby avoiding the deep learning model from forgetting the old task knowledge during the training process, thereby causing possible overfitting, underfitting and low training efficiency problems, so that the model can more accurately adapt to the needs of the current task, thereby achieving the effect of improving the model's generalization ability and training stability.

[0099] For example, to help understand the implementation process of the defect detection method obtained by combining the above-mentioned embodiment 1 and embodiment 2, please refer to Figure 5 , Figure 5 A brief flow chart of a defect detection method is provided, specifically: The whole defect detection system includes a multi-scale convolution residual enhancement module, a multi-scale attention integration module, a multi-scale continuous adaptation module and a defect result output module. The multi-scale convolution residual enhancement module is used to perform dilated convolution operations with different dilation rates on the collected input image group (i.e., the OLED film surface image), and perform residual connection at the output end to obtain a candidate feature map, and selectively adjust the shape of the candidate feature map to obtain a feature map of the OLED film surface; then, the channel attention mechanism and the spatial attention mechanism in the multi-scale attention integration module are used to process the feature map respectively to weight the feature map; then, an elastic weight sharing strategy is adopted in the multi-scale continuous adaptation module to realize continuous learning of the image data set of the current defect detection training task; finally, in the defect result output module, the scores of each preset defect category converted from the feature map are converted into a probability distribution through the softmax function, so as to perform classification decisions, and can clearly identify the target defect category with the highest probability, and output it as the defect detection result of the OLED film surface to be detected.

[0100] For further information, please refer to Figure 6 ,The elastic weight sharing strategy in the multi-scale continuous adaptation module is as follows: By maintaining the Fisher information matrix and weights of the old task (previous defect detection task), the update of the weights of the old task is adjusted during the learning process of the new task (defect detection task), reducing the forgetting of the knowledge of the old task. When training a new task, this module calculates a penalty term and adds it to the loss function to protect the weights that are important to the old task. In addition, by continuously adjusting the weight factors contributed by the contrast loss and noise contrast estimation loss, the loss function of the new task can be made more accurate.

[0101] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the defect detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0102] This application also provides a defect detection device, please refer to Figure 7 , the defect detection device comprises: An image acquisition module 10 is used to acquire multiple surface images of the OLED film to be inspected; A feature extraction module 20 is used to extract features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected; The defect detection module 30 is used to predict the defects of the characteristic graph based on the pre-trained deep learning model to obtain the defect detection result of the surface of the OLED film to be detected.

[0103] Optionally, the image acquisition module 10 is further used for: Collecting multi-illumination images and multi-viewing angle images of the OLED film to be detected, wherein the multi-illumination images are multiple images of the OLED film to be detected collected under multiple color lights, and the multi-viewing angle images are multiple images of the OLED film to be detected collected under different angles of illumination based on preset color lights; Based on the multi-illumination images and the multi-viewing angle images, a plurality of surface images of the OLED film to be inspected are composed.

[0104] Optionally, the pre-trained deep learning model is configured with a plurality of scalable convolutional layers with different dilation rates, and the feature extraction module 20 is further used to: For any scalable convolutional layer in the pre-trained deep learning model, extracting target scale features of each surface image at a current dilation rate through the scalable convolutional layer; After traversing each scalable convolutional layer, a feature map of the surface of the OLED film to be detected is constructed based on each target scale feature.

[0105] Optionally, the feature extraction module 20 is further used for: Performing residual connection on each target scale feature at the output end to obtain a candidate feature map of the surface of the OLED film to be detected; Detecting the image size of the candidate feature map, and when the size of the candidate feature map is inconsistent with the size of the surface image, adjusting the shape of the candidate feature map by a preset convolution to obtain a feature map of the surface of the OLED film to be detected; In the case where the size of the candidate feature map is consistent with the size of the surface image, the candidate feature map is used as the feature map of the surface of the OLED film to be detected.

[0106] Optionally, the feature extraction module 20 is further used for: Processing the feature map based on a channel attention mechanism and a spatial attention mechanism respectively, and constructing a new feature map based on an output of the channel attention mechanism after processing the feature map and an output of the spatial attention mechanism after processing the feature map; The characteristic map of the surface of the OLED film to be detected is updated based on the new characteristic map.

[0107] Optionally, the defect detection module 30 is further used for: Mapping the feature map based on the fully connected layer in the pre-trained deep learning model to obtain scores for each preset defect category; Convert each score into a probability distribution, and determine a target defect category among the preset defect categories based on the converted probability distribution, wherein the score corresponding to the target defect category has the highest probability in the probability distribution; The target defect category is used as the defect detection result of the surface of the OLED film to be detected.

[0108] Optionally, the defect has multiple defect categories, and any defect category corresponds to a defect detection training task. The model training module 40 in the defect detection device is used to: For any defect detection training task among the defect detection training tasks, determine a historical deep learning model obtained after the previous defect detection training task of the defect detection training task is completed, and obtain the Fisher information matrix and loss function corresponding to the historical deep learning model, the historical weight corresponding to the previous defect detection training task, and the current weight corresponding to the defect detection training task; Optimizing the loss function according to the Fisher information matrix, the historical weights and the current weights, and training the historical deep learning model based on the optimized loss function; After each defect detection training task is completed, the pre-trained deep learning model is obtained.

[0109] Optionally, the model training module 40 is further used for: A sample data set of multi-view and multi-illumination images is obtained, and the sample data set is input into the historical deep learning model to train the historical deep learning model, wherein during the training process of the historical deep learning model, the model parameters in the historical deep learning model are adjusted according to the optimized loss function.

[0110] The defect detection device provided by the present application adopts the defect detection method in the above embodiment, which can solve the technical problem of how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films. Compared with the prior art, the beneficial effects of the defect detection device provided by the present application are the same as the beneficial effects of the defect detection method provided by the above embodiment, and the other technical features in the defect detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0111] The present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the defect detection method in the above-mentioned embodiment one.

[0112] Reference below Figure 8, which shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present application. The electronic devices in the embodiments of the present application may include but are not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0113] like Figure 8 As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0114] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0115] The electronic device provided by the present application adopts the defect detection method in the above embodiment, which can solve the technical problem of how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the defect detection method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0116] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0117] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0118] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the defect detection method in the above-mentioned embodiment.

[0119] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0120] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0121] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: collects multiple surface images of the OLED film to be detected; extracts the features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected; predicts defects of the feature map based on the pre-trained deep learning model to obtain defect detection results of the surface of the OLED film to be detected.

[0122] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0123] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0124] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0125] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned defect detection method, and can solve the technical problem of how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the defect detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0126] The present application also provides a computer program product, including a computer program, which implements the steps of the defect detection method as described above when executed by a processor.

[0127] The computer program product provided in this application can solve the technical problem of how to improve the adaptability and generalization ability of the detection model to multi-scale defects of OLED films. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the defect detection method provided in the above embodiment, which will not be repeated here.

[0128] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A defect detection method, characterized in that: The defect detection method comprises: Collect multiple surface images of the OLED film to be inspected; Extracting features of each surface image at different scales through a pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected; The defects of the characteristic graph are predicted based on the pre-trained deep learning model to obtain defect detection results of the surface of the OLED film to be detected.

2. The defect detection method according to claim 1, characterized in that: The step of collecting multiple surface images of the OLED film to be inspected comprises: Collecting multi-illumination images and multi-viewing angle images of the OLED film to be detected, wherein the multi-illumination images are multiple images of the OLED film to be detected collected under multiple color lights, and the multi-viewing angle images are multiple images of the OLED film to be detected collected under different angles of illumination based on preset color lights; Based on the multi-illumination images and the multi-viewing angle images, a plurality of surface images of the OLED film to be inspected are composed.

3. The defect detection method according to claim 1, characterized in that: The pre-trained deep learning model is configured with a plurality of scalable convolutional layers with different dilation rates. The step of extracting features of each surface image at different scales through the pre-trained deep learning model to obtain a feature map of the surface of the OLED film to be detected comprises: For any scalable convolutional layer in the pre-trained deep learning model, extracting target scale features of each surface image at a current dilation rate through the scalable convolutional layer; After traversing each scalable convolutional layer, a feature map of the surface of the OLED film to be detected is constructed based on each target scale feature.

4. The defect detection method according to claim 3, characterized in that: The step of constructing a characteristic map of the surface of the OLED film to be detected based on each target scale feature comprises: Performing residual connection on each target scale feature at the output end to obtain a candidate feature map of the surface of the OLED film to be detected; Detecting the image size of the candidate feature map, and when the size of the candidate feature map is inconsistent with the size of the surface image, adjusting the shape of the candidate feature map by a preset convolution to obtain a feature map of the surface of the OLED film to be detected; In the case where the size of the candidate feature map is consistent with the size of the surface image, the candidate feature map is used as the feature map of the surface of the OLED film to be detected.

5. The defect detection method according to claim 1, characterized in that: After the step of extracting the features of each surface image at different scales by the pre-trained deep learning model to obtain the feature map of the surface of the OLED film to be detected, the method further includes: Processing the feature map based on a channel attention mechanism and a spatial attention mechanism respectively, and constructing a new feature map based on an output of the channel attention mechanism after processing the feature map and an output of the spatial attention mechanism after processing the feature map; The characteristic map of the surface of the OLED film to be detected is updated based on the new characteristic map.

6. The defect detection method according to claim 1, characterized in that: The step of predicting the defects of the characteristic graph based on the pre-trained deep learning model to obtain the defect detection result of the surface of the OLED film to be detected includes: Mapping the feature map based on the fully connected layer in the pre-trained deep learning model to obtain scores for each preset defect category; Convert each score into a probability distribution, and determine a target defect category among the preset defect categories based on the converted probability distribution, wherein the score corresponding to the target defect category has the highest probability in the probability distribution; The target defect category is used as the defect detection result of the surface of the OLED film to be detected.

7. The defect detection method according to claim 1, characterized in that: The defect has multiple defect categories, and any defect category corresponds to a defect detection training task. The defect detection method also includes: For any defect detection training task among the defect detection training tasks, determine a historical deep learning model obtained after the previous defect detection training task of the defect detection training task is completed, and obtain the Fisher information matrix and loss function corresponding to the historical deep learning model, the historical weight corresponding to the previous defect detection training task, and the current weight corresponding to the defect detection training task; Optimizing the loss function according to the Fisher information matrix, the historical weights and the current weights, and training the historical deep learning model based on the optimized loss function; After each defect detection training task is completed, the pre-trained deep learning model is obtained.

8. The defect detection method according to claim 7, characterized in that: The step of training the historical deep learning model based on the optimized loss function includes: A sample data set of multi-view and multi-illumination images is obtained, and the sample data set is input into the historical deep learning model to train the historical deep learning model, wherein during the training process of the historical deep learning model, the model parameters in the historical deep learning model are adjusted according to the optimized loss function.

9. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the defect detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the defect detection method according to any one of claims 1 to 7 are implemented.

Images