Neural network, method and system for OLED abnormal display defect detection

Abstract

The invention provides a neural network, method and system for OLED abnormal display defect detection, and relates to the technical field of defect detection, the neural network comprises a backbone network, a Canny edge detection operator, a dense connection network, an attention network and a full connection layer; the backbone network carries out defect feature extraction on the OLED display image to obtain a shallow layer feature map and a deep layer feature map; a Canny edge detection operator carries out edge contour detection on the shallow layer feature map to obtain an edge feature map; performing defect feature multiplexing on the deep feature map by the dense connection network to obtain a gradient feature map; the attention network performs dependency relationship construction on the deep feature map to obtain a regional feature map; and the full connection layer performs defect type prediction on a fusion feature map of the edge feature map, the gradient feature map and the regional feature map to obtain an abnormal display defect type prediction result. According to the method, OLED abnormal display defect detection is carried out in a mode of combining deep learning and edge detection, and the problems of missing detection and over-detection are not likely to occur.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a neural network, method and system for detecting abnormal display defects of OLEDs. Background Art

[0002] Organic laser displays (OLED, Organic Light-Emitting Diode) have gradually become the first choice for displays due to their advantages such as flexible screens, low energy consumption, uniform image quality, and self-luminescence, and their operating range is becoming increasingly wide. During the production and lighting process of organic laser displays, there is an abnormal display defect, which is manifested as abnormal lighting color, overall dimming, blurred display, and block display.

[0003] Since abnormal display defects have a high probability of occurrence and the feature types are diverse, traditional computer vision technology cannot cover all types, which can easily lead to missed detections. Also, since the defect area is large, when using traditional target detection algorithms for detection, some background images with similar colors are easily detected as defects, which can easily lead to over-detection. Summary of the invention

[0004] In order to solve the problem that abnormal display defects of OLED are prone to missed detection and over-detection, the embodiments of the present invention provide a neural network, method and system for abnormal display defect detection of OLED.

[0005] In a first aspect, an embodiment of the present invention provides a neural network for detecting abnormal display defects of OLED, wherein the neural network includes a backbone network, a Canny edge detection operator, a densely connected network, an attention network, and a fully connected layer;

[0006] The backbone network extracts defect features from the OLED display image to obtain a shallow feature map and a deep feature map;

[0007] The edge detection operator performs edge contour detection on the shallow feature map to obtain an edge feature map;

[0008] The densely connected network reuses defect features on the deep feature map to obtain a gradient feature map;

[0009] The attention network constructs dependency relationships on the deep feature map to obtain a regional feature map;

[0010] The fully connected layer predicts the defect type of the edge feature map, the gradient feature map, and the fused feature map of the regional feature map to obtain the prediction result of the abnormal display defect type of the OLED.

[0011] In the above embodiment, the present invention takes into account that the abnormal display defect of OLED itself has a strong block distribution feature, and therefore combines the Canny edge detection operator and the deep learning model to detect the defect. The Canny edge detection operator can effectively detect the edge contour information of the OLED display image, and the deep learning model can learn the characteristics of different types of morphology. Combining the two can effectively represent the characteristics of abnormal defects with a large defect area and diverse feature types, thereby realizing the prediction of the type of abnormal display defect of OLED.

[0012] As some optional implementations of the present application, the neural network also includes a feature map fusion layer, which performs feature map addition fusion and feature map connection fusion on the edge feature map, the gradient feature map and the regional feature map to obtain a fused feature map.

[0013] In the above embodiment, the present invention first performs feature map addition and fusion on the edge feature map and the gradient feature map to obtain a feature map with edge information and semantic information; then performs feature map connection and fusion on the image and the regional feature map after the feature map addition and fusion to obtain a fused feature map with more obvious blocking and edge feature information.

[0014] As some optional implementations of the present application, the feature map fusion layer performs feature map addition and fusion on the edge feature map and the gradient feature map, and performs feature map connection and fusion on the image and the regional feature map after the feature map addition and fusion to obtain a fused feature map.

[0015] In the above embodiment, the feature map addition fusion of the present invention is to add two feature maps with the same size element by element at corresponding positions. This operation occurs after the feature extraction stage of the deep learning model, and aims to enhance the representation ability of the model by fusing feature information from different sources or different levels; feature map connection fusion is to directly connect feature maps from different sources or different levels in a certain dimension (usually the channel dimension) to form a wider feature map, thereby enhancing the expression ability of the model.

[0016] As some optional implementations of the present application, before the feature map fusion layer performs feature map addition and fusion on the edge feature map and the gradient feature map, it is necessary to perform bilinear interpolation processing on the gradient feature map.

[0017] In the above embodiment, the present invention uses a bilinear interpolation method to adjust the image size of the edge feature map and the gradient feature map, so that the edge feature map and the gradient feature map have the same size, ensuring that the subsequent feature map addition and fusion can proceed smoothly.

[0018] As some optional implementations of the present application, the feature map fusion layer needs to perform bilinear difference processing on the regional feature maps before performing feature map connection fusion on the image after the feature maps are added and fused and on the regional feature maps.

[0019] In the above embodiment, the present invention uses a bilinear interpolation method to adjust the image size of the image and the regional feature map after the feature map addition and fusion, so that the image and the regional feature map after the feature map addition and fusion have the same size, ensuring that the subsequent feature map connection and fusion can proceed smoothly.

[0020] In a second aspect, the present invention provides a method for detecting abnormal display defects of OLED, the method using the neural network, the method comprising the following steps:

[0021] Acquire an OLED display image to be detected, and preprocess the OLED display image to be detected;

[0022] The pre-processed OLED display image to be detected is input into the neural network for defect type prediction to obtain the OLED abnormal display defect type prediction result.

[0023] In the above embodiment, the present invention uses an improved neural network to predict the defect type of the OLED display image to be detected, which can effectively detect abnormal display defects and is less likely to cause problems of missed detection and over-detection.

[0024] As some optional implementations of the present application, the preprocessing includes filtering out interference fringes and enhancing image sharpness on the OLED display image.

[0025] In the above embodiment, the present invention pre-processes the OLED display image and can highlight the image edge by filtering out interference fringes, thereby effectively reducing the impact of horizontal and vertical lines inherent in image capture; and can enhance the high-frequency information in the image by image sharpening enhancement, thereby achieving a sharpening effect.

[0026] As some optional implementations of the present application, before the pre-processed OLED display image to be inspected is input into the neural network for defect type prediction, it is necessary to perform model training on the neural network.

[0027] In the above embodiment, the present invention enables the neural network to predict the defect type of the OLED display image through model training.

[0028] As some optional implementation methods of the present application, the process of model training for the neural network is as follows:

[0029] Acquire historical OLED display images, and pre-process the historical OLED display images;

[0030] The historical OLED display images after preprocessing are input into the neural network for binary cross entropy loss calculation to achieve model training.

[0031] In the above embodiment, the present invention adopts a binary cross entropy loss calculation method to perform model training, that is, the difference between the probability distribution predicted by the neural network and the actual training label can be measured by binary cross entropy loss calculation.

[0032] In a third aspect, the present invention provides a system for detecting abnormal display defects of OLED, wherein the functions of the system correspond one-to-one to the method, the system uses the neural network, and the system comprises:

[0033] A display image acquisition unit, the display image acquisition unit is used to acquire the OLED display image to be detected and pre-process the OLED display image to be detected;

[0034] A neural network prediction unit is used to input the pre-processed OLED display image to be detected into a neural network for defect type prediction to obtain an OLED abnormal display defect type prediction result.

[0035] In a fourth aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for detecting abnormal display defects of OLED.

[0036] In a fifth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for detecting abnormal display defects of OLED is implemented.

[0037] The beneficial effects of the present invention are as follows: The present invention adopts a method of combining deep learning with the Canny edge detection operator to detect abnormal display defects. Considering that shallower feature maps can represent stronger edge information, deeper feature maps can represent stronger semantic relationships, and the Canny edge detection operator can effectively obtain the edge information of the image and strengthen the block feature representation of the image, therefore, combining deep learning with the Canny edge detection operator can effectively detect abnormal display defects, and is not prone to problems of missed detection and over-detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 is a schematic diagram of the computer device structure of the hardware operating environment described in the embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of the structure of a neural network for detecting abnormal display defects of OLED according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] In order to solve the problem of missed detection and over-detection of abnormal display defects of OLED, the present application provides a neural network, method and system for abnormal display defect detection of OLED. Before introducing the specific technical solution of the present application, the hardware operating environment involved in the embodiment of the present application is first introduced.

[0043] See also Figure 1 , Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0044] like Figure 1 As shown, the computer device may include: a processor, such as a central processing unit (CPU), a communication bus, a user interface, a network interface, and a memory. Among them, the communication bus is used to realize the connection and communication between these components. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), and optionally, the user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity interface). The memory may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory may optionally be a storage device independent of the aforementioned processor.

[0045] Those skilled in the art will understand that Figure 1The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0046] like Figure 1 As shown, the memory as a storage medium may include an operating system module, a network communication module, a user interface module and a data storage module.

[0047] exist Figure 1 In the computer device shown, the network interface is mainly used for data communication with the network server; the user interface is mainly used for data interaction with the user; the processor and memory in the computer device of the present application can be set in the computer device, and the computer device calls the system for OLED abnormal display defect detection stored in the data storage module in the memory through the processor, and executes the method for OLED abnormal display defect detection provided in the embodiment of the present application.

[0048] Based on the hardware environment of the aforementioned embodiment, the embodiment of the present application first provides a neural network for detecting abnormal display defects of OLED, see Figure 2 , Figure 2 The schematic diagram is a structural diagram of the neural network for detecting abnormal display defects of the OLED, wherein the neural network includes a backbone network, a Canny edge detection operator, a densely connected network, an attention network, a feature map fusion layer, and a fully connected layer.

[0049] The backbone network extracts defect features from the OLED display image to obtain a shallow feature map and a deep feature map.

[0050] In an embodiment of the present invention, the backbone network may be a VGG16 backbone network, which is a classic convolutional neural network architecture in deep learning technology. The VGG16 backbone network is composed of multiple convolutional layers and maximum pooling layers stacked alternately. Among them, the convolutional layer includes five convolutional blocks, and each convolutional block contains 2 to 3 convolutional layers. The backbone network has a powerful feature extraction capability and takes a short time. Specifically, the shallow feature map is a feature map formed after the third convolutional block, and the deep feature map is a feature map after the final maximum pooling layer.

[0051] The Canny edge detection operator performs edge contour detection on the shallow feature map to obtain an edge feature map; wherein the Canny edge detection operator can be a Canny edge detection operator, the core idea of ​​which is to find the position with the largest gradient change in the shallow feature map, so as to determine the position of the edge.

[0052] The densely connected network reuses defect features of deep feature maps to obtain gradient feature maps. The densely connected network is an innovative convolutional neural network structure, and its core feature is that each layer directly exchanges features with all previous and subsequent layers, forming a tightly connected network that can maximize the transmission efficiency of information flow and gradient flow in the network, and improve parameter efficiency through feature reuse. Compared with other structures, the densely connected network has higher parameter efficiency, is easier to train, and has less data requirements.

[0053] The attention network constructs dependencies on the deep feature map to obtain a regional feature map. The core feature of the attention network is that it can more effectively locate different block regions, thereby achieving effective region division.

[0054] The fully connected layer predicts the defect type of the edge feature map, the gradient feature map, and the fused feature map of the regional feature map to obtain the prediction result of the abnormal display defect type of the OLED.

[0055] Specifically, the feature map fusion layer performs feature map addition and fusion on the edge feature map and the gradient feature map to obtain a feature map with edge information and semantic information, and performs feature map connection and fusion on the image and the regional feature map after the feature map addition and fusion to obtain a fused feature map with obvious blocking and edge feature information.

[0056] In an embodiment of the present invention, before the feature map fusion layer performs feature map addition and fusion on the edge feature map and the gradient feature map, it is necessary to perform two bilinear interpolation processes on the gradient feature map, and use the bilinear interpolation method to adjust the image size of the edge feature map and the gradient feature map so that the edge feature map and the gradient feature map have the same size, ensuring that the subsequent feature map addition and fusion can proceed smoothly.

[0057] In an embodiment of the present invention, before the feature map fusion layer performs feature map connection fusion on the image after feature map addition and fusion and the regional feature map, it is necessary to perform two bilinear interpolation processes on the regional feature map, and use the bilinear interpolation method to adjust the image size of the image after feature map addition and fusion and the regional feature map, so that the sizes of the image after feature map addition and fusion and the regional feature map are the same, ensuring that the subsequent feature map connection fusion can proceed smoothly.

[0058] Specifically, the fully connected layer uses a normalized exponential function to predict the scores of different defect types for the fused feature map, and outputs the score prediction results.

[0059] In the above embodiment, the embodiment of the present invention improves the structure of the neural network in view of the characteristics of the organic laser display having a large abnormal display defect area and various defect feature types. Considering that the characteristics of the abnormal display defect itself have a strong block distribution, the Canny edge detection operator and the deep learning model are combined to detect the defect. The Canny edge detection operator can effectively detect the edge of the image, and the deep learning model can learn the characteristics of different types of morphology. Combining the two can effectively represent the characteristics of larger abnormal display defects.

[0060] Based on the above neural network, an embodiment of the present application provides a method for detecting abnormal display defects of OLED, the method comprising the following steps:

[0061] (1) Acquire an OLED display image to be detected, and preprocess the OLED display image to be detected.

[0062] In an embodiment of the present invention, preprocessing the OLED display image includes filtering interference fringes and enhancing image sharpness. Interference fringes filtering can highlight the edge of the image, effectively reducing the impact of the horizontal and vertical lines inherent in image capture; and image sharpening enhancement can highlight the characteristics of gradient variation, wherein interference fringes filtering can use fast Fourier transform to achieve frequency domain filtering, and remove or suppress signal components within a specific frequency range to achieve frequency domain processing of the signal, thereby eliminating weak vertical or horizontal lines in the picture. Secondly, image sharpening enhancement mainly refers to using a high-pass filter to enhance the high-frequency information in the image, thereby achieving a sharpening effect.

[0063] (2) The pre-processed OLED display image to be tested is input into a neural network for defect type prediction to obtain a prediction result of the OLED abnormal display defect type.

[0064] In an embodiment of the present invention, before the pre-processed OLED display image to be detected is input into a neural network for defect type prediction, it is necessary to perform model training on the neural network. Through model training, the neural network can predict the defect type of the OLED display image.

[0065] Specifically, the process of training the neural network model is as follows:

[0066] ① Acquire historical OLED display images and preprocess the historical OLED display images.

[0067] ② Input the preprocessed historical OLED display images into the neural network for binary cross entropy loss calculation to achieve model training.

[0068] In the above embodiment, the model training is performed by calculating the binary cross entropy loss, that is, the difference between the probability distribution predicted by the neural network and the actual training label can be measured by calculating the binary cross entropy loss.

[0069] In summary, the present invention uses an improved neural network to predict the defect type of the OLED display image to be inspected, which can effectively detect abnormal display defects and is less likely to cause problems of missed inspection and over-inspection.

[0070] In addition, in one embodiment, based on the same inventive concept as the above embodiment, the embodiment of the present invention provides a system for detecting abnormal display defects of OLED, the system corresponds to the method one by one, and the system includes:

[0071] A display image acquisition unit, the display image acquisition unit is used to acquire the OLED display image to be detected and pre-process the OLED display image to be detected;

[0072] A neural network prediction unit is used to input the pre-processed OLED display image to be detected into a neural network for defect type prediction to obtain an OLED abnormal display defect type prediction result.

[0073] It should be noted that each unit in the system for detecting abnormal display defects of OLED in this embodiment corresponds one-to-one to each step in the method for detecting abnormal display defects of OLED in the aforementioned embodiment. Therefore, the specific implementation manner and technical effects achieved by this embodiment can refer to the implementation manner of the aforementioned method for detecting abnormal display defects of OLED, and will not be repeated here.

[0074] In addition, in one embodiment, the present application further provides a computer device, which includes a processor, a memory, and a computer program stored in the memory, and the computer program implements the method in the aforementioned embodiment when executed by the processor.

[0075] In addition, in one embodiment, the present application further provides a computer storage medium, on which a computer program is stored, and the computer program implements the method in the aforementioned embodiment when executed by a processor.

[0076] In some embodiments, the computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories. The computer may be various computing devices including intelligent terminals and servers.

[0077] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0078] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0079] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0080] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0081] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0083] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A neural network for detecting abnormal display defects of OLED, characterized in that: The neural network includes a backbone network, a Canny edge detection operator, a densely connected network, an attention network, and a fully connected layer; The backbone network extracts defect features from the OLED display image to obtain a shallow feature map and a deep feature map; The Canny edge detection operator performs edge contour detection on the shallow feature map to obtain an edge feature map; The densely connected network reuses defect features on the deep feature map to obtain a gradient feature map; The attention network constructs dependency relationships on the deep feature map to obtain a regional feature map; The fully connected layer predicts the defect type of the edge feature map, the gradient feature map, and the fused feature map of the regional feature map to obtain the prediction result of the abnormal display defect type of the OLED.

2. The neural network for detecting abnormal display defects of OLED according to claim 1, characterized in that: The neural network also includes a feature map fusion layer, which performs feature map addition fusion and feature map connection fusion on the edge feature map, the gradient feature map and the regional feature map to obtain a fused feature map.

3. The neural network for detecting abnormal display defects of OLED according to claim 2, characterized in that: The feature map fusion layer performs feature map addition fusion on the edge feature map and the gradient feature map, and performs feature map connection fusion on the image and the regional feature map after the feature map addition fusion, so as to obtain a fused feature map.

4. The neural network for detecting abnormal display defects of OLED according to claim 3, characterized in that: Before the feature map fusion layer performs feature map addition and fusion on the edge feature map and the gradient feature map, it is necessary to perform bilinear interpolation processing on the gradient feature map.

5. The neural network for detecting abnormal display defects of OLED according to claim 3, characterized in that: The feature map fusion layer needs to perform bilinear interpolation processing on the regional feature maps before performing feature map connection fusion on the image after the feature maps are added and fused and on the regional feature maps.

6. A method for detecting abnormal display defects of OLED, characterized in that: The method uses the neural network described in claims 1 to 5, and the method comprises the following steps: Acquire an OLED display image to be detected, and preprocess the OLED display image to be detected; The pre-processed OLED display image to be detected is input into the neural network for defect type prediction to obtain the OLED abnormal display defect type prediction result.

7. The method for detecting abnormal display defects of OLED according to claim 6, characterized in that: The preprocessing includes filtering out interference fringes and enhancing image sharpness on the OLED display image.

8. The method for detecting abnormal display defects of OLED according to claim 7, characterized in that: Before the pre-processed OLED display image to be inspected is input into the neural network for defect type prediction, the neural network needs to be trained.

9. The method for detecting abnormal display defects of OLED according to claim 8, characterized in that: The process of training a neural network model is as follows: Acquire historical OLED display images, and pre-process the historical OLED display images; The historical OLED display images after preprocessing are input into the neural network for binary cross entropy loss calculation to achieve model training.

10. A system for detecting abnormal display defects of OLED, characterized in that: The system uses the neural network described in claims 1 to 5, and the system comprises: A display image acquisition unit, the display image acquisition unit is used to acquire the OLED display image to be detected and pre-process the OLED display image to be detected; A neural network prediction unit is used to input the pre-processed OLED display image to be detected into a neural network for defect type prediction to obtain an OLED abnormal display defect type prediction result.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for detecting abnormal display defects of OLED as described in any one of claims 6 to 9 is implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for detecting abnormal display defects of OLED according to any one of claims 6 to 9 is implemented.

Images