Display panel flaw detection method and device, electronic equipment and storage medium

Abstract

The application provides a display panel defect detection method, device, electronic equipment and storage medium. The display panel defect detection method comprises: performing feature extraction on a target panel display image corresponding to a display panel to be detected to obtain a plurality of first feature maps corresponding to the target panel display image; performing convolution processing on a target feature map in the plurality of first feature maps to obtain a second feature map; performing feature fusion processing on the other feature maps to obtain a plurality of third feature maps; the plurality of third feature maps correspond to different scales respectively; performing target detection on the target panel display image according to the second feature map and the plurality of third feature maps to determine a target defect position corresponding to a display panel defect in the target panel display image and a target defect category corresponding to the display panel defect. The technology can improve the detection efficiency and accuracy of display panel defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a display panel defect detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] In order to ensure the quality of the display panel, in the production process of the display panel, the display panel defect (Mura) detection is often needed to determine whether there is a display panel defect in the display panel.

[0003] In the related art, the detection method used in the display panel defect detection is generally to detect the display panel defect by the human eye when the display panel displays in the dark environment. This way of detecting the display panel defect not only has low detection efficiency and low detection accuracy, but also can cause harm to the human eye. Therefore, how to detect the display panel defect has become a technical problem to be solved.

[0004] In addition, the way of detecting the display panel defect in the related art can also have the above technical problems. SUMMARY

[0005] To solve one or more of the above technical problems, the present application provides a display panel defect detection method and device, electronic equipment and storage medium.

[0006] According to a first aspect of the present application, a display panel defect detection method is provided, comprising:

[0007] performing feature extraction on a target panel display image corresponding to a display panel to be detected to obtain a plurality of first feature maps corresponding to the target panel display image; the plurality of first feature maps correspond to different scales respectively;

[0008] performing convolution processing on a target feature map in the plurality of first feature maps to obtain a second feature map; the scale of the target feature map is greater than the scales of other feature maps in the plurality of first feature maps;

[0009] performing feature fusion processing on the other feature maps to obtain a plurality of third feature maps; the plurality of third feature maps correspond to different scales respectively;

[0010] performing target detection on the target panel display image according to the second feature map and the plurality of third feature maps to determine a target defect position corresponding to the display panel defect in the target panel display image and a target defect category corresponding to the display panel defect.

[0011] According to a second aspect of the present application, another display panel defect detection method is provided, comprising:

[0012] The target panel display image corresponding to the display panel to be detected is input to the trained detection model, a backbone network in the trained detection model is used to perform feature extraction on the target panel display image, and a plurality of first feature maps corresponding to the target panel display image are obtained; the plurality of first feature maps correspond to different scales respectively;

[0013] A neck network in the trained detection model is used to perform convolution processing on a target feature map in the plurality of first feature maps, a second feature map is obtained, and feature fusion processing is performed on other feature maps in the plurality of first feature maps, and a plurality of third feature maps are obtained; the plurality of third feature maps correspond to different scales respectively; the scale of the target feature map is greater than the scale of the other feature maps;

[0014] The second feature map and the plurality of third feature maps are input to a head network in the trained detection model, target defect positions corresponding to display panel defects in the target panel display image output by the head network and target defect categories corresponding to the display panel defects are obtained; the head network is used to determine the target defect positions and the target defect categories according to the second feature map and the plurality of third feature maps.

[0015] According to a third aspect of the present application, a display panel defect detection device is provided, comprising:

[0016] A first feature map acquisition module is configured to perform feature extraction on a target panel display image corresponding to a display panel to be detected, and obtain a plurality of first feature maps corresponding to the target panel display image; the plurality of first feature maps correspond to different scales respectively;

[0017] A second feature map acquisition module is configured to perform convolution processing on a target feature map in the plurality of first feature maps, and obtain a second feature map; the scale of the target feature map is greater than the scale of other feature maps in the plurality of first feature maps;

[0018] A third feature map acquisition module is configured to perform feature fusion processing on the other feature maps, and obtain a plurality of third feature maps; the plurality of third feature maps correspond to different scales respectively;

[0019] A target detection module is configured to perform target detection on the target panel display image according to the second feature map and the plurality of third feature maps, and determine target defect positions corresponding to display panel defects in the target panel display image and target defect categories corresponding to the display panel defects.

[0020] According to a fourth aspect of the present application, another display panel defect detection device is provided, comprising:

[0021] The feature map acquisition module is configured to input a target panel display image corresponding to a display panel to be detected into the trained detection model, perform feature extraction on the target panel display image by using a backbone network in the trained detection model, and obtain a plurality of first feature maps corresponding to the target panel display image; the plurality of first feature maps correspond to different scales respectively.

[0022] The feature map processing module is configured to perform convolution processing on a target feature map in the plurality of first feature maps based on a neck network in the trained detection model, obtain a second feature map, and perform feature fusion processing on other feature maps in the plurality of first feature maps, to obtain a plurality of third feature maps; the plurality of third feature maps correspond to different scales respectively; the scale of the target feature map is greater than the scale of the other feature maps.

[0023] The target detection module is configured to input the second feature map and the plurality of third feature maps into a head network in the trained detection model, and obtain a target defect position corresponding to a display panel defect in the target panel display image and a target defect category corresponding to the display panel defect output by the head network; the head network is configured to determine the target defect position and the target defect category according to the second feature map and the plurality of third feature maps.

[0024] According to a fifth aspect of the present application, an electronic device is provided, comprising:

[0025] at least one processor; and

[0026] a memory connected with the at least one processor in communication; wherein

[0027] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method in any embodiment of the present application.

[0028] According to a sixth aspect of the present application, a non-transitory computer readable storage medium storing computer instructions is provided, the computer instructions being used to enable a computer to perform the method in any embodiment of the present application.

[0029] The technology of the present application first performs feature extraction on a target panel display image corresponding to a display panel to be detected, to obtain a plurality of first feature maps, then obtains a second feature map according to a target feature map in the plurality of first feature maps, and a plurality of third feature maps according to other feature maps in the plurality of first feature maps, and after obtaining the second feature map and the plurality of third feature maps, performs target detection on the target panel display image according to the second feature map and the plurality of third feature maps, to determine a target defect position corresponding to a display panel defect in the target panel display image and a target defect category corresponding to the display panel defect.

[0030] By performing target detection on the target panel display image based on the second feature map and multiple third feature maps, the location and category of the target defect corresponding to the display panel defect in the target panel display image can be determined. Therefore, display panel defect detection can be completed without human intervention. This not only improves the detection efficiency and accuracy of display panel defect detection but also avoids harm to the human eye.

[0031] Furthermore, since the target feature map used to obtain the second feature map is larger in scale than the other feature maps, it has a relatively stronger ability to perceive details in the target panel display image. It can better represent the shallow, fine-grained pixel structure information of the target panel display image, thus enabling better detection of small-scale display panel defects. Meanwhile, since the multiple third feature maps are obtained through feature fusion processing of other feature maps, although their ability to perceive details in the target panel display image is relatively weaker, they can better represent the deep, abstract, and specialized semantic information of the target panel display image. This allows for better detection of larger-scale display panel defects. In other words, by using the second feature map and multiple third feature maps to perform target detection on the target panel display image, it is possible to detect display panel defects of different scales, thereby improving the detection performance of display panel defects.

[0032] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0033] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0034] Figure 1 A flowchart of a method for detecting defects in a display panel provided in an embodiment of this application is shown;

[0035] Figure 2 A schematic diagram of the structure of a detection model provided in an embodiment of this application is shown;

[0036] Figure 3 A schematic diagram of a target defect location provided in an embodiment of this application is shown;

[0037] Figure 4 A flowchart illustrating another method for detecting display panel defects provided in an embodiment of this application is shown;

[0038] Figure 5A schematic diagram illustrating a display panel defect detection process provided in an embodiment of this application is shown.

[0039] Figure 6 A schematic diagram of a display panel defect detection device provided in an embodiment of this application is shown;

[0040] Figure 7 A schematic diagram of another display panel defect detection device provided in an embodiment of this application is shown; and

[0041] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0042] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0043] This application provides a method for detecting defects in display panels; please refer to the following for details. Figure 1 This is a flowchart illustrating a method for detecting defects in a display panel provided in an embodiment of this application. The method may include the following steps:

[0044] Step S101: Perform feature extraction on the target panel display image corresponding to the display panel to be detected to obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales.

[0045] Step S102: Perform convolution processing on the target feature map in multiple first feature maps to obtain a second feature map; the scale of the target feature map is larger than the scale of other feature maps in multiple first feature maps.

[0046] Step S103: Perform feature fusion processing on other feature maps to obtain multiple third feature maps; the multiple third feature maps correspond to different scales.

[0047] Step S104: Based on the second feature map and multiple third feature maps, perform target detection on the target panel display image to determine the target defect location and target defect category corresponding to the display panel defect in the target panel display image.

[0048] The first method for detecting display panel defects provided in this application embodiment first extracts features from the target panel display image corresponding to the display panel to be detected, obtaining multiple first feature maps. Then, based on the target feature map in the multiple first feature maps, a second feature map is obtained, and based on other feature maps in the multiple first feature maps, multiple third feature maps are obtained. After obtaining the second feature map and multiple third feature maps, target detection is performed on the target panel display image based on the second feature map and multiple third feature maps, thereby determining the target defect location and target defect category corresponding to the display panel defect in the target panel display image.

[0049] By performing target detection on the target panel display image based on the second feature map and multiple third feature maps, the location and category of the target defect corresponding to the display panel defect in the target panel display image can be determined. Therefore, display panel defect detection can be completed without human intervention. This not only improves the detection efficiency and accuracy of display panel defect detection but also avoids harm to the human eye.

[0050] Furthermore, since the target feature map used to obtain the second feature map is larger in scale than the other feature maps, it has a relatively stronger ability to perceive details in the target panel display image. It can better represent the shallow, fine-grained pixel structure information of the target panel display image, thus enabling better detection of small-scale display panel defects. Meanwhile, since the multiple third feature maps are obtained through feature fusion processing of other feature maps, although their ability to perceive details in the target panel display image is relatively weaker, they can better represent the deep, abstract, and specialized semantic information of the target panel display image. This allows for better detection of larger-scale display panel defects. In other words, by using the second feature map and multiple third feature maps to perform target detection on the target panel display image, it is possible to detect display panel defects of different scales, thereby improving the detection performance of display panel defects.

[0051] The display panel to be inspected refers to the display panel for which defect detection is required. The display panel to be inspected is generally an OLED (Organic Light-Emitting Diode) display panel, but can also be an LCD (Liquid Crystal Display) display panel. In this embodiment, the type of display panel to be inspected is not specifically limited. The following description uses an OLED display panel as an example to illustrate the display panel defect detection method provided in this embodiment.

[0052] The target panel display image refers to the panel display image captured when the display panel under test is in use. Since panel defects are more noticeable when the display panel under test is displayed at low grayscale and low brightness, the target panel display image is, in most cases, a panel display image captured when the display panel under test is displayed at low grayscale and low brightness. In one example, the target panel display image is a panel display image captured when the display panel under test is displayed at a set grayscale. The set grayscale generally refers to a low grayscale level between 0 and 255 grayscale levels; for example, grayscale levels less than 64 grayscale levels can be considered low grayscale levels. Specifically, the set grayscale level can be 0 grayscale.

[0053] In practical applications, panel display images acquired using image acquisition devices often suffer from problems such as inappropriate image scale and excessive redundant information. Therefore, after acquiring the initial panel display image (i.e., the panel display image acquired using an image acquisition device) of the display panel to be detected for grayscale display settings, further image preprocessing can be performed on this initial image to obtain the target panel display image. Image preprocessing of the initial panel display image can eliminate irrelevant information and restore useful, true information, thereby enhancing the detectability of the target panel display image and simplifying its information content. Simultaneously, image preprocessing can also give the target panel display image a specific image scale. Image preprocessing includes, but is not limited to, image cropping, rotation, scaling, and translation.

[0054] The display panel under inspection may have only one defect or multiple defects. Each defect can be categorized as a line mura, circle mura, rectangle mura, or sandy mura. In one example, the defect categories include only line mura, circle mura, rectangle mura, and sandy mura.

[0055] In this embodiment, when extracting features from the target panel display image corresponding to the display panel to be detected to obtain multiple first feature maps, feature extraction can be performed on the target panel display image first to obtain the target feature map among the multiple first feature maps. Then, downsampling processing is performed on the target feature map to obtain other feature maps. The number of channels corresponding to the target feature map and other feature maps are also different; that is, when performing downsampling processing on the target feature map, the number of channels of the feature map needs to be adjusted simultaneously.

[0056] The target panel typically displays multi-channel images, such as RGB (Red, Green, Blue) three-channel images. Additionally, multiple primary features... Figure 1 Generally, this refers to four first feature maps. That is, in this embodiment, when extracting features from the target panel display image to obtain multiple first feature maps, four first feature maps of different scales are generally obtained. It should be noted that this embodiment does not specifically limit the number of channels of the target panel display image or the number of first feature maps. The number of channels and the number of first feature maps can be pre-configured based on detection requirements and prior values. Furthermore, this embodiment does not specifically limit the scale of the target panel display image (scale generally refers to width and height), but it needs to ensure that the width and height of the target panel display image are both multiples of 2. The following example uses a 416*416 RGB three-channel image as the target panel display image and four first feature maps to illustrate the display panel defect detection method provided in this embodiment.

[0057] When extracting features from the target panel display image to obtain four first feature maps, we can first extract features from the target panel display image to obtain a feature map with a scale of 104*104 and 128 channels as the first first feature map (i.e., the target feature map). Then, we downsample the target feature map to obtain a feature map with a scale of 52*52 and 256 channels as the second first feature map. Next, we downsample the second first feature map again to obtain a feature map with a scale of 26*26 and 512 channels as the third first feature map. Finally, we downsample the third first feature map to obtain a feature map with a scale of 13*13 and 1024 channels as the fourth first feature map.

[0058] In one example, such as Figure 2 As shown, the DarkNet53 network structure can be used to extract features from the target panel display image to obtain the four first feature maps mentioned above. Figure 2In the diagram, (32*3*3) in the 2D convolutional layer indicates that the 2D convolutional layer includes 32 convolutional kernels of size 3*3; (416, 416, 32) in the 2D convolutional layer indicates that the scale of the output feature map is 416*416 and the number of channels is 32; 1*64 in the residual layer indicates that the residual layer with 64 channels is executed once; 2*128 in the residual layer indicates that the residual layer with 128 channels is executed twice; and 8*256 in the residual layer indicates that the residual layer with 128 channels is executed twice. A residual layer with 256 channels is executed 8 times. The 8*512 in the residual layer represents a residual layer with 512 channels executed 8 times, and the 4*1024 in the residual layer represents a residual layer with 1024 channels executed 4 times. The (N, N, M) in the residual layer represents the scale of the output feature map as N*N and the number of channels as M. For example, the (104, 104, 128) in the residual layer represents the scale of the output feature map as 104*104 and the number of channels as 128.

[0059] In this embodiment, convolving a target feature map among multiple first feature maps to obtain a second feature map means performing multiple convolutions on the target feature map using a 1*1 convolution kernel to obtain a second feature map with the same scale and number of channels as the target feature map. For example, when the target feature map has a scale of 104*104 and 128 channels, the second feature map obtained by convolving the target feature map will also have a scale of 104*104 and 128 channels.

[0060] In one example, such as Figure 2 As shown, a convolutional layer with 5 layers and 128 channels can be used (in... Figure 2 The convolutional layer 1) performs convolution processing on the target feature map with a scale of 104*104 and a channel number of 128 to obtain a second feature map with a scale of 104*104 and a channel number of 128.

[0061] In this embodiment of the application, feature fusion processing is performed on other feature maps to obtain multiple third features. Figure 1 Generally, this means first performing convolution and upsampling on the small-scale feature maps in other feature maps to obtain a feature map to be fused that has the same scale as the large-scale feature maps in other feature maps. Then, the feature map to be fused is fused with the large-scale feature map to obtain multiple third feature maps.

[0062] In one example, there are four first feature maps, and the other feature maps in the multiple first feature maps include a second first feature map with a scale of 52*52 and 256 channels, a third first feature map with a scale of 26*26 and 512 channels, and a fourth first feature map with a scale of 13*13 and 1024 channels. In this case, the multiple third feature maps obtained may include a first third feature map with a scale of 26*26 and 256 channels, and a second third feature map with a scale of 52*52 and 128 channels.

[0063] Specifically, the process of obtaining the first third feature map with a scale of 26*26 and 256 channels can be as follows:

[0064] First, the fourth feature map with a scale of 13*13 and 1024 channels is subjected to convolution and upsampling to obtain a feature map to be fused with a scale of 26*26 and 256 channels. For example... Figure 2 As shown, a convolutional layer with 5 layers and 1024 channels can be used first (in... Figure 2 The convolutional layer 2) performs convolution processing on the fourth first feature map to obtain a feature map with a scale of 13*13 and 1024 channels. Then, the first upsampling layer (in...) is used to perform convolution processing on the fourth first feature map. Figure 2 The upsampling layer 1) performs upsampling on the feature map with a scale of 13*13 and a number of channels of 1024 to obtain a feature map to be fused with a scale of 26*26 and a number of channels of 256.

[0065] Second, the feature map to be fused, with a scale of 26*26 and 256 channels, is fused with the third first feature map to obtain a feature map with a scale of 26*26 and 768 channels. For example... Figure 2 As shown, the first feature fusion layer (in) can be used Figure 2 The first feature map, denoted as the feature fusion layer 1), is fused with the third first feature map to obtain a feature map with a scale of 26*26 and a channel number of 256.

[0066] Third, the feature map with a scale of 26*26 and 768 channels is convolved to obtain a feature map with a scale of 26*26 and 256 channels, which is used as the first third feature map.

[0067] like Figure 2 As shown, a convolutional layer with 5 layers and 256 channels can be used (in... Figure 2 The convolutional layer 3) performs convolution processing on the feature map with a scale of 26*26 and a channel number of 768 to obtain the first third feature map with a scale of 26*26 and a channel number of 256.

[0068] Furthermore, the process of obtaining the second third feature map with a scale of 52*52 and 128 channels can be as follows:

[0069] First, the first third feature map with a scale of 26*26 and 256 channels is upsampled to obtain a feature map with a scale of 52*52 and 128 channels. For example... Figure 2 As shown, a second upsampling layer can be used (in Figure 2 The first third feature map with a scale of 26*26 and a number of channels of 256 is upsampled to obtain a feature map with a scale of 52*52 and a number of channels of 128.

[0070] Second, the feature map with a scale of 52*52 and 128 channels is fused with the second first feature map to obtain a feature map with a scale of 52*52 and 384 channels. For example... Figure 2 As shown, the second feature fusion layer (in) can be utilized Figure 2 The feature fusion layer 2) fuses the feature map with a scale of 52*52 and a channel number of 128 with the second first feature map to obtain a feature map with a scale of 52*52 and a channel number of 384.

[0071] Third, convolution is performed on the feature map with a scale of 52*52 and 384 channels to obtain a second third feature map with a scale of 52*52 and 128 channels. For example... Figure 2 As shown, a convolutional layer with 5 layers and 128 channels can be used (in... Figure 2 The convolutional layer 4) performs convolution processing on the feature map with a scale of 52*52 and a channel number of 384 to obtain the second and third feature maps with a scale of 52*52 and a channel number of 128.

[0072] In this embodiment, when performing target detection on the target panel display image based on the second feature map and multiple third feature maps to determine the target defect location and target defect category corresponding to the display panel defect in the target panel display image, the second feature map and multiple third feature maps can be convolved first to obtain the fourth feature map corresponding to the second feature map and the fourth feature maps corresponding to the multiple third feature maps respectively. Then, the target defect location and target defect category are determined based on the fourth feature map corresponding to the second feature map and the fourth feature maps corresponding to the multiple third feature maps respectively. The feature values ​​in the fourth feature map are used to represent the center point position corresponding to the multiple set anchor boxes, the scale corresponding to the multiple set anchor boxes, the confidence level corresponding to the multiple set anchor boxes, and the defect category probability value corresponding to each set anchor box in the multiple set anchor boxes.

[0073] Multiple set anchor frames are obtained by clustering multiple panel display images, based on a clustering algorithm, by pre-labeling the locations of display panel defects. In one example, there are three set anchor frames, with dimensions of 7*7, 11*11, and 24*24 respectively.

[0074] In one example, a possible implementation for obtaining the fourth feature map corresponding to the second feature map is as follows: First, obtain the pre-configured target number of channels. Then, perform convolution processing on the second feature map using a 3*3 convolution kernel and a 1*1 convolution kernel to obtain the fourth feature map corresponding to the second feature map. Here, the number of channels in the fourth feature map corresponding to the second feature map is the target number of channels. It should be noted that the process for obtaining the fourth feature map corresponding to multiple third feature maps is the same as the process for obtaining the fourth feature map corresponding to the second feature map.

[0075] The target number of channels is calculated based on the number of multiple anchor frames, the defect categories of the display panel defects, the confidence level of each anchor frame, the scale of each anchor frame, and the center position of each anchor frame. For example, when the multiple anchor frames include three anchor frames with scales of 7*7, 11*11, and 24*24 respectively, and the defect categories of the display panel defects include linear defects, circular defects, rectangular defects, and sand-shaped defects, the target number of channels is 27. Here, 27 = 3*(4+1+4), where 3 represents the number of multiple anchor frames, the first 4 represents the center point (x, y) and scale (width and height) of each anchor frame, 1 represents the confidence level of each anchor frame, and the second 4 represents the defect category probability value of each anchor frame corresponding to the four defect categories.

[0076] like Figure 3As shown, after obtaining the second feature map with a scale of 104*104 and 128 channels, convolutional layer 5 is used to convolve the second feature map using a 3*3 kernel and a 1*1 kernel to obtain a feature map with a scale of 104*104 and 27 channels, which serves as the fourth feature map corresponding to the second feature map. After obtaining the first third feature map with a scale of 26*26 and 256 channels, convolutional layer 6 is used to convolve the first third feature map using a 3*3 kernel and a 1*1 kernel to obtain a feature map with a scale of 26*26 and 27 channels, which serves as the fourth feature map corresponding to the first third feature map. After obtaining the second and third feature maps with a scale of 52*52 and 128 channels, convolutional layer 7 can be used to perform convolution processing on the second and third feature maps using convolutional kernels with a kernel size of 3*3 and 1*1 to obtain a feature map with a scale of 52*52 and 27 channels, which serves as the fourth feature map corresponding to the second and third feature maps.

[0077] In this embodiment of the application, the target defect location generally refers to the location box corresponding to the display panel defect on the target panel display image. For example... Figure 2 As shown, this is a schematic diagram of a target defect location provided in an embodiment of this application. The target defect location can also refer to the coordinates of the position of the display panel defect on the target panel display image.

[0078] In this embodiment, to further improve the detection efficiency and accuracy of display panel defect detection, a pre-trained detection model can be obtained, and then the target panel display image can be input into the pre-trained detection model. That is, the pre-trained detection model determines the location and category of the target defect corresponding to the display panel defect. The pre-trained detection model includes a backbone network, a neck network, and a head network. The backbone network extracts features from the target panel display image to obtain multiple first feature maps. The neck network performs convolution processing on the target feature maps to obtain second feature maps and performs feature fusion processing on other feature maps to obtain multiple third feature maps. The head network performs target detection on the target panel display image based on the second feature maps and multiple third feature maps to determine the location and category of the target defect.

[0079] In one example, the trained detection model is an improvement upon the YOLO (You Only Look Once) v3 model. The YOLOv3 backbone network structure is not modified; only its output is optimized. That is, the backbone network in the trained detection model is based on an improvement upon the YOLOv3 backbone network; in this case, the backbone network in the trained detection model is a DarkNet53 network structure. When the number of first feature maps is four, the DarkNet53 network structure can output the first first feature map in the second residual layer, the first first feature map in the third residual layer, the first third feature map in the fourth residual layer, and the first first feature map in the fifth residual layer. For example... Figure 2 As shown, when the target panel displays a 416*416 RGB three-channel image, the DarkNet53 network structure outputs a first feature map (i.e., the target feature map) with a scale of 104*104 and 128 channels in the second residual layer, a second first feature map with a scale of 52*52 and 256 channels in the third residual layer, a third first feature map with a scale of 26*26 and 512 channels in the fourth residual layer, and a fourth first feature map with a scale of 13*13 and 1024 channels in the fifth residual layer.

[0080] The neck network of YOLOv3 needs improvement; that is, the neck network in the trained detection model is based on an improved version of the YOLOv3 neck network. For example... Figure 2 As shown, the neck network in the trained detection model is used to obtain a second feature map with a scale of 104*104 and 128 channels, a first third feature map with a scale of 26*26 and 256 channels, and a second third feature map with a scale of 52*52 and 128 channels for the four first feature maps mentioned above.

[0081] The YOLOv3 head network needs improvement; that is, the head network in the trained detection model is based on an improved version of the YOLOv3 head network. For example... Figure 2 As shown, the neck network in the trained detection model is used to obtain a feature map with a scale of 104*104 and 27 channels as the fourth feature map corresponding to the second feature map, a feature map with a scale of 26*26 and 27 channels as the fourth feature map corresponding to the first third feature map, and a feature map with a scale of 52*52 and 27 channels as the fourth feature map corresponding to the second third feature map. Based on the fourth feature map corresponding to the second feature map and the fourth feature maps corresponding to the multiple third feature maps, the location and category of the target defect are determined.

[0082] It should be noted that, Figure 4 In the convolutional layers, residual layers, feature fusion layers, and upsampling layers, (N, N, M) are used to represent that the scale of the feature map output by the network layer is N*N and the number of channels is M. For example, (104, 104, 128) in the convolutional layer is used to represent that the scale of the output feature map is 104*104 and the number of channels is 128.

[0083] In one possible implementation, the trained detection model is obtained through the following steps:

[0084] First, a set of sample target panel display images is obtained, along with annotations for the location and type of panel defects in each sample target panel display image within the set. Then, based on the set of sample target panel display images and the corresponding annotations for the panel defect locations and types, the detection model to be trained is trained to obtain the trained detection model.

[0085] Corresponding to the first method for detecting display panel defects provided in the embodiments of this application, the embodiments of this application also provide another method for detecting display panel defects. Figure 5 A flowchart illustrating another method for detecting display panel defects provided in an embodiment of this application is shown, which may include steps S401-S403.

[0086] Step S401: Input the target panel display image corresponding to the display panel to be detected into the trained detection model, and use the backbone network in the trained detection model to extract features from the target panel display image to obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales.

[0087] Step S402: Based on the neck network in the trained detection model, perform convolution processing on the target feature map in multiple first feature maps to obtain a second feature map, and perform feature fusion processing on other feature maps in multiple first feature maps to obtain multiple third feature maps; the multiple third feature maps correspond to different scales; the scale of the target feature map is larger than the scale of other feature maps.

[0088] Step S403: Input the second feature map and multiple third feature maps into the head network of the trained detection model to obtain the target defect location and target defect category corresponding to the display panel defect in the target panel display image output by the head network; the head network is used to determine the target defect location and target defect category based on the second feature map and multiple third feature maps.

[0089] The second method for detecting display panel defects provided in this application embodiment first extracts features from the target panel display image corresponding to the display panel to be detected, obtaining multiple first feature maps. Then, based on the target feature map in the multiple first feature maps, a second feature map is obtained, and other feature maps in the multiple first feature maps are used to obtain multiple third feature maps. After obtaining the second feature map and multiple third feature maps, target detection is performed on the target panel display image based on the second feature map and multiple third feature maps, thereby determining the target defect location and target defect category corresponding to the display panel defect in the target panel display image.

[0090] By performing target detection on the target panel display image based on the second feature map and multiple third feature maps, the location and category of the target defect corresponding to the display panel defect in the target panel display image can be determined. Therefore, display panel defect detection can be completed without human intervention. This not only improves the detection efficiency and accuracy of display panel defect detection but also avoids harm to the human eye.

[0091] Furthermore, since a trained detection model is used to determine the location and type of the target defect, the speed and efficiency of determining the location and type of the target defect can be further improved.

[0092] In practical applications, the trained detection model can be deployed on terminal devices or servers, and it can exist in the form of functional modules, applications, and software.

[0093] like Figure 6 As shown, the process of determining the location and category of the target defect using the trained detection model is as follows:

[0094] The first step is model training data acquisition, which involves obtaining a set of sample target panel display images, along with annotations indicating the location and category of panel defects for each image. The second step is model training, which involves training the detection model based on the set of sample target panel display images and the corresponding annotations of panel defect locations and categories, resulting in a trained detection model. The third step is acquiring target panel display images. The fourth step is inputting the target panel display images into the trained detection model to obtain the target defect locations and categories output by the trained detection model.

[0095] Corresponding to the first method for detecting display panel defects provided in the embodiments of this application, the embodiments of this application also provide a device for detecting display panel defects. This device, as follows... Figure 6 As shown, Figure 7This illustration shows a structural block diagram of a display panel defect detection device provided in an embodiment of this application. The device may include:

[0096] The first feature map acquisition module 601 is used to extract features from the target panel display image corresponding to the display panel to be detected, and obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales respectively.

[0097] The second feature map acquisition module 602 is used to perform convolution processing on the target feature map in multiple first feature maps to obtain the second feature map; the scale of the target feature map is larger than the scale of other feature maps in multiple first feature maps;

[0098] The third feature map acquisition module 603 is used to perform feature fusion processing on other feature maps to obtain multiple third feature maps; the multiple third feature maps correspond to different scales.

[0099] The target detection module 604 is used to perform target detection on the target panel display image based on the second feature map and multiple third feature maps, and to determine the target defect location and target defect category corresponding to the display panel defect in the target panel display image.

[0100] In one possible implementation, the target detection module 604 includes:

[0101] The fourth feature map acquisition submodule is used to perform convolution processing on the second feature map and multiple third feature maps to obtain the fourth feature map corresponding to the second feature map and the fourth feature maps corresponding to the multiple third feature maps respectively. The feature values ​​in the fourth feature map are used to represent the center point position, scale, confidence level, and defect category probability value of each set anchor box in the multiple set anchor boxes respectively.

[0102] The target detection submodule is used to determine the location and category of the target defect based on the fourth feature map corresponding to the second feature map and the fourth feature maps corresponding to multiple third feature maps.

[0103] In one possible implementation, the fourth feature map acquisition submodule includes:

[0104] The target channel number acquisition submodule is used to acquire the pre-configured target channel number;

[0105] The convolution processing submodule is used to perform convolution processing on the second feature map using a 3*3 convolution kernel and a 1*1 convolution kernel to obtain the fourth feature map corresponding to the second feature map; the number of channels of the fourth feature map corresponding to the second feature map is the target number of channels.

[0106] In one possible implementation, the device further includes:

[0107] The image acquisition module is used to acquire the initial panel display image of the display panel to be detected in order to set the grayscale display.

[0108] The image preprocessing module is used to preprocess the initial panel display image to obtain the target panel display image.

[0109] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0110] Corresponding to the second method for detecting display panel defects provided in the embodiments of this application, the embodiments of this application also provide another device for detecting display panel defects. This device is as follows: Figure 7 As shown, Figure 8 This invention illustrates a structural block diagram of another display panel defect detection device provided in an embodiment of this application. The device may include:

[0111] The feature map acquisition module 701 is used to input the target panel display image corresponding to the display panel to be detected into the trained detection model, and use the backbone network in the trained detection model to extract features from the target panel display image to obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales respectively;

[0112] The feature map processing module 702 is used to perform convolution processing on the target feature map in multiple first feature maps based on the neck network in the trained detection model to obtain the second feature map, and to perform feature fusion processing on other feature maps in multiple first feature maps to obtain multiple third feature maps; the multiple third feature maps correspond to different scales; the scale of the target feature map is larger than the scale of the other feature maps;

[0113] The target detection module 703 is used to input the second feature map and multiple third feature maps into the head network of the trained detection model to obtain the target defect location and target defect category corresponding to the display panel defect in the target panel display image output by the head network; the head network is used to determine the target defect location and target defect category based on the second feature map and multiple third feature maps.

[0114] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0115] The functions of each unit, module, or sub-module in the various devices of this disclosure embodiment can be found in the corresponding descriptions in the above method embodiments, and they have corresponding beneficial effects, which will not be repeated here.

[0116] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] Figure 8 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 8 As shown, the electronic device includes a memory 801 and a processor 802. The memory 801 stores a computer program that can run on the processor 802. When the processor 802 executes the computer program, it implements the method described in the above embodiments. The number of memories 801 and processors 802 can be one or more.

[0118] The electronic device also includes:

[0119] The communication interface 803 is used to communicate with external devices and exchange and transmit data.

[0120] If the memory 801, processor 802, and communication interface 803 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, ​ The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0121] Optionally, in a specific implementation, if the memory 801, processor 802, and communication interface 803 are integrated on a single chip, then the memory 801, processor 802, and communication interface 803 can communicate with each other through an internal interface.

[0122] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0123] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this application.

[0124] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0125] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0126] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0127] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0128] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0129] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0130] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0131] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

Claims

1. A method for detecting defects in a display panel, characterized in that, include: Acquire the initial panel display image of the display panel to be tested for setting grayscale display; The initial panel display image is preprocessed to obtain the target panel display image; Feature extraction is performed on the target panel display image corresponding to the display panel to be detected to obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales. A second feature map is obtained by convolution processing the target feature map in the plurality of first feature maps; the scale of the target feature map is larger than the scale of the other feature maps in the plurality of first feature maps. The other feature maps are subjected to feature fusion processing to obtain multiple third feature maps; the multiple third feature maps correspond to different scales. Based on the second feature map and the plurality of third feature maps, target detection is performed on the target panel display image to determine the target defect location and the target defect category corresponding to the display panel defect in the target panel display image.

2. The method according to claim 1, characterized in that, The step of performing target detection on the target panel display image based on the second feature map and the plurality of third feature maps, and determining the target defect location and target defect category corresponding to the display panel defect in the target panel display image, includes: The second feature map and the plurality of third feature maps are convolved to obtain the fourth feature map corresponding to the second feature map and the fourth feature map corresponding to the plurality of third feature maps respectively; the feature values ​​in the fourth feature map are used to represent the center point position corresponding to the plurality of set anchor boxes respectively, the scale corresponding to the plurality of set anchor boxes respectively, the confidence level corresponding to the plurality of set anchor boxes respectively, and the defect category probability value corresponding to each set anchor box in the plurality of set anchor boxes. Based on the fourth feature map corresponding to the second feature map and the fourth feature maps corresponding to the plurality of third feature maps respectively, the location of the target defect and the category of the target defect are determined.

3. The method according to claim 2, characterized in that, The fourth feature map corresponding to the second feature map is obtained through the following steps: Get the pre-configured target number of channels; The second feature map is convolved using a 3*3 kernel and a 1*1 kernel to obtain a fourth feature map corresponding to the second feature map; the number of channels in the fourth feature map corresponding to the second feature map is the target number of channels.

4. The method according to claim 1, characterized in that, Before performing feature extraction on the target panel display image corresponding to the display panel to be detected to obtain multiple first feature maps corresponding to the target panel display image, the method further includes: The target panel display image is input into a trained detection model; the trained detection model includes a backbone network, a neck network, and a head network; the backbone network is used to extract features from the target panel display image to obtain the plurality of first feature maps; the neck network is used to perform convolution processing on the target feature maps to obtain second feature maps, and to perform feature fusion processing on the other feature maps to obtain the plurality of third feature maps; the head network is used to perform target detection on the target panel display image based on the second feature maps and the plurality of third feature maps, and to determine the location and category of the target defect.

5. The method according to claim 4, characterized in that, The trained detection model was obtained through the following steps: Obtain a set of sample target panel display images, and label the panel defect locations and defect categories for each sample target panel display image in the set; Based on the set of sample target panel display images and the corresponding labeled panel defect locations and defect categories, the detection model to be trained is trained to obtain the trained detection model.

6. A method for detecting defects in a display panel, characterized in that, include: Acquire the initial panel display image of the display panel to be tested for setting grayscale display; The initial panel display image is preprocessed to obtain the target panel display image; The target panel display image corresponding to the display panel to be detected is input into the trained detection model. The backbone network in the trained detection model is used to extract features from the target panel display image to obtain multiple first feature maps corresponding to the target panel display image. The multiple first feature maps correspond to different scales. Based on the neck network in the trained detection model, the target feature map in the plurality of first feature maps is convolved to obtain a second feature map, and the other feature maps in the plurality of first feature maps are fused to obtain a plurality of third feature maps; the plurality of third feature maps correspond to different scales respectively; The scale of the target feature map is larger than the scale of the other feature maps; The second feature map and the plurality of third feature maps are input into the head network of the trained detection model to obtain the target defect location and the target defect category corresponding to the display panel defect in the target panel display image output by the head network. The head network is used to determine the location and category of the target defect based on the second feature map and the plurality of third feature maps.

7. A device for detecting defects in a display panel, characterized in that, include: The image acquisition module is used to acquire the initial panel display image of the display panel to be detected in order to set the grayscale display. The image preprocessing module is used to preprocess the initial panel display image to obtain the target panel display image; The first feature map acquisition module is used to extract features from the target panel display image corresponding to the display panel to be detected, and obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales respectively. The second feature map acquisition module is used to perform convolution processing on the target feature map in the plurality of first feature maps to obtain a second feature map; the scale of the target feature map is larger than the scale of the other feature maps in the plurality of first feature maps; The third feature map acquisition module is used to perform feature fusion processing on the other feature maps to obtain multiple third feature maps; the multiple third feature maps correspond to different scales respectively; The target detection module is used to perform target detection on the target panel display image based on the second feature map and the plurality of third feature maps, and to determine the target defect location and the target defect category corresponding to the display panel defect in the target panel display image.

8. A device for detecting defects in a display panel, characterized in that, include: The image acquisition module is used to acquire the initial panel display image of the display panel to be detected in order to set the grayscale display. The image preprocessing module is used to preprocess the initial panel display image to obtain the target panel display image; The feature map acquisition module is used to input the target panel display image corresponding to the display panel to be detected into the trained detection model, and use the backbone network in the trained detection model to extract features from the target panel display image to obtain multiple first feature maps corresponding to the target panel display image; the multiple first feature maps correspond to different scales respectively; The feature map processing module is used to perform convolution processing on the target feature maps in the plurality of first feature maps based on the neck network in the trained detection model to obtain second feature maps, and to perform feature fusion processing on other feature maps in the plurality of first feature maps to obtain a plurality of third feature maps; the plurality of third feature maps correspond to different scales respectively; The scale of the target feature map is larger than the scale of the other feature maps; The target detection module is used to input the second feature map and the plurality of third feature maps into the head network of the trained detection model to obtain the target defect location and the target defect category corresponding to the display panel defect in the target panel display image output by the head network. The head network is used to determine the location and category of the target defect based on the second feature map and the plurality of third feature maps.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; characterized in that the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method described in any one of claims 1 to 6.

Images