Panel defect detection method, storage medium and terminal device

Abstract

This application discloses a panel defect detection method, storage medium, and terminal device. The method includes: determining a plurality of candidate panel images based on defect location information of a defective region in a panel image to be detected and the panel image to be detected; and determining a defect category corresponding to the panel image based on the plurality of candidate panel images. After obtaining the defect location information of the defective region, the application selects a plurality of candidate panel images from the panel image based on the defect location information, and determines the defect category of the panel image based on the selected plurality of candidate panel images. This method can increase the feature information of the acquired panel image, thereby improving the accuracy of the defect classification.

Description

Technical Field

[0001] The present application relates to the field of panel processing technology, and in particular to a panel defect detection method, storage medium and terminal equipment. Background Art

[0002] Product quality is one of the most important production indicators in the manufacturing industry. To ensure product quality, defect detection during the production process has become an indispensable process. For example, display panels (such as thin-film transistor-liquid crystal displays (TFT-LCDs)) require defect detection on every production line. However, defect detection of display panels is currently generally performed manually. However, due to visual fatigue caused by repetitive work, inspectors may make misjudgments, which in turn affects the quality of the display panels. Summary of the Invention

[0003] The technical problem to be solved by this application is to provide a panel defect detection method, storage medium and terminal device in response to the shortcomings of the existing technology.

[0004] In order to solve the above technical problems, a first aspect of an embodiment of the present application provides a panel defect detection method, the method comprising:

[0005] determining a plurality of candidate panel images according to defect position information of a defect area in the panel image to be inspected and the panel image to be inspected;

[0006] Determine defect categories corresponding to the panel images based on the plurality of candidate panel images.

[0007] The panel defect detection method, wherein, among the plurality of candidate panel images, at least two candidate panel images have different image sizes.

[0008] The panel defect detection method, wherein determining the defect category corresponding to the panel image based on the plurality of candidate panel images specifically includes:

[0009] determining a target panel image based on the plurality of candidate panel images;

[0010] The target panel image is input into a trained defect classification model; and the defect category corresponding to the panel image is determined by the defect classification model.

[0011] The panel defect detection method, wherein determining the target panel image based on the plurality of candidate panel images specifically includes:

[0012] Adjusting the plurality of candidate panel images respectively to obtain a plurality of reference panel images; wherein the image size of each reference panel image in the plurality of reference panel images is the same;

[0013] Several reference panel images are spliced ​​together to obtain a target panel image, wherein the number of channels of the target panel image is equal to the sum of the number of channels of the several reference panel images.

[0014] The panel defect detection method, wherein the defect classification model includes a channel attention module and a classification module, wherein the inputting of the target panel image into the trained defect classification model; and determining the defect category corresponding to the panel image through the defect classification model specifically includes:

[0015] Inputting the target image into a channel attention module, and determining a target feature vector through the channel attention module;

[0016] The target feature vector is input into a classification module, and the defect category corresponding to the panel image is determined by the classification module.

[0017] The panel defect detection method, wherein the channel attention module includes a convolution unit, an attention unit, and a fusion unit; inputting the target image into the channel attention module, and determining the channel target feature vector through the channel attention module specifically includes:

[0018] Inputting the target image into a convolution unit, and determining a first eigenvector by the convolution unit;

[0019] Inputting the first eigenvector into the attention unit, and determining a second eigenvector through the attention unit, wherein the second eigenvector is used to reflect weight information of each channel in the target image;

[0020] The first feature vector and the second feature vector are input into the fusion unit, and the target feature vector is determined by the fusion unit.

[0021] The panel defect detection method, wherein the defect classification model is obtained by training based on a preset training sample set, wherein the process of determining the preset training sample set specifically includes:

[0022] Acquire several panel images;

[0023] For each panel image among the plurality of panel images, determining defect location information of a defective area in the panel image;

[0024] selecting a plurality of reference panel images from the panel image according to the defect position information, wherein the plurality of reference panel images each include a portion or all of the defect area;

[0025] selecting a designated panel image from a plurality of reference panel images, wherein the designated panel image is a reference panel image with the smallest image size among the plurality of reference panel images;

[0026] Rotating the designated panel image according to a preset angle, and determining a reference panel image group based on the designated panel image obtained by rotation and reference panel images other than the designated panel image among the plurality of reference panel images;

[0027] All generated reference panel image groups are used as the preset training sample sets.

[0028] In the panel defect detection method, determining the reference panel image group based on the designated panel image obtained by rotation and the reference panel images other than the designated panel image among the plurality of reference panel images specifically includes:

[0029] performing enhancement processing on the rotated designated panel image according to a first preprocessing method to obtain a first panel image;

[0030] performing enhancement processing on each reference panel image except the designated panel image in the plurality of reference panel images according to a second preset processing method to obtain each second panel image, wherein the first preprocessing method is different from the second preprocessing method;

[0031] A reference panel image group is determined based on the first panel and all second panel images.

[0032] In the panel defect detection method, the image sizes of the reference panel images among the plurality of reference panel images are different from each other, and the designated panel image is the reference panel image with the smallest image size among the plurality of reference panel images.

[0033] The panel defect detection method, wherein the acquiring defect position information of the defect area in the panel image to be detected specifically includes:

[0034] The panel image to be inspected is input into a trained defect recognition model, and the defect position information of the defect area in the panel image is determined by the defect recognition model.

[0035] The panel defect detection method, wherein the defect location information includes the area size of the defect area and positioning information of the defect area.

[0036] A second aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the panel defect detection methods described above.

[0037] A third aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor;

[0038] The communication bus realizes the connection and communication between the processor and the memory;

[0039] When the processor executes the computer-readable program, the steps in any of the above panel defect detection methods are implemented.

[0040] Beneficial Effects: Compared to the prior art, the present application provides a panel defect detection method, storage medium, and terminal device. The method includes: determining a plurality of candidate panel images based on defect location information of a defective region in a panel image to be detected and the panel image to be detected; and determining the defect category corresponding to the panel image based on the plurality of candidate panel images. After obtaining the defect location information of the defective region, the present application selects a plurality of candidate panel images from the panel image based on the defect location information, and determines the defect category of the panel image based on the selected plurality of candidate panel images. This method can increase the feature information of the acquired panel image, thereby improving the accuracy of the defect classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0042] Figure 1 This is a flow chart of the panel defect detection method provided in this application.

[0043] Figure 2 This is an example flow chart of the panel defect detection method provided in this application.

[0044] Figure 3 This is a schematic diagram of the channel attention module in the panel defect detection method provided in this application.

[0045] Figure 4 This is a schematic diagram of the terminal device structure provided in this application. DETAILED DESCRIPTION

[0046] This application provides a panel defect detection method, storage medium, and terminal device. To clarify the purpose, technical solution, and effects of this application, the application is further described below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to explain this application and are not intended to limit this application.

[0047] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0048] After research, the inventors found that product quality is one of the most important production indicators in the manufacturing industry. In order to ensure product quality, defect detection of products during the production process has become an indispensable process. For example, taking display panels (such as thin film transistor liquid crystal displays (TFT-LCD, Thin Film Transistor-Liquid Crystal Display)), each production line needs to perform defect detection on the display panels. The currently commonly used defect detection process for display panels is generally as follows: first, the display panels in the panel production process are inspected by automatic optical inspection equipment, and the display panels with defects are preliminarily photographed, and then the images of the defective areas of the panels are handed over to technicians with professional knowledge for inspection. This method that relies on manual inspection faces problems such as high labor costs and low efficiency. At the same time, a large amount of repetitive labor can easily cause visual fatigue of personnel and lead to misjudgment.

[0049] To avoid the problems associated with manual inspection, technicians are continuously researching automatic classification methods. Currently, commonly used automatic classification methods mainly include traditional methods and deep learning methods. Traditional methods design a mask template and perform image difference on the defect image after filtering to obtain the location and morphology information of the defect image, and then classify the defects according to the set rules. However, when using traditional methods for automatic defect classification, technical personnel with professional knowledge are required to design the algorithm and design different recognition rules based on different defect shapes, making traditional methods difficult and inflexible. Furthermore, traditional methods have slow processing speeds and are only suitable for simple scenarios. Deep learning methods generally use data containing defect location and category information to train detection models (such as YOLO and SSD). When inspecting the panel image, they obtain the defect location and defect category of the defect area in the panel image. However, this method requires reducing the entire panel image to a specified size and inputting the reduced panel image into the network model. This results in the loss of detailed information about the defective panel image, resulting in poor detection results.

[0050] To address the above issues, in an embodiment of the present application, several candidate panel images are determined based on the defect location information of the defect area in the panel image to be inspected and the panel image to be inspected; and the defect category corresponding to the panel image is determined based on the several candidate panel images. After obtaining the defect location information of the defect area, the present application selects several candidate panel images from the panel image based on the defect location information, and determines the defect category of the panel image based on the selected candidate panel images. This can increase the feature information of the acquired panel image, thereby improving the accuracy of the defect classification.

[0051] The application content will be further explained below through description of embodiments in conjunction with the accompanying drawings.

[0052] Figure 1 This is a flow chart of a panel defect detection method provided by this embodiment. This method can be performed by a panel defect detection device, which can be implemented by software and applied to electronic devices such as PCs, tablet computers, servers, or personal digital assistants. Figure 1 and Figure 2 The panel defect detection method provided in this embodiment specifically includes:

[0053] S10 , determining a plurality of candidate panel images according to defect position information of a defect area in a panel image to be detected and the panel image.

[0054] Specifically, the panel image to be inspected may be a panel image acquired in real time or at preset intervals during the panel production process by an image acquisition device (such as a camera or a still camera) pre-installed on a production line for producing panels; or a panel image acquired from a local storage space of an electronic device that runs a method for generating a panel defect detection model; or an image acquisition request is sent to an image storage server, and a panel image is received that is returned by the server based on the image acquisition request; of course, the panel image may also be acquired by other means, and the specific acquisition method is not limited here. The panel image may include a TFT-LCD panel, an LCD panel, an integrated circuit panel, or a chip panel, and the panel to be inspected may include a circuit area and a non-circuit area.

[0055] The defect location information is used to reflect the regional location and regional size of the panel defect. After the defect location information is obtained, an image region can be determined in the panel image based on the defect location information. The image region can be a rectangular region or a square region, etc. Therefore, the defect location information can include the regional size of the defect region and the positioning information of the defect region, wherein the positioning information of the defect region is used to reflect the position of the defect region in the panel image, and the regional size of the defect region is used to reflect the regional size of the defect region. The positioning information of the defect region can be the coordinate information of the center point of the defect region, or the coordinate information of the upper left corner of the defect region, or the coordinate information of the upper right corner of the defect region, etc.; the regional size includes the regional width and regional height of the defect region. In a specific implementation of this embodiment, the positioning information is the coordinate information of the center point of the defect region, so that the image center of the candidate image region determined by the candidate coincides with the image center of the defect region, and the candidate region includes image information of the periphery of the defect region, thereby improving the correlation between the background information obtained based on the candidate panel image and the defect region.

[0056] For example, the defect location information includes the coordinate information of the center point of the defect area and the area width and area height of the defect area. When the defect location information is {(200,200),50,60}, it means that the coordinates of the center point of the defect area are (200,200), the area width is 50, and the area height is 60. Therefore, the defect location information determines that the image area is an image area centered on the pixel point with coordinate point (200,200) in the panel image, with a width of 50 and a height of 60.

[0057] Of course, in actual applications, the panel image may include one defect area or multiple defect areas. When the panel image includes one defect area, the defect location information is one; when the panel image includes multiple defect areas, the defect location information is multiple, and the multiple defect areas correspond to the multiple defect location information one-to-one, and the area shapes of the image areas determined by the defect location information in the panel image can be the same or different. For example, the image area determined by some defect location information is a rectangular area, and the image area determined by some defect location information is a square area, etc., or the image areas determined by all defect location information are all rectangular areas, or the image areas determined by all defect location information are all square areas, etc.

[0058] In one implementation of this embodiment, the process of acquiring the defect position information of the defect area in the panel image to be detected may specifically include:

[0059] The panel image to be inspected is input into a trained defect recognition model, and the defect position information of the defect area in the panel image is determined by the defect recognition model.

[0060] Specifically, the defect recognition model is a trained deep learning model, through which the defect location information of the defect area in the panel image can be determined. It can be understood that the input item of the defect recognition model is the panel image to be detected, the output item of the defect recognition model is the defect location information, and the number of defect location information output by the defect recognition model is the same as the number of defect areas identified by the recognition model, and the defect location information corresponds one-to-one to the defect areas carried by the panel image. For example, the defective panel image includes defect area A, defect area B and defect area C, then the defect location information output by the defect recognition model includes defect location information 1, defect location information 2 and defect location information 3, and defect location information 1 corresponds to defect area A, defect location information 2 corresponds to defect area B, and defect location information 3 corresponds to defect area C. In practical applications, the defect recognition model can adopt the yolo v3 model, etc.

[0061] Each of the plurality of candidate panel images includes the defect area, and at least two of the plurality of candidate panel images have different image sizes. It can be understood that each candidate panel image is a partial image area in the panel image, and each candidate panel image includes the same defect area. In other words, each candidate panel image is determined based on the same defect location information. For example, defect area A in the panel image corresponds to defect location information 1, and the plurality of candidate panel images include candidate panel image a, candidate panel image b, and candidate panel image c. Then, candidate panel image a, candidate panel image b, and candidate panel image c are all determined based on defect location information 1, and candidate panel image a, candidate panel image b, and candidate panel image c all include defect area A. In this embodiment, candidate panel images of different image sizes are selected. Since candidate panel images of different image sizes carry different image information, large-sized candidate panel images can provide global features of the panel image, and small-sized candidate panel images can provide detailed features of the defect area of ​​the panel image. In this way, when fusing multiple candidate panel images, the larger candidate panel images among the candidate panel images can provide global features of the panel image, while the smaller candidate panel images among the candidate panel images can provide detailed features of the defective areas of the panel image. This improves the image information included in the fused target image, thereby improving the accuracy of defect classification determined based on the target image. In one specific implementation of this embodiment, the image sizes of the candidate panel images are different.

[0062] Based on this, when determining several candidate panel images, the number of defect location information can be determined. When the number of defect location information is equal to 1, several candidate panel images are determined based on the defect location information and the panel image. When the number of defect location information is greater than 1, for each defect location information, several candidate panel images corresponding to the defect location information are determined based on the defect location information and the panel image. When the number of defect location information is 0, it indicates that the panel image does not contain a defective area, and the panel image is directly determined to be a defect-free panel, completing the panel defect detection. In addition, when the number of defect location information is greater than 1, the number of candidate panel images corresponding to each defect location information is the same.

[0063] In an implementation of this embodiment, the defect location information includes positioning information and area size, and determining a number of candidate panel images based on the defect location information and the panel image specifically includes:

[0064] reading the region size in the defect location information, and determining a plurality of target region sizes according to the region size, wherein each of the plurality of target region sizes is larger than the region size;

[0065] reading positioning information in the defect position information, and determining a plurality of image areas in the panel image according to the positioning information and a plurality of target area sizes;

[0066] The determined image regions are used as candidate panel images.

[0067] Specifically, each of the several target region sizes corresponds to a candidate panel image, and the target region size is the image size of the corresponding candidate panel image. Each of the several target region sizes is larger than the region size of the defect region, so that each determined candidate panel region can include a defect region. The target region sizes being larger than the region size of the defect region means that the width of the target region size is larger than the width of the region size, and the height of the target region size is larger than the height of the region size. For example, if the target region size is 50*50 and the region size is 30*40, then the target region size is larger than the region size.

[0068] In a specific implementation of the embodiment, the process of determining the sizes of several target regions may include: after obtaining the region size of the defect region, reading the region width and region height in the region size, selecting a target width from the region width and region height, wherein the target width is the larger of the region width and region height; after obtaining the target width, determining several reference widths based on the target width, wherein each of the several reference widths is larger than the target width and smaller than the width and height of the panel image; and using the target width as the width and height and each reference width as the width and height to obtain several target region sizes. For example, if the target width is 40 and the several reference widths include 50 and 60, then the target region sizes are divided into {40,40}, {50,50} and {60,60}

[0069] The determination process of the several reference widths may be randomly selecting several widths greater than the target width as the several reference widths. For example, if the target width is 40, 45 and 50 are randomly selected as the reference widths; or, the target width is enlarged according to several preset ratios (such as 10%, 20%, 30%, etc.) to obtain several reference widths. For example, if the target width is 40, the target width is enlarged by 10% and 20% respectively to obtain the reference widths of 44 and 48; or, the target width is enlarged according to a preset ratio (such as 10%, etc.) to obtain a reference width, and this reference width is used as the target width, and the step of enlarging the target width according to the preset ratio to obtain a reference width is re-executed until the number of re-executions is greater than a preset number threshold (such as 2 times, etc.) to obtain several reference widths. For example, the target width is 100 and the preset ratio is 10%, and the reference widths obtained are 110 and 121. Of course, in practical applications, the reference widths can also be selected in other ways, which will not be elaborated here one by one.

[0070] In an implementation manner of this embodiment, the determination process of several target region sizes may include: after obtaining the region size of the defect region, reading the region width and region height in the region size, and determining a target region size according to the region width and region height. Among them, the region width and region height of the target region size are equal, and the region width of the target region is the larger one of the region width and region height; determining the ratio of the image size of the panel image to the region size of the defect region. If the ratio is greater than or equal to a preset ratio threshold, then 1 / N times the image size of the panel image to M / N times the image size of the panel image are respectively used as the target region sizes. Here, N is determined according to the number of the several target region sizes, M is equal to the number of the target region sizes minus one and is a positive integer and M < N. For example, the number of the target region sizes is 3 and the preset ratio threshold is 3, then the target region sizes determined according to the panel image are one-third of the image size of the panel image and one-third of the image size of the panel image respectively. In addition, if the ratio is less than the preset ratio threshold, a preset number of target region sizes can be randomly selected, and the preset number is equal to the number of the several target region sizes minus one.

[0071] S30. Determine the defect category corresponding to the panel image according to the several candidate panel images.

[0072] Specifically, the defect category is used to reflect the defect shape of the panel defect, and the defect category may include triangle, lightning shape, dot shape, sand shape, etc. For example, for the panel image A, the shape of the panel defect on the panel image A is triangle, then the defect category is triangle defect. The several candidate panel images correspond to the same defect region, and the defect category is the defect category corresponding to this defect region.

[0073] In an implementation of this embodiment, determining the defect category corresponding to the panel image based on the plurality of candidate panel images specifically includes:

[0074] determining a target panel image based on the plurality of candidate panel images;

[0075] The target panel image is input into a trained defect classification model; and the defect category corresponding to the panel image is determined by the defect classification model.

[0076] Specifically, the defect classification model is a trained deep learning model, and the defect recognition model can be used to determine the defect category of the defect area in the target panel image. It is understood that the input of the defect classification model is the target panel image, and the output of the defect classification model is the defect category, so that the defect classification model can determine the specific category corresponding to the target panel image.

[0077] In one implementation of this embodiment, since a panel image may include multiple defect regions, the multiple candidate panel images may include multiple candidate panel image groups. Each candidate panel image group in the multiple candidate panel image groups includes multiple candidate panel images. The number of candidate panel image groups is the same as the number of defect regions carried by the panel image to be detected, and each candidate panel image group corresponds to one defect region. Each candidate panel image group determines a target panel image. It is understood that the multiple candidate panel images in each candidate panel image group are determined based on the defect location information corresponding to their corresponding defect regions, and each candidate panel image includes the defect region. For example, if a defect panel image includes defect region A, defect region B, and defect region C, where defect region A corresponds to defect location information 1, defect region B corresponds to defect location information 2, and defect region C corresponds to defect location information 3, then the multiple candidate panel image groups include candidate panel image group a, candidate panel image group b, and candidate panel image group c. Candidate panel image group a is determined based on defect location information 1, candidate panel image group b is determined based on defect location information 2, and candidate panel image group c is determined based on defect location information 3.

[0078] In an implementation of this embodiment, determining the target panel image based on the plurality of candidate panel images specifically includes:

[0079] Adjusting a plurality of candidate panel images respectively to obtain a plurality of reference panel images;

[0080] Several reference panel images are spliced ​​together to obtain a target panel image, wherein the number of channels of the target panel image is equal to the sum of the number of channels of the several reference panel images.

[0081] Specifically, the plurality of reference panel images correspond one-to-one to the plurality of candidate panel images, and each of the plurality of reference panel images is obtained by preprocessing the candidate panel image corresponding to the reference panel image, and the image size of each of the plurality of reference panel images is the same. For example, if the plurality of candidate panel images include candidate panel image A, candidate panel image B, and candidate panel image C, then the plurality of reference panel images include three reference panel images, respectively denoted as reference panel image a, reference panel image b, and reference panel image c, where reference panel image a is obtained by preprocessing candidate panel image A, reference panel image b is obtained by preprocessing candidate panel image B, and reference panel image c is obtained by preprocessing candidate panel image C. In a specific implementation of this embodiment, the preprocessing may include pixel value normalization and image size scaling, and the preprocessing process for each candidate panel image is the same.

[0082] In one implementation of this embodiment, the defect classification model includes a channel attention module and a classification module. Inputting the target panel image into the trained defect classification model; and determining the defect category corresponding to the panel image through the defect classification model specifically includes:

[0083] Inputting the target image into a channel attention module, and determining a target feature vector through the channel attention module;

[0084] The target feature vector is input into a classification module, and the defect category corresponding to the panel image is determined by the classification module.

[0085] Specifically, the classification module is used to determine the defect category corresponding to the panel image based on the target feature vector. The input of the classification model is the target feature vector, and the output is the defect category. The classification module can adopt an existing network model for detecting categories, such as a classification network model composed of VGG, ResNet, and DenseNet. The channel attention module is used to determine the target feature vector corresponding to the target image. The channel attention module is configured with a channel attention mechanism, and the target feature vector corresponding to the target image is determined by the channel attention mechanism. The target feature map is obtained by fusion of several candidate panel images according to the channel, and the image sizes corresponding to each candidate panel image are different. In this way, the channel attention mechanism can make different defect areas pay attention to the image information of different channels of the target image, so that the model can obtain image information related to the defect area, thereby improving the accuracy of defect category prediction. For example, for defects with tiny shapes, the image details in the small-sized candidate panel image can be relied upon, and position-related defects can rely on the global information of the large-sized candidate panel image.

[0086] In one implementation of this embodiment, Figure 3 As shown, the channel attention module includes a convolution unit, an attention unit, and a fusion unit; inputting the target image into the channel attention module, and determining the channel target feature vector through the channel attention module specifically includes:

[0087] Inputting the target image into a convolution unit, and determining a first eigenvector by the convolution unit;

[0088] Inputting the first eigenvector into the attention unit, and determining a second eigenvector through the attention unit, wherein the second eigenvector is used to reflect weight information of each channel in the target image;

[0089] The first feature vector and the second feature vector are input into the fusion unit, and the target feature vector is determined by the fusion unit.

[0090] Specifically, the attention unit includes a global average pooling layer, a first fully connected layer, a first activation function layer, a second fully connected layer, and a second activation function layer. The convolution unit includes a convolution layer, wherein the input of the convolution layer is the target image and the output is the first eigenvector. The input of the global average pooling layer is the first eigenvector, and the output is the input of the first fully connected layer. The output of the first fully connected layer is passed through the first activation function layer and then input into the second fully connected layer. The output of the second fully connected layer is passed through the second activation function and then outputs the second eigenvector. The fusion unit takes the first eigenvector and the second eigenvector as inputs and multiplies the first and second eigenvectors according to the channel dimension to obtain a target eigenvector. By multiplying the first and second eigenvectors, since the second eigenvector reflects the weight information of each channel in the target image, when each channel of the first eigenvector is weighted, the weight coefficients of each channel of each first eigenvector are different, thereby ensuring that each channel of each target image receives different attention. In addition, the dimension of the second eigenvector is equal to the channel dimension of the first eigenvector.

[0091] For example: the image scale of the target image is 90*299*299, the parameters of the convolution layer can be 3*3*128, and the target image outputs the first eigenvector of the data dimension of 128*299*299 through the convolution layer. The first eigenvector obtains a 128*1*1 tensor through local average pooling, and the 128*1*1 tensor obtains a 32-dimensional vector through the fully connected layer. The 32-dimensional vector is activated by Relu configured in the second activation function layer and then enters and exits the second fully connected layer, and the output dimension of the second fully connected layer is a 128-dimensional vector. The 128-dimensional vector is normalized by the sigmoid activation configured in the second activation function layer to obtain a 128-dimensional normalized vector. The fusion layer multiplies the 128-dimensional normalized vector and the first eigenvector according to the channel dimension to obtain the target eigenvector.

[0092] In one embodiment of the present invention, the defect classification model is obtained by training based on a preset training sample set, wherein the process of determining the preset training sample set specifically includes:

[0093] Acquire several panel images;

[0094] For each panel image among the plurality of panel images, determining defect location information of a defective area in the panel image;

[0095] selecting a plurality of reference panel images from the panel image according to the defect position information, wherein the plurality of reference panel images each include a portion or all of the defect area;

[0096] selecting a designated panel image from a plurality of reference panel images, wherein the designated panel image is a reference panel image with the smallest image size among the plurality of reference panel images;

[0097] Rotating the designated panel image according to a preset angle, and determining a reference panel image group based on the designated panel image obtained by rotation and reference panel images other than the designated panel image among the plurality of reference panel images;

[0098] All generated reference panel image groups are used as the preset training sample sets.

[0099] Specifically, the plurality of reference panel images corresponding to each panel image include at least two reference panel images including the same defect region, and the plurality of reference panel images include at least a first reference panel image and a second reference panel image, wherein the image size of the first reference panel image is different from the image size of the second reference panel image. It is understood that the plurality of reference panel images each include at least two reference panel images, and each of the plurality of reference panel images corresponds to the same defect region. It should be noted that the plurality of reference panel images each may include a portion or all of the defect region. In one implementation, each of the plurality of reference panel images may include the entire defect region, i.e., the entire defect region. Furthermore, the plurality of reference panel images include at least a first reference panel image and a second reference panel image, and the image size of the first reference panel image is different from the image size of the second reference panel image. For example, the plurality of reference panel images include reference panel image A and reference panel image B, and reference panel image A and reference panel image B correspond to the same image region, but reference panel image A and reference panel image B each correspond to a different image size. It is worth noting that the process for determining the number of reference panel images corresponding to each panel image is the same as the process for determining the number of reference panel images corresponding to each defect area in the above-described embodiment, and will not be further elaborated here. For details, please refer to the above description. Furthermore, the image sizes of the reference panel images in each reference panel image group are different.

[0100] In one implementation of this embodiment, the image sizes of each reference panel image in a plurality of reference panel images are different from each other, and the designated panel image can be the reference panel image with the smallest image size in a group of reference panel images. For example, the reference panel image group includes reference panel image A, reference panel image B, and reference panel image C, and the image sizes of reference panel image A, reference panel image B, and reference panel image C increase in sequence, then reference panel image A is the designated panel image.

[0101] The preset angle is pre-set and is used to limit the rotation angle of the specified panel image. The preset angle can be one or more of 0°, 90°, 180° and 270°, and when there are multiple preset angles, the specified panel image is rotated according to the multiple preset angles to obtain multiple enhanced reference panel image groups, which can improve the diversity of the training image set. At the same time, by rotating the reference panel image according to 0°, 90°, 180° or 270°, the model can avoid overfitting while being easier to learn features. This is because the printed circuit of the liquid crystal panel is usually composed of horizontal or vertical lines. Since the right-angle rotation can make the defect area in the reference panel image in the horizontal direction or vertical direction, it is easier to learn features and avoid overfitting of the model caused by defect areas at different angles.

[0102] In an implementation of this embodiment, determining the reference panel image group according to the designated panel image obtained by rotation and the reference panel images other than the designated panel image among the plurality of reference panel images specifically includes:

[0103] performing enhancement processing on the rotated designated panel image according to a first preprocessing method to obtain a first panel image;

[0104] performing enhancement processing on each reference panel image except the designated panel image in the plurality of reference panel images according to a second preset processing method to obtain each second panel image, wherein the first preprocessing method is different from the second preprocessing method;

[0105] A reference panel image group is determined based on the first panel and all second panel images.

[0106] Specifically, the first preset processing method is used to rotate the obtained designated panel image for enhancement processing to improve the diversity of training samples. The first pre-processing method includes various processing methods: random mirroring, random color jittering, equal-proportional random cropping, pixel value normalization, and image size scaling, wherein the equal-proportional random cropping can be an equal-proportional random cropping between 0.8 and 1.2. The second pre-processing method includes random color jittering, pixel value normalization, and image size scaling. The image size of the designated panel image after image size scaling in the first pre-processing method is the first image size, and the image size of each remaining candidate image after image size scaling in the second pre-processing method is the second image size, the first image size is the same as the second image size, and the remaining candidate images are reference panel images other than the designated panel image among the multiple reference panel images.

[0107] In summary, this embodiment provides a panel defect detection method, storage medium, and terminal device. The method includes: determining a plurality of candidate panel images based on defect location information of a defective region in a panel image to be detected and the panel image to be detected; and determining a defect category corresponding to the panel image based on the plurality of candidate panel images. After obtaining the defect location information of the defective region, the present application selects a plurality of candidate panel images from the panel image based on the defect location information, and determines the defect category of the panel image based on the selected plurality of candidate panel images. This increases the feature information of the acquired panel image, thereby improving the accuracy of the defect classification.

[0108] In one embodiment, since many different defects may exist on a display panel, and the locations of these defects in the display panel may be very close, this may result in defect frames detected during defect detection of the display panel containing intersecting or nested defect location information. These intersecting frames or nested defect location information may affect the defect detection results, and therefore it is necessary to filter out the intersecting or nested frames to reduce repeated or overlapping defect location information, thereby improving the effect of panel defect detection. Based on this, in one implementation of the embodiment, when the defect detection area is a plurality of defect detection areas; after inputting the panel image to be detected into the detection model and determining the detection panel image through the detection model, the method further includes:

[0109] Acquire confidence levels corresponding to each piece of defect location information included in the inspection panel image, and select target defect location information from each piece of defect location information based on the confidence levels;

[0110] The selected target defect position information is deleted from the detection panel image, and the deleted detection panel image is used as the detection panel image.

[0111] Specifically, the defect location information is used to reflect the regional location and size of the panel defect. After obtaining the defect location information, an image region can be determined in the panel image based on the defect location information, where the image region can be a rectangular region or a square region, etc. Some of the defect location information may intersect or nest. The defect region corresponding to each of the plurality of defect location information can be in a shape such as a rectangular frame or a square frame, and the shape of the defect region corresponding to each of the plurality of defect location information can be different. For example, some defect location information may be rectangular candidate regions, while others may be square candidate regions. Furthermore, each piece of defect location information includes a corresponding confidence level, which reflects the degree of confidence that the image region corresponding to the defect location information is defective. In one implementation of this embodiment, the confidence level ranges from 0 to 1, and a larger confidence level indicates a higher degree of confidence in the defect location information; conversely, a smaller confidence level indicates a lower degree of confidence in the defect location information. For example, the credibility of the defect location information when the confidence level is 1 is higher than the credibility of the defect location information when the confidence level is 0.1.

[0112] Furthermore, based on the confidence level corresponding to each defect position information, all defect position information can be filtered by using a non-maximum suppression algorithm to filter all defect position information detected, so as to determine the defect position information corresponding to the panel image. For example, all defect location information is taken as a defect location information set, and all defect location information with confidence less than or equal to a first threshold (such as 0.1, etc.) in the defect location information set is deleted to update the defect location information set; each defect location information in the updated defect location information set is arranged from large to small according to the confidence, the defect location information with the highest confidence is selected, and the intersection-and-union ratio of other defect location information in the defect location information set to the defect location information with the highest confidence is determined (the intersection-and-union ratio can be understood as the ratio of the intersection and union of two detection frames), the defect location information with an intersection-and-union ratio greater than a second threshold (such as 0.3, etc.) is filtered out from the updated defect location information set, and the filtered defect location information set is taken as the updated defect location information set, and the defect location information in the updated defect location information set is continued to be arranged from large to small according to the confidence, and the defect location information set after the filter does not contain defect location information; all the selected defect location information is taken as the panel defect location information corresponding to the panel.

[0113] In an implementation of this embodiment, selecting target defect location information from each defect location information based on the confidence level specifically includes:

[0114] Obtain the position parameters corresponding to each defect location information;

[0115] Based on the confidence and location parameters corresponding to each piece of defect location information, target defect location information is selected from each piece of defect location information.

[0116] Specifically, the position parameter is used to reflect the position of the defect area corresponding to the defect position information in the panel image and the size of the defect area. It can be understood that the position parameter is the area information of the image area corresponding to the defect area corresponding to the defect position information in the panel image. The position parameter includes position information and size information. The size information is used to reflect the size of the defect position information, and the position information is used to reflect the position of the defect position information in the panel image. For example, the size information is the area of ​​the defect position information, and the position information is the distance between the center of the defect position information and the image center of the panel image. Of course, in actual applications, the position information can also be determined in other ways, such as the distance from the center point of the defect position information to the upper left corner of the panel image, the distance from the upper left corner of the defect position information to the upper left corner of the panel image, etc. The size information can be determined in other ways, such as the circumference of the defect position information.

[0117] Furthermore, in one implementation of the present embodiment, the defect area corresponding to the defect location information is a rectangular candidate area, the size information is the area of ​​the defect location information, and the location information is the distance between the center of the defect location information and the image center of the panel image. The process of determining the location parameters of the defect location information can be as follows: first, obtain the vertex coordinates of the four vertices of the defect location information; second, determine the four side lengths and the center point coordinates of the defect location information based on the four vertex coordinates; finally, determine the size information of the defect location information based on the four side lengths, and determine the location information of the defect location information based on the center point coordinates. For example, the image center coordinates of the panel image are (0,0), and the four vertex coordinates of the defect location information A are (0,2), (2,2), (2,0) and (0,0), respectively. Then, the location information of the defect location information A is The size information of the defect position information A is 4.

[0118] The selection of target defect location information from each defect location information refers to screening each defect location information from each defect location information to select some defect location information that does not meet the filtering conditions from all defect location information, and use the selected defect location information as the target defect location information, wherein the filtering conditions are determined based on the confidence and position parameters corresponding to each defect location information. It can be understood that by using the two factors of confidence and position parameters as filtering factors for the defect location information, on the one hand, whether there is a panel area in the defect location information, and on the other hand, considering the regional information of the area where the defect location information is located in the panel image, when filtering intersecting or nested defect location information, while retaining defect location information with high confidence, it is also possible to retain candidate areas whose size and position meet the conditions by limiting the position parameters, thereby improving the accuracy of the retained defect location information, thereby improving the accuracy of panel defect detection.

[0119] The reason why this embodiment uses confidence and position parameters as filtering factors for defect location information is that the defects of the display panel are generally some discrete small defect particles, so that the defect location information is the defect location information corresponding to each small defect particle, and the defect location information corresponding to each small defect particle will have a very high confidence (for example, taking a confidence value of 0-1 as an example, a very high confidence can be understood as a confidence greater than a preset confidence, for example, it can be a confidence greater than 0.7, etc.). Then, when filtering the defect location information based on the confidence, the defect location information can be filtered to be the defect location information corresponding to each small defect particle, but a large detection frame containing the defects of these small defect particles cannot be obtained, and thus the defect category corresponding to the display panel cannot be determined based on the defect location information obtained by filtering, which leads to the problem of low defect detection accuracy.

[0120] In one implementation of this embodiment, selecting target defect location information from each piece of defect location information based on the confidence level and location parameter corresponding to each piece of defect location information specifically includes:

[0121] Determine the confidence score corresponding to each defect location information according to the confidence level and location parameters corresponding to each defect location information;

[0122] According to the confidence scores corresponding to the defect location information, target defect location information is selected from the defect location information.

[0123] Specifically, the confidence score is used to reflect the possibility of the defect location information being selected, and the larger the value of the confidence score, the higher the credibility of the defect location information; conversely, the smaller the value of the confidence score, the lower the credibility of the defect location information. For example, when the confidence score is 5, the possibility of the defect location information being selected is higher than the possibility of the defect location information when the confidence score is 1. The target defect location information set is a set of candidate areas composed of defect location information obtained by filtering based on the confidence score, wherein the target defect location information set is a subset of the defect location information set. When the defect location information is filtered out based on the confidence score, the target defect candidate set is a true subset of the defect candidate set; when the defect location information is not filtered out based on the confidence score, the target defect candidate set is the defect candidate set. For example, the defect location information set includes defect location information A, defect location information B, defect location information C, and defect location information D. Based on the confidence scores corresponding to each defect location information, defect location information A and defect location information D are filtered out, then the target defect location information set includes defect location information B and defect location information D.

[0124] Furthermore, in an implementation of this embodiment, the location parameters include location information and size information; and the confidence score corresponding to each defect location information is determined based on the confidence level and location parameters corresponding to each defect location information as follows:

[0125] The confidence level, position information, and size information corresponding to the defect position information are weighted to obtain a confidence score corresponding to the defect position information.

[0126] Specifically, the size information is used to reflect the size of the defect location information. For example, the size information is the area of ​​the defect location information, the length of the left side of the subsequent box of the size information defect, etc. The position information is used to indicate the location of the defect location information in the panel image. For example, the size information is the area of ​​the defect location information, the position information is the distance between the center of the defect location information and the image center of the panel image, the distance from the center point of the defect location information to the upper left corner of the panel image, the distance from the upper left corner of the defect location information to the upper left corner of the panel image, etc. The confidence level is the confidence level carried by the defect location information.

[0127] Furthermore, before weighted processing is performed on the confidence, position information, and size information, it is necessary to determine the weight coefficients corresponding to the confidence, position information, and size information. The weight coefficients corresponding to the confidence and the weight coefficients corresponding to the size information are both pre-set. For example, the weight coefficient of the confidence is 1, and the weight coefficient of the size information is 0.5. In addition, in practical applications, the weight coefficient corresponding to the size information can be set according to the actual detection situation, and the weight coefficient corresponding to the size information can be determined according to the importance of the size in the detection task. When the importance of the size is high, the weight coefficient corresponding to the size information is large; conversely, when the importance of the size is low, the weight coefficient corresponding to the size information is small. In this way, when the confidence of the defect location information with a larger size is low, the confidence score corresponding to the defect location information with a larger size will also increase, so that the probability of the defect location information with a larger size being selected increases. For example, in a detection task that focuses on the size of the defect location information, the value of the weight coefficient corresponding to the size information is greater than the value of the weight coefficient corresponding to the size information in a detection task that focuses on the defect location information. For example, in a detection task that focuses on the size of the defect location information, the value of the weight coefficient corresponding to the size information is 0.8; in a detection task that focuses on the defect location information, the value of the weight coefficient corresponding to the size information is 0.4, and so on.

[0128] Furthermore, in an implementation of this embodiment, the process of obtaining the weight parameter corresponding to the location information specifically includes:

[0129] For each edge in the defect location information, determining a distance between the edge and a target edge, wherein the target edge is an edge of the panel image corresponding to the edge;

[0130] A second weight coefficient corresponding to the position information is determined based on all determined distances.

[0131] Specifically, the distance refers to the distance between the region edge and the target edge, and the region edge is parallel to the target edge. Therefore, the process of determining the distance can be: select a starting point on the region edge, draw a perpendicular line from the starting point to the target edge, and use the distance between the starting point and the perpendicular point as the distance between the region edge and the target edge. For example, for edge a in the defect position information, there are borders a and borders b translated in the panel image, the distance between border a and edge a is A, the distance between border b and edge a is B, A>B, and border b is the target edge corresponding to edge a. Of course, it is worth noting that the defect position information can have the same shape as the edge frame of the panel image, for example, both are rectangular frames, etc.

[0132] Furthermore, the distance refers to the distance between two parallel edges. One of the two parallel edges is designated as the initial edge, and the other as the target edge. To determine the distance between the initial edge and the target edge, a point can be selected on the initial edge, a perpendicular line drawn from the point to the target edge, and the distance between the point and the perpendicular line is used as the distance between the initial edge and the target edge. For example, the defect location information is a rectangular box abcd, and the edge box of the panel image is a rectangular width ABCD. The target edge corresponding to the ab side of the rectangular box abcd is AB, and the distance between the ab side and the AB side is d_h1; the target edge corresponding to the bc side is BC, and the distance between the bc side and the BC side is d_w2; the target edge corresponding to the cd side is CD, and the distance between the cd side and the CD side is d_h2; the target edge corresponding to the da side is DA, and the distance between the da side and the DA side is d_w2.

[0133] Furthermore, in one implementation of this example, the defect location information is a rectangular frame; after obtaining the distance corresponding to each side in the defect location information, the four sides of the defect location information are divided into a first side group and a second side group; the first side group includes a first wide side and a first high side, the first wide side intersecting with the first high side; the second side group includes a second wide side and a second high side, the second wide side and the second high side intersecting. After obtaining the first side group and the second side group, the ratio of the sum of the first wide side and the first high side in the first side group to the sum of the target side corresponding to the first wide side and the target side corresponding to the first high side is calculated; and the ratio of the sum of the second wide side and the second high side in the second side group to the sum of the target side corresponding to the second wide side and the target side corresponding to the second high side is calculated. Finally, based on the two ratios obtained by the calculation, a weight coefficient corresponding to the location information is determined. For example, the smaller of the two ratios is used as the weight coefficient corresponding to the location information, or the larger of the two ratios is used as the weight coefficient corresponding to the location information.

[0134] In a specific implementation, the smaller of the two ratios is used as the weight coefficient corresponding to the position information. This can preserve the influence of the confidence score corresponding to the position information while preventing the position information from having too high a proportion in the confidence score, thereby affecting the proportion of the confidence and causing the wrong selection of the defect position information. Therefore, the calculation formula for the weight coefficient corresponding to the position information can be:

[0135]

[0136] Among them, W1 is the weight coefficient corresponding to the position information, d w1 is the distance from the first wide side to its corresponding target side; d h1 is the distance from the first high side to its corresponding target side; d w2 is the distance from the second widest side to its corresponding target side; d h2is the distance from the second wide side to its corresponding target side; W is the width of the edge frame of the panel image, and H is the height of the edge frame of the panel image.

[0137] Furthermore, after determining the confidence level, size information, location information, the weight coefficient corresponding to the confidence level, the weight coefficient corresponding to the size information, and the weight coefficient corresponding to the location information, the calculation formula for the confidence score may be:

[0138] ComBox=az+W0s+W1p

[0139] Among them, ComBox is the confidence score, z is the confidence level, a is the weight coefficient corresponding to the confidence level, W0 is the weight coefficient corresponding to the size information, s is the size information, W1 is the weight coefficient corresponding to the position information, and p is the position information.

[0140] Furthermore, in an implementation of this embodiment, selecting target defect location information from each piece of defect location information according to the confidence score corresponding to each piece of defect location information specifically includes:

[0141] Determine, based on the confidence scores corresponding to the defect candidate regions, a target candidate region and a reference candidate region set corresponding to each defect location information, wherein the reference candidate region set includes the remaining defect candidate regions in the defect candidate region set excluding the target candidate region, and the target candidate region is the defect location information with the highest confidence score among the defect location information;

[0142] For each reference candidate region in the reference candidate region set, determining a first area parameter and a second area parameter between the target candidate region and the reference candidate region;

[0143] filtering the reference candidate region set according to the first area parameter and the second area parameter corresponding to each reference candidate region to obtain filtered reference candidate regions;

[0144] The filtered reference candidate area is used as the defect location information, and the step of determining the target candidate area and the reference candidate area set corresponding to each defect location information according to the confidence score corresponding to each defect location information is continued; until the filtered reference candidate area does not contain the reference candidate area;

[0145] The defect position information of each panel defect except all the determined target candidate areas is used as the target defect position information.

[0146] Specifically, the target candidate area is a defect location information in the defect location information set, and the reference candidate area set includes the defect location information excluding the remaining defect location information outside the target candidate area. It can be understood that all defect location information in the target candidate area and the reference candidate area set are all defect location information in the defect location information set. Therefore, it can be seen that the process of determining the target candidate area corresponding to each defect location information and the reference candidate area set based on the confidence score corresponding to each defect location information in each defect location information can be: according to the confidence score of each defect location information, select a defect location information as the target candidate area in the defect location information set; remove the target candidate area from each defect location information to obtain the reference candidate area set, wherein the target candidate area can be the defect location information with the largest confidence score in the defect location information set. For example, the defect location information includes defect location information A, defect location information B, defect location information C, defect location information D and defect location information E, where the confidence score corresponding to defect location information A is 5, the confidence score corresponding to defect location information B is 4.8, the confidence score corresponding to defect location information C is 4.9, the confidence score corresponding to defect location information D is 3.8 and the confidence score corresponding to defect location information E is 5.8, then defect location information E is the target candidate area, and defect location information A, defect location information B, defect location information C and defect location information D constitute the reference candidate area set.

[0147] Furthermore, the first area parameter is used to reflect the ratio of the intersection area and the combined area of ​​the target candidate area and the reference candidate area, and the second area parameter is used to reflect the ratio of the intersection area and the reference candidate area to the target candidate area. Thus, for each reference candidate area in the reference candidate area set, determining the first area parameter and the second area parameter between the target candidate area and the reference candidate area specifically includes:

[0148] For each reference candidate region in the reference candidate region set, determining an intersection region and a union region between the target candidate region and the reference candidate region;

[0149] Determining a first area parameter corresponding to the reference candidate region according to the intersecting region and the merged region;

[0150] A second area parameter corresponding to the reference candidate region is determined according to the intersection region and the image region corresponding to the target candidate region.

[0151] Specifically, the intersection area between the target candidate area and the reference candidate area refers to the intersection area of ​​the image area with the edge frame of the target candidate area and the image area with the reference candidate area as the edge frame; the merged area between the target candidate area and the reference candidate area refers to the merged area of ​​the image area with the edge frame of the target candidate area and the image area with the reference candidate area as the edge frame. The first area parameter is the ratio of the area of ​​the intersection area to the area of ​​the merged area. In addition, the image area corresponding to the target candidate area is the image area with the target candidate area as the edge frame; the second area parameter is the ratio of the area of ​​the intersection area to the area of ​​the image area corresponding to the target candidate area. Of course, it is worth noting that the intersection area, the merged area, and the image area corresponding to the target candidate area can all be partial image areas of the panel image.

[0152] Furthermore, in an implementation of this embodiment, filtering the reference candidate region set according to the first area parameter and the second area parameter corresponding to each reference candidate region to obtain the filtered reference candidate regions is specifically:

[0153] For each reference candidate region, if the first area parameter or the second area parameter corresponding to the reference candidate region is greater than a preset threshold, the reference candidate region is filtered out from the reference candidate region set to obtain a filtered reference candidate region.

[0154] Specifically, the preset threshold is a pre-set criterion for determining whether each reference candidate region should be filtered out. It is understood that the preset threshold is a filtering basis for filtering the reference candidate region set. Thus, after obtaining the first area parameter R1 and the second area parameter R2, the first area parameter R1 and the second area parameter R2 can be measured based on the preset threshold to determine whether the reference candidate region should be filtered out.

[0155] In a specific implementation of this embodiment, the condition for filtering out the reference candidate region may be that the first area parameter R1 or the second area parameter R2 is greater than a preset threshold. Thus, after obtaining the first area parameter R1 and the second area parameter R2, the first area parameter R1 and the second area parameter R2 may be compared with the preset threshold, respectively. If the first area parameter R1 is greater than the preset threshold or the second area parameter R2 is greater than the preset threshold, the reference candidate region is used as a reference candidate region to be filtered out, and the reference candidate region is filtered out from the reference candidate region set. In this way, the step of comparing the first area parameter R1 and the second area parameter R2 with the preset threshold is performed for each reference candidate region, so that all reference candidate regions to be filtered out of the reference candidate region set can be filtered out to obtain filtered reference candidate regions.

[0156] Furthermore, after obtaining the filtered reference candidate set, it can be determined whether the filtered reference candidate set contains defect location information; if the defect location information is included, the filtered reference candidate set is used as the defect location information, and the steps of determining the target candidate area and the reference candidate area set corresponding to each defect location information according to the confidence score corresponding to each defect location information are continued; if the defect location information is not included, the determined target candidate areas are obtained, and the defect location information of each panel defect except all the determined target candidate areas is used as the target defect location information.

[0157] Based on the above-mentioned panel defect detection method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the panel defect detection method as described in the above-mentioned embodiment.

[0158] Based on the above panel defect detection method, the present application also provides a terminal device, such as Figure 4 As shown, it includes at least one processor 20; a display screen 21; and a memory 22. It may also include a communications interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communications interface 23 can communicate with each other via bus 24. The display screen 21 is configured to display a preset user guidance interface in the initial setup mode. The communications interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0159] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0160] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0161] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal device. In addition, the memory 22 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0162] In addition, the specific process of loading and executing the multiple instructions in the storage medium and the processor in the terminal device has been described in detail in the above method and will not be described here one by one.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A panel defect detection method, characterized in that: The method comprises: determining a plurality of candidate panel images according to defect position information of a defect area in the panel image to be inspected and the panel image to be inspected; determining, based on the plurality of candidate panel images, defect categories corresponding to the panel images; The step of determining the defect category corresponding to the panel image according to the plurality of candidate panel images specifically includes: determining a target panel image based on the plurality of candidate panel images; Inputting the target panel image into a trained defect classification model, and determining the defect category corresponding to the panel image through the defect classification model; The defect classification model is obtained by training based on a preset training sample set, wherein the process of determining the preset training sample set specifically includes: Acquire several panel images; For each panel image among the plurality of panel images, determining defect location information of a defective area in the panel image; selecting a plurality of reference panel images from the panel image according to the defect position information, wherein the plurality of reference panel images each include a portion or all of the defect area; selecting a designated panel image from a plurality of reference panel images; Rotating the designated panel image according to a preset angle, and determining a reference panel image group based on the designated panel image obtained by rotation and reference panel images other than the designated panel image among the plurality of reference panel images; All generated reference panel image groups are used as the preset training sample sets.

2. The panel defect detection method according to claim 1, characterized in that: At least two of the plurality of candidate panel images have different image sizes.

3. The panel defect detection method according to claim 1, characterized in that: Determining a target panel image according to the plurality of candidate panel images specifically includes: Adjusting the plurality of candidate panel images respectively to obtain a plurality of reference panel images; wherein the image size of each reference panel image in the plurality of reference panel images is the same; Several reference panel images are spliced ​​together to obtain a target panel image, wherein the number of channels of the target panel image is equal to the sum of the number of channels of the several reference panel images.

4. The panel defect detection method according to claim 1, wherein: The defect classification model includes a channel attention module and a classification module, and the target panel image is input into the trained defect classification model; Determining the defect category corresponding to the panel image by using the defect classification model specifically includes: Inputting the target panel image into a channel attention module, and determining a target feature vector through the channel attention module; The target feature vector is input into a classification module, and the defect category corresponding to the panel image is determined by the classification module.

5. The panel defect detection method according to claim 4, characterized in that: The channel attention module includes a convolution unit, an attention unit, and a fusion unit; inputting the target panel image into the channel attention module, and determining the channel target feature vector through the channel attention module specifically includes: Inputting the target panel image into a convolution unit, and determining a first eigenvector by the convolution unit; Inputting the first eigenvector into the attention unit, and determining a second eigenvector through the attention unit, wherein the second eigenvector is used to reflect weight information of each channel in the target panel image; The first feature vector and the second feature vector are input into the fusion unit, and the target feature vector is determined by the fusion unit.

6. The panel defect detection method according to claim 1, characterized in that: The designated panel image is a reference panel image with the smallest image size among a plurality of reference panel images.

7. The panel defect detection method according to claim 1, characterized in that: The step of determining the reference panel image group based on the designated panel image obtained by rotation and the reference panel images other than the designated panel image among the plurality of reference panel images specifically includes: performing enhancement processing on the rotated designated panel image according to a first preprocessing method to obtain a first panel image; performing enhancement processing on each reference panel image except the designated panel image in the plurality of reference panel images according to a second preset processing method to obtain each second panel image, wherein the first preprocessing method is different from the second preprocessing method; A reference panel image group is determined based on the first panel and all second panel images.

8. The panel defect detection method according to claim 1, characterized in that: The image sizes of the reference panel images among the plurality of reference panel images are different from each other, and the designated panel image is the reference panel image with the smallest image size among the plurality of reference panel images.

9. The panel defect detection method according to claim 1, characterized in that: Determining defect location information of a defective area in the panel image to be inspected specifically includes: The panel image to be inspected is input into a trained defect recognition model, and the defect position information of the defect area in the panel image is determined by the defect recognition model.

10. The panel defect detection method according to any one of claims 1 or 9, characterized in that: The defect location information includes the area size of the defect area and positioning information of the defect area.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the panel defect detection method according to any one of claims 1 to 10.

12. A terminal device, characterized in that: include: A processor, a memory and a communication bus; the memory stores a computer-readable program that can be executed by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps of the panel defect detection method according to any one of claims 1 to 10 are implemented.

Images