Image processing method, device, equipment, and storage medium

Abstract

The present application discloses an image processing method and apparatus, a device, and a storage medium, which belong to the field of display technology. The method includes: obtaining an image containing a display panel acquired by an AOI device; performing resolution enhancement processing on the image to obtain a target image; and determining bad information of the display panel in the target image. By performing resolution enhancement processing on the image acquired by the AOI device to improve the resolution of the target image obtained after the resolution enhancement processing, it can be ensured that the clarity of the target image is high and the image quality is good. When the bad information of the display panel is subsequently determined in the target image with higher clarity, the accuracy of the determined bad information of the display panel is higher, which reduces the probability that the bad phenomenon existing in the display panel cannot be identified.

Description

Technical Field

[0001] The present application relates to the field of display technology, and in particular to an image processing method and apparatus, device, and storage medium. Background Art

[0002] Currently, display panels are usually manufactured through a complex process flow, during which circuits of the display panels are prone to open circuits or short circuits.

[0003] Therefore, after each process in the manufacturing process of the display panel, it is necessary to use automatic optical inspection (AOI) equipment to obtain images of the display panel, and identify defects on these images to obtain the location and type of defects in the display panel after each process. The subsequent repair of the display panel with defects can greatly improve the product yield of the display panel.

[0004] However, when the quality of the image acquired by the AOI device is poor, the accuracy of defect recognition of the image is also poor, which may easily lead to the inability to recognize the defective phenomenon in the display panel, thereby resulting in a low product yield of the display panel. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, equipment, and storage medium. The problem of low product yield of current display panels can be solved. The technical solution is as follows:

[0006] In one aspect, an image processing method is provided, the method comprising:

[0007] Acquire an image including a display panel captured by an automatic optical inspection (AOI) device;

[0008] Performing resolution enhancement processing on the image to obtain a target image;

[0009] Defective information of the display panel in the target image is determined.

[0010] Optionally, performing resolution enhancement processing on the image to obtain a target image includes:

[0011] The image is input into the enhancement model to obtain the target image output by the enhancement model, and the enhancement model is used to enhance the resolution of the input image.

[0012] Optionally, performing resolution enhancement processing on the image to obtain a target image includes:

[0013] An image interpolation algorithm or an image super-resolution reconstruction algorithm is used to perform resolution enhancement processing on the image to obtain the target image.

[0014] Optionally, performing resolution enhancement processing on the image to obtain a target image includes:

[0015] After determining that the image is not a clear image, performing at least one resolution enhancement process on the image until the obtained image is a clear image;

[0016] The obtained image is determined as the target image.

[0017] Optionally, performing at least one resolution enhancement process on the image includes:

[0018] Performing resolution enhancement processing on the image to obtain an intermediate image;

[0019] Detecting whether the intermediate image is a clear image;

[0020] After determining that the intermediate image is not a clear image, a resolution enhancement process is performed on the intermediate image to obtain a new intermediate image, and an operation of detecting whether the new intermediate image is a clear image is performed.

[0021] Optionally, the method further includes:

[0022] Performing a definition feature extraction process on the image to obtain a first definition feature of the image;

[0023] The first clarity feature is compared with a second clarity feature of a clear reference image to detect whether the image is a clear image.

[0024] Optionally, comparing the first clarity feature with a second clarity feature of a clear reference image to detect whether the image is a clear image, wherein detecting whether the image is a clear image includes:

[0025] determining a similarity between the first definition feature and the second definition feature;

[0026] When it is determined that the similarity is greater than a similarity threshold, determining that the image is a clear image;

[0027] When it is determined that the similarity is not greater than the similarity threshold, it is determined that the image is not a clear image.

[0028] Optionally, determining the defect information of the display panel in the target image includes:

[0029] Performing bad feature extraction processing on the target image to obtain a first bad feature of the target image;

[0030] The first defect feature is compared with a second defect feature of a defective image to determine defect information of the display panel in the target image, wherein the defective image is an image including a defective display panel.

[0031] Optionally, before inputting the image into the enhancement model to obtain the image output by the enhancement model, the method further includes:

[0032] receiving the enhanced model;

[0033] Alternatively, the artificial intelligence model is trained multiple times to obtain the enhanced model;

[0034] Wherein, each training process includes:

[0035] Input a blurred sample image into the artificial intelligence model to obtain an image output by the artificial intelligence model;

[0036] Based on the image output by the artificial intelligence model and the clear label image corresponding to the sample image, the parameters of the artificial intelligence model are adjusted.

[0037] In another aspect, an image processing device is provided, the device comprising:

[0038] An acquisition module, used to acquire an image including a display panel acquired by an automatic optical inspection (AOI) device;

[0039] An enhancement module, used for performing resolution enhancement processing on the image to obtain a target image;

[0040] A determination module is used to determine defective information of the display panel in the target image.

[0041] Optionally, the enhancement module is used to:

[0042] The image is input into the enhancement model to obtain the target image output by the enhancement model, and the enhancement model is used to enhance the resolution of the input image.

[0043] Optionally, the enhancement module is used to:

[0044] An image interpolation algorithm or an image super-resolution reconstruction algorithm is used to perform resolution enhancement processing on the image to obtain the target image.

[0045] Optionally, the enhancement module includes:

[0046] an enhancement unit, configured to, after determining that the image is not a clear image, perform at least one resolution enhancement process on the image until the obtained image is a clear image;

[0047] A determination unit is used to determine the obtained image as the target image.

[0048] On the other hand, an image processing device is provided, comprising: a processor, and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned image processing method.

[0049] On the other hand, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a processing component, the processing component executes the above-mentioned image processing method.

[0050] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0051] After acquiring the image containing the display panel captured by the AOI device, the image is subjected to resolution enhancement processing to improve the resolution of the target image obtained after the resolution enhancement processing, thereby ensuring that the target image has high clarity and good image quality. When the bad information of the display panel is subsequently determined in the target image with higher clarity, the accuracy of the bad information of the display panel determined is higher, reducing the probability that the bad phenomenon in the display panel cannot be identified. After repairing the bad phenomenon in the display panel, the product yield of the display panel can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 is a structural schematic diagram of an image processing system involved in an image processing method provided in an embodiment of the present application;

[0054] Figure 2 is a flow chart of an image processing method provided by an embodiment of the present application;

[0055] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present application;

[0056] Figure 4is a flow chart of a method for detecting whether an image is a clear image provided by an embodiment of the present application;

[0057] Figure 5 is a flow chart of a method for determining bad information of a display panel in a target image provided by an embodiment of the present application;

[0058] Figure 6 is a structural block diagram of an image processing device provided in an embodiment of the present application;

[0059] Figure 7 is a structural block diagram of an enhancement module provided in an embodiment of the present application;

[0060] Figure 8 It is a structural block diagram of another image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0062] Please refer to Figure 1 , Figure 1 1 is a schematic diagram of the structure of an image processing system involved in an image processing method provided in an embodiment of the present application. The image processing system 100 may include: an AOI device 101 and an image processing device 102.

[0063] The AOI device 101 is used to collect images of the display panel 103. Thus, during the manufacturing process of the display panel 103, the AOI device 101 can collect images of the display panel 103 after each process.

[0064] The image processing device 102 may be a computer device, or a server, or a server cluster consisting of several servers, or a cloud computing service center, etc.

[0065] The AOI device 101 is in communication connection with the image processing device 102. The AOI device 101 can send the image it has collected to the image processing device 102.

[0066] Optionally, the image processing system 100 may further include: a storage server 104. The storage server 104 may be a single server, or a server cluster consisting of a plurality of servers, or a cloud computing service center, etc.

[0067] The storage server 104 is respectively connected to the AOI device 101 and the image processing device 102. In this way, the AOI device 101 can be connected to the image processing device 102 through the storage server 104. After the AOI device 101 acquires an image, the AOI device 101 can send the acquired image to the storage server 104 for storage. The image processing device 102 can download the image to be processed from the storage server 104.

[0068] It should be noted that the so-called communication connection in the embodiments of the present application may be a communication connection established through a wired network or a wireless network.

[0069] Please refer to Figure 2 , Figure 2 is a flowchart of an image processing method provided in an embodiment of the present application. The image processing method is applied to Figure 1 The image processing device 102 in the image processing system 100 is shown. The image processing method may include:

[0070] Step 201: Obtain an image including a display panel captured by an AOI device.

[0071] Step 202: Perform resolution enhancement processing on the image to obtain a target image.

[0072] Step 203: Determine defect information of the display panel in the target image.

[0073] In summary, the image processing method provided in the embodiment of the present application, after acquiring the image containing the display panel captured by the AOI device, performs resolution enhancement processing on the image to improve the resolution of the target image obtained after the resolution enhancement processing, thereby ensuring that the clarity of the target image is high and the image quality is good. When the bad information of the display panel is subsequently determined in the target image with higher clarity, the accuracy of the bad information of the display panel determined is higher, reducing the probability that the bad phenomenon existing in the display panel cannot be identified. After the bad phenomenon existing in the display panel is repaired, the product yield of the display panel can be effectively improved.

[0074] Please refer to Figure 3 , Figure 3 is a flowchart of another image processing method provided in an embodiment of the present application. The image processing method is applied to Figure 1 The image processing device 102 in the image processing system 100 is shown. The image processing method may include:

[0075] Step 301: Acquire an image including a display panel captured by an AOI device.

[0076] In the embodiment of the present application, the image processing device can obtain the image containing the display panel captured by the AOI device.

[0077] In the present application, during the manufacturing process of the display panel, the AOI device can collect images of the display panel after each process. Moreover, the AOI device can send the collected images containing the display panel to the image processing device, so that the image processing device can obtain the images containing the display panel.

[0078] For example, after the AOI device captures an image containing a display panel, the AOI device can send the captured image to a storage server so that the storage server can store the image containing the display panel. Later, if it is necessary to detect defects in the display panel after a certain process, the image processing device downloads the corresponding image from the storage server and detects the defects in the display panel in the image.

[0079] It should be noted that the AOI device can automatically photograph the display panel after each process, and during the photographing process of the AOI device, the AOI device can automatically focus to ensure the clarity of the image containing the display panel. However, during the automatic focusing process of the AOI device, when the focus of the AOI device is inaccurate, the clarity of the image captured by it is relatively low. For this reason, after the image processing device obtains the image captured by the AOI device, the image processing device also needs to detect whether the image captured by the AOI device is a clear image through the following step 302.

[0080] Step 302: Detect whether the image captured by the AOI device is a clear image.

[0081] In the embodiment of the present application, after the image processing device obtains the image containing the display panel captured by the AOI device, the image processing device also needs to detect whether the image captured by the AOI device is a clear image.

[0082] The clear image refers to an image with a resolution greater than a resolution threshold. When the image resolution is greater than the resolution threshold, the image has a high definition, and even if the bad information of the display panel is directly determined in the image, the accuracy of the determined bad information of the display panel can be ensured to be high.

[0083] To this end, in the present application, after the image processing device determines that the image captured by the AOI device is not a clear image (that is, a blurred image), it is necessary to perform corresponding processing on the image, and then determine the poor information of the display panel in the processed image, that is, execute step 303; after the image processing device determines that the image captured by the AOI device is a clear image, the poor information of the display panel can be directly determined in the image, that is, execute step 305.

[0084] In this application, please refer to Figure 4 , Figure 4 This is a flow chart of a method for detecting whether an image is a clear image provided by an embodiment of the present application. The image processing device may detect whether the image captured by the AOI device is a clear image, which may include the following steps:

[0085] Step 3021: Perform clarity feature extraction processing on the image collected by the AOI device to obtain a first clarity feature of the image.

[0086] In an embodiment of the present application, the image processing device may perform clarity feature extraction processing on the image captured by the AOI device to obtain a first clarity feature of the image.

[0087] The clarity feature of the image is used to reflect the clarity of the image, and may include the resolution information of the image and the information of the difference between any two adjacent pixels, etc. The clarity feature may be an array or a vector.

[0088] In an optional implementation, the clarity feature extraction process can be implemented by a convolutional neural network (CNN). For example, an image can be directly input into the CNN, and the CNN calculates and outputs the clarity features of the image. In another optional implementation, the clarity feature extraction process can also be implemented by other computing modules or feature extractors. For example, a convolution operation can be performed on the image, and the result of the operation is used as the clarity feature of the image. It should be noted that there can be other ways to extract the clarity feature, and the embodiments of the present application are not limited to this.

[0089] Step 3022: Compare the first clarity feature with the second clarity feature of the clear reference image to detect whether the image is a clear image.

[0090] In an embodiment of the present application, the image processing device may compare the first clarity feature with the second clarity feature of a clear reference image to detect whether the image is a clear image.

[0091] In the present application, the second clarity feature of the clear reference image is obtained in advance. After the image processing device obtains the first clarity feature of the image captured by the AOI device each time, the first clarity feature is directly compared with the second clarity feature to detect whether the image captured by the AOI device is a clear image.

[0092] It should be noted that the clear reference image is a high-definition image collected in advance by the AOI device. The clear reference image can be manually screened from a large number of images collected by the AOI device. After performing clarity feature extraction processing on a large number of high-definition images and performing an average operation on the extracted clarity features, the result of the operation can be used as the second clarity feature of the clear reference image.

[0093] For example, the image processing device may compare the first clarity feature with the second clarity feature of the clear reference image to detect whether the image is a clear image, which may include the following steps:

[0094] Step A1: Determine the similarity between the first definition feature and the second definition feature.

[0095] In the embodiment of the present application, after the image processing device determines the first clarity feature of the image captured by the AOI device, the image processing device may determine the similarity between the first clarity feature and the second clarity feature of the clear reference image.

[0096] For example, the image processing device may use a similarity calculation formula to calculate the similarity between the first definition feature and the second definition feature.

[0097] Step B1: Detect whether the similarity is greater than a similarity threshold.

[0098] In the embodiment of the present application, after the image processing device determines the similarity between the first definition feature and the second definition feature, the image processing device may detect whether the similarity is greater than a similarity threshold.

[0099] For example, when the image processing device determines that the similarity between the first clarity feature and the second clarity feature is greater than the similarity threshold, step C1 is executed; when the image processing device determines that the similarity between the first clarity feature and the second clarity feature is not greater than the similarity threshold, step D1 is executed.

[0100] Step C1: when it is determined that the similarity is greater than the similarity threshold, the image is determined to be a clear image.

[0101] In the embodiment of the present application, when the image processing device determines that the similarity between the first clarity feature and the second clarity feature is greater than the similarity threshold, the image processing device can determine that the image captured by the AOI device is a clear image. At this time, step 305 can be executed.

[0102] Step D1: When it is determined that the similarity is not greater than the similarity threshold, the image is determined to be an unclear image.

[0103] In the embodiment of the present application, when the image processing device determines that the similarity between the first clarity feature and the second clarity feature is not greater than the similarity threshold, the image processing device may determine that the image captured by the AOI device is not a clear image, and step 305 may be executed.

[0104] It should be noted that if the operator finds that the clarity of the image collected by the AOI device is low during the process of collecting images containing the display panel, the operator can mark the image with low clarity collected by the AOI device so that the image can carry a mark indicating that the image has low clarity. In this case, after the image processing device obtains the image collected by the AOI device, the image processing device can detect whether the image carries a mark indicating that the image has low clarity. After the image processing device detects that the image carries this mark, the image processing device can directly execute step 303 without executing step 302; and after the image processing device does not detect that the image carries this mark, the image processing device executes step 302. In this way, the amount of calculation of the image processing device can be effectively reduced and the calculation performance of the image processing device can be improved.

[0105] Step 303: After determining that the image captured by the AOI device is not a clear image, perform resolution enhancement processing on the image to obtain a target image.

[0106] In an embodiment of the present application, after the image processing device determines that the image captured by the AOI device is not a clear image, that is, after the image processing device determines that the image captured by the AOI device is a blurred image, the image processing device can perform resolution enhancement processing on the image to obtain a target image.

[0107] In the present application, the image processing device performs resolution enhancement processing on the image captured by the AOI device, and there are multiple ways to obtain the target image. The embodiments of the present application are schematically described by taking the following two optional implementation methods as examples:

[0108] In a first optional implementation, the image processing device performs resolution enhancement processing on the image collected by the AOI device to obtain the target image, which may include:

[0109] The image processing device inputs the image collected by the AOI device into the enhancement model to obtain the image output by the enhancement model, and determines the image output by the enhancement model as the target image.

[0110] In the embodiment of the present application, the enhancement model is used to enhance the resolution of the input image. Thus, after the image processing device inputs the image captured by the AOI device to the enhancement model, the resolution of the target image output by the enhancement model is higher than the resolution of the image captured by the AOI device, so that the clarity of the target image is higher than the clarity of the image captured by the AOI device.

[0111] Optionally, the enhanced model may be an artificial intelligence (AI) model obtained by learning and training using a deep neural network or a convolutional neural network.

[0112] It should be noted that before inputting the image captured by the AOI device into the enhancement model and obtaining the target image output by the enhancement model, the image processing device also needs to obtain the enhancement model. In this application, the image processing device performs multiple training processes on the artificial intelligence model to obtain the enhancement model.

[0113] However, since it is necessary to process a large amount of data in the process of performing multiple training processes on the artificial intelligence model to obtain an enhanced model, in order to improve the training efficiency, in this application, a training server with stronger computing power can be used to train the artificial intelligence model to obtain an enhanced model. After the training is completed, the enhanced model can be sent to the image processing device so that the image processing device can receive the enhanced model. In this case, the training server can perform multiple training processes on the artificial intelligence model to obtain an enhanced model.

[0114] For example, whether the image processing device performs multiple training processes on the artificial intelligence model or the training server performs multiple training processes on the artificial intelligence model, the method of each training process is the same.

[0115] Each training process may include:

[0116] Step A2: input a blurred sample image into the artificial intelligence model to obtain an image output by the artificial intelligence model.

[0117] Step B2: Adjust the parameters of the artificial intelligence model based on the image output by the artificial intelligence model and the clear label image corresponding to the sample image.

[0118] The blurred sample image and the clear label image corresponding to the sample image display the same image content. The sample image and the corresponding label image are only different in clarity.

[0119] In this application, the clarity features of the image output by the artificial intelligence model and the clarity features of the label image corresponding to the sample image can be compared, and the parameters of the artificial intelligence model can be adjusted according to the comparison results. As the number of training times increases, the gap between the clarity of the image output by the artificial intelligence model and the clarity of the label image corresponding to the sample image becomes smaller and smaller.

[0120] It should be noted that during multiple training processes of the artificial intelligence model, if the cutoff condition is met, the artificial intelligence model after the last training process can be determined as an enhanced model.

[0121] Optionally, the cutoff condition may include: executing the training process for a specified number of times; or, when the training process is executed n times in succession, the similarity between the clarity feature of the image output by the artificial intelligence model each time and the clarity feature of the label image corresponding to the sample image is greater than the similarity threshold, where n is an integer greater than 1.

[0122] After the artificial intelligence model is trained to obtain an enhanced model, the resolution of the input image can be enhanced through the enhanced model. In this way, if a blurred image is input to the enhanced model, the clarity of the image output by the enhanced model is much higher than the clarity of the blurred image.

[0123] In a second optional implementation, the image processing device performs resolution enhancement processing on the image collected by the AOI device to obtain the target image, which may include:

[0124] The image collected by the AOI device is processed with resolution enhancement using an image interpolation algorithm or an image super-resolution reconstruction algorithm to obtain the target image.

[0125] In an embodiment of the present application, an image interpolation algorithm or an image super-resolution reconstruction algorithm is used to perform resolution enhancement processing on the image captured by the AOI device, and the resolution of the target image obtained is higher than the resolution of the image captured by the AOI device, so that the clarity of the target image is higher than the clarity of the image captured by the AOI device.

[0126] It should be noted that both of the above two optional implementations can effectively improve the resolution of the image captured by the AOI device. However, if the resolution of the image captured by the AOI device is low, it is difficult to process it into a higher resolution image through a single resolution enhancement process.

[0127] To this end, in an embodiment of the present application, after the image processing device determines that the image captured by the AOI device is not a clear image, the image processing device needs to perform at least one resolution enhancement process on the image captured by the AOI device until the obtained image is a clear image, and determine the obtained image as the target image.

[0128] In the present application, after each resolution enhancement process is performed on an image, the image processing device needs to detect whether the image obtained after the resolution enhancement process is a clear image. If the image obtained after the resolution enhancement process is a clear image, the image obtained after the resolution enhancement process is determined as the target image; if the image obtained after the resolution enhancement process is not a clear image, the resolution enhancement process is performed again.

[0129] It should be noted that the process of the image processing device detecting whether the image obtained after resolution enhancement processing is a clear image each time can refer to the process of detecting whether the image captured by the AOI device is a clear image in step 302, and the embodiments of the present application will not be repeated here.

[0130] In the embodiment of the present application, there are multiple ways for the image processing device to perform at least one resolution enhancement process on the image captured by the AOI device. The embodiment of the present application takes the following two exemplary implementation methods as examples for explanation:

[0131] In a first exemplary implementation, the image processing device performing at least one resolution enhancement process on the image captured by the AOI device may include:

[0132] Step A3: Perform resolution enhancement processing on the image processing device to obtain an intermediate image.

[0133] In the embodiment of the present application, the image processing device performs resolution enhancement processing on the image captured by the AOI device to obtain an intermediate image. The resolution enhancement processing method for the image in step A3 can adopt any one of the above two optional implementation methods, and the embodiment of the present application will not be repeated here.

[0134] Step B3: Detect whether the intermediate image is a clear image.

[0135] In the embodiment of the present application, the image processing device can detect whether the intermediate image is a clear image. The detection process can refer to the process of detecting whether the image captured by the AOI device is a clear image in step 302, and the embodiment of the present application will not be repeated here.

[0136] For example, after the image processing device determines that the middle image is a clear image, the middle image is determined as the target image. After the image processing device determines that the middle image is not a clear image, step C3 is performed.

[0137] Step C3: after determining that the intermediate image is not a clear image, perform resolution enhancement processing on the intermediate image to obtain a new intermediate image, and perform an operation of detecting whether the new intermediate image is a clear image.

[0138] In the embodiment of the present application, after the image processing device determines that the intermediate image is not a clear image, the image processing device can perform resolution enhancement processing on the intermediate image to obtain a new intermediate image. Afterwards, the image processing device performs an operation of detecting whether the new intermediate image is a clear image, that is, repeating step B3. Among them, the resolution enhancement processing method for the intermediate image in step C3 can adopt any one of the above two optional implementation methods, and the embodiment of the present application will not be repeated here.

[0139] It should be noted that after the image collected by the AOI device is subjected to resolution enhancement processing to obtain an intermediate image, the clarity of the intermediate image is higher than the clarity of the image collected by the AOI device; after the intermediate image is subjected to resolution enhancement processing to obtain a new intermediate image, the clarity of the new intermediate image is higher than the clarity of the intermediate image. In this way, the more times the resolution enhancement processing is performed on the image collected by the AOI device, the higher the clarity of the processed image.

[0140] In a second exemplary implementation, the image processing device performing at least one resolution enhancement process on the image captured by the AOI device may include:

[0141] Step A4: Perform resolution enhancement processing on the image processing device to obtain an intermediate image.

[0142] This step can refer to the above-mentioned step A3, and the embodiment of the present application will not be repeated here.

[0143] Step B4: Detect whether the intermediate image is a clear image.

[0144] This step can refer to the above-mentioned step B3, and the embodiment of the present application will not be repeated here.

[0145] For example, after the image processing device determines that the middle image is a clear image, the middle image is determined as the target image. After the image processing device determines that the middle image is not a clear image, step C4 is performed.

[0146] Step C4: after determining that the intermediate image is not a clear image, re-perform resolution enhancement processing on the image captured by the AOI device to obtain a new intermediate image, and perform an operation of detecting whether the new intermediate image is a clear image.

[0147] In the embodiment of the present application, after the image processing device determines that the intermediate image is not a clear image, the image processing device can re-perform resolution enhancement processing on the image captured by the AOI device to obtain a new intermediate image. Afterwards, the image processing device performs an operation of detecting whether the new intermediate image is a clear image, that is, repeating step B4. Among them, the resolution enhancement processing method for the intermediate image in step C4 can adopt any one of the above two optional implementation methods, and the embodiment of the present application will not be repeated here.

[0148] It should be noted that the degree of resolution enhancement processing performed on the image collected by the AOI device this time is higher than the degree of resolution enhancement processing performed on the image collected by the AOI device last time. Thus, the clarity of the intermediate image obtained after the resolution enhancement processing is performed on the image collected by the AOI device this time is higher than the clarity of the intermediate image obtained after the resolution enhancement processing is performed on the image collected by the AOI device last time. Thus, the more times the resolution enhancement processing is performed on the image collected by the AOI device, the higher the clarity of the processed image.

[0149] In the present application, through the above two exemplary implementation methods, after the image processing device performs multiple resolution enhancement processes on the image collected by the AOI device, the obtained target image has a higher clarity.

[0150] Step 304: Determine defect information of the display panel in the target image.

[0151] In the embodiment of the present application, after the image processing device performs resolution enhancement processing on the image captured by the AOI device to obtain the target image, the image processing device can determine the defective information of the display panel in the target image.

[0152] The defect information of the display panel may include: type information of defective phenomena existing in the display panel, and location information of a part of the display panel where the defective phenomenon exists in the display panel.

[0153] For example, Figure 5 As shown, Figure 5 1 is a flow chart of a method for determining bad information of a display panel in a target image provided by an embodiment of the present application. The image processing device may determine bad information of a display panel in a target image, including the following steps:

[0154] Step 3041: Perform bad feature extraction processing on the target image to obtain the first bad feature of the target image.

[0155] In the embodiment of the present application, the image processing device may perform bad feature extraction processing on the target image to obtain the first bad feature of the target image.

[0156] The bad feature of the image is used to reflect bad information of the display panel in the image, which may include grayscale information and boundary information of the display content in the image, etc. The bad feature may be an array or a vector.

[0157] For example, when the undesirable features include boundary information of the display content in the image, the image processing device may obtain the boundary information by: smoothing the image with a Gaussian filter; calculating the amplitude and direction of the gradient with the finite difference of the first-order partial derivative; performing non-maximum suppression on the gradient amplitude; and detecting and connecting edges with a dual threshold algorithm.

[0158] In an optional implementation, the bad feature extraction process can be implemented by CNN. For example, the image can be directly input into CNN, and CNN calculates and outputs the bad features of the image. In another optional implementation, the bad feature extraction process can also be implemented by other computing modules or feature extractors. For example, a convolution operation can be performed on the image, and the result of the operation is used as the bad feature of the image. It should be noted that there can be other ways to extract the bad features, and the embodiments of the present application are not limited to this.

[0159] Step 3042: Compare the first defective feature with the second defective feature of the defective image to determine defective information of the display panel in the target image.

[0160] In the embodiment of the present application, the image processing device may compare the first defective feature with the second defective feature of the defective image to determine the defective information of the display panel in the image, wherein the defective image is an image containing a defective display panel.

[0161] In the present application, the second bad feature of the bad image is acquired in advance. After the image processing device acquires the first bad feature of the target image, the first bad feature is directly compared with the second bad feature to determine whether the display panel in the target image has a bad phenomenon, and when the display panel in the target image has a bad phenomenon, the bad information of the display panel in the target image is determined.

[0162] It should be noted that the bad image is an image of a display panel with high definition and containing a bad image that is collected in advance by an AOI device. The bad image can be manually screened out from a large number of images collected by the AOI device. After performing bad feature extraction processing on a large number of images of display panels with high definition and containing a bad image, and performing an average operation on the extracted bad features, the result of the operation can be used as the second bad feature of the bad image.

[0163] In the present application, when the types of bad phenomena in the display panel are different, the bad features in the bad image are also different. For this reason, the number of second bad features of the bad image is multiple, and the multiple second bad features correspond to the types of multiple bad phenomena in different images. Moreover, the number of first bad features obtained after bad feature extraction of the target image is also multiple. After comparing each first bad feature with each second bad feature in turn, the multiple types of bad phenomena that may exist in the display panel can be detected and checked.

[0164] For example, if the similarity between each first defective feature and each second defective feature of the target image is not greater than a similarity threshold, the image processing device can determine that there is no defective phenomenon on the display panel in the target image.

[0165] If the similarity between a first defective feature and a second defective feature of the target image is greater than a similarity threshold, the image processing device can determine that the display panel in the target image has a defective phenomenon. The image processing device can determine the type of defective phenomenon of the display panel in the target image based on the second defective feature. Moreover, the image processing device determines the location information of the defective phenomenon of the display panel in the target image based on the first defective feature. In this way, the image processing device can determine the defective information of the display panel in the target image.

[0166] Step 305: After determining that the image captured by the AOI device is a clear image, determine defect information of the display panel in the image.

[0167] In the embodiment of the present application, after the image processing device determines that the image captured by the AOI device is a clear image, the image processing device can determine the bad information of the display panel in the image. It should be noted that the way in which the image processing device determines the bad information of the display panel in the image captured by the AOI device can refer to the way in which the image processing device determines the bad information of the display panel in the target image in step 304, and the embodiment of the present application will not be repeated here.

[0168] It should be noted that the order of the steps of the image processing method provided in the embodiment of the present application can be appropriately adjusted, and the steps can be increased or decreased accordingly according to the circumstances. Any technician familiar with the technical field can easily think of the changed methods within the technical scope disclosed in the present application, and they should be included in the protection scope of the present application, so they will not be repeated here.

[0169] In summary, the image processing method provided in the embodiment of the present application, after acquiring the image containing the display panel captured by the AOI device, performs resolution enhancement processing on the image to improve the resolution of the target image obtained after the resolution enhancement processing, thereby ensuring that the clarity of the target image is high and the image quality is good. When the bad information of the display panel is subsequently determined in the target image with higher clarity, the accuracy of the bad information of the display panel determined is higher, reducing the probability that the bad phenomenon existing in the display panel cannot be identified. After the bad phenomenon existing in the display panel is repaired, the product yield of the display panel can be effectively improved.

[0170] The present application also provides an image processing device. Figure 6 As shown, Figure 6 is a structural block diagram of an image processing device provided in an embodiment of the present application. The image processing device 400 can be integrated in Figure 1 The image processing device 102 in the image processing system 100 is shown. The image processing apparatus 400 may include:

[0171] The acquisition module 401 is used to acquire an image including a display panel captured by an automatic optical inspection (AOI) device.

[0172] The enhancement module 402 is used to perform resolution enhancement processing on the image to obtain a target image.

[0173] The determination module 403 is used to determine defect information of the display panel in the target image.

[0174] In summary, the image processing device provided in the embodiment of the present application, after acquiring the image containing the display panel captured by the AOI device, performs resolution enhancement processing on the image to improve the resolution of the target image obtained after the resolution enhancement processing, thereby ensuring that the clarity of the target image is high and the image quality is good. When the bad information of the display panel is subsequently determined in the target image with higher clarity, the accuracy of the bad information of the display panel determined is higher, reducing the probability that the bad phenomenon existing in the display panel cannot be identified. After the bad phenomenon existing in the display panel is repaired, the product yield of the display panel can be effectively improved.

[0175] Optionally, the enhancement module 402 is used to input an image into an enhancement model to obtain a target image output by the enhancement model, and the enhancement model is used to enhance the resolution of the input image.

[0176] Optionally, the enhancement module 402 is used to adopt an image interpolation algorithm or an image super-resolution reconstruction algorithm to perform resolution enhancement processing on the image to obtain a target image.

[0177] Optional, such as Figure 7 As shown, Figure 7 402 is a structural block diagram of an enhancement module provided in an embodiment of the present application. The enhancement module 402 may include:

[0178] The enhancement unit 4021 is used to perform at least one resolution enhancement process on the image after determining that the image acquired by the AOI device is not a clear image, until the obtained image is a clear image;

[0179] The determination unit 4022 is used to determine the obtained image as a target image.

[0180] Optionally, the enhancement unit 4021 is used to perform resolution enhancement processing on the image acquired by the AOI device to obtain an intermediate image; detect whether the intermediate image is a clear image; after determining that the intermediate image is not a clear image, perform resolution enhancement processing on the intermediate image to obtain a new intermediate image, and execute an operation of detecting whether the new intermediate image is a clear image.

[0181] Optional, such as Figure 8 As shown, Figure 8 400 is a structural block diagram of another image processing device provided in an embodiment of the present application. The image processing device 400 may also include:

[0182] The definition feature extraction module 404 is used to perform definition feature extraction processing on the image acquired by the AOI device to obtain a first definition feature of the image.

[0183] The comparison module 405 is used to compare the first clarity feature with the second clarity feature of the clear reference image to detect whether the image is a clear image.

[0184] Optionally, the comparison module 405 is used to: determine the similarity between the first clarity feature and the second clarity feature; when it is determined that the similarity is greater than a similarity threshold, determine that the image captured by the AOI device is a clear image; when it is determined that the similarity is not greater than the similarity threshold, determine that the image captured by the AOI device is not a clear image.

[0185] Optionally, the determination module 403 is used to: perform bad feature extraction processing on the target image to obtain a first bad feature of the target image; compare the first bad feature with the second bad feature of the bad image to determine bad information of the display panel in the target image, the bad image being an image containing a bad display panel.

[0186] Optionally, the image processing device further includes: a receiving module or a training module.

[0187] Wherein, the receiving module is used to receive the enhanced model.

[0188] The training module is used to perform multiple training processes on the artificial intelligence model to obtain an enhanced model. Each training process includes: inputting a blurred sample image into the artificial intelligence model to obtain an image output by the artificial intelligence model; and adjusting the parameters of the artificial intelligence model based on the image output by the artificial intelligence model and the clear label image corresponding to the sample image.

[0189] In summary, the image processing device provided in the embodiment of the present application, after acquiring the image containing the display panel captured by the AOI device, performs resolution enhancement processing on the image to improve the resolution of the target image obtained after the resolution enhancement processing, thereby ensuring that the clarity of the target image is high and the image quality is good. When the bad information of the display panel is subsequently determined in the target image with higher clarity, the accuracy of the bad information of the display panel determined is higher, reducing the probability that the bad phenomenon existing in the display panel cannot be identified. After the bad phenomenon existing in the display panel is repaired, the product yield of the display panel can be effectively improved.

[0190] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0191] The present application also provides an image processing device. The image processing device may include: a processor, and a memory for storing executable instructions of the processor. The processor is configured to execute Figure 2 or Figure 3 The image processing method is shown.

[0192] The present application also provides a computer-readable storage medium. The computer-readable storage medium stores instructions that, when the computer-readable storage medium is run on a processing component, cause the processing component to execute Figure 2 or Figure 3 The image processing method is shown.

[0193] In the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more than two, unless otherwise clearly defined.

[0194] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0195] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image including a display panel captured by an automatic optical inspection (AOI) device; Performing resolution enhancement processing on the image to obtain a target image; Performing bad feature extraction processing on the target image to obtain a first bad feature of the target image, wherein the first bad feature includes grayscale information and boundary information of the target image; The first defective feature is compared with the second defective feature of the defective image to determine the defective information of the display panel in the target image, the defective information of the display panel includes type information and location information of the defective phenomenon of the display panel, and the defective image is an image containing a defective display panel. If the similarity between one of the first defective features and one of the second defective features of the target image is greater than a similarity threshold, it is determined that the display panel in the target image has a defective phenomenon. If the similarity between each of the first defective features and each of the second defective features of the target image is not greater than the similarity threshold, it is determined that the display panel in the target image does not have a defective phenomenon.

2. The method according to claim 1, characterized in that Performing resolution enhancement processing on the image to obtain a target image includes: The image is input into the enhancement model to obtain the target image output by the enhancement model, and the enhancement model is used to enhance the resolution of the input image.

3. The method according to claim 1, characterized in that Performing resolution enhancement processing on the image to obtain a target image includes: An image interpolation algorithm or an image super-resolution reconstruction algorithm is used to perform resolution enhancement processing on the image to obtain the target image.

4. The method according to any one of claims 1 to 3, characterized in that: Performing resolution enhancement processing on the image to obtain a target image includes: After determining that the image is not a clear image, performing at least one resolution enhancement process on the image until the obtained image is a clear image; The obtained image is determined as the target image.

5. The method according to claim 4, characterized in that Performing at least one resolution enhancement process on the image, comprising: Performing resolution enhancement processing on the image to obtain an intermediate image; Detecting whether the intermediate image is a clear image; After determining that the intermediate image is not a clear image, a resolution enhancement process is performed on the intermediate image to obtain a new intermediate image, and an operation of detecting whether the new intermediate image is a clear image is performed.

6. The method according to claim 4, characterized in that The method further comprises: Performing a definition feature extraction process on the image to obtain a first definition feature of the image; The first clarity feature is compared with a second clarity feature of a clear reference image to detect whether the image is a clear image.

7. The method according to claim 6, characterized in that Comparing the first clarity feature with a second clarity feature of a clear reference image to detect whether the image is a clear image, and detecting whether the image is a clear image includes: determining a similarity between the first definition feature and the second definition feature; When it is determined that the similarity is greater than a similarity threshold, determining that the image is a clear image; When it is determined that the similarity is not greater than the similarity threshold, it is determined that the image is not a clear image.

8. The method according to claim 2, characterized in that: Before inputting the image into the enhancement model to obtain the image output by the enhancement model, the method further includes: receiving the enhanced model; Alternatively, the artificial intelligence model is trained multiple times to obtain the enhanced model; Wherein, each training process includes: Input a blurred sample image into the artificial intelligence model to obtain an image output by the artificial intelligence model; Based on the image output by the artificial intelligence model and the clear label image corresponding to the sample image, the parameters of the artificial intelligence model are adjusted.

9. An image processing device, characterized in that: The device comprises: An acquisition module, used to acquire an image including a display panel acquired by an automatic optical inspection (AOI) device; An enhancement module, used for performing resolution enhancement processing on the image to obtain a target image; A determination module is used to determine defective information of the display panel in the target image.

10. The device according to claim 9, characterized in that The enhancement module is used to: The image is input into the enhancement model to obtain the target image output by the enhancement model, and the enhancement model is used to enhance the resolution of the input image.

11. The device according to claim 9, characterized in that The enhancement module is used to: An image interpolation algorithm or an image super-resolution reconstruction algorithm is used to perform resolution enhancement processing on the image to obtain the target image.

12. The device according to any one of claims 9 to 11, characterized in that: The enhancement module comprises: an enhancement unit, configured to, after determining that the image is not a clear image, perform at least one resolution enhancement process on the image until the obtained image is a clear image; A determination unit is used to determine the obtained image as the target image.

13. An image processing device, characterized in that: include: A processor, and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the image processing method as described in any one of claims 1 to 8.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the computer-readable storage medium is run on a processing component, the processing component executes the image processing method according to any one of claims 1 to 8.

Images