Image Processing Method and Apparatus

Abstract

Embodiments of the present application provide an image processing method and apparatus, which relate to the field of computer technology. The image processing method in the embodiments of the present application includes: performing clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain a plurality of image clustering sets; selecting the target image clustering set with the largest number of images from the plurality of image clustering sets, and calculating the average color channel parameters of the images in the target image clustering set based on the color channel parameters of each image in the target image clustering set; performing calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain the calibrated image to be detected. The technical solution of the embodiments of the present application enables the image defect localization model to accurately localize defects in liquid crystal panel images aggregated in a specific color pattern, improving the accuracy of defect localization by the image defect localization model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly, to an image processing method and apparatus. Background Art

[0002] During the manufacturing process of a liquid crystal panel, it is necessary to photograph the liquid crystal panel processed through each processing step to obtain a liquid crystal panel image, and an image defect localization model is used to process the liquid crystal panel image to obtain the defect location of the liquid crystal panel image.

[0003] The image defect localization model needs to be trained with a large number of liquid crystal panel images. In related technologies, generally, the liquid crystal panel images captured during the manufacturing process of the liquid crystal panel are directly used as the training sample data of the image defect localization model to train the image defect localization model.

[0004] Since the photographing of the liquid crystal panel is affected by various factors such as environmental illumination, camera imaging parameters, and the reflectivity of the liquid crystal panel, the liquid crystal panel images used to train the image defect localization model show a specific color pattern aggregation, such as light yellow, yellow-green, brown-yellow, or orange-yellow, etc. Directly using these liquid crystal panel images as the training sample data of the image defect localization model will cause the image defect localization model to be unable to accurately locate the defects in the liquid crystal panel images with specific color pattern aggregation, and the accuracy of defect localization is not high. Summary of the Invention

[0005] Embodiments of this application provide an image processing method and apparatus, which can, to a certain extent, solve the technical problem that the image defect localization model cannot accurately locate the defects in the liquid crystal panel images with specific color pattern aggregation, and the accuracy of defect localization is not high.

[0006] Other features and advantages of this application will become apparent through the following detailed description, or be learned in part through the practice of this application.

[0007] According to one aspect of the embodiments of this application, an image processing method is provided, including: performing clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain a plurality of image clustering sets; selecting the target image clustering set with the largest number of images from the plurality of image clustering sets, and calculating the average color channel parameters of the images in the target image clustering set based on the color channel parameters of each image in the target image clustering set; performing calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain the calibrated image to be detected.

[0008] According to one aspect of the embodiments of the present application, an image processing apparatus is provided, including: a clustering unit configured to perform clustering processing on images in the image dataset based on the color channel parameters of each image in the image dataset to obtain a plurality of image clustering sets; a calculation unit configured to select a target image clustering set with the largest number of images from the plurality of image clustering sets, and calculate the average color channel parameters of the images in the target image clustering set based on the color channel parameters of the images in the target image clustering set; a first calibration unit configured to perform calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain a calibrated image to be detected.

[0009] In some embodiments of the present application, based on the foregoing solution, the clustering unit is configured to: calculate the average color channel parameters corresponding to each image in the image dataset respectively based on the color channel parameters corresponding to all pixels included in each image in the image dataset; perform clustering processing on the images in the image dataset based on the average color channel parameters corresponding to each image in the image dataset to obtain a plurality of image clustering sets.

[0010] In some embodiments of the present application, based on the foregoing solution, the clustering unit is configured to: calculate the average color channel parameters of each image in the image dataset in each color channel respectively based on the color channel parameters of all pixels included in each image in the image dataset in each color channel.

[0011] In some embodiments of the present application, based on the foregoing solution, the first calibration unit is configured to: determine the average color channel parameters corresponding to the image to be detected based on the color channel parameters of all pixels included in the image to be detected; generate a calibration parameter based on the average color channel parameters of the images in the target image clustering set and the average color channel parameters corresponding to the image to be detected; perform calibration processing on the color channel parameters of all pixels included in the image to be detected based on the calibration parameter to obtain a calibrated image to be detected.

[0012] In some embodiments of the present application, based on the foregoing solution, the first calibration unit is configured to: calculate the ratio between the average color channel parameters of the images in the target image clustering set and the average color channel parameters corresponding to the image to be detected; generate a calibration parameter based on the ratio.

[0013] In some embodiments of the present application, based on the foregoing solution, the image processing apparatus further includes: an input unit configured to input the calibrated image to be detected into a pre-trained image defect localization model; a localization unit configured to perform defect localization processing on the calibrated image to be detected through the image defect localization model and output a defect localization result.

[0014] In some embodiments of the present application, based on the foregoing solution, the image processing device further includes: a second calibration unit, configured to calibrate the color channel parameters of the images in the image dataset based on the average color channel parameters of the images in the target image clustering set, to obtain a calibrated image dataset; an annotation unit, configured to perform image defect annotation processing on the images in the calibrated image dataset, to obtain an annotated image dataset; a generation unit, configured to generate training sample data based on the annotated image dataset; and a training unit, configured to train a machine learning model based on the generated training sample data, to obtain the image defect localization model.

[0015] In some embodiments of the present application, based on the foregoing solution, the generation unit is configured to: select a target image from the annotated image dataset; perform data augmentation processing on the target image, to obtain a processed image; and generate training sample data based on the processed image and the annotated image dataset.

[0016] In some embodiments of the present application, based on the foregoing solution, the generation unit is configured to: obtain a data augmentation probability threshold; assign random numbers to the images in the annotated image dataset; and determine the images with the assigned random numbers less than or equal to the data augmentation probability threshold as the target images.

[0017] According to one aspect of the embodiments of the present application, there is provided a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the image processing method as described in the above embodiments.

[0018] According to one aspect of the embodiments of the present application, there is provided an electronic device, including: one or more processors; a storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the image processing method as described in the above embodiments.

[0019] In the technical solutions provided by some embodiments of the present application, by clustering the images in the image dataset based on the color channel parameters of each image in the image dataset, a plurality of image clustering sets are obtained, and the target image clustering set with the largest number of images is selected from the plurality of image clustering sets. Then, based on the color channel parameters of the images in the target image clustering set, the average color channel parameters of the images in the target image clustering set are calculated. Next, based on the average color channel parameters of the images in the target image clustering set, the color channel parameters of the image to be detected are calibrated to obtain the calibrated image to be detected. By first performing data preprocessing of image calibration on the image to be detected, and then having the image defect localization model perform defect localization detection based on the calibrated image to be detected, compared with directly inputting the image to be detected without data preprocessing of image calibration into the image defect localization model, when the image to be detected is a liquid crystal panel image aggregated in a specific color mode, the image defect localization model can also perform accurate defect localization, improving the accuracy of defect localization of the image defect localization model.

[0020] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the accompanying drawings in the following description are only some embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts. In the drawings:

[0022] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied is shown.

[0023] Figure 2 A flowchart showing an image processing method according to an embodiment of the present application is shown.

[0024] Figure 3 A specific flowchart showing step S210 of the image processing method according to an embodiment of the present application is shown.

[0025] Figure 4 A specific flowchart showing step S230 of the image processing method according to an embodiment of the present application is shown.

[0026] Figure 5 A specific flowchart showing step S410 of the image processing method according to an embodiment of the present application is shown.

[0027] Figure 6Shows a flowchart of an image processing method according to an embodiment of the present application.

[0028] Figure 7 Shows a flowchart of an image processing method according to an embodiment of the present application.

[0029] Figure 8 Shows a detailed flowchart of step S720 of an image processing method according to an embodiment of the present application.

[0030] Figure 9 Shows a detailed flowchart of step S810 of an image processing method according to an embodiment of the present application.

[0031] Figure 10 Shows a block diagram of an image processing apparatus according to an embodiment of the present application.

[0032] Figure 11 Shows a schematic structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application. Detailed implementation manners

[0033] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more complete and comprehensive, and will fully convey the concept of the example embodiments to those skilled in the art.

[0034] In addition, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0035] The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0037] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solution of the embodiment of the present application can be applied is shown.

[0038] As Figure 1 shown, the system architecture may include a client 101, a network 102, and a server 103. The network 102 is used to provide a medium for a communication link between the client 101 and the server 103. The network 102 may include various connection types, such as a wired communication link, a wireless communication link, and so on.

[0039] It should be understood that Figure 1 the numbers of the client 101, the network 102, and the server 103 in

[0040] are merely illustrative. According to the implementation requirements, there may be any number of clients 101, networks 102, and servers 103. For example, the server 103 may be a server cluster composed of multiple servers, etc.

[0041] The client 101 performs clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset, obtains multiple image clustering sets, selects the target image clustering set with the largest number of images from the multiple image clustering sets, calculates the average color channel parameters of the images in the target image clustering set based on the color channel parameters of the images in the target image clustering set, and then calibrates the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain the calibrated image to be detected. By first performing data preprocessing of image calibration on the image to be detected and then having the image defect localization model perform defect localization detection based on the calibrated image to be detected, compared with directly inputting the image to be detected without data preprocessing of image calibration into the image defect localization model, when the image to be detected is a liquid crystal panel image aggregated in a specific color mode, the image defect localization model can also perform accurate defect localization, improving the accuracy of defect localization of the image defect localization model.

[0042] It should be noted that the image processing method provided by the embodiments of the present application is generally executed by the client 101. Correspondingly, the image processing device is generally disposed in the client 101. However, in other embodiments of the present application, the server 103 may also have a similar function as the client 101, so as to execute the solution of the image processing method provided by the embodiments of the present application.

[0043] The implementation details of the technical solutions of the embodiments of the present application are elaborated in detail below.

[0044] Figure 2 The flowchart of an image processing method according to an embodiment of the present application is shown. The image processing method can be executed by a client, and the client can be Figure 1 the client 101 shown in Figure 2 As shown, the image processing method at least includes steps S210 to S230, which are introduced in detail as follows.

[0045] In step S210, based on the color channel parameters of each image in the image dataset, the images in the image dataset are clustered to obtain a plurality of image cluster sets.

[0046] In one embodiment, during the manufacturing process of a liquid crystal panel, when the liquid crystal panel is processed through a certain processing technology, a processed liquid crystal panel can be photographed by a specific image capturing device to obtain a liquid crystal panel image, and the obtained liquid crystal panel image is used as an image for color defect analysis of the liquid crystal panel. The image dataset is a set of images obtained by photographing the liquid crystal panel. It should be noted that due to the objective differences in the shooting environments of each liquid crystal panel image, for example, the shooting environment differences caused by various different factors such as environmental light, camera imaging parameters, and the reflectivity of the liquid crystal panel, the colors of the liquid crystal panel images are different. Therefore, the set of liquid crystal panel images included in the image dataset generally refers to a set of liquid crystal panel images with color characteristic differences.

[0047] In one embodiment, the color channel parameter of an image is the parameter value in the color channel corresponding to the image in a specific color mode. The color channel parameter of an image can be a parameter characterizing the color characteristics of the image. An image generally includes a plurality of pixels, and each pixel has a corresponding color channel parameter. The color channel parameters of all the pixels included in the image constitute the color channel parameter of the image.

[0048] The color mode corresponding to the image can be the color mode of a single color channel, such as a grayscale image. For an image with a single color channel, the color channel parameter value corresponding to each pixel has only one parameter value in one color channel. The color mode corresponding to the image can also be the color mode of multiple color channels, such as the Red-Green-Blue (RGB) color mode. For example, in the RGB color mode, the corresponding color channel parameters are three parameter values, namely the parameter values in the R color channel, the G color channel, and the B color channel.

[0049] In one embodiment, in order to determine the common color characteristics of all the images included in the image dataset, clustering processing can be performed on the images in the image dataset based on the color channel parameters of each image in the image dataset, to obtain multiple image clustering sets, and each image clustering set can be used as an image set representing images with the same color characteristics.

[0050] Optionally, the clustering algorithm used for clustering the images in the image dataset can be the Mean-Shift clustering algorithm or the k-means clustering algorithm, etc. Of course, it can also be other clustering algorithms, which are not limited herein.

[0051] Optionally, when using the color channel parameters of each image in the image dataset as the input data for clustering processing, the color channel parameters corresponding to all the pixels included in the image can be directly used as the input data for clustering, to perform clustering processing on the images in the image dataset, to obtain multiple image clustering sets, so as to cluster the images in the image dataset according to the color characteristics included in the images.

[0052] Reference Figure 3 , Figure 3 shows a specific flowchart of step S210 of the image processing method according to an embodiment of the present application. In this embodiment, step S210 can specifically include steps S310 to S320, which are described in detail as follows.

[0053] In step S310, based on the color channel parameters corresponding to all the pixels included in each image in the image dataset, the average color channel parameter corresponding to each image in the image dataset is calculated respectively.

[0054] In one embodiment, when clustering the images in the image dataset based on the color channel parameters of each image in the image dataset, the average color channel parameter corresponding to each image in the image dataset can also be calculated first according to the color channel parameters corresponding to all the pixels included in each image in the image dataset. In other words, for each image in the image dataset, the average color channel parameter corresponding to the image is calculated according to the color channel parameters corresponding to all the pixels included in the image, and the average color channel parameter is used as a parameter reflecting the overall color characteristics of the image.

[0055] Specifically, the color channel parameters of all the pixels can be added up and divided by the number of pixels included in the image to obtain the average color channel parameter corresponding to the image.

[0056] Optionally, when the color mode corresponding to the image is a single-color-channel color mode, the average color channel parameter corresponding to the image can be obtained by adding up the single-color-channel parameters corresponding to all the pixels included in each image and dividing by the number of pixels included in the image, and the average color channel parameter corresponding to the image obtained is only one.

[0057] Optionally, when the color mode of the image is a multi-color-channel color mode, step S310 may specifically include: calculating the average color channel parameter of each image in the image dataset in each color channel based on the color channel parameters of all the pixels included in each image in the image dataset in each color channel.

[0058] When the color mode of the image is a multi-color-channel color mode, the average color channel parameter corresponding to the image is the color channel parameter in multiple color channels. Taking an image in the RGB color mode as an example, the color channel parameters of all the pixels included in each image in the R color channel can be added up and divided by the number of pixels included in the image to obtain the average color channel parameter of the image in the R color channel. Similarly, the average color channel parameter of the image in the G color channel and the average color channel parameter of the image in the B color channel can be calculated respectively. Furthermore, the average color channel parameters of the image in the R color channel, the G color channel, and the B color channel are calculated, and the average color channel parameters of the image in the three different color channels constitute the average color channel parameter corresponding to the image.

[0059] In step S320, based on the average color channel parameter corresponding to each image in the image dataset, the images in the image dataset are clustered to obtain a plurality of image cluster sets.

[0060] In one embodiment, after obtaining the average color channel parameters corresponding to each image in the image dataset, they are used as the input data for the clustering algorithm. The input data is processed by the clustering algorithm, and the clustering results of multiple image clustering sets are output, thereby realizing the clustering process of the images in the image dataset according to the color channel parameters of the images in the image dataset.

[0061] Figure 3 In the technical solution of the illustrated embodiment, by first calculating the average color channel parameters corresponding to each image in the image dataset according to the color channel parameters corresponding to all pixels included in each image in the image dataset, and then performing clustering processing on the images in the image dataset based on the average color channel parameters corresponding to each image in the image dataset. Compared with the method of directly clustering the images in the image dataset according to the color channel parameters of all pixels included in the image, the dimension of the calculation data corresponding to each image input into the clustering algorithm can be significantly reduced, thereby effectively reducing the complexity of the clustering operation, reducing the calculation amount of the system, and saving system resources.

[0062] Please continue to refer to Figure 2 , in step S220, select the target image clustering set with the largest number of images from multiple image clustering sets, and calculate the average color channel parameters of the images in the target image clustering set based on the color channel parameters of the images in the target image clustering set.

[0063] In one embodiment, after obtaining multiple image clustering sets, the image clustering set with the largest number of images can be selected from the multiple image clustering sets as the target image clustering set, and this target image clustering set is used as the image set that can best represent the common color characteristics of all the images in the image dataset.

[0064] It can be understood that if there are two image clustering sets with the same number of images and the number of images in both is more than that of other image clustering sets, then either one of the two image clustering sets can be selected as the target image clustering set.

[0065] In one embodiment, after determining the target image clustering set, the average color channel parameters of the images in the target image clustering set can be calculated based on the color channel parameters of the images in the target image clustering set.

[0066] Specifically, the average color channel parameters corresponding to each image in the image dataset can be calculated respectively according to the color channel parameters corresponding to all pixels included in each image in the target image cluster. In other words, for each image in the target image cluster, the average color channel parameters corresponding to the image are calculated according to the color channel parameters corresponding to all pixels included in the image. After calculating the average color channel parameters corresponding to each image in the target image cluster, the average color channel parameters corresponding to each image in the target image cluster are summed, and then the sum result is divided by the number of images in the target image cluster to calculate the average color channel parameters of the images in the target image cluster.

[0067] As mentioned above, the color channel parameters of the image under each color channel need to be calculated separately. Therefore, when the color mode corresponding to the image is a single color channel color mode, the average color channel parameter corresponding to the obtained image is only one. Correspondingly, the average color channel parameter of the images in the target image cluster is also only one; while when the color mode of the image is a multi-color channel color mode, the average color channel parameters corresponding to the obtained image are multiple. Correspondingly, the average color channel parameters of the images in the target image cluster are also multiple.

[0068] In step S230, based on the average color channel parameters of the images in the target image cluster, the color channel parameters of the image to be detected are calibrated to obtain the calibrated image to be detected.

[0069] In one embodiment, after determining the average color channel parameters of the images in the target image cluster, the color channel parameters of the image to be detected can be calibrated based on the average color channel parameters of the images in the target image cluster to obtain the calibrated image to be detected.

[0070] Specifically, the color channel parameters of all pixels included in the image to be detected can be calibrated according to the average color channel parameters of the images in the target image cluster, so that the image to be detected can be adjusted into an image with the common color characteristics of all images in the image dataset as the calibrated image to be detected.

[0071] Compared with the method of directly inputting the image to be detected without image calibration into the image defect localization model, by first performing data preprocessing of image calibration on the image to be detected, and then the image defect localization model performs defect localization detection according to the calibrated image to be detected, when the image to be detected is a liquid crystal panel image aggregated in a specific color mode, the image defect localization model can also perform accurate defect localization, improving the accuracy of defect localization of the image defect localization model.

[0072] Reference Figure 4, Figure 4 FIG. 230 shows a specific flowchart of step S230 of an image processing method according to an embodiment of the present application. In this embodiment, step S230 may specifically include steps S410 to S430, which are described in detail as follows.

[0073] In step S410, based on the color channel parameters of all pixels included in the image to be detected, the average color channel parameters corresponding to the image to be detected are determined.

[0074] In one embodiment, when calibrating the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set, the average color channel parameters corresponding to the image to be detected may be first determined based on the color channel parameters of all pixels included in the image to be detected.

[0075] Specifically, the color channel parameters of all pixels may be added together and divided by the number of pixels included in the image to obtain the average color channel parameters corresponding to the image.

[0076] It can be understood that when the color mode corresponding to the image is a single-color-channel color mode, the average color channel parameters corresponding to the image to be detected may be obtained by adding the single-color-channel parameters corresponding to all pixels included in the image to be detected and dividing by the number of pixels included in the image. The average color channel parameters corresponding to the image to be detected obtained are only one.

[0077] When the color mode of the image to be detected is a multi-color-channel color mode, the average color channel parameters corresponding to the image to be detected are the color channel parameters in multiple color channels. Taking an image in the RGB color mode as an example, the color channel parameters of all pixels included in the image to be detected in the R color channel may be added together and divided by the number of pixels included in the image to be detected to obtain the average color channel parameters of the image to be detected in the R color channel. Similarly, the average color channel parameters of the image to be detected in the G color channel and the B color channel may be calculated respectively. Furthermore, the average color channel parameters of the image to be detected in the R color channel, the G color channel, and the B color channel are calculated, and the average color channel parameters of the image to be detected in the three different color channels constitute the average color channel parameters corresponding to the image to be detected.

[0078] In step S420, based on the average color channel parameters of the images in the target image clustering set and the average color channel parameters corresponding to the image to be detected, a calibration parameter is generated.

[0079] In one embodiment, when generating calibration parameters based on the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected, specifically, the calibration parameters for calibrating the image can be generated according to the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected, as well as the corresponding relationship between the calibration parameters for calibrating the image and the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image, so as to realize the calibration process for the image to be detected.

[0080] Reference Figure 5 , Figure 5 FIG. shows a specific flowchart of step S410 of an image processing method according to an embodiment of the present application. In this embodiment, step S410 may specifically include steps S510 to S520, which are described in detail as follows.

[0081] In step S510, calculate the ratio between the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected.

[0082] In one embodiment, when generating calibration parameters for calibrating the image to be detected, the ratio between the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected can be calculated first.

[0083] It can be understood that when the color mode of the image is a multi-color channel color mode, both the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected include multiple average color channel parameters under different color channels. Therefore, it is necessary to calculate the ratios between the average color channel parameters of the image under different color channels respectively.

[0084] Taking an image in the RGB color mode as an example, the average color channel parameters of the images in the target image cluster set include the average color channel parameter R1 under the R color channel, the average color channel parameter G1 under the G color channel, and the average color channel parameter B1 under the B color channel. The average color channel parameters corresponding to the image to be detected include the average color channel parameter R2 under the R color channel, the average color channel parameter G2 under the G color channel, and the average color channel parameter B2 under the B color channel. Then the ratios between the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected include those corresponding to the R color channel Those corresponding to the G color channel And those corresponding to the B color channel

[0085] In step S520, calibration parameters are generated based on the ratio.

[0086] In one embodiment, after obtaining the ratio between the average color channel parameters of the images in the target image cluster set and the average color channel parameters corresponding to the image to be detected, calibration parameters for calibrating the image to be detected are generated based on the corresponding relationship between the ratio, the ratio and the calibration parameters.

[0087] Optionally, the corresponding relationship between the ratio and the calibration parameters can be a positively correlated linear relationship. For example, the ratio can be directly used as the calibration parameter for calibrating the image.

[0088] It can be understood that when the color mode of the image to be detected is a multi-color channel color mode, when generating the calibration parameters based on the ratio, it is necessary to determine the calibration parameters in different color channels according to the ratio between the average color channel parameters of the two images in different color channels respectively.

[0089] Taking an image in the RGB color mode as an example, the ratio corresponding to the R color channel can be directly used as the calibration parameter of the image in the corresponding R color channel for calibrating the color channel parameters in the R color channel. Similarly, the ratio corresponding to the G color channel can be used as the calibration parameter of the image in the corresponding G color channel and the ratio corresponding to the B color channel can be used as the calibration parameter of the image in the corresponding B color channel.

[0090] Please continue to refer to Figure 4 In step S430, based on the calibration parameters, the color channel parameters of all pixels included in the image to be detected are calibrated to obtain the calibrated image to be detected.

[0091] In one embodiment, after generating the calibration parameters for calibrating the image to be detected, the color channel parameters of all pixels included in the image to be detected can be calibrated according to the calibration parameters corresponding to the image to be detected to obtain a calibrated image dataset.

[0092] Specifically, the color channel parameters of each pixel of the image to be detected in the corresponding color channel are calibrated according to the calibration parameters of the image to be detected in each color channel. The method of calibration can specifically be to calculate the product of the calibration parameter of the image in each color channel and the color channel parameter of the pixel of the image in the corresponding color channel, and this product is used as the calibrated color channel parameter, thereby realizing the calibration of the color channel parameters of each pixel of the image in the corresponding color channel.

[0093] After the color channel parameters of all pixels included in the image are calibrated, the calibrated image to be detected can be obtained.

[0094] As can be seen from the above, by clustering the images in the image dataset based on the color channel parameters of each image in the image dataset, multiple image clustering sets are obtained, and the target image clustering set with the largest number of images is selected from the multiple image clustering sets. Then, based on the color channel parameters of each image in the target image clustering set, the average color channel parameters of the images in the target image clustering set are calculated. Further, based on the average color channel parameters of the images in the target image clustering set, the color channel parameters of the image to be detected are calibrated to obtain the calibrated image to be detected. By first performing data preprocessing of image calibration on the image to be detected and then having the image defect localization model perform defect localization detection based on the calibrated image to be detected, compared with directly inputting the image to be detected without data preprocessing of image calibration into the image defect localization model, when the image to be detected is a liquid crystal panel image aggregated in a specific color mode, the image defect localization model can also perform accurate defect localization, improving the accuracy of defect localization by the image defect localization model.

[0095] Reference Figure 6 , Figure 6 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method in the embodiment of the present application may further include steps S610 to S620, which are described in detail as follows.

[0096] In step S610, the calibrated image to be detected is input into a pre-trained image defect localization model.

[0097] In one embodiment, when performing color defect analysis on a liquid crystal panel, the liquid crystal panel image can be input into an image defect localization model obtained through training. The image defect localization model is obtained by training a machine learning model. The machine learning model can be a CNN (Convolutional Neural Network) model, or it can also be a deep neural network model, etc. The pre-trained image defect localization model can perform defect localization processing on the input calibrated image to be detected to detect the defect position in the image to be detected.

[0098] In step S620, the calibrated image to be detected is subjected to defect localization processing by the image defect localization model, and a defect localization result is output.

[0099] In one embodiment, the calibrated image to be detected is subjected to defect location processing by an image defect location model, and a defect location result is output, which is defect location information obtained by performing defect location processing on the calibrated image to be detected.

[0100] Figure 6 In the technical solution of the illustrated embodiment, by performing calibration processing on the color channel parameters of the image to be detected, a calibrated image to be detected is obtained. By first performing data preprocessing of image calibration on the image to be detected, and then the image defect location model performs defect location detection based on the calibrated image to be detected. Compared with directly inputting the image to be detected without data preprocessing of image calibration into the image defect location model, when the image to be detected is a liquid crystal panel image aggregated in a specific color mode, the image defect location model can also perform accurate defect location, improving the accuracy of defect location by the image defect location model.

[0101] Reference Figure 7 , Figure 7 shows a flowchart of an image processing method according to an embodiment of the present application. The image processing method in the embodiment of the present application may further include steps S710 to S740, which are described in detail as follows.

[0102] In step S710, based on the average color channel parameters of the images in the target image clustering set, the color channel parameters of the images in the image dataset are calibrated to obtain a calibrated image dataset.

[0103] In one embodiment, when generating training sample data for training a machine learning model according to the image dataset, the color channel parameters of the images in the image dataset can be calibrated first based on the average color channel parameters of the images in the target image clustering set to obtain a calibrated image dataset. Among them, the method of calibrating the color channel parameters of each image in the image dataset based on the average color channel parameters of the images in the target image clustering set is the same as the method of calibrating the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set, and will not be elaborated here.

[0104] In step S720, image defect annotation processing is performed on each image in the calibrated image dataset to obtain an annotated image dataset.

[0105] In one embodiment, when training an image defect location model according to the calibrated image dataset, image defect annotation processing can also be performed on each image in the calibrated image dataset, that is, annotation processing is pre-performed at the actual defect positions of each image.

[0106] In step S730, based on the labeled image dataset, training sample data is generated.

[0107] In one embodiment, after performing annotation processing on the calibrated image dataset to obtain the labeled image dataset, the training sample data for training the machine learning model can be generated according to the labeled image dataset.

[0108] Reference Figure 8 , Figure 8 shows a specific flowchart of step S720 of the image processing method according to an embodiment of the present application. Step S720 in the embodiment of the present application may include steps S810 to S830, which are described in detail as follows.

[0109] In step S810, target images are selected from the labeled image dataset.

[0110] In one embodiment, after performing annotation processing on the calibrated image dataset to obtain the labeled image dataset, in order to improve the effect of the machine learning model, some images can also be selected from the labeled image dataset as the target images for data augmentation processing.

[0111] Optionally, some images can be randomly selected directly from the labeled image dataset as the target images for data augmentation processing.

[0112] Optionally, reference Figure 9 , Figure 9 shows a specific flowchart of step S810 of the image processing method according to an embodiment of the present application. Step S810 in the embodiment of the present application may include steps S910 to S930, which are described in detail as follows.

[0113] In step S910, a data augmentation probability threshold is obtained.

[0114] In step S920, random numbers are assigned to the images in the labeled image dataset.

[0115] In step S930, the images with the assigned random numbers less than or equal to the data augmentation probability threshold are determined as the target images.

[0116] In one embodiment, when selecting the target images for data augmentation processing from the labeled image dataset, to ensure randomness, a data augmentation probability threshold for image selection can be preset first, and image selection is performed according to this data augmentation probability threshold. The data augmentation probability threshold can be a preset probability parameter, for example, it can be a probability parameter between 0.5 and 0.8.

[0117] Specifically, a random number can be preselected and assigned to each image in the labeled image dataset. This random number is generally a certain parameter between 0 and 1.

[0118] Compare the random number assigned to each image with the data augmentation probability threshold. Determine the images for which the assigned random number is less than or equal to the data augmentation probability threshold as the target images, thereby achieving the ability to randomly select the target images that need to be data-augmented from the labeled image dataset.

[0119] Please continue to refer to Figure 8 , in step S820, perform data augmentation on the target images to obtain the processed images.

[0120] In one embodiment, after obtaining the target images, data augmentation can be performed on the selected target images to obtain the processed images.

[0121] Specifically, for any one of the selected target images, various types of data augmentation such as color transformation, rotation, scaling, and adding noise can be performed on the target image. It can be understood that for each type of data augmentation, it can be determined whether each target image needs to perform the corresponding data augmentation according to a certain probability. Thus, each target image may perform one or more types of data augmentation simultaneously.

[0122] In step S830, based on the processed images and the labeled image dataset, generate training sample data.

[0123] The processed images and the original images that have not undergone data augmentation together constitute the training sample data for training the machine learning model.

[0124] Figure 8 In the technical solution of the illustrated embodiment, by using the images that have not undergone data augmentation and the images that have undergone data augmentation together to generate the training sample data for training the machine learning model, the generalization ability of the trained image defect localization model can be effectively improved, and the performance of the image defect localization model for image defect localization can be enhanced.

[0125] Please continue to refer to Figure 7 , in step S740, train the machine learning model based on the generated training sample data to obtain the image defect localization model.

[0126] In one embodiment, the machine learning model is trained based on the generated training sample data to obtain an image defect localization model. The process of training the machine learning model is to adjust the coefficients in the network structure corresponding to the machine learning model so that for the input image to be detected, after the operations of the coefficients in the network structure corresponding to the machine learning model, the output result is the determined defect localization information.

[0127] Figure 7 In the technical solution of the illustrated embodiment, when the image to be detected is a liquid crystal panel image aggregated in a specific color mode, data preprocessing of color calibration is performed on each image in the image dataset to be detected, so as to reduce the color difference between the image and the images in the calibrated image dataset, so that the image defect localization model trained with the images in the calibrated image dataset can accurately locate the defects in the liquid crystal panel image aggregated in a specific color mode, and improve the accuracy of defect localization of the image defect localization model.

[0128] The following introduces the device embodiments of the present application, which can be used to execute the image processing method in the above embodiments of the present application. For the details not disclosed in the device embodiments of the present application, please refer to the embodiments of the above image processing method of the present application.

[0129] Figure 10 The block diagram of an image processing device according to an embodiment of the present application is shown.

[0130] Refer to Figure 10 As shown, an image processing device 1000 according to an embodiment of the present application includes: a clustering unit 1010, a calculation unit 1020, and a first calibration unit 1030. Among them, the clustering unit 1010 is configured to perform clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain a plurality of image clustering sets; the calculation unit 1020 is configured to select the target image clustering set with the largest number of images from the plurality of image clustering sets, and calculate the average color channel parameters of the images in the target image clustering set based on the color channel parameters of the images in the target image clustering set; the first calibration unit 1030 is configured to perform calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain the calibrated image to be detected.

[0131] In some embodiments of the present application, based on the foregoing solution, the clustering unit 1010 is configured to: respectively calculate the average color channel parameters corresponding to each image in the image dataset based on the color channel parameters corresponding to all pixels included in each image in the image dataset; perform clustering processing on the images in the image dataset based on the average color channel parameters corresponding to each image in the image dataset to obtain a plurality of image clustering sets.

[0132] In some embodiments of the present application, based on the foregoing solution, the clustering unit 1010 is configured to: calculate the average color channel parameters of each image in the image dataset under each color channel respectively based on the color channel parameters of all pixels included in each image in the image dataset under each color channel.

[0133] In some embodiments of the present application, based on the foregoing solution, the first calibration unit 1030 is configured to: determine the corresponding average color channel parameters of the image to be detected based on the color channel parameters of all pixels included in the image to be detected; generate calibration parameters based on the average color channel parameters of the images in the target image clustering set and the corresponding average color channel parameters of the image to be detected; perform calibration processing on the color channel parameters of all pixels included in the image to be detected based on the calibration parameters to obtain the calibrated image to be detected.

[0134] In some embodiments of the present application, based on the foregoing solution, the first calibration unit 1030 is configured to: calculate the ratio between the average color channel parameters of the images in the target image clustering set and the corresponding average color channel parameters of the image to be detected; generate calibration parameters based on the ratio.

[0135] In some embodiments of the present application, based on the foregoing solution, the image processing device further includes: an input unit for inputting the calibrated image to be detected into a pre-trained image defect localization model; a localization unit for performing defect localization processing on the calibrated image to be detected through the image defect localization model and outputting a defect localization result.

[0136] In some embodiments of the present application, based on the foregoing solution, the image processing device further includes: a second calibration unit for performing calibration processing on the color channel parameters of the images in the image dataset based on the average color channel parameters of the images in the target image clustering set to obtain a calibrated image dataset; an annotation unit for performing image defect annotation processing on the images in the calibrated image dataset to obtain an annotated image dataset; a generation unit for generating training sample data based on the annotated image dataset; a training unit for training a machine learning model based on the generated training sample data to obtain an image defect localization model.

[0137] In some embodiments of the present application, based on the foregoing solution, the generation unit is configured to: select a target image from the annotated image dataset; perform data augmentation processing on the target image to obtain a processed image; generate training sample data based on the processed image and the annotated image dataset.

[0138] In some embodiments of the present application, based on the foregoing solution, the generating unit is configured to: obtain a data augmentation probability threshold; assign random numbers to the images in the labeled image dataset; and determine the images whose assigned random numbers are less than or equal to the data augmentation probability threshold as target images.

[0139] Figure 11 FIG. shows a schematic structural diagram of a computer system of an electronic device suitable for implementing embodiments of the present application.

[0140] It should be noted that Figure 11 The computer system 1100 of the shown electronic device is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0141] As Figure 11 shown, the computer system 1110 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage section 1108 into a random access memory (RAM) 1103, such as executing the method described in the above embodiments. In the RAM 1103, various programs and data required for system operations are also stored. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0142] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read from it can be installed into the storage section 1108 as needed.

[0143] In particular, according to an embodiment of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, various functions defined in the system of the present application are executed.

[0144] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device. The computer program included on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0146] The units involved in the embodiments described in the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these units do not constitute a limitation to the unit itself in some cases.

[0147] On the other hand, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the methods described in the above embodiments.

[0148] It should be noted that although several modules or units of devices for performing actions are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0149] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the methods according to the embodiments of the present application.

[0150] Other embodiments of the present application will be readily contemplated by those skilled in the art after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common general knowledge or conventional technical means in the technical field not disclosed in the present application.

[0151] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An image processing method, characterized in that, Including: Performing clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain multiple image clustering sets; Selecting the target image clustering set with the largest number of images from the multiple image clustering sets, and calculating the average color channel parameters of the images in the target image clustering set based on the color channel parameters of each image in the target image clustering set; Performing calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain the calibrated image to be detected; wherein, the calibrated image to be detected is used for defect localization.

2. The image processing method according to claim 1, wherein The performing clustering processing on the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain multiple image clustering sets includes: Respectively calculating the average color channel parameters corresponding to each image in the image dataset based on the color channel parameters corresponding to all pixels included in each image in the image dataset; Performing clustering processing on the images in the image dataset based on the average color channel parameters corresponding to each image in the image dataset to obtain multiple image clustering sets.

3. The image processing method according to claim 2, characterized in that, The respectively calculating the average color channel parameters corresponding to each image in the image dataset based on the color channel parameters corresponding to all pixels included in each image in the image dataset includes: Respectively calculating the average color channel parameters of each image in the image dataset in each color channel based on the color channel parameters of all pixels included in each image in the image dataset in each color channel.

4. The image processing method according to claim 1, wherein The performing calibration processing on the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain the calibrated image to be detected includes: Determining the average color channel parameters corresponding to the image to be detected based on the color channel parameters of all pixels included in the image to be detected; Generating a calibration parameter based on the average color channel parameters of the images in the target image clustering set and the average color channel parameters corresponding to the image to be detected; Performing calibration processing on the color channel parameters of all pixels included in the image to be detected based on the calibration parameter to obtain the calibrated image to be detected.

5. The image processing method according to claim 4, wherein The generating a calibration parameter based on the average color channel parameters of the images in the target image clustering set and the average color channel parameters corresponding to the image to be detected includes: Calculating the ratio between the average color channel parameters of the images in the target image clustering set and the average color channel parameters corresponding to the image to be detected; Generating a calibration parameter based on the ratio.

6. The image processing method according to any one of claims 1-5, characterized in that The image processing method further includes: Inputting the calibrated image to be detected into a pre-trained image defect localization model; Performing defect localization processing on the calibrated image to be detected through the image defect localization model and outputting a defect localization result.

7. The image processing method according to claim 6, wherein The image processing method further includes: Performing calibration processing on the color channel parameters of the images in the image dataset based on the average color channel parameters of the images in the target image clustering set to obtain a calibrated image dataset; Perform image defect annotation processing on each image in the calibrated image dataset to obtain an annotated image dataset; Generate training sample data based on the annotated image dataset; Train a machine learning model based on the generated training sample data to obtain the image defect localization model.

8. The image processing method according to claim 7, characterized in that The generating training sample data based on the annotated image dataset includes: Select a target image from the annotated image dataset; Perform data augmentation processing on the target image to obtain a processed image; Generate training sample data based on the processed image and the annotated image dataset.

9. The image processing method according to claim 8, wherein the selecting a target image from the annotated image dataset includes: Obtain a data augmentation probability threshold; Assign a random number to the images in the annotated image dataset; Determine the images with the assigned random number less than or equal to the data augmentation probability threshold as the target images.

10. An image processing apparatus, characterized in that, including: A clustering unit for clustering the images in the image dataset based on the color channel parameters of each image in the image dataset to obtain a plurality of image clustering sets; A calculation unit for selecting the target image clustering set with the largest number of images from the plurality of image clustering sets, and calculating the average color channel parameters of the images in the target image clustering set based on the color channel parameters of the images in the target image clustering set; A first calibration unit for calibrating the color channel parameters of the image to be detected based on the average color channel parameters of the images in the target image clustering set to obtain a calibrated image to be detected; wherein, the calibrated image to be detected is used for defect localization.

11. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the image processing method according to any one of claims 1-9 is implemented.

12. An electronic device, characterized in that, including: One or more processors; A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the image processing method according to any one of claims 1-9.

Images