A method, device, and storage medium for training a display screen image pollution filtering model

Abstract

The present application discloses a method, device and storage medium for training a display screen image pollution filtering model to improve the effect of display screen image pollution filtering. Obtain a display screen image set, a dirty area set and a filtered image set as a training sample set; construct a U-Net network model; perform feature encoding processing on the training sample set; add noise to the display screen image features to generate a noise state within a corresponding time interval; according to the noise state, the filtered image features and time parameters, and in combination with a weighted loss function, train a noise network to generate fitting parameters; generate denoised features according to the fitting parameters, and then input the denoised features into a decoder, and enhance the transmission of features through skip layer connections to generate repaired features; calculate the encoder / decoder loss value through the loss function and the repaired features, and update the encoder and decoder through the encoder / decoder loss value; calculate the noise network loss value through the fitting parameters and the weighted loss function, and update the noise network through the noise network loss value.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of display screen dirt filtering, and in particular, to a method, an apparatus, and a storage medium for training a display screen image dirt filtering model. Background Art

[0002] With the update and iteration of electronic products, the display screen is used more and more frequently, and consumers have higher requirements for the quality of products. This has led to an increasing amount of detection content for the display screen and an increasing detection accuracy.

[0003] As the application fields of electronic devices are more and more extensive and the application scenarios are more and more complex, the precision of the display screen is getting higher and higher, and the shapes are also ever-changing. In the field of display screen manufacturing, due to factors such as manufacturing processes, materials, and the environment, the phenomenon of uneven brightness or color, specifically in the display screen field, is called the mura defect. In order to reduce the mura defect, the Demura process, as a traditional mura defect elimination method, has become a key process step in the display screen production and manufacturing process, aiming to correct the brightness and chromaticity uniformity of each pixel point on the display screen, so as to ensure the accuracy and consistency of the displayed image. By effectively implementing the Demura process, the overall uniformity of the calibrated panel can be significantly improved, thereby increasing the product yield rate and panel grade. However, since the Demura process relies on a camera to capture the panel to obtain brightness data, if there are interfering objects such as stains, dust, and protective film fragments on the display panel, collectively referred to as dirt interference, these interfering objects will be misinterpreted by the algorithm as mura. If these interfering objects are not specially processed, it may ultimately lead to over-compensation or mis-compensation phenomena at the corresponding positions on the panel after Demura. This not only affects the display effect but also may reduce the product quality.

[0004] The detection and recovery of dirt on the display screen usually rely on traditional image processing methods and image generation methods based on deep learning. Traditional image processing methods rely on pre-set thresholds to process images, thereby determining the dirt area and eliminating it. However, the dirt determined by this type of method is not precise enough, and for some complex situations, the dirt positioning effect and repair effect are poor.

[0005] Nowadays, using deep learning models for detection and repair has greatly improved the efficiency and accuracy. However, currently, the pixel structure of the display screen is complex, and nowadays, there are curved surfaces on the display screen that can be bent and folded, and the types of display screen dirt are constantly increasing, resulting in a decline in the dirt filtering effect on the display screen image. Summary of the Invention

[0006] The present application discloses a method, an apparatus, and a storage medium for training a display screen image dirt filtering model, which are used to improve the dirt filtering effect of the display screen image.

[0007] The first aspect of the present application discloses a method for training a display screen image pollution filtering model, including:

[0008] Obtain a display screen image set, a dirty area set, and a filtered image set as a training sample set. The display screen image set is a plurality of captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty pollution filtering;

[0009] Construct a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer links. The loss functions of the encoder and the encoder are set in the U-Net network model;

[0010] Input the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a number of downsampled display screen image features, dirty area features, and filtered image features;

[0011] Input the display screen image features into the noise addition module to add noise to the display screen image features to generate the noise state within the corresponding time interval;

[0012] Input the noise state and the filtered image features into the noise network, and train the noise network to generate fitting parameters according to the noise state, the filtered image features, and the time parameters, in combination with the weighted loss function;

[0013] Input the fitting parameters into the denoising module to generate denoised features, and then input the denoised features into the decoder to enhance the transmission of features through skip layer links to generate repaired features;

[0014] Calculate the encoder / decoder loss value through the loss function and the repaired features, and update the encoder and the decoder through the encoder / decoder loss value;

[0015] Construct a weighted loss function through the dirty area features and the backward stochastic differential equation, and calculate the noise network loss value through the fitting parameters and the weighted loss function;

[0016] Update the noise network through the noise network loss value.

[0017] Optionally, obtaining the display screen image set, the dirty area set, and the filtered image set as the training sample set includes:

[0018] During the Demura process, perform image acquisition on the display screen to be processed to generate a display screen image set;

[0019] Perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set;

[0020] Perform dirt filtering on the display screen image set according to the dirt area set to generate a filtered image set;

[0021] Determine the display screen image set, the dirt area set, and the filtered image set as the training sample set.

[0022] Optionally, after performing image acquisition on the display screen to be processed during the Demura process to generate a display screen image set, and before performing threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirt area set, the method further includes:

[0023] Perform edge filtering on the display screen images in the display screen image set using the sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering.

[0024] Optionally, after obtaining the display screen image set, the dirt area set, and the filtered image set as the training sample set, and before inputting the training sample set into the encoder of the U-Net network model for feature encoding processing to generate downsampled display screen image features, dirt area features, and filtered image features, the method further includes:

[0025] Perform data augmentation processing on the images in the training sample set, and the data augmentation processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0026] Optionally, after constructing a weighted loss function through the dirt area features and the backward stochastic differential equation, and after calculating the noise network loss value through the fitting parameters and the weighted loss function, and before updating the noise network through the noise network loss value, the method further includes:

[0027] Generate a loss value correction parameter according to the envelope area, dirt category, and dirt area distribution of the current dirt area features, the dirt category includes dust on the film, dust under the film, and debris on the film, and the dirt area distribution includes a curved surface area and a flat surface area;

[0028] Correct the noise network loss value according to the loss value correction parameter.

[0029] The second aspect of this application discloses a device for training a display screen image dirt filtering model, including:

[0030] An acquisition unit, configured to acquire a display screen image set, a dirt area set, and a filtered image set as a training sample set, the display screen image set is several captured images of the display screen during the Demura process, the dirt area set includes the dirt areas of each display screen image in the display screen image set, and the filtered image set includes the images of each display screen image in the display screen image set after dirt filtering;

[0031] A construction unit for constructing a U-Net network model, the U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder and skip connections, and a loss function of the encoder and the encoder is set in the U-Net network model;

[0032] A processing unit for inputting a training sample set into the encoder of the U-Net network model for feature encoding processing to generate a plurality of downsampled display screen image features, dirty area features and filtered image features;

[0033] A noise addition unit for inputting the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval;

[0034] A first generation unit for inputting the noise state and the filtered image features into the noise network, and training the noise network to generate fitting parameters according to the noise state, the filtered image features and time parameters, and in combination with a weighted loss function;

[0035] A second generation unit for inputting the fitting parameters into the denoising module to generate denoised features, and then inputting the denoised features into the decoder to enhance the transmission of features through skip connections to generate repaired features;

[0036] A first update unit for calculating an encoder / decoder loss value through the loss function and the repaired features, and updating the encoder and the decoder through the encoder / decoder loss value;

[0037] A calculation unit for constructing a weighted loss function through the dirty area features and the backward stochastic differential equation, and calculating the noise network loss value through the fitting parameters and the weighted loss function;

[0038] A second update unit for updating the noise network through the noise network loss value.

[0039] Optionally, the acquisition unit includes:

[0040] A first generation module for performing image acquisition on the display screen to be processed during the Demura process to generate a display screen image set;

[0041] A second generation module for performing threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set;

[0042] A third generation module for performing dirty area filtering processing on the display screen image set according to the dirty area set to generate a filtered image set;

[0043] A determination module for determining the display screen image set, the dirty area set and the filtered image set as the training sample set.

[0044] Optionally, after the first generation module and before the second generation module, the device further includes:

[0045] A filtering module, configured to perform edge filtering on the display screen images in the display screen image set using the sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering.

[0046] Optionally, after the acquisition unit and before the processing unit, the device further includes:

[0047] An enhancement unit, configured to perform data enhancement processing on the images in the training sample set, and the data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0048] Optionally, after the calculation unit and before the second update unit, the device further includes:

[0049] A third generation unit, configured to generate a loss value correction parameter according to the envelope area, dirt class, and dirt area distribution of the current dirt area feature, where the dirt class includes dust on the film, dust under the film, and debris on the film, and the dirt area distribution includes a curved surface area and a flat surface area;

[0050] A compensation unit, configured to correct the noise network loss value according to the loss value correction parameter.

[0051] A third aspect of the present application provides a device for training a display screen image dirt filtering model, including:

[0052] A processor, a memory, an input / output unit, and a bus;

[0053] The processor is connected to the memory, the input / output unit, and the bus;

[0054] The memory stores a program, and the processor calls the program to execute the methods in the first aspect and any optional methods of the first aspect.

[0055] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, it executes the methods in the first aspect and any optional methods of the first aspect.

[0056] From the above technical solutions, it can be seen that the embodiments of the present application have the following advantages:

[0057] In this application, first, a display screen image set, a dirty area set, and a filtered image set are obtained as a training sample set. The display screen image set is several captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty removal. Next, a U-Net network model is constructed. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip connections. The loss functions of the encoder and the encoder are set in the U-Net network model. The training sample set is input into the encoder of the U-Net network model for feature encoding processing to generate several downsampled display screen image features, dirty area features, and filtered image features. The downsampling process by the encoder can reduce the resolution of the display screen image, reducing the training time and training cost. Then, the display screen image features are input into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval. The noise state and the filtered image features are input into the noise network. According to the noise state, the filtered image features, and the time parameter, and combined with the weighted loss function, the noise network is trained to generate fitting parameters. The fitting parameters are input into the denoising module to generate denoised features, and then the denoised features are input into the decoder. Through the skip connections, the transmission of features is enhanced to generate repaired features. The encoder / decoder loss value is calculated through the loss function and the repaired features, and the encoder and the decoder are updated through the encoder / decoder loss value. A weighted loss function is constructed through the dirty area features and the backward stochastic differential equation. The noise network loss value is calculated through the fitting parameters and the weighted loss function. The noise network is updated through the noise network loss value.

[0058] By pre-determining the dirty areas of the training samples and constructing the weighted loss function of the noise network accordingly, the loss function of the U-Net network model is weighted using the dirty areas as prior knowledge, enabling the U-Net network model to quickly notice the dirty areas. Even if the dirty areas are located in curved surface areas or the types of dirt increase, the U-Net network model can effectively train and detect the dirty areas and the corresponding types of dirt, improving the effect of dirty removal from the display screen images. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 It is a schematic diagram of an embodiment of the method for training the dirty removal model of the display screen image of the present application;

[0061] Figure 2 Schematic diagram of an embodiment of the method for determining the training sample set for this application;

[0062] Figure 3 Schematic diagram of an embodiment of the method for processing the display screen image set for this application;

[0063] Figure 4 Schematic diagram of an embodiment of the method for enhancing the training sample data for this application;

[0064] Figure 5 Schematic diagram of an embodiment of the method for generating the scratch defect area for this application;

[0065] Figure 6 Schematic diagram of an embodiment of the device for training the display screen image decontamination model for this application;

[0066] Figure 7 Schematic diagram of an embodiment of the device for training the display screen image decontamination model for this application;

[0067] Figure 8 Schematic diagram of the U-Net network model for this application. Detailed implementation manner

[0068] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of this application. However, those skilled in the art should clearly understand that this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of this application.

[0069] It should be understood that when used in the specification of this application and the appended claims, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0070] It should also be understood that the term " / and" as used in the specification of this application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0071] As used in the specification of this application and the appended claims, the term "if" may be construed, depending on the context, as "when", "once", "in response to determining", or "in response to detecting". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]".

[0072] In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0073] References to "one embodiment" or "some embodiments" or the like described in the specification of this application mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of this application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having", and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0074] In the prior art, the detection and recovery of dirt on the display screen usually rely on traditional image processing methods and image generation methods based on deep learning. The traditional image processing method relies on pre-set thresholds to process images, thereby determining the dirty area and eliminating it. However, the dirt determined by this type of method is not precise enough, and for some complex situations, the positioning effect and repair effect of the dirt are poor.

[0075] Nowadays, using deep learning models for detection and repair has greatly improved the efficiency and accuracy. However, currently, the pixel structure of the display screen is complex, and nowadays, there are curved surfaces on the display screen that can be bent and folded, and the types of dirt on the display screen are increasing continuously, so the dirt filtering effect on the display screen image will decrease.

[0076] Based on this, this application discloses a method, device, and storage medium for training a display screen image dirt filtering model to improve the dirt filtering effect of the display screen image.

[0077] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0078] The method of the present application can be applied to a server, a device, a terminal, or other devices with logical processing capabilities. In this regard, the present application does not make any limitations. For the convenience of description, the following will take the execution entity as a terminal as an example for description.

[0079] Please refer to Figure 1 , an embodiment of a method for training a display screen image decontamination model provided by the present application includes:

[0080] 101. Obtain a display screen image set, a dirty area set, and a filtered image set as a training sample set. The display screen image set is a set of captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty removal.

[0081] In this embodiment, first, during the Demura process of the existing process, image acquisition is performed for each display screen to be processed to generate a display screen image set. , and then the dirty areas on the display screen image are determined through the existing process.

[0082] In this embodiment, the types of dirt mainly include dust on the screen, dust under the screen, and so on.

[0083] In this embodiment, based on the display panel to be processed, an acquisition camera is used to take pictures of it to obtain the image data of the display panel, and a display screen image set is obtained. The display screen image .

[0084] Then, a dirty area set and a filtered image set are obtained. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty removal.

[0085] 102. Construct a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. The loss function of the encoder and the encoder is set in the U-Net network model.

[0086] In this embodiment, the terminal constructs a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip connections. In the U-Net network model, an encoder and a loss function of the encoder are set. Please refer to Figure 8 , Figure 8 which is a schematic diagram of the architecture of the U-Net network model.

[0087] 103. Input the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a number of downsampled display screen image features, dirty area features, and filtered image features;

[0088] The terminal inputs the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a number of downsampled display screen image features, dirty area features, and filtered image features. Specifically, when performing Demura processing, very high-resolution image data is usually generated, which will cause general diffusion models to be unable to be effectively trained and fitted, and significantly increase the running time of the model. To solve this problem, the terminal performs feature encoding on the image through the U-Net network model, so as to train the image in the low-resolution latent space. This method can not only effectively reduce the computational burden, but also accelerate the convergence process of the model.

[0089] The U-Net network model is a convolutional neural network for image segmentation, with a symmetric encoder and decoder structure, and enhanced feature transmission through skip connections. The constructed U-Net network model has an encoder E , a decoder D and skip connections, and also includes a noise addition module, a noise network, and a denoising module. The encoder is used to perform feature encoding on the image. The downsampled display screen image features, dirty area features, and filtered image features are respectively defined as F = E(I) , , , where I is the display screen image set, Y is the filtered image feature, is the dirty area feature, and E() represents the feature encoding operation of inputting the picture into the encoder and then outputting the image feature.

[0090] 104. Input the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval;

[0091] In this embodiment, the terminal inputs the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval.

[0092] Specifically, first define the stochastic differential equation: , where f(F, t) is the drift term, g(t) is the diffusion term, dw is a Wiener process that follows a normal distribution, t is the time. Set the time parameter T and discretize the stochastic differential equation to add random noise to the feature F:

[0093]

[0094] Through multiple iterations, the noise state F of the feature within the entire time interval [0, T] can be obtained .

[0095] 105. Input the noise state and the filtered image feature into the noise network. According to the noise state, the filtered image feature, and the time parameter, and in combination with the weighted loss function, train the noise network to generate fitting parameters;

[0096] The terminal inputs the noise state and the filtered image feature into the noise network. According to the noise state, the filtered image feature, and the time parameter, and in combination with the weighted loss function, train the noise network to generate fitting parameters.

[0097] Specifically, the terminal needs to construct a noise network to fit the unknown parameter , and according to the marginal distribution , where represents the expectation under the distribution, represents the true probability distribution of the feature F, represents the probability distribution at time t when the initial state is . Therefore, it can be concluded that:

[0098]

[0099] where is the clean image feature we need after denoising, that is, , where is with respect to the gradient.

[0100] Construct a noise network by combining the multi-layer perception mechanism and technology with the convolutional neural network t , input the time parameter and the noise state to fit , design a weighted loss function to optimize the noise network, and its weighted loss is:

[0101] ​

[0102] where is the expected value of the joint distribution of and . refers to the fitting result of the noise network input time parameter t and the noise state after that, * is the dot product. Using prior knowledge to weight the loss enables the model to quickly notice the dirty areas and the degree of dirtiness at the positions. This weighting strategy makes the model assign higher loss weights to the areas marked as dirty according to the degree of dirtiness during the training process, thus prompting it to give priority to focusing on and repairing these important areas during the learning process. This not only improves the learning efficiency of the model but also enhances its denoising ability in practical applications, ensuring the quality and accuracy of the restoration result.

[0103] 106. Input the fitting parameters into the denoising module to generate denoising features, and then input the denoising features into the decoder. Enhance the transmission of the features through skip connections to generate repaired features;

[0104] The terminal inputs the fitting parameters into the denoising module to generate denoising features, and then inputs the denoising features into the decoder. Enhance the transmission of the features through skip connections to generate repaired features.

[0105] The specific denoising process is defined as:

[0106]

[0107] After discretizing it, the process can be described as:

[0108]

[0109] Replace the noise network result into the denoising process where represents the discretized time step:

[0110]

[0111] Restore the noise through multiple iterations to obtain the repaired features .

[0112] The terminal obtains the repaired features and then uses the U-Net decoder D to decode the repaired features to restore them to the original size. Optimize the U-Net network model by introducing the reconstruction loss to ensure that the features can be restored to the original image with high quality. The form of the reconstruction loss function is:

[0113]

[0114] where, Represent the image features of the display screen after dimensionality reduction The input decoder performs downsampling and outputs the decoding result Represent the features of the dirty area after dimensionality reduction The input decoder performs downsampling and outputs the decoding result. N is the total number of pixels in the image. By minimizing this reconstruction loss, it is possible to effectively guide the U-Net network to learn how to accurately restore image features, thereby improving the restoration quality and ensuring the clarity and details of the final output image.

[0115] 107. Calculate the encoder / decoder loss value through the loss function and the repaired features, and update the encoder and decoder through the encoder / decoder loss value;

[0116] Subsequently, the terminal first calculates the encoder / decoder loss value through the loss function and the repaired features, and updates the weight parameters of the encoder and decoder through the encoder / decoder loss value.

[0117] 108. Construct a weighted loss function through the features of the dirty area and the reverse stochastic differential equation, and calculate the loss value of the noise network through the fitting parameters and the weighted loss function;

[0118] When the terminal subsequently first calculates the encoder / decoder loss value through the loss function and the repaired features, and updates the weight parameters of the encoder and decoder through the encoder / decoder loss value, the terminal constructs a weighted loss function through the features of the dirty area and the reverse stochastic differential equation, and calculates the loss value of the noise network through the fitting parameters and the weighted loss function.

[0119] 109. Update the noise network through the loss value of the noise network.

[0120] The terminal updates the weight parameters of the noise network through the loss value of the noise network.

[0121] The terminal trains the network using the training set and the validation set to ensure that the model can effectively fit the data. After the model training is completed, we use the test set to evaluate its image restoration performance to ensure that its performance on unseen data meets the expected standards. Once the model passes the test, we can directly apply it to the Demura process for dirt removal operations without complex parameter adjustment. The user only needs to input the image to be repaired, and the model can quickly generate high-quality repair results, simplifying the usage process and improving the convenience of the application.

[0122] In this application, first, a display screen image set, a dirty area set, and a filtered image set are obtained as a training sample set. The display screen image set is a number of captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty filtering. Next, a U-Net network model is constructed. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. The loss functions of the encoder and the encoder are set in the U-Net network model. The training sample set is input into the encoder of the U-Net network model for feature encoding processing to generate a number of downsampled display screen image features, dirty area features, and filtered image features. The downsampling process of the encoder can reduce the resolution of the display screen image, reducing the training time and training cost. Then, the display screen image features are input into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval. The noise state and the filtered image features are input into the noise network. According to the noise state, the filtered image features, and time parameters, and in combination with a weighted loss function, the noise network is trained to generate fitting parameters. The fitting parameters are input into the denoising module to generate denoised features, and then the denoised features are input into the decoder. Through skip layer connections, the transmission of features is enhanced to generate repaired features. The encoder / decoder loss value is calculated through the loss function and the repaired features, and the encoder and decoder are updated through the encoder / decoder loss value. A weighted loss function is constructed through the dirty area features and an inverse stochastic differential equation. The noise network loss value is calculated through the fitting parameters and the weighted loss function. The noise network is updated through the noise network loss value.

[0123] By pre-determining the dirty areas of the training samples and constructing a weighted loss function for the noise network with this, the loss function of the U-Net network model is weighted using the dirty areas as prior knowledge, enabling the U-Net network model to quickly notice the dirty areas. Even if the dirty areas are located in curved surface areas or the types of dirt increase, the U-Net network model can effectively train and detect the dirty areas and the corresponding dirt types, improving the effect of filtering dirt from the display screen images.

[0124] Please refer to Figure 2 , an embodiment of a method for determining a training sample set provided by this application includes:

[0125] 201. During the Demura process, image acquisition is performed on the display screen to be processed to generate a display screen image set;

[0126] 202. Threshold segmentation processing and normalization processing are performed on the display screen images in the display screen image set to generate a dirty area set;

[0127] In this embodiment, during the Demura process, the terminal captures images of the display screen to be processed, generates a set of display screen images, and then designs a grayscale threshold , performs threshold segmentation on the filtering result, then performs morphological opening operation to extract the rough position of the dust (initial positioning of the dirty area).

[0128] Normalize the rough position of the dust , and the specific method is as follows: First, calculate the pixel mean within the position .

[0129]

[0130] where W is the position coordinate range, N is the total number of pixels within the window, is the point coordinate. Then calculate its standard deviation .

[0131]

[0132] Finally, normalize the value of each pixel:

[0133]

[0134] where, is a small constant used to avoid division by zero, and its normalization result (set of dirty areas) can represent the degree of dirt on the display panel.

[0135] 203. Perform dirt filtering processing on the set of display screen images according to the set of dirty areas to generate a set of filtered images;

[0136] Next, the terminal performs dirt filtering processing on the set of display screen images according to the set of dirty areas to generate a set of filtered images, and can obtain the corresponding set of filtered dirt images (set of filtered images).

[0137] 204. Determine the set of display screen images, the set of dirty areas, and the set of filtered images as the training sample set.

[0138] Finally, determine the set of display screen images, the set of dirty areas, and the set of filtered images as the training sample set.

[0139] Please refer to Figure 3 , this application provides an embodiment of a method for processing a set of display screen images, including:

[0140] 301. Use the sobel operator to perform edge filtering on the display screen images in the set of display screen images to extract image gradient information, and use mean filtering to smooth the extracted edges.

[0141] The terminal uses the Sobel operator to perform edge filtering on the display screen images in the display screen image set to extract image gradient information, and uses mean filtering to smooth the extracted edges. Specifically, the terminal uses the Sobel operator to perform edge filtering on the image set to extract image gradient information, uses mean filtering to smooth the extracted edges to reduce noise, and designs a grayscale threshold , performs threshold segmentation on the filtering result, and then performs morphological opening operation to extract the rough position of the dust .

[0142] Please refer to Figure 4 , this application provides an embodiment of a method for enhancing training sample data, including:

[0143] 401. Perform data augmentation processing on the images in the training sample set. The data augmentation processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0144] The terminal performs data augmentation processing on the images in the training sample set. The data augmentation processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0145] The terminal first collects display panel image data containing dust and the corresponding image data after removing the dust , ensuring that the two are consistent in content and resolution. To enhance the diversity of the data set, we adopt data augmentation techniques such as random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping. Then, the terminal reasonably divides the entire data set into a training set (70%), a validation set (15%), and a test set (15%) to optimize the parameters and evaluate the performance of the model during the model training process. These steps help improve the accuracy and robustness of the model in the dust removal task.

[0146] The training set is used for model training, and the validation set is used to select the most accurate parameters during training to extract the model robustness. After training, the test set is used to detect whether the model meets the standard.

[0147] Please refer to Figure 5 , this application provides an embodiment of a method for generating a scratch defect area, including:

[0148] 501. Generate a loss value correction parameter according to the envelope area, dirt type, and dirt area distribution of the current dirt area characteristics. The dirt types include dust on the film, dust under the film, and debris on the film, and the dirt area distribution includes a curved surface area and a flat surface area;

[0149] 502. Calibrate the loss value of the noise network according to the loss value calibration parameter.

[0150] Generally, there is dirt on the surface of the display screen. However, in this application, the types of dirt mainly include dust on the film and dust in the interlayer. Secondly, there is also a special type of dirt. In the prior art, in order to reduce the collision and abrasion caused by the grasping device during the movement of the display screen, a thin protective film is pasted on the surface of the display screen, which can prevent scratches on the surface of the display screen during the detection process. However, the grasping device will inevitably collide and rub against the protective film, thereby generating some film fragment dirt that is not large in volume but affects defect detection. Compared with dust dirt, this type of film fragment dirt will affect the reflection of light. In the area of film fragment dirt, a part of the light source will be refracted and deviated from the acquisition device, thereby affecting defect detection and increasing the possibility of passing inspection.

[0151] In this embodiment, although weighting is performed through the defect area, the U-Net network model has a good repair effect on the dirt in a specific area. However, due to the presence of the protective film, a new type of dirt is introduced. Therefore, the terminal first determines the envelope area, dirt category, and dirt area distribution of the current dirt area characteristics, and calibrates the loss value through different dirt data.

[0152] First, divide the display screen into a curved surface area and a flat surface area, and respectively count the minimum envelope area of dust on the film, dust in the interlayer, and film fragment dirt on the curved surface area, denoted as 、 and , and count the envelope area of dust on the film, dust in the interlayer, and film fragment dirt on the flat surface area, denoted as 、 and . The repair difficulty of dust on the film, dust in the interlayer, and film fragment dirt on the flat surface area gradually increases, and the repair difficulty of dust on the film, dust in the interlayer, and film fragment dirt on the curved surface area also gradually increases. When setting up the model, repair expectations are set for these dirt types. 、 and The repair expectations of are 、 and respectively, from high to low in sequence, and all are values greater than 0 and less than 1. 、 and The repair expectations of are 、 and respectively, from high to low in sequence, and all are values greater than 0 and less than 1.

[0153] The terminal generates a loss value correction parameter according to the envelope area, dirt type, and dirt area distribution of the current dirt area feature, and the formula is as follows:

[0154]

[0155] Then, according to the loss value correction parameter correct the noise network loss value and multiply it by the weighted loss to correct the loss value. This method corrects the loss value through multiple dirt types, envelope area, and repair expectations. When updating the weights using the loss value later, it can be more accurate, and the detection of several types of defects such as dust on the film, dust in the interlayer, and broken film dirt can be more accurate.

[0156] Please refer to Figure 6 , this application provides an embodiment of a device for training a display screen image dirt filtering model, including:

[0157] An acquisition unit 601, configured to acquire a display screen image set, a dirt area set, and a filtered image set as a training sample set. The display screen image set is several captured images of the display screen during the Demura process. The dirt area set includes the dirt areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirt filtering;

[0158] Optionally, the acquisition unit 601 includes:

[0159] A first generation module 6011, configured to perform image acquisition on the display screen to be processed during the Demura process to generate a display screen image set;

[0160] A filtering module 6012, configured to perform edge filtering on the display screen images in the display screen image set using a sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering;

[0161] A second generation module 6013, configured to perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirt area set;

[0162] A third generation module 6014, configured to perform dirt filtering processing on the display screen image set according to the dirt area set to generate a filtered image set;

[0163] A determination module 6015, configured to determine the display screen image set, the dirt area set, and the filtered image set as a training sample set.

[0164] An enhancement unit 602, configured to perform data enhancement processing on the images in the training sample set. The data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing;

[0165] A construction unit 603 for constructing a U-Net network model, where the U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip connections. An encoder and a loss function of the encoder are set in the U-Net network model;

[0166] A processing unit 604 for inputting a training sample set into the encoder of the U-Net network model for feature encoding processing to generate a plurality of downsampled display screen image features, dirty area features, and filtered image features;

[0167] A noise addition unit 605 for inputting the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval;

[0168] A first generation unit 606 for inputting the noise state and the filtered image features into the noise network, and training the noise network to generate fitting parameters according to the noise state, the filtered image features, and time parameters, in combination with a weighted loss function;

[0169] A second generation unit 607 for inputting the fitting parameters into the denoising module to generate denoised features, and then inputting the denoised features into the decoder to enhance the transmission of features through skip connections to generate repaired features;

[0170] A first update unit 608 for calculating an encoder / decoder loss value through the loss function and the repaired features, and updating the encoder and the decoder through the encoder / decoder loss value;

[0171] A calculation unit 609 for constructing a weighted loss function through the dirty area features and the backward stochastic differential equation, and calculating a noise network loss value through the fitting parameters and the weighted loss function;

[0172] A second update unit 610 for updating the noise network through the noise network loss value;

[0173] A third generation unit 611 for generating a loss value correction parameter according to the envelope area, the dirt category, and the dirt area distribution of the current dirty area features, where the dirt category includes dust on the film, dust under the film, and debris on the film, and the dirt area distribution includes a curved surface area and a flat surface area;

[0174] A compensation unit 612 for correcting the noise network loss value according to the loss value correction parameter.

[0175] Please refer to Figure 7 , this application provides a device for training a display screen image dirt filtering model, including:

[0176] A processor 701, a memory 702, an input / output unit 703, and a bus 704.

[0177] The processor 701 is connected to the memory 702, the input / output unit 703, and the bus 704.

[0178] The memory 702 stores a program, and the processor 701 calls the program to execute methods such as Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 in it.

[0179] This application provides a computer-readable storage medium with a program stored thereon. When the program is executed on a computer, it executes methods such as Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 in it.

[0180] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0181] In several embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, in each embodiment of this application, the functional units can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0184] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs.

Claims

1. A method for training a display screen image pollution filtering model, characterized in that, Including: Obtain a display screen image set, a dirty area set, and a filtered image set as a training sample set. The display screen image set is several captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty area filtering; Construct a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip connections. The loss function of the encoder and the encoder is set in the U-Net network model; Input the training sample set into the encoder of the U-Net network model for feature encoding processing to generate several dimensionality-reduced display screen image features, dirty area features, and filtered image features; Input the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval; Input the noise state and the filtered image features into the noise network. According to the noise state, the filtered image features, and time parameters, and in combination with a weighted loss function, train the noise network to generate fitting parameters; Input the fitting parameters into the denoising module to generate denoised features, and then input the denoised features into the decoder. Enhance the transmission of features through the skip connections to generate repaired features; Calculate the encoder / decoder loss value through the loss function and the repaired features, and update the encoder and the decoder through the encoder / decoder loss value; Construct a weighted loss function through the dirty area features and an inverse stochastic differential equation, and calculate the noise network loss value through the fitting parameters and the weighted loss function; Update the noise network through the noise network loss value.

2. The method according to claim 1, characterized in that Obtaining a display screen image set, a dirty area set, and a filtered image set as a training sample set includes: During the Demura process, perform image acquisition on the display screen to be processed to generate a display screen image set; Perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set; Perform dirty area filtering processing on the display screen image set according to the dirty area set to generate a filtered image set; Determine the display screen image set, the dirty area set, and the filtered image set as the training sample set.

3. The method according to claim 2, wherein Before performing threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set after image acquisition of the display screen to be processed during the Demura process to generate a display screen image set, the method further includes: Perform edge filtering on the display screen images in the display screen image set using a sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering.

4. The method according to claim 3, characterized in that, Before obtaining the display screen image set, the dirty area set, and the filtered image set as the training sample set and inputting the training sample set into the encoder of the U-Net network model for feature encoding processing to generate the dimension-reduced display screen image features, dirty area features, and filtered image features, the method further includes: Performing data augmentation processing on the images in the training sample set, where the data augmentation processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

5. The method according to any one of claims 1 to 4, characterized in that Before constructing a weighted loss function through the dirty area features and the reverse stochastic differential equation, calculating the noise network loss value through the fitting parameter and the weighted loss function, and then updating the noise network with the noise network loss value, the method further includes: Generating a loss value correction parameter according to the envelope area, dirty category, and dirty area distribution of the current dirty area features, where the dirty category includes dust on the film, dust under the film, and debris on the film, and the dirty area distribution includes a curved surface area and a flat surface area; Correcting the noise network loss value according to the loss value correction parameter.

6. An apparatus for training a display screen image pollution filtering model, characterized in that, Including: An acquisition unit for acquiring a display screen image set, a dirty area set, and a filtered image set as the training sample set, where the display screen image set is a plurality of captured images of the display screen during the Demura process, the dirty area set includes the dirty areas of each display screen image in the display screen image set, and the filtered image set includes the images of each display screen image in the display screen image set after dirty filtering; A construction unit for constructing a U-Net network model, where the U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and a skip connection, and the loss function of the encoder and the encoder is set in the U-Net network model; A processing unit for inputting the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a plurality of dimension-reduced display screen image features, dirty area features, and filtered image features; A noise addition unit for inputting the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval; A first generation unit for inputting the noise state and the filtered image features into the noise network, training the noise network according to the noise state, the filtered image features, and the time parameter, and combining with the weighted loss function to generate a fitting parameter; A second generation unit for inputting the fitting parameter into the denoising module to generate denoised features, and then inputting the denoised features into the decoder, and enhancing the transmission of features through the skip connection to generate repaired features; A first update unit for calculating the encoder / decoder loss value through the loss function and the repaired features, and updating the encoder and the decoder with the encoder / decoder loss value; A calculation unit for constructing a weighted loss function through the dirty area features and the reverse stochastic differential equation, and calculating the noise network loss value through the fitting parameter and the weighted loss function; A second update unit, configured to update the noise network according to the noise network loss value.

7. The device according to claim 6, characterized in that, An acquisition unit, comprising: A first generation module, configured to perform image acquisition on a display screen to be processed during a Demura process, and generate a display screen image set; A second generation module, configured to perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set, and generate a set of dirty regions; A third generation module, configured to perform dirty region filtering processing on the display screen image set according to the set of dirty regions, and generate a filtered image set; A determination module, configured to determine the display screen image set, the set of dirty regions, and the filtered image set as a training sample set.

8. The device according to claim 7, wherein After the first generation module and before the second generation module, the apparatus further comprises: A filtering module, configured to perform edge filtering on the display screen images in the display screen image set by using a sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges by using mean filtering.

9. An apparatus for training a display screen image pollution filtering model, characterized in that, Comprising: A processor, a memory, an input / output unit, and a bus, where the processor is connected to the memory, the input / output unit, and the bus, the memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, A program is stored on the computer-readable storage medium, and when the program is executed on a computer, it executes the method according to any one of claims 1 to 5.

Images