Display screen image pollution filtering model training method and device and storage medium

Abstract

The invention discloses a display screen image pollution filtering model training method and device and a storage medium. The method and device are used for improving the display screen image pollution filtering effect. Obtaining a display screen image set, a dirty area set and a filtered image set as a training sample set; a U-Net network model is constructed; performing feature coding processing on the training sample set; noise addition is carried out on the image features of the display screen, and a noise state in a corresponding time interval is generated; according to the noise state, image features and time parameters are filtered out, and a weighted loss function is combined to train a noise network to generate fitting parameters; denoising features are generated according to the fitting parameters, then the denoising features are input into a decoder, transmission of the features is enhanced through a skip link, and repairing features are generated; calculating the loss value of the encoder / decoder through the loss function and the repair feature, and updating the encoder and the decoder through the loss value of the encoder / decoder; and calculating a noise network loss value through the fitting parameter and the weighted loss function, and updating the noise network through the noise network loss value.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of display screen dirt removal, and in particular, to a method, an apparatus, and a storage medium for training a display screen image dirt removal model. Background Art

[0002] With the update and iteration of electronic products, the display screen is used more frequently, and consumers have higher requirements for the quality of products. This has led to an increasing amount of detection content for the display screen, and the detection accuracy is also constantly rising.

[0003] As the application fields of electronic devices are more and more, and the application scenarios are more and more complex, the precision of the display screen is getting higher and higher, and the shape is also ever-changing. In the field of display screen manufacturing, due to factors such as manufacturing processes, materials, and environments, the phenomenon of uneven brightness or color, specifically in the display screen field, is called the mura defect. In order to reduce the mura defect, as a traditional mura defect elimination method, the Demura process has become a key process step in the display screen production and manufacturing process, aiming to correct the brightness and chromaticity uniformity of each pixel point on the display screen, so as to ensure the accuracy and consistency of the displayed image. By effectively implementing the Demura process, the overall uniformity of the corrected panel can be significantly improved, thereby increasing the product yield rate and panel grade. However, since the Demura process relies on a camera to capture the panel to obtain brightness data, if there are interference objects such as stains, dust, and protective film fragments on the display panel, collectively referred to as dirt interference, these interference objects will be misinterpreted by the algorithm as mura. If these interference objects are not specially processed, it may ultimately lead to over-compensation or mis-compensation phenomena at the corresponding positions on the panel after Demura. This not only affects the display effect but also may reduce the product quality.

[0004] The detection and restoration of dirt on the display screen usually rely on traditional image processing methods and image generation methods based on deep learning. Traditional image processing methods rely on pre-set thresholds to process images, thereby determining the dirt area and eliminating it. However, the dirt determined by this type of method is not precise enough, and for some complex situations, the dirt positioning effect and repair effect are poor.

[0005] Nowadays, using deep learning models for detection and repair has greatly improved the efficiency and accuracy. However, currently, the pixel structure of the display screen is complex, and nowadays, there are curved and foldable display screens, and the types of display screen dirt are constantly increasing, so the dirt removal effect on the display screen image will decline. Summary of the Invention

[0006] The present application discloses a method, an apparatus, and a storage medium for training a display screen image dirt removal model, which are used to improve the dirt removal effect of the display screen image.

[0007] The first aspect of the present application discloses a method for training a display screen image decontamination model, including: Obtain a display screen image set, a dirty area set, and a filtered image set as a training sample set. The display screen image set is a plurality of captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty decontamination; Construct a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip connections. The loss function of the encoder and the encoder is set in the U-Net network model; Input the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a plurality of dimensionality-reduced display screen image features, dirty area features, and filtered image features; Input the display screen image features into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval; Input the noise state and the filtered image features into the noise network, and train the noise network to generate fitting parameters according to the noise state, the filtered image features, and time parameters, and in combination with a weighted loss function; Input the fitting parameters into the denoising module to generate denoised features, and then input the denoised features into the decoder to enhance the transmission of features through skip connections to generate repaired features; Calculate the encoder / decoder loss value through the loss function and the repaired features, and update the encoder and the decoder through the encoder / decoder loss value; Construct a weighted loss function through the dirty area features and the backward stochastic differential equation, and calculate the noise network loss value through the fitting parameters and the weighted loss function; Update the noise network through the noise network loss value.

[0008] Optionally, obtaining a display screen image set, a dirty area set, and a filtered image set as a training sample set includes: During the Demura process, perform image acquisition on the display screen to be processed to generate a display screen image set; Perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set; Perform dirty decontamination processing on the display screen image set according to the dirty area set to generate a filtered image set; Determine the display screen image set, the dirty area set, and the filtered image set as the training sample set.

[0009] Optionally, after performing image acquisition on the display screen to be processed during the Demura process and generating a display screen image set, before performing threshold segmentation processing and normalization processing on the display screen images in the display screen image set and generating a set of stain area captures, the method further includes: Perform edge filtering on the display screen images in the display screen image set using the sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering.

[0010] Optionally, after obtaining the display screen image set, the set of stain areas, and the filtered image set as a training sample set, before inputting the training sample set into the encoder of the U-Net network model for feature encoding processing to generate reduced-dimensional display screen image features, stain area features, and filtered image features, the method further includes: Perform data augmentation processing on the images in the training sample set. The data augmentation processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0011] Optionally, after constructing a weighted loss function through the stain area features and the backward stochastic differential equation, and calculating the noise network loss value through the fitting parameters and the weighted loss function, before updating the noise network through the noise network loss value, the method further includes: Generate a loss value correction parameter according to the envelope area, stain category, and stain area distribution of the current stain area features. The stain categories include dust on the film, dust under the film, and debris on the film, and the stain area distributions include curved surface areas and flat surface areas; Correct the noise network loss value according to the loss value correction parameter.

[0012] The second aspect of the present application discloses an apparatus for training a display screen image stain removal model, including: An acquisition unit for acquiring a display screen image set, a set of stain areas, and a filtered image set as a training sample set. The display screen image set is a plurality of captured images of the display screen during the Demura process. The set of stain areas includes the stain areas of each display screen image in the display screen image set, and the filtered image set includes the images of each display screen image in the display screen image set after stain removal; A construction unit for constructing a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. The loss function of the encoder and the encoder is set in the U-Net network model; A processing unit for inputting the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a plurality of reduced-dimensional display screen image features, stain area features, and filtered image features; A noise adding unit, configured to input the display screen image features into a noise adding module, add noise to the display screen image features, and generate a noise state within a corresponding time interval; A first generating unit, configured to input the noise state and the filtered image features into a noise network, and train the noise network to generate fitting parameters according to the noise state, the filtered image features, and time parameters, in combination with a weighted loss function; A second generating unit, configured to input the fitting parameters into a denoising module to generate denoised features, and then input the denoised features into a decoder, and enhance the transmission of the features through skip connections to generate repaired features; A first updating unit, configured to calculate an encoder / decoder loss value through a loss function and the repaired features, and update the encoder and the decoder through the encoder / decoder loss value; A calculating unit, configured to construct a weighted loss function through the dirty area features and an inverse stochastic differential equation, and calculate a noise network loss value through the fitting parameters and the weighted loss function; A second updating unit, configured to update the noise network through the noise network loss value.

[0013] Optionally, the obtaining unit includes: A first generating module, configured to perform image acquisition on the display screen to be processed during the Demura process to generate a display screen image set; A second generating module, configured to perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set; A third generating module, configured to perform dirty area filtering processing on the display screen image set according to the dirty area set to generate a filtered image set; A determining module, configured to determine the display screen image set, the dirty area set, and the filtered image set as a training sample set.

[0014] Optionally, after the first generating module and before the second generating module, the apparatus further includes: A filtering module, configured to perform edge filtering on the display screen images in the display screen image set using a sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering.

[0015] Optionally, after the obtaining unit and before the processing unit, the apparatus further includes: An enhancing unit, configured to perform data enhancement processing on the images in the training sample set, and the data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0016] Optionally, after the calculating unit and before the second updating unit, the apparatus further includes: A third generation unit, configured to generate a loss value correction parameter according to the area of the envelope region of the current soiled area feature, the soiled category, and the soiled area distribution, where the soiled category includes dust on the film, dust under the film, and debris on the film, and the soiled area distribution includes a curved surface area and a flat surface area; A compensation unit, configured to correct the noise network loss value according to the loss value correction parameter.

[0017] A third aspect of the present application provides an apparatus for training a display screen image anti-soiling model, including: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the methods as described in the first aspect and any optional methods of the first aspect.

[0018] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, it executes the methods as described in the first aspect and any optional methods of the first aspect.

[0019] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages: In this application, first, a display screen image set, a dirty area set, and a filtered image set are obtained as a training sample set. The display screen image set is a number of captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty removal. Next, a U-Net network model is constructed. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. The loss functions of the encoder and the encoder are set in the U-Net network model. The training sample set is input into the encoder of the U-Net network model for feature encoding processing to generate a number of downsampled display screen image features, dirty area features, and filtered image features. The downsampling process of the encoder can reduce the resolution of the display screen image, reducing the training time and training cost. Then, the display screen image features are input into the noise addition module to add noise to the display screen image features to generate a noise state within a corresponding time interval. The noise state and the filtered image features are input into the noise network. According to the noise state, the filtered image features, and the time parameters, and combined with the weighted loss function, the noise network is trained to generate fitting parameters. The fitting parameters are input into the denoising module to generate denoised features, and then the denoised features are input into the decoder. Through the skip layer connections, the transmission of features is enhanced to generate repaired features. The encoder / decoder loss value is calculated through the loss function and the repaired features, and the encoder and the decoder are updated through the encoder / decoder loss value. A weighted loss function is constructed through the dirty area features and the backward stochastic differential equation, and the noise network loss value is calculated through the fitting parameters and the weighted loss function. The noise network is updated through the noise network loss value.

[0020] By pre-determining the dirty areas of the training samples and constructing the weighted loss function of the noise network with this, the loss function of the U-Net network model is weighted using the dirty areas as prior knowledge, enabling the U-Net network model to quickly notice the dirty areas. Even if the dirty areas are located in curved surface areas or the types of dirt increase, the U-Net network model can effectively train and detect the dirty areas and the corresponding dirt types, improving the effect of dirty removal from the display screen images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 Schematic diagram of an embodiment of the method for training the display screen image dirt removal model of the present application; Figure 2 Schematic diagram of an embodiment of the method for determining the training sample set in this application; Figure 3 Schematic diagram of an embodiment of the method for processing the display screen image set in this application; Figure 4 Schematic diagram of an embodiment of the method for enhancing the training sample data in this application; Figure 5 Schematic diagram of an embodiment of the method for generating the scratch defect area in this application; Figure 6 Schematic diagram of an embodiment of the device for training the display screen image decontamination model in this application; Figure 7 Schematic diagram of an embodiment of the device for training the display screen image decontamination model in this application; Figure 8 Schematic diagram of the U-Net network model in this application. Detailed implementation manners

[0023] In the following description, for the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to thoroughly understand the embodiments of this application. However, those skilled in the art should clearly understand that this application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of this application.

[0024] It should be understood that when used in the specification and appended claims of this application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0025] It should also be understood that the term "and / or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when", "once", "in response to determining", or "in response to detecting" according to the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined", "in response to determining", "once [the described condition or event] is detected", or "in response to detecting [the described condition or event]" according to the context.

[0027] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", etc. are only used for differential description and should not be construed as indicating or implying relative importance.

[0028] The reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that in one or more embodiments of the present application, specific features, structures or characteristics described in connection with that embodiment are included. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0029] In the prior art, the detection and recovery of dirt on a display screen usually rely on traditional image processing methods and image generation methods based on deep learning. Traditional image processing methods rely on pre-set thresholds to process images, so as to determine the dirty areas and eliminate them. However, the dirt determined by such methods is not precise enough, and for some complex situations, the positioning effect and repair effect of the dirt are poor.

[0030] Nowadays, using deep learning models for detection and repair has greatly improved the efficiency and accuracy. However, currently the pixel structure of the display screen is complex, and nowadays there are curved surfaces on the display screen that can be bent and folded, and the types of dirt on the display screen are increasing continuously, so the dirt filtering effect on the display screen image will decrease.

[0031] Based on this, the present application discloses a method, device and storage medium for training a display screen image dirt filtering model, which is used to improve the dirt filtering effect of the display screen image.

[0032] Next, the technical solutions in the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0033] The method of the present application can be applied to a server, a device, a terminal or other devices with logical processing capabilities. In this regard, the present application makes no limitation. For the sake of convenience of description, the following will take the execution subject as the terminal for description.

[0034] Please refer to Figure 1, an embodiment of a method for training a display screen image pollution filtering model provided by this application includes: 101. Obtain a display screen image set, a dirty area set, and a filtered image set as a training sample set. The display screen image set is a set of several captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty pollution filtering; In this embodiment, first, during the Demura process of the existing process, image acquisition is performed for each display screen to be processed to generate a display screen image set , and then the dirty areas on the display screen images are determined through the existing process.

[0035] In this embodiment, the types of dirt mainly include dust on the screen, dust under the screen, and so on.

[0036] In this embodiment, based on the display panel to be processed, an acquisition camera is used to take pictures of it to obtain the image data of the display panel, and a display screen image set is obtained. The display screen image .

[0037] Then, the dirty area set and the filtered image set are obtained. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty pollution filtering.

[0038] 102. Construct a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. An encoder and a loss function of the encoder are set in the U-Net network model; In this embodiment, the terminal constructs a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. An encoder and a loss function of the encoder are set in the U-Net network model. Please refer to Figure 8 , Figure 8 as the architecture schematic diagram of the U-Net network model.

[0039] 103. Input the training sample set into the encoder of the U-Net network model for feature encoding processing to generate several downsampled display screen image features, dirty area features, and filtered image features; The terminal inputs the training sample set into the encoder of the U-Net network model for feature encoding processing, generating several downsampled display screen image features, dirt area features, and filtered image features. Specifically, when performing Demura processing, extremely high-resolution image data is usually generated, which causes general diffusion models to be unable to effectively train and fit, and significantly increases the running time of the model. To solve this problem, the terminal performs feature encoding on the image through the U-Net network model, thereby training the image in the latent space of low resolution. This method can not only effectively reduce the computational burden but also accelerate the convergence process of the model.

[0040] The U-Net network model is a convolutional neural network for image segmentation, with a symmetric encoder and decoder structure, and skip connections to enhance feature transmission. The constructed U-Net network model has an encoder E , a decoder D , and skip connections, and also includes a noise addition module, a noise network, and a denoising module. The encoder is used to perform feature encoding on the image, and the downsampled display screen image features, dirt area features, and filtered image features are respectively defined as F = E(I) , , , where I is the display screen image set, Y is the filtered image feature, is the dirt area feature, and E() represents the feature encoding operation of inputting the picture into the encoder and then outputting the image feature.

[0041] 104. Input the display screen image feature into the noise addition module, add noise to the display screen image feature, and generate the noise state within the corresponding time interval; In this embodiment, the terminal inputs the display screen image feature into the noise addition module, adds noise to the display screen image feature, and generates the noise state within the corresponding time interval.

[0042] Specifically, first define the stochastic differential equation: , where f(F, t) is the drift term, g(t) is the diffusion term, dw is the Wiener process subject to the normal distribution, t is the time. Set the time parameter T, and discretize the stochastic differential equation to add random noise to the feature F:

[0043] Through multiple iterations, the noise state of the feature F within the entire time interval [0, T] can be obtained .

[0044] 105. Input the noise state and the filtered image features into the noise network. According to the noise state, the filtered image features, and the time parameter, and in combination with the weighted loss function, train the noise network to generate fitting parameters. The terminal inputs the noise state and the filtered image features into the noise network. According to the noise state, the filtered image features, and the time parameter, and in combination with the weighted loss function, train the noise network to generate fitting parameters.

[0045] Specifically, the terminal needs to construct a noise network to fit the unknown parameters , and according to the marginal distribution , where represents the expectation under the distribution, represents the true probability distribution of the feature F, represents that in the case where the initial state is , at time t the probability distribution. Therefore, it can be obtained that:

[0046] where is the clean image feature we need after denoising, that is, , where, is the gradient with respect to .

[0047] Combine the multi-layer perception mechanism and technology with the convolutional neural network to construct the noise network , input the time parameter t and the noise state to fit , design a weighted loss function to optimize the noise network, and its weighted loss is:

[0048] where is the joint distribution expectation of the distributions and , refers to the fitting result after the noise network inputs the time parameter t and the noise state , and * is the dot product. Use prior knowledge to weight the loss so that the model can quickly notice the dirty areas and the degree of dirtiness at the positions. This weighting strategy makes the model give higher loss weights to the areas marked as dirty according to the degree of dirtiness during the training process, thereby prompting it to give priority to focusing on and repairing these important areas during the learning process. This not only improves the learning efficiency of the model but also enhances its denoising ability in practical applications, ensuring the quality and accuracy of the restoration result.

[0049] ​106. Input the fitting parameters into the denoising module to generate denoised features, and then input the denoised features into the decoder. Enhance the feature transmission through skip connections to generate restored features. The terminal inputs the fitting parameters into the denoising module to generate denoised features, and then inputs the denoised features into the decoder. Enhance the feature transmission through skip connections to generate restored features.

[0050] The specific denoising process is defined as:

[0051] After discretizing it, the process can be described as:

[0052] Replace the result of the noise network into the denoising process, where represents the discretized time step:

[0053] Restore the noise through multiple iterations to obtain the restored features .

[0054] The terminal obtains the restored features After that, use the U-Net decoder D to decode the restored features to restore them to the original size. Optimize the U-Net network model by introducing the reconstruction loss to ensure that the features can be restored to the original image with high quality. The form of the reconstruction loss function is:

[0055] where, represents the downsampled display screen image features Input into the decoder for downsampling to output the decoded result, represents the downsampled dirty area features Input into the decoder for downsampling to output the decoded result. N is the total number of pixels in the image. By minimizing this reconstruction loss, it can effectively guide the U-Net network to learn how to accurately restore image features, thereby improving the restoration quality and ensuring the clarity and details of the final output image.

[0056] 107. Calculate the encoder / decoder loss value through the loss function and the restored features, and update the encoder and decoder through the encoder / decoder loss value; Subsequently, the terminal first calculates the encoder / decoder loss value through the loss function and the restored features, and updates the weight parameters of the encoder and decoder through the encoder / decoder loss value.

[0057] 108. Construct a weighted loss function through the dirty area features and the backward stochastic differential equation, and calculate the loss value of the noise network through the fitting parameters and the weighted loss function; After the terminal subsequently calculates the encoder / decoder loss value through the loss function and the repair features and updates the weight parameters of the encoder and the decoder through the encoder / decoder loss value, the terminal constructs a weighted loss function through the dirty area features and the backward stochastic differential equation, and calculates the loss value of the noise network through the fitting parameters and the weighted loss function.

[0058] 109. Update the noise network through the loss value of the noise network.

[0059] The terminal updates the weight parameters of the noise network through the loss value of the noise network.

[0060] The terminal trains the network using the training set and the validation set to ensure that the model can effectively fit the data. After the model training is completed, we use the test set to evaluate its image restoration performance to ensure that its performance on unseen data meets the expected standards. Once the model passes the test, we can directly apply it to the Demura process for dirt removal without complex parameter adjustment. The user only needs to input the image to be repaired, and the model can quickly generate high-quality repair results, simplifying the usage process and improving the convenience of the application.

[0061] In this application, first, a display screen image set, a dirty area set, and a filtered image set are obtained as a training sample set. The display screen image set is several captured images of the display screen during the Demura process. The dirty area set includes the dirty areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirty filtering. Next, a U-Net network model is constructed. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. The loss functions of the encoder and the encoder are set in the U-Net network model. The training sample set is input into the encoder of the U-Net network model for feature encoding processing to generate several downsampled display screen image features, dirty area features, and filtered image features. The downsampling process by the encoder can reduce the resolution of the display screen image, reducing the training time and training cost. Then, the display screen image features are input into the noise addition module to add noise to the display screen image features to generate the noise state within the corresponding time interval. The noise state and the filtered image features are input into the noise network. According to the noise state, the filtered image features, and the time parameter, and combined with the weighted loss function, the noise network is trained to generate fitting parameters. The fitting parameters are input into the denoising module to generate denoised features, and then the denoised features are input into the decoder. Through the skip layer connections, the transmission of features is enhanced to generate repaired features. The encoder / decoder loss value is calculated through the loss function and the repaired features, and the encoder and the decoder are updated through the encoder / decoder loss value. A weighted loss function is constructed through the dirty area features and the backward stochastic differential equation, and the noise network loss value is calculated through the fitting parameters and the weighted loss function. The noise network is updated through the noise network loss value.

[0062] By pre-determining the dirty areas of the training samples and constructing the weighted loss function of the noise network with this, the loss function of the U-Net network model is weighted using the dirty areas as prior knowledge, enabling the U-Net network model to quickly notice the dirty areas. Even if the dirty areas are located in the curved surface area or the types of dirt increase, the U-Net network model can effectively train and detect the dirty areas and the corresponding types of dirt, improving the effect of dirty filtering of the display screen image.

[0063] Please refer to Figure 2 , an embodiment of a method for determining a training sample set provided by this application includes: 201. During the Demura process, image acquisition is performed on the display screen to be processed to generate a display screen image set; 202. Threshold segmentation processing and normalization processing are performed on the display screen images in the display screen image set to generate a dirty area set; In this embodiment, the terminal performs image acquisition on the display screen to be processed during the Demura process to generate a display screen image set, and then designs a grayscale threshold Perform threshold segmentation on the filtering result, and then perform morphological opening operation to extract the rough position of the dust (Initial positioning of the dirty area).

[0064] For the rough position of the dust Perform normalization processing. The specific method is as follows: First, calculate the mean value of the pixels within the position .

[0065]

[0066] where W is the position coordinate range and N is the total number of pixels within the window, is the point coordinate. Then calculate its standard deviation .

[0067]

[0068] Finally, normalize the value of each pixel:

[0069] where, is a small constant used to avoid division by zero, and its normalization result (dirty area set) can represent the degree of dirt on the display panel.

[0070] 203. Perform dirt filtering processing on the display screen image set according to the dirty area set to generate a filtered image set; Next, the terminal performs dirt filtering processing on the display screen image set according to the dirty area set to generate a filtered image set, and can obtain the corresponding set of images with dirt filtered out (filtered image set).

[0071] 204. Determine the display screen image set, the dirty area set, and the filtered image set as the training sample set.

[0072] Finally, determine the display screen image set, the dirty area set, and the filtered image set as the training sample set.

[0073] Please refer to Figure 3 , this application provides an embodiment of a method for processing a display screen image set, including: 301. Use the sobel operator to perform edge filtering on the display screen images in the display screen image set to extract image gradient information, and use mean filtering to smooth the extracted edges.

[0074] The terminal uses the Sobel operator to perform edge filtering on the display screen images in the display screen image set to extract image gradient information, and uses mean filtering to smooth the extracted edges. Specifically, the terminal uses the Sobel operator to perform edge filtering on the image set to extract image gradient information, and uses mean filtering to smooth the extracted edges to reduce noise, and designs a grayscale threshold , performs threshold segmentation on the filtering result, and then performs morphological opening operation to extract the rough position of the dust .

[0075] Please refer to Figure 4 , this application provides an embodiment of a method for enhancing training sample data, including: 401. Perform data enhancement processing on the images in the training sample set. The data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0076] The terminal performs data enhancement processing on the images in the training sample set. The data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing.

[0077] The terminal first collects display panel image data containing dust and the corresponding image data after removing dust , ensuring that the two are consistent in content and resolution. To enhance the diversity of the data set, we adopt data enhancement techniques such as random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping. Then, the terminal reasonably divides the entire data set into a training set (70%), a validation set (15%), and a test set (15%) to optimize parameters and evaluate the performance of the model during the model training process. These steps help improve the accuracy and robustness of the model in the dust removal task.

[0078] The training set is used for model training, and the validation set is used to select the most accurate parameters during training to extract the robustness of the model. After training, the test set is used to detect whether the model meets the standard.

[0079] Please refer to Figure 5 , this application provides an embodiment of a method for generating a scratch defect area, including: 501. Generate a loss value correction parameter according to the envelope area, dirt category, and dirt area distribution of the current dirt area features. The dirt categories include dust on the film, dust under the film, and debris on the film. The dirt area distribution includes a curved surface area and a flat surface area; 502. Correct the noise network loss value according to the loss value correction parameter.

[0080] Generally, there is dirt on the surface of a display screen, but in this application, the types of dirt mainly include dust on the film and interlayer dust. Secondly, there is also a special kind of dirt. In the prior art, in order to reduce the collision and scratches caused by the grabbing device during the movement of the display screen, a thin protective film is affixed to the surface of the display screen. During the detection process, it can prevent scratches on the surface of the display screen. However, the grabbing device will inevitably collide and rub against the protective film, thereby generating some small-volume film fragments and dirt that affect defect detection. Compared with dust dirt, this type of film fragment dirt will affect the reflection of the light source. In the film fragment dirt area, a part of the light source will be refracted to avoid the acquisition equipment, thereby affecting defect detection and increasing the possibility of over-inspection.

[0081] In this embodiment, although the U-Net network model has a good repair effect on the dirt in a specific area by weighting the defect area, the presence of the protective film introduces a new type of dirt. In this regard, the terminal first determines the envelope area, dirt category and dirt area distribution of the current dirt area feature, and corrects the loss value through different dirt data.

[0082] First, the display screen is divided into a curved area and a flat area, and the minimum envelope area of ​​dust on the film, interlayer dust, and film fragments in the curved area is calculated respectively, and is set as , and , the enveloping area of ​​the dust on the membrane, interlayer dust and membrane fragments in the plane area is counted and set as , and The difficulty of repairing the dust, interlayer dust and membrane fragments on the flat surface area is gradually increasing, and the difficulty of repairing the dust, interlayer dust and membrane fragments on the curved surface area is also gradually increasing. When the model is established, the repair expectations for these contaminants are set. , and The repair expectations are , and , from high to low, all are values ​​greater than 0 and less than 1. , and The repair expectations are , and , from high to low, all are values ​​greater than 0 and less than 1.

[0083] The terminal generates the loss value correction parameter according to the envelope area, dirt category and dirt area distribution of the current dirty area characteristics. The formula is as follows:

[0084] Then, correct the parameters according to the loss value Correct the loss value of the noise network and multiply the weighted loss to correct the loss value. This method corrects the loss value through multiple dirt types, the area of the envelope region, and the repair expectation. When updating the weights using the loss value later, it can be more accurate, and the detection of defects such as dust on the film, dust in the interlayer, and broken film dirt can be more accurate.

[0085] Please refer to Figure 6 , this application provides an embodiment of a device for training a display screen image anti-fouling model, including: An acquisition unit 601, configured to acquire a display screen image set, a dirt area set, and a filtered image set as a training sample set. The display screen image set is a set of captured images of the display screen during the Demura process. The dirt area set includes the dirt areas of each display screen image in the display screen image set. The filtered image set includes the images of each display screen image in the display screen image set after dirt removal; Optionally, the acquisition unit 601 includes: A first generation module 6011, configured to perform image acquisition on the display screen to be processed during the Demura process to generate a display screen image set; A filtering module 6012, configured to perform edge filtering on the display screen images in the display screen image set using a sobel operator to extract image gradient information, and perform smoothing processing on the extracted edges using mean filtering; A second generation module 6013, configured to perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirt area set; A third generation module 6014, configured to perform dirt removal processing on the display screen image set according to the dirt area set to generate a filtered image set; A determination module 6015, configured to determine the display screen image set, the dirt area set, and the filtered image set as a training sample set.

[0086] An enhancement unit 602, configured to perform data enhancement processing on the images in the training sample set. The data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment, and mirror flipping processing; A construction unit 603, configured to construct a U-Net network model. The U-Net network model includes an encoder, a noise addition module, a noise network, a denoising module, a decoder, and skip layer connections. The loss functions of the encoder and the encoder are set in the U-Net network model; A processing unit 604, configured to input the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a plurality of dimension-reduced display screen image features, dirt area features, and filtered image features; The noise adding unit 605 is configured to input the display screen image features into a noise adding module, add noise to the display screen image features, and generate a noise state within a corresponding time interval. The first generating unit 606 is configured to input the noise state and the filtered image features into a noise network, and train the noise network to generate fitting parameters according to the noise state, the filtered image features, and time parameters, in combination with a weighted loss function. The second generating unit 607 is configured to input the fitting parameters into a denoising module to generate denoised features, and then input the denoised features into a decoder, and enhance the transmission of the features through skip connections to generate repaired features. The first updating unit 608 is configured to calculate an encoder / decoder loss value through a loss function and the repaired features, and update the encoder and the decoder through the encoder / decoder loss value. The calculating unit 609 is configured to construct a weighted loss function through the dirty area features and an inverse stochastic differential equation, and calculate a noise network loss value through the fitting parameters and the weighted loss function. The second updating unit 610 is configured to update the noise network through the noise network loss value. The third generating unit 611 is configured to generate a loss value correction parameter according to the envelope area, the dirt category, and the dirt area distribution of the current dirty area features, where the dirt category includes dust on the film, dust under the film, and debris on the film, and the dirt area distribution includes a curved surface area and a flat surface area. The compensation unit 612 is configured to correct the noise network loss value according to the loss value correction parameter.

[0087] Please refer to Figure 7 , this application provides an apparatus for training a display screen image dirt filtering model, including: A processor 701, a memory 702, an input / output unit 703, and a bus 704.

[0088] The processor 701 is connected to the memory 702, the input / output unit 703, and the bus 704.

[0089] The memory 702 stores a program, and the processor 701 calls the program to execute the methods as described in Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5 .

[0090] This application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, it executes the methods as described in Figure 1 , Figure 2 and Figure 3 , Figure 4 , Figure 5The method in

[0091] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0092] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0094] In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0095] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

Claims

1. A method for training a display screen image pollution filtering model, characterized in that: include: Acquire a display screen image set, a dirty area set and a filtered image set as training sample sets, wherein the display screen image set is a plurality of images of the display screen taken during the demura process, the dirty area set includes the dirty area of ​​each display screen image in the display screen image set, and the filtered image set includes each display screen image in the display screen image set after the dirty area is filtered out; Constructing a U-Net network model, the U-Net network model includes an encoder, a noise adding module, a noise network, a noise removal module, a decoder and a skip link, and the U-Net network model is provided with the encoder and the loss function of the encoder; Inputting the training sample set into the encoder of the U-Net network model for feature encoding processing to generate a number of reduced-dimensional display screen image features, dirty area features, and filtered image features; Inputting the display screen image feature into the noise adding module, adding noise to the display screen image feature, and generating a noise state in a corresponding time interval; Inputting the noise state and the filtered image features into the noise network, and training the noise network to generate fitting parameters based on the noise state, the filtered image features and time parameters in combination with a weighted loss function; Input the fitting parameters into the denoising module to generate denoising features, then input the denoising features into the decoder, enhance the transmission of features through the skip-layer link, and generate repair features; Calculate a codec loss value by using the loss function and the restoration feature, and update the encoder and the decoder by using the codec loss value; Constructing a weighted loss function through the dirty area characteristics and the inverse stochastic differential equation, and calculating the noise network loss value through the fitting parameters and the weighted loss function; The noise network is updated by the noise network loss value.

2. The method according to claim 1, characterized in that Obtain a display screen image set, a dirty area set, and a filtered image set as a training sample set, including: In the Demura process, the image of the display screen to be processed is collected to generate a display screen image set; Performing threshold segmentation and normalization processing on the display screen images in the display screen image set to generate a dirty area set; Performing a dirt removal process on the display screen image set according to the dirt region set to generate a filtered image set; The display screen image set, the dirty region set and the filtered image set are determined as training sample sets.

3. The method according to claim 2, characterized in that In the Demura process, after collecting images of the display screen to be processed and generating a display screen image set, threshold segmentation and normalization processing are performed on the display screen images in the display screen image set, and before the dirty area set is captured, the method further includes: The display screen images in the display screen image set are subjected to edge filtering using a sobel operator to extract image gradient information, and the extracted edges are smoothed using a mean filter.

4. The method according to claim 3, characterized in that After obtaining the display screen image set, the dirty area set and the filtered image set as the training sample set, the training sample set is input into the encoder of the U-Net network model for feature encoding processing, and before generating the reduced-dimensional display screen image features, the dirty area features and the filtered image features, the method further includes: Data enhancement processing is performed on the images in the training sample set, and the data enhancement processing includes random rotation, random cropping, brightness adjustment, contrast adjustment and mirror flipping processing.

5. The method according to any one of claims 1 to 4, characterized in that After constructing a weighted loss function by using the dirty area features and the inverse stochastic differential equation, and calculating a noise network loss value by using the fitting parameters and the weighted loss function, and before updating the noise network by using the noise network loss value, the method further includes: Generate a loss value correction parameter according to the envelope area, dirt category and dirt area distribution of the current dirt area feature, wherein the dirt category includes dust on the film, dust under the film and debris on the film, and the dirt area distribution includes curved surface area and flat surface area; The noise network loss value is corrected according to the loss value correction parameter.

6. A device for training a display screen image pollution filtering model, characterized in that: include: an acquisition unit, configured to acquire a display screen image set, a dirty region set, and a filtered image set as training sample sets, wherein the display screen image set is a plurality of images of the display screen taken during the demura process, the dirty region set includes the dirty region of each display screen image in the display screen image set, and the filtered image set includes each display screen image in the display screen image set after the dirty region is filtered out; A construction unit, used to construct a U-Net network model, wherein the U-Net network model includes an encoder, a noise adding module, a noise network, a noise removing module, a decoder and a skip-layer link, and the U-Net network model is provided with the encoder and the loss function of the encoder; A processing unit, used for inputting the training sample set into the encoder of the U-Net network model for feature encoding processing, and generating a plurality of reduced-dimensional display screen image features, dirty area features and filtered image features; A noise adding unit, used for inputting the display screen image feature into the noise adding module, adding noise to the display screen image feature, and generating a noise state in a corresponding time interval; A first generating unit, configured to input the noise state and the filtered image feature into the noise network, and train the noise network to generate fitting parameters according to the noise state, the filtered image feature and a time parameter in combination with a weighted loss function; A second generating unit is used to input the fitting parameters into the denoising module to generate denoising features, and then input the denoising features into the decoder, enhance the transmission of features through the skip-layer link, and generate restoration features; A first updating unit, configured to calculate a codec loss value by using the loss function and the restoration feature, and update the coder and the decoder by using the codec loss value; A calculation unit, used to construct a weighted loss function through the dirty area characteristics and the inverse stochastic differential equation, and calculate the noise network loss value through the fitting parameters and the weighted loss function; The second updating unit is configured to update the noise network according to the noise network loss value.

7. The device according to claim 6, characterized in that Get unit, including: The first generation module is used to collect images of the display screen to be processed during the Demura process and generate a display screen image set; A second generating module is used to perform threshold segmentation processing and normalization processing on the display screen images in the display screen image set to generate a dirty area set; A third generating module, configured to perform a dirt removal process on the display screen image set according to the dirt region set to generate a filtered image set; A determination module is used to determine the display screen image set, the dirty area set and the filtered image set as a training sample set.

8. The device according to claim 7, characterized in that After the first generation module and before the second generation module, the device further includes: The filtering module is used to perform edge filtering on the display screen images in the display screen image set using the Sobel operator to extract image gradient information, and to smooth the extracted edges using the mean filter.

9. A device for training a display screen image pollution filtering model, characterized in that: include: A processor, a memory, an input-output unit and a bus, wherein the processor is connected to the memory, the input-output unit and the bus, the memory stores a program, and the processor calls the program to execute the method as claimed in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 5 is performed.

Images