A method and device for detecting a single classification anomaly of a display screen and a storage medium

Abstract

The application discloses a display screen single classification anomaly detection method and device and a storage medium, which are used for improving the accuracy of single classification detection and anomaly detection of a new display screen defect detection model. A single classification detection model and a single classification loss function are constructed; a training image is input into the single classification detection model; shallow feature enhancement processing is performed on the training image to generate a feature map; shallow features of the training image and the feature map are extracted, feature fusion is performed, and deep features are extracted; a binary classification prediction result is generated for the deep features; a first loss value is calculated, and the single classification detection model is updated according to the first loss value; a sub-class pseudo label is generated for each training image; a multi-classification head is expanded on the single classification detection model, and a multi-classification loss function and a sub-class classification target are constructed; and the single classification detection model is jointly trained through the deep features, the multi-classification head, the multi-classification loss function and the sub-class classification target until the single classification head reaches a training target.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of display screen detection, and more particularly to a method, device, and storage medium for detecting single-category anomalies in a display screen. Background Art

[0002] With the development of industry, surface defect and anomaly detection in display screens has become a hot research topic within the industry. Traditional surface anomaly detection methods rely on manual operations performed by qualified experts. However, this approach is not only inefficient but also highly dependent on the subjective judgment of the operator, making it difficult to guarantee detection accuracy. To this end, existing technologies incorporate automated anomaly detection equipment into industrial production lines to ensure product quality, reduce manual testing costs, and significantly improve production efficiency. With the rapid development of computer image processing technology, many advanced algorithms have been widely used in the field of display material anomaly detection, greatly improving detection accuracy. In recent years, deep learning has also been widely used in display material anomaly detection, with remarkable results.

[0003] With the advancement of display screens, new display screens are becoming increasingly functional and structurally complex. Examples include foldable, flexible, and tiled screens, as well as displays with multiple internal microcircuit layers. Furthermore, the current development of display screens favors the precise integration of multiple sophisticated structures, leading to increased complexity in defects found in these new displays. These defects are becoming more difficult to detect due to their signatures being less perceptible to the naked eye. Currently, deep learning-based algorithms have demonstrated superior performance compared to traditional methods in single-category visual inspection of defects in many new display screens. However, existing single-category visual inspection algorithms for defects in new display screens typically rely on supervised learning, which requires a large amount of labeled data. Due to the rapid evolution of new display screens and the increasing specificity of defects across different display types, the core training samples (anomaly samples) in existing training datasets are difficult to reuse. Furthermore, due to the difficulty in labeling, images of displays with new anomaly labels are often difficult to obtain, resulting in a limited supply of labeled anomaly or defect training data. This results in inefficient training of single-category anomaly detection models for new display screens and limits their application.

[0004] Secondly, in the field of single-class anomaly detection for new display screens, supervised anomaly detection methods are used. These supervised anomaly detection methods typically rely on training sets containing labeled data and are trained using classic machine learning models or deep neural networks (such as support vector machines, random forests, and XGBoost). By learning the characteristic differences between normal and abnormal data, single-class anomaly detection models can detect anomalies. The advantage of supervised methods is that when there is sufficient labeled data, the models can achieve high-accuracy anomaly detection. However, a drawback of single-class anomaly detection models for new display screens trained using supervised methods is that most current deep learning-based single-class detection methods directly utilize high-dimensional features from pre-trained networks (such as features extracted by ResNet). These high-dimensional features for new display screens are typically trained in multi-class scenarios and lack adaptability to the target single-class data. The lack of dedicated optimization for single-class abnormal data during feature extraction results in a dispersed distribution of features within the class. This in turn makes it difficult for single-class detection models for new display screens to effectively distinguish abnormal and normal samples in the feature space, reducing the accuracy of single-class and anomaly detection in new display screen defect detection models. Summary of the Invention

[0005] The present application discloses a method, device and storage medium for single-category anomaly detection of a display screen, which are used to improve the accuracy of single-category detection and anomaly detection of a new display screen defect detection model.

[0006] In a first aspect, an embodiment of the present application provides a method for single-class anomaly detection of a display screen, comprising: constructing a single-class detection model and a single-class loss function; obtaining a training image of a new display screen marked with a single-class label, and inputting the training image into the single-class detection model; performing shallow feature enhancement processing on the training image according to the single-class label to generate a feature map; extracting shallow features of the training image and the feature map respectively, performing feature fusion on the two sets of shallow features, and then extracting deep features, and generating a binary classification prediction result for the deep features according to the single-classification head; calculating a first loss value according to the single-classification loss function and the binary classification prediction result, and updating the single-classification detection model according to the first loss value until the initial training target is reached; clustering the deep features to generate a sub-class pseudo-label for each training image; extending a multi-classification head on the single-classification detection model, and constructing a multi-classification loss function and a sub-classification classification target; jointly training the feature extractor, two classification heads, and sub-class pseudo-labels in the single-classification detection model through the deep features, the multi-classification head, the multi-classification loss function, and the sub-classification target until the single-classification head reaches the training target.

[0007] Optionally, after the step of jointly training the feature extractor, two classification heads and subcategory pseudo labels in the single classification detection model through deep features, multi-classification heads, multi-classification loss functions and subcategory classification targets, the method also includes: removing the extended multi-classification head to obtain an unlabeled image in a new scene; copying the unlabeled image and perturbing the copied unlabeled image; performing shallow feature enhancement processing on the unlabeled image to generate a scene feature map; extracting shallow features of the unlabeled image, the perturbed unlabeled image and the scene feature map to generate a first feature, a second feature and a third feature respectively; fusing the first feature and the third feature to generate a first fused feature, and then performing deep feature extraction to generate a first deep feature; fusing the second feature and the third feature to generate a second fused feature, and then performing deep feature extraction to generate a second deep feature; inputting the first depth feature and the second depth feature into the single classification head to generate a conventional prediction result and a perturbed prediction result; constructing a consistency loss function to generate a consistency loss based on the conventional prediction result, the perturbed prediction result and the consistency loss function; and updating the parameters applicable to the new scene of the single classification detection model based on the consistency loss.

[0008] Optionally, after the step of updating the parameters of the single-category detection model applicable to the new scenario based on the consistency loss, the method further includes: when the conventional prediction results and the perturbation prediction results show that there are unlabeled images at the classification boundary, these unlabeled images are precisely labeled; the precisely labeled unlabeled images are determined as untrained images, and the untrained images are used to optimize the single-category detection model during the production process.

[0009] Optionally, the new display screen is a folding screen with a microcircuit layer in the folding area, the single classification detection model is a folding screen single classification detection model, and the unlabeled image is a real-time image of the folding area of ​​the folding screen on the production line; the steps of copying the unlabeled image and perturbation processing the copied unlabeled image include: copying the unlabeled image; introducing light source disturbance into the copied unlabeled image according to the curvature parameters of the folding screen in the folding area; and introducing noise disturbance to the disturbed unlabeled image by cropping and rotating.

[0010] Optionally, a training image of the new display screen marked with a single category label is obtained, and the training image is input into a single classification detection model, including: obtaining a captured image of the new display screen, and dividing the captured image into an abnormal image and a non-abnormal image; combining the abnormal image with the non-abnormal image, and expanding the data volume through image enhancement technology to generate a first abnormal expanded image; identifying a reference area in the non-abnormal image, replacing the reference area with a corresponding abnormal area in the abnormal image, and assimilating the replaced abnormal area to generate a second abnormal expanded image, where the reference area is an area where the authority influence weight reaches a preset value; performing a disturbance process on the non-abnormal image to generate a third abnormal expanded image; annotating the abnormal image, the non-abnormal image, the first abnormal expanded image, the second abnormal expanded image and the third abnormal expanded image with a single category label, and determining them as training images; and inputting the training image into the single classification detection model.

[0011] Optionally, the new display screen is a folding screen with a microcircuit layer in the folding area, the single-category detection model is a folding screen single-category detection model, the training image is a captured image of the folding area of ​​the folding screen, and the single-category label is a folding area single-category label; shallow feature enhancement processing is performed on the training image according to the single-category label to generate a feature map, including: determining the gain data of the training image as the folding screen curvature and the folding screen microcircuit reflection according to the folding area single-category label; determining the curvature parameters of the folding area of ​​the training image; setting the first folding screen and the second folding screen according to the curvature parameters, and projecting light of a preset brightness value to the first folding screen and the second folding screen through an external light source, the first folding screen is a folding screen without a microcircuit layer in the folding area, and the second folding screen is a folding screen with a microcircuit layer in the folding area; collecting images of the first folding screen and the second folding screen respectively, and calculating the microcircuit reflection parameters according to the folding areas of the two images; constructing a shallow feature gain convolution kernel according to the curvature parameters and the circuit reflection parameters; shallow feature enhancement processing is performed on the training image through the shallow feature gain convolution kernel to generate a feature map.

[0012] Optionally, the steps of extending a multi-classification head on the single-classification detection model and constructing a multi-classification loss function and a sub-category classification target include: extending a multi-classification head on the single-classification detection model for probability distribution detection of sub-category pseudo-labels; determining the grayscale discrimination weight value of each sub-category pseudo-label on the new display screen; generating a grayscale error weight value based on the curvature parameter and microcircuit reflection parameter of the new display screen; constructing a multi-classification cross entropy function based on the grayscale discrimination weight value and the grayscale error weight value; and constructing a sub-category classification target.

[0013] In the second aspect, an embodiment of the present application provides a device for single-category anomaly detection of a display screen, comprising: a first construction unit for constructing a single-category detection model and a single-category loss function; a first acquisition unit for acquiring a training image of a new display screen marked with a single-category label, and inputting the training image into the single-category detection model; a first generation unit for performing shallow feature enhancement processing on the training image according to the single-category label to generate a feature map; a second generation unit for respectively extracting shallow features of the training image and the feature map, performing feature fusion on the two sets of shallow features, and then extracting deep features, and generating a binary classification prediction result for the deep features according to the single-category head; the first training unit The first unit is used to calculate the first loss value according to the single-class loss function and the binary classification prediction results, and update the single-class detection model according to the first loss value until the initial training target is reached; the third generation unit is used to cluster the deep features and generate subcategory pseudo labels for each training image; the second construction unit is used to expand a multi-classification head on the single-class detection model and construct a multi-classification loss function and subcategory classification targets; the second training unit is used to jointly train the feature extractor, two classification heads and subcategory pseudo labels in the single-class detection model through deep features, multi-classification heads, multi-classification loss functions and subcategory classification targets until the single-classification head reaches the training target.

[0014] Optionally, after the second training unit, the device further includes: a second acquisition unit for removing the expanded multi-classification head to acquire an unlabeled image in a new scene; a perturbation unit for copying the unlabeled image and performing perturbation processing on the copied unlabeled image; a fourth generation unit for performing shallow feature enhancement processing on the unlabeled image to generate a scene feature map; a fifth generation unit for extracting shallow features from the unlabeled image, the perturbed unlabeled image, and the scene feature map to generate a first feature, a second feature, and a third feature respectively; a sixth generation unit for fusing the first feature and the third feature to generate a fourth feature map; A first fusion feature is extracted, and then a depth feature is extracted to generate a first depth feature; a seventh generation unit is used to fuse the second feature and the third feature to generate a second fusion feature, and then extract the depth feature to generate a second depth feature; an eighth generation unit is used to input the first depth feature and the second depth feature into the single classification head to generate a conventional prediction result and a perturbation prediction result; a ninth generation unit is used to construct a consistency loss function and generate a consistency loss according to the conventional prediction result, the perturbation prediction result and the consistency loss function; an update unit is used to update the parameters of the single classification detection model applicable to the new scenario according to the consistency loss.

[0015] Optionally, after the updating unit, the device further includes: a labeling unit, for precisely labeling the unlabeled images when the conventional prediction results and the perturbation prediction results show that there are unlabeled images at the classification boundary; an optimization unit, for determining the precisely labeled unlabeled images as untrained images, and using the untrained images to optimize the single-classification detection model during the production process.

[0016] Optionally, the new display screen is a folding screen with a microcircuit layer in the folding area, the single-classification detection model is a folding screen single-classification detection model, and the unlabeled image is a real-time image of the folding area of ​​the folding screen on the production line; the perturbation unit includes: copying the unlabeled image; introducing light source perturbation into the copied unlabeled image according to the curvature parameters of the folding screen in the folding area; and introducing noise perturbation to the disturbed unlabeled image by cropping and rotating.

[0017] Optionally, the first acquisition unit includes: acquiring a captured image of the new display screen, and dividing the captured image into an abnormal image and a non-abnormal image; combining the abnormal image with the non-abnormal image, and expanding the data volume through image enhancement technology to generate a first abnormal expanded image; identifying a reference area in the non-abnormal image, replacing the reference area with a corresponding abnormal area in the abnormal image, and assimilating the replaced abnormal area to generate a second abnormal expanded image, where the reference area is an area where the authority influence weight reaches a preset value; disturbing the non-abnormal image to generate a third abnormal expanded image; annotating the abnormal image, the non-abnormal image, the first abnormal expanded image, the second abnormal expanded image and the third abnormal expanded image with single category labels, and determining them as training images; and inputting the training images into a single classification detection model.

[0018] Optionally, the new display screen is a folding screen with a microcircuit layer in the folding area, the single-category detection model is a folding screen single-category detection model, the training image is a captured image of the folding area of ​​the folding screen, and the single-category label is a folding area single-category label; the first generation unit includes: determining the gain data of the training image as the folding screen curvature and the folding screen microcircuit reflection according to the folding area single-category label; determining the curvature parameters of the folding area of ​​the training image; setting the first folding screen and the second folding screen according to the curvature parameters, and projecting light of a preset brightness value to the first folding screen and the second folding screen through an external light source, the first folding screen is a folding screen without a microcircuit layer in the folding area, and the second folding screen is a folding screen with a microcircuit layer in the folding area; collecting images of the first folding screen and the second folding screen respectively, and calculating the microcircuit reflection parameters according to the folding areas of the two images; constructing a shallow feature gain convolution kernel according to the curvature parameters and the circuit reflection parameters; performing shallow feature enhancement processing on the training image through the shallow feature gain convolution kernel to generate a feature map.

[0019] Optionally, the second construction unit includes: extending a multi-classification head for performing probability distribution detection on sub-class pseudo labels on the basis of a single-classification detection model;

[0020] Determine the grayscale discrimination weight value of each subcategory pseudo label on the new display screen; generate the grayscale error weight value based on the curvature parameters and microcircuit reflection parameters of the new display screen; construct a multi-classification cross entropy function based on the grayscale discrimination weight value and the grayscale error weight value; and construct the subcategory classification target.

[0021] In a third aspect, an embodiment of the present application provides a device for detecting single-category anomalies in a display screen, comprising:

[0022] processor, memory, input and output units, and buses;

[0023] The processor is connected to the memory, input and output units, and the bus;

[0024] The memory stores a program, and the processor calls the program to execute the first aspect and any optional method of the first aspect.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the program executes the first aspect and any optional method of the first aspect.

[0026] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0027] This application first constructs a single-classification detection model and a single-classification loss function. Next, a training image of the new display screen marked with a single-class label is obtained, and the training image is input into the single-classification detection model. Then, shallow feature enhancement processing is performed on the training image according to the single-class label to generate a feature map. Then, shallow features of the training image and the feature map are extracted respectively, and then the two sets of shallow features are fused, and then deep features are extracted, and binary classification prediction results are generated for the deep features according to the single-classification head. The first loss value is calculated based on the single-classification loss function and the binary classification prediction results, and the single-classification detection model is updated according to the first loss value until the initial training target is reached. The deep features are clustered to generate subcategory pseudo-labels for each training image. A multi-classification head is expanded on the single-classification detection model, and a multi-classification loss function and sub-classification classification target are constructed. The feature extractor, two classification heads and sub-class pseudo-labels in the single-classification detection model are jointly trained by deep features, multi-classification heads, multi-classification loss functions and sub-classification classification targets until the single-classification head reaches the training target.

[0028] By performing specific shallow feature enhancement processing on the training images, a feature map is generated, and the shallow features of the training images and the feature map are extracted respectively, and then the two groups of shallow features are fused, and the deep features are extracted. At this time, the deep features extracted have completed the special optimization of single-class abnormal data, and the situation of relatively dispersed intra-class feature distribution has been preliminarily solved. Next, a multi-classification head is expanded on the single-classification detection model, and a multi-classification loss function and a sub-class classification target are constructed, and then the feature extractor, the two classification heads and the sub-class pseudo label in the single-classification detection model are jointly trained through the detection result of the multi-classification head until the single-classification head completely reaches the training target. Through the further optimization of the feature representation by the multi-classification head, the optimized enhanced feature representation can help the single-classification detection model to further refine the classification of the abnormal images in the training images, thereby providing the single-classification head branch with more fine-grained abnormal detection capability and improving the accuracy of single-classification detection and abnormal detection of the new display screen defect detection model. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0030] Figure 1 An embodiment schematic diagram of the display screen single-classification abnormality detection method of the present application;

[0031] Figure 2 An embodiment schematic diagram of the model updating method based on a new scene of the present application;

[0032] Figure 3 An embodiment schematic diagram of the model optimization method of the present application;

[0033] Figure 4 An embodiment schematic diagram of the perturbation processing method based on unlabeled images of the present application;

[0034] Figure 5 An embodiment schematic diagram of the training image processing method of the present application;

[0035] Figure 6 An embodiment schematic diagram of the shallow feature enhancement processing method of the present application;

[0036] Figure 7 An embodiment schematic diagram of the expansion method of the single-classification detection model of the present application;

[0037] Figure 8An embodiment of the device for display screen single category anomaly detection of the present application;

[0038] Figure 9 Another embodiment of the device for display screen single category anomaly detection of the present application;

[0039] Figure 10 An embodiment of the device for display screen single category anomaly detection of the present application; DETAILED DESCRIPTION

[0040] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0041] It will be understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0042] It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0043] As used in the description of the application and the appended claims, the term "if' can be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon [the described condition or event] being detected" or "in response to [the described condition or event] being detected," depending on the context.

[0044] In addition, the terms "first," "second," "third," etc. are used herein only to describe different instances of an element, and do not imply relative importance of the elements.

[0045] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specifications are not necessarily all referring to the same embodiment, however, are meant to signify that a particular feature, structure, or characteristic being referred to is included in at least one embodiment of the application. The terms "including," "comprising," "having," and variations thereof are meant to encompass the terms "including but not limited to."

[0046] In the prior art, in the field of single-classification anomaly detection of new display screens, a supervised anomaly detection method is used. Such a supervised anomaly detection method usually relies on a training set containing labeled data, and uses a classic machine learning model or a deep neural network (such as a support vector machine, a random forest, XGBoost, etc.) for training. Through learning the feature difference between normal data and abnormal data, a single-classification anomaly detection model can realize anomaly detection. The advantage of the supervised method is that when the labeled data is sufficient, the model can realize high-precision anomaly detection. However, the defect of the single-classification anomaly detection model of the new display screen trained based on the supervised method is that: most of the current single-classification detection methods based on deep learning directly use high-dimensional features (such as features extracted by ResNet) of a pre-trained network. These high-dimensional features of the new display screen are usually trained in a multi-classification scenario, and lack adaptability to target single-classification data. The feature extraction process lacks special optimization for single-classification abnormal data, resulting in a relatively dispersed intra-class feature distribution, and thus the single-classification detection model of the new display screen cannot effectively distinguish between abnormal samples and normal samples in the feature space, reducing the accuracy of single-classification detection and anomaly detection of the new display screen defect detection model.

[0047] Based on this, the application discloses a display screen single-classification anomaly detection method, device and storage medium, which is used to improve the accuracy of single-classification detection and anomaly detection of the new display screen defect detection model.

[0048] The technical solutions in the application will be described clearly and completely below in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0049] The method of the application can be applied to a server, a device, a terminal or other devices with logical processing capability, and the application is not limited thereto. For convenience of description, the following will be described taking a terminal as an example.

[0050] See also Figure 1 The present application provides an embodiment of a method for detecting single-category anomalies in a display screen, comprising:

[0051] 101. Build a single-class detection model and single-class loss function.

[0052] Please refer to Figure 10 , the single-class detection model has multiple stages, one is the training stage (upper half), and the other is the production detection stage (lower half). The deep learning model (single-class detection model) is used for image classification tasks, with the goal of dividing training images (labeled images, where the labels are single-class labels) into normal images and abnormal images (abnormal images include abnormal data and disturbed data). In terms of model design, this embodiment adopts the traditional convolutional neural network (CNN) framework (such as ResNet50) as the backbone network, and improves it on this basis. In this embodiment, it is mainly the training stage of the upper half, including the shallow feature map extraction network, the shallow training image extraction network above the shallow feature map, the feature fusion module, the deep feature extraction module (for feature extraction), the binary classification head module (single-class head), the multi-classification head module and the clustering analysis module ( Figure 10 The K-means method is used in the Figure 10 The K-means method (K-means clustering algorithm) in [1] is a classic unsupervised learning algorithm used to divide a data set into K different clusters, so that the similarity of data points in the same cluster is high, while the similarity between different clusters is low. Figure 10 The ResNet50 in this example is a deep convolutional neural network (CNN) that belongs to the ResNet (Residual Network) family. The core innovation of ResNet is the residual block, which has the following structure: after the input passes through two convolutional layers, it is added to the original input (skip connection). The ResNet50 in this example allows gradients to be directly backpropagated through skip connections, alleviating the vanishing gradient problem in deep networks.

[0053] In order to alleviate the data imbalance problem, in the supervised training process, in this embodiment, the single classification loss function adopts the weighted binary cross entropy loss function. For a small amount of abnormal data, it is necessary to give a higher training weight to ensure that the model can better learn the characteristics of the abnormal data, thereby improving the single classification detection model's ability to recognize abnormal images. The single classification loss function L constructed in this embodiment is BCE As shown below:

[0054]

[0055] Where N is the number of labeled images (training images), is the weight of the single-class anomaly class, is the single-class normal class weight, is the single-category label of the i-th training image, is the model single classification prediction result (binary classification prediction result) of the i-th training image.

[0056] 102. Obtain training images of the new display screen that are labeled with a single category label, and input the training images into a single classification detection model.

[0057] In this embodiment, the terminal collects training images from the new display screen, determines defects on the training images, and labels the training images. Each training image now has a corresponding single-category label. Next, the terminal inputs the training images into the single-category detection model.

[0058] 103. Perform shallow feature enhancement on the training image according to the single category label to generate a feature map.

[0059] The terminal determines the enhancement parameters and enhancement methods of shallow feature enhancement processing based on the types existing in the single category label, and then performs shallow feature enhancement processing on the training image to generate a feature map of shallow feature enhancement. The specific determination and implementation details of the shallow feature enhancement processing will be explained in subsequent embodiments.

[0060] 104. Extract shallow features of the training image and feature map respectively, perform feature fusion on the two sets of shallow features, extract deep features, and generate binary classification prediction results for the deep features based on the single classification head.

[0061] In this embodiment, shallow features of the feature map and training images (labeled images) are extracted using a shallow feature map extraction network and a shallow series image extraction network, respectively. Feature fusion is then performed using a feature fusion module. Deep feature extraction is then performed using a deep feature extraction module. The resulting deep features are specifically optimized for single-class anomaly data, addressing the issue of dispersed feature distribution within a class. The deep features are then fed into a single-class classification head (binary classification head) for single-class defect anomaly analysis, yielding a binary classification prediction result.

[0062] In this embodiment, by combining the features of traditional image processing with deep learning methods and adopting attention mechanisms and multiple shallow feature extraction modules, the single-classification detection model's ability to recognize abnormal images is enhanced, thereby further improving the accuracy of anomaly detection.

[0063] 105. Calculate a first loss value based on the single-class loss function and the binary classification prediction results, and update the single-class detection model based on the first loss value until the initial training target is reached.

[0064] In this embodiment, after the terminal obtains the binary classification prediction result, it inputs the binary classification prediction result into the single classification loss function L BCE A loss value is calculated to obtain a first loss value, and the single-class detection model is updated based on the first loss value until the initial training target is reached. For small amounts of abnormal data, this embodiment assigns a higher training weight to ensure that the model can better learn the characteristics of the abnormal data, thereby improving the single-class detection model's ability to recognize abnormal images. Specifically, in this embodiment, the weight of the single-class abnormal class is set to 3, and the weight of the single-class normal class is set to 1.

[0065] 106. Cluster the deep features and generate subcategory pseudo labels for each training image.

[0066] When the single-classification head is preliminarily trained, the single-classification detection model's single-classification detection capability cannot fully adapt to the defects and anomalies of the new display screen. It is necessary to further refine the single-classification detection model's ability to classify abnormal images in the training images. Specifically, the terminal first clusters the extracted deep features. In this embodiment, K-mean clustering is used to generate subcategory pseudo-labels for each training image. Specifically, after the preliminary training is completed, the single-classification detection model will further establish subcategories for each image by distinguishing image features. The core idea of ​​this stage is to cluster the feature vectors learned through the network to identify different types of abnormal images and assign a specific subcategory to each image. For example, the K-means clustering algorithm can be used to cluster the features of normal images and abnormal images separately to determine K subcategories. Based on the clustering results, we assign a pseudo-label to each training image (the feature map corresponds to the pseudo-label), and the pseudo-label represents the subcategory index to which the image belongs.

[0067] 107. Expand a multi-classification head on the single-classification detection model and construct a multi-classification loss function and sub-classification objectives.

[0068] 108. The feature extractor, two classification heads and subcategory pseudo labels in the single classification detection model are jointly trained through deep features, multi-classification heads, multi-classification loss functions and subcategory classification targets until the single classification head reaches the training target.

[0069] Next, the terminal adds a multi-classification head to the entire single-class detection model network, constructs a sub-classification objective, and constructs a multi-classification loss function. This multi-classification network (multi-classification head) is then jointly trained with the binary classification network (binary classification head). It is important to note that the sub-classification network and the binary classification network share the same feature extractor, using a common set of deep features, ensuring uniform feature extraction. By iteratively updating the feature extractor, the two classifiers, and the sub-class pseudo-labels, the terminal continuously optimizes the feature representation, thereby improving classification performance.

[0070] Use multi-class cross entropy loss L sub To constrain, the formula is:

[0071]

[0072] Where C is the number of subcategories, is the label of the image corresponding to the i-th subcategory, The model prediction result for the image corresponding to the i-th subcategory.

[0073] The advantage of this approach is that the enhanced feature representation helps the single-class detection model further refine its classification of abnormal images, providing more fine-grained anomaly detection capabilities. The single-class detection model can not only distinguish between normal and abnormal images, but also identify and distinguish different types of anomalies, improving the accuracy and robustness of single-class detection for anomaly detection.

[0074] First, a single-class detection model and a single-class loss function are constructed. Next, training images of the new display screen labeled with a single class label are obtained and fed into the single-class detection model. Shallow features of the training images are then enhanced based on the single-class label to generate a feature map. Shallow features are then extracted from the training images and feature map, respectively. These two sets of shallow features are then fused, and deep features are extracted. Binary classification predictions are generated for the deep features using the single-class head. A first loss value is calculated based on the single-class loss function and the binary classification predictions. The single-class detection model is updated based on this first loss value until the initial training target is reached. The deep features are clustered to generate sub-class pseudo-labels for each training image. A multi-classification head is then extended to the single-class detection model, and a multi-classification loss function and sub-class classification target are constructed. The feature extractor, two classification heads, and sub-class pseudo-labels in the single-class detection model are jointly trained using the deep features, the multi-classification head, the multi-classification loss function, and the sub-class classification target until the single-classification head reaches the training target.

[0075] First, the feature parameters of traditional image detection are usually represented by an image of the same size as the original image, thereby obtaining the image's parameter features and retaining the feature values ​​at the corresponding positions in the image. This embodiment introduces an attention mechanism based on this and designs a shallow feature extraction module to extract shallow features of the image. This shallow feature extraction module can be integrated with the global features of the image, guiding the single-class detection model to focus on the most representative features in the image, thereby improving the accuracy of single-class image classification.

[0076] Specifically, the training image is enhanced with specific shallow features to generate a feature map. The shallow features of both the training image and the feature map are then extracted separately. These two sets of shallow features are then fused together to extract deep features. The extracted deep features are specifically optimized for single-class anomalies, providing a preliminary solution to the problem of dispersed feature distribution within a class.

[0077] Next, a multi-classification head is added to the single-classification detection model, and a multi-classification loss function and sub-classification objectives are constructed. The detection results of the multi-classification head are then used to jointly train the feature extractor, two classification heads, and sub-class pseudo-labels in the single-classification detection model until the single-classification head fully achieves the training objectives. The multi-classification head further optimizes the feature representation. The optimized and enhanced feature representation helps the single-classification detection model further refine its classification of anomalies in training images, thereby providing the single-classification head with more fine-grained anomaly detection capabilities, thereby improving the accuracy of both single-class and anomaly detection in the new display defect detection model.

[0078] To this end, this example introduces an active learning strategy and incorporates feature clustering to divide sub-labels, optimizing the feature representation of the target category, ensuring compactness while also enhancing generalization capabilities. This approach fully utilizes data resources and significantly improves the model's adaptability to unknown scenarios.

[0079] See also Figure 2 , the present application provides an embodiment of a model updating method based on a new scenario, comprising:

[0080] 201. Remove the extended multi-classification head to obtain unlabeled images in the new scene.

[0081] In this embodiment, although traditional image processing methods can extract image features and perform threshold control, they cannot effectively deal with the diverse and complex display panel defects and have difficulty handling constantly changing anomaly types and new scenarios. Furthermore, existing technologies lack the ability to handle new scenarios and unlabeled data, resulting in the model's inability to quickly adapt to new scenarios and requiring a significant amount of manual intervention and data labeling. To this end, this embodiment introduces an active learning strategy and incorporates feature clustering to divide sublabels, optimizing the feature representation of the target category while ensuring compactness and enhanced generalization capabilities. This approach fully utilizes data resources and significantly improves the model's adaptability to unknown scenarios.

[0082] Specifically, the terminal removes the extended multi-classification head and obtains unlabeled images in the new scene, because auxiliary training of the multi-classification head is no longer required in the actual production process.

[0083] 202. Copy the unlabeled image and perform perturbation processing on the copied unlabeled image.

[0084] The terminal copies the unlabeled image and perturbs the copied unlabeled image. Specifically, the terminal perturbs the copied unlabeled image based on the defect type targeted by the single-classification detection model. The specific steps will be described in detail later.

[0085] 203. Perform shallow feature enhancement on the unlabeled image to generate a scene feature map.

[0086] In this embodiment, the same shallow feature enhancement processing method as in Example 1 is used. If there are multiple processing methods, it is necessary to copy multiple unlabeled images and then use multiple shallow feature enhancement processing methods to process them separately, and then generate several scene feature maps (in Figure 10 The steps in the lower half are also called feature maps).

[0087] 204. Extract shallow features from the unlabeled image, the perturbed unlabeled image, and the scene feature map to generate a first feature, a second feature, and a third feature, respectively.

[0088] 205. Fuse the first feature and the third feature to generate a first fused feature, and then extract a depth feature to generate a first depth feature.

[0089] 206. Fuse the second feature and the third feature to generate a second fused feature, and then perform depth feature extraction to generate a second depth feature.

[0090] After the terminal generates a disturbed unlabeled image and several scene feature maps, it is necessary to perform shallow feature extraction on the disturbed unlabeled image, the unperturbed unlabeled image, and the scene feature map to generate the first feature, the second feature, and the third feature, as in Example 1.

[0091] The terminal then fuses the first feature and the third feature to generate a first fused feature, and then extracts the depth feature to generate a first depth feature. The second feature and the third feature are then fused to generate a second fused feature, and then the depth feature is extracted to generate a second depth feature. The specific extraction method is similar to that in Example 1. One of the first depth feature and the second depth feature is disturbed and the other is not disturbed. It should be noted that, because of how long the scene feature map is, it is necessary to analyze each scene feature map separately, so that there are actually multiple groups of first depth features and second depth features, and the number is similar to the number of scene feature maps.

[0092] 207. Input the first depth feature and the second depth feature into a single classification head to generate a normal prediction result and a perturbation prediction result.

[0093] The terminal inputs the first depth feature and the second depth feature into a single classification head to generate a normal prediction result and a perturbation prediction result. It also generates multiple groups of normal prediction results and perturbation prediction results according to the number of scene feature maps.

[0094] 208. Construct a consistency loss function and generate consistency loss according to the conventional prediction result, the perturbation prediction result and the consistency loss function.

[0095] At this time, the terminal constructs a consistency loss function and generates a consistency loss based on the conventional prediction results, the perturbation prediction results and the consistency loss function. In order to enable the model to respond to new scenarios more quickly and overcome the dependence of abnormal states on the subjective judgment of the human eye, the present invention adopts an active learning training method. This method can help the model identify abnormal states more accurately, and continue to learn and optimize its performance during the test production process. In the test production process, most of the data are unlabeled unlabeled images, so they need to be processed through the following steps: First, the unlabeled image and its feature parameters after image processing are input into the model to obtain the feature ; Then, the data is perturbed, and after perturbation processing, it is input into the model to obtain new features The perturbation method may include: adding the same noise to the feature parameters of the annotated image and its processed image when the image is input, or processing it through data augmentation methods such as cropping and rotation; in addition, perturbation processing can also be performed directly at the feature level, adding noise or other forms of perturbation.

[0096] After obtaining the processed features, the data is fed into the binary classification head for prediction and compared with the original prediction results. For samples with large discrepancies in the results, consistency loss is used to ensure that the model output is consistent, ensuring that the classification results are consistent between the perturbed input and the original input.

[0097] The consistency loss L U for:

[0098]

[0099] M is the number of unlabeled images, is the prediction result of the i-th unlabeled image, is the prediction result after perturbation of the i-th unlabeled image.

[0100] It should be noted that because there are multiple sets of conventional prediction results and perturbation prediction results, there are multiple consistency losses. The weights can be superimposed according to different processing methods of the scene feature map, and then normalized to generate a unique consistency loss.

[0101] 209. Update the parameters of the single-class detection model to adapt to new scenarios based on consistency loss.

[0102] Finally, the terminal updates the parameters of the single-class detection model applicable to the new scenario based on the consistency loss.

[0103] The above method is used to train the single-class detection model based on new scenarios. The beneficial effects are as follows:

[0104] 1. Adopt an active learning strategy: Through the active learning mechanism, the model can automatically select the most challenging unlabeled samples for labeling. This not only reduces manual intervention but also enables continuous learning to cope with new scenarios and changes, thereby improving the model's adaptability.

[0105] 2. Data augmentation methods alleviate the problem of data imbalance: Through methods such as image augmentation, transplantation of abnormal parts, and introduction of disturbed data, the dataset is expanded and the model's ability to identify abnormal data is improved, thus overcoming the challenge of data imbalance.

[0106] 3. Model self-iteration and optimization: Through iterative training and continuous learning, the model can be continuously optimized and adjusted during the test production process, quickly adapting to new production scenarios and anomaly types, and improving the reliability and accuracy of anomaly detection.

[0107] See also Figure 3 , the present application provides an embodiment of a model optimization method, comprising:

[0108] 301. When the conventional prediction results and the perturbation prediction results show that there are unlabeled images at the classification boundary, these unlabeled images are precisely labeled.

[0109] 302. Determine the precisely annotated unlabeled images as untrained images, and use the untrained images to optimize the single-class detection model during the production process.

[0110] In this embodiment, if some unlabeled images are near the classification boundary and the prediction results are uncertain, these unlabeled images are output as uncertain samples and handed over to experts for manual labeling. These labeled samples are then used to further optimize the single-class detection model. In the active learning process, training batches can be set. Typically, after training 10 batches, card control is performed based on a preset threshold, and samples located at the classification boundary are collected and sent to experts for labeling. This process can effectively improve the generalization ability of the single-class detection model and ensure that the single-class detection model can continuously improve itself in practical applications, gradually improving the accuracy of anomaly detection.

[0111] Through active learning, the single-class detection model can not only quickly adapt to new environments without a large amount of labeled data, but also be optimized through continuous feedback in the actual production process, thereby enhancing the robustness and accuracy of anomaly detection.

[0112] See also Figure 4 , the present application provides an embodiment of a disturbance processing method based on unlabeled images, comprising:

[0113] 401. Copy the unlabeled image.

[0114] 402. Introduce light source disturbance into the copied unlabeled image according to the curvature parameter of the folding screen in the folding area.

[0115] 403. Noise perturbation is introduced into the disturbed unlabeled image by cropping and rotating.

[0116] In this embodiment, the novel display screen is a foldable screen with a microcircuit layer within its folding area (the folding area refers to an area on the novel display screen that is composed of multiple display layers, including a microcircuit layer and a pixel display layer). The single-class detection model is for the folding screen, and the unlabeled image is a real-time image of the folding area of ​​the folding screen captured on the production line. The novel display screen incorporates a flexible, foldable and bendable structure, allowing pixels to stretch during folding and exhibiting structural recovery capabilities. The microcircuit layer is an improvement to the hierarchical structure of the novel display screen. To increase the functionality of the pixel layer (display layer) and other structural layers of the novel display screen, microcircuits can be controlled by configuring microcircuits. Furthermore, multiple microcircuits can be integrated into a single display screen structural layer (microcircuit layer) and added to the novel display screen's hierarchical structure, essentially adding a new microcircuit layer to the existing layer. Because the microcircuits in the microcircuit layer contain mostly metal traces that can reflect light, the microcircuit layer experiences light reflection in the circuit area. Whether the pixels in the novel display layer illuminate or external light illuminates the novel display screen, the circuit area reflects light to a certain extent, resulting in visual unevenness on the display screen. To mitigate this, the microcircuits in the microcircuit layer are typically placed on the curved edges of both sides of the flexible screen. However, for foldable screens, which are also flexible screens, the circuit area is typically located in the folding area, as this area is less likely to affect the user's visual experience during use. However, this increases the difficulty of defect detection for these new display types. These new displays feature increasingly higher pixel integration and dense, orderly pixel arrangement. In foldable screens, the initial pixel arrangement in the folding area changes as the curvature of the fold increases, causing the pixels in the folding area to emit light in different directions. However, the circuit area affects the display light source based on its own reflectivity, which also varies with curvature. One method, through bending, increases scattering, reducing the light source's divergence (the greater the folding, the more severe the scattering), while the other, through reflection, enhances the display light source. This makes defects in the folding area more susceptible to the influence of both the folding area and the circuit area. This reduces detection quality for single-class detection models for these new displays, especially since the folding area is a critical defect detection area.

[0117] In the defect detection of new folding screens, it is necessary to capture images of the folding area with different folding degrees (control curvature) and perform defect detection in the captured images.

[0118] In this embodiment, after the terminal copies the unlabeled image, it is necessary to introduce light source disturbance into the copied unlabeled image according to the curvature parameters of the folding screen in the folding area. Because when the folding screen is folded, the external light source can affect both the circuit area on the display screen and the hierarchical reflection of the folding area. Therefore, the light source characteristics of the external light source are extracted and introduced into the training image for disturbance processing. Then, noise disturbance is introduced into the disturbed unlabeled image by cropping and rotating. After the disturbance processing, the accurate consistency loss can be better obtained. After the single-classification detection model is updated and iterated through the consistency loss, a better model detection effect can be obtained.

[0119] See also Figure 5 , the present application provides an embodiment of a method for processing a training image, comprising:

[0120] 501. Obtain a captured image of the novel display screen, and divide the captured image into an abnormal image and a non-abnormal image.

[0121] 502. Combine the abnormal image with the non-abnormal image, and expand the data volume using image enhancement technology to generate a first abnormal expanded image.

[0122] 503. Identify a reference region in the non-abnormal image, replace the reference region with the corresponding abnormal region in the abnormal image, and perform assimilation processing on the replaced abnormal region to generate a second abnormal expanded image, where the reference region is a region where the authority influence weight reaches a preset value.

[0123] 504. Perform a disturbance process on the non-abnormal image to generate a third abnormal expanded image.

[0124] 505 . Annotate the abnormal image, the non-abnormal image, the first abnormal expanded image, the second abnormal expanded image, and the third abnormal expanded image with single-category labels and determine them as training images.

[0125] 506. Input the training image into the single-classification detection model.

[0126] Existing technologies are expensive to label, and obtaining labeled data containing accurate anomaly samples is extremely difficult. This is especially true for complex and dynamically changing datasets, where both the cost and time consumption are extremely high. Furthermore, there is sample imbalance. In actual production scenarios, anomaly data is typically very rare, far less than normal data. This class imbalance makes supervised models prone to bias toward normal data during training, affecting detection effectiveness.

[0127] Therefore, in this embodiment, for supervised learning data, the image and its feature parameters after image processing and the label are used for training. For unsupervised learning data, only the image and its feature parameters after image processing are used. In the application of supervised data, in order to solve the problem of lack of abnormal data, a series of image enhancement techniques are used to increase the diversity and richness of the data set, including the following methods:

[0128] 1. Combination of normal data and abnormal data: combine a small amount of abnormal images with normal images, and expand the data amount through image enhancement techniques (such as image cutting, rotation, color transformation, etc.). This method artificially creates more variant samples, effectively improving the generalization ability of the model.

[0129] 2. Abnormal part transplantation: transplant a small amount of abnormal parts into normal images to artificially synthesize abnormal data. The specific method is: first identify the typical areas in the normal image, then replace these areas with real abnormal areas, and then enhance the diversity of abnormal samples. In this way, various types of abnormal situations can be effectively simulated.

[0130] 3. Disturbance data introduction: introduce disturbance data, which does not represent any specific normal or abnormal state, but is made into "non-normal" image data by adding noise or other forms of image transformation (such as image blurring, random cropping, etc.). Disturbance data is considered as "non-normal data", and its role is to help the model better identify and distinguish normal and abnormal images during training.

[0131] Through these image enhancement techniques, this embodiment can effectively overcome the problem of data deficiency, improve the training effect of the model and the accuracy of abnormal detection, and thus realize more reliable abnormal detection.

[0132] Only using normal image data for training, in order to improve the generalization ability of the model, combined with image enhancement methods and cross-domain data to enrich the data set.

[0133] Please refer to Figure 6 , an embodiment of a method for shallow feature enhancement processing is provided, which includes:

[0134] 601. According to the single-class label of the folding area, the gain data of the training image is determined as the folding screen curvature and the folding screen microcircuit reflection.

[0135] In this embodiment, the new display screen is a folding screen with a microcircuit layer in the folding area. At this time, the training image is the captured image of the folding area of the folding screen when it is folded, the single classification detection model is the folding screen single classification detection model, and the single classification label is the folding area single classification label. The terminal determines the gain data of the training image according to the folding area single classification label as the folding screen curvature and the folding screen microcircuit reflection, because the folding screen curvature and the folding screen microcircuit reflection are the parameters that are most affected by defects in this new type of folding screen.

[0136] 602, determine the curvature parameter of the folding area of the training image.

[0137] After the terminal determines the gain data of the training image according to the folding area single classification label as the folding screen curvature and the folding screen microcircuit reflection, the curvature parameter of the folding area of the training image is determined.

[0138] 603, set the first folding screen and the second folding screen according to the curvature parameter, and project light rays with a preset brightness value to the first folding screen and the second folding screen through an external light source. The first folding screen is a folding screen without a microcircuit layer in the folding area, and the second folding screen is a folding screen with a microcircuit layer in the folding area.

[0139] 604, respectively collect images of the first folding screen and the second folding screen, and calculate the microcircuit reflection parameter according to the folding areas of the two images.

[0140] Next, the terminal needs to fold the two folding screens to a corresponding degree according to the curvature parameter. The first folding screen is a folding screen without a microcircuit layer in the folding area (a common folding screen of the same model), and the second folding screen is a folding screen with a microcircuit layer in the folding area (a new folding screen of the same model). Then, the external light source projects light rays with a preset brightness value to the first folding screen and the second folding screen.

[0141] Next, the terminal respectively collects images of the first folding screen and the second folding screen, and calculates the brightness value according to the folding areas of the two images, and then calculates the microcircuit reflection parameter according to the brightness difference.

[0142] 605, construct a shallow feature gain convolution kernel according to the curvature parameter and the circuit reflection parameter.

[0143] 606, perform shallow feature enhancement processing on the training image through the shallow feature gain convolution kernel to generate a feature map.

[0144] Next, the terminal constructs a shallow feature gain convolution kernel according to the curvature parameter and the circuit reflection parameter:

[0145] When constructing shallow feature gain convolution kernels for foldable displays, in addition to considering spatial proximity and pixel similarity, we also incorporate the curvature parameters (curvature information) and circuit reflection parameters (reflectivity information) of the new foldable display into the convolution kernel. Curvature information helps preserve edge information in areas with large curvature variations, such as fold edges, while reflectivity information can be used to adjust the filter strength in areas with different circuit reflectivity, thereby reducing the filter strength in areas with large reflectivity differences (such as highlights or shadows) to preserve details.

[0146] The curvature and reflectivity information of a foldable display includes the curvature parameter κ(p) and the circuit reflectivity parameter R(p). The curvature parameter κ(p) represents the curvature value at pixel point p. A larger curvature indicates a more curved area. The circuit reflectivity parameter R(p) represents the reflectivity value at pixel point p. Reflectivity can reflect the glossiness or light intensity of a surface. Areas with large differences in reflectivity may contain important details or edges.

[0147] Construct a shallow feature gain convolution kernel that is sensitive to curvature parameters and circuit reflection parameters. In order to incorporate curvature and reflectivity information into the traditional convolution kernel, the traditional convolution kernel makes its weight depend not only on spatial proximity and pixel similarity, but also on curvature difference and reflectivity difference. Specifically, the shallow feature gain convolution kernel G(p,q) can be expressed as:

[0148]

[0149] in:

[0150] is the spatial proximity Gaussian kernel, which measures the spatial distance between p and q.

[0151] : Pixel similarity Gaussian kernel, which measures the difference in pixel values ​​between p and q, and are the pixel values ​​of points p and q respectively.

[0152] is the curvature similarity Gaussian kernel, which measures the curvature difference between p and q, and are the curvatures at points p and q respectively.

[0153] is the reflectivity similarity Gaussian kernel, which measures the reflectivity difference between p and q, and are the reflectivities at points p and q, respectively.

[0154] The specific form is as follows:

[0155] Spatial proximity Gaussian kernel :

[0156]

[0157] pixel similarity gaussian kernel :

[0158]

[0159] curvature similarity gaussian kernel :

[0160]

[0161] reflectivity similarity gaussian kernel :

[0162]

[0163] where, is the standard deviation of spatial distance, is used to control the influence of spatial proximity, a larger will result in a more extensive smoothing. is the standard deviation of pixel value difference, is used to control the influence of pixel similarity, a smaller will enhance the sensitivity to pixel value difference, thus better preserving edges. And is the standard deviation of curvature, is used to control the influence of curvature similarity, a smaller will enhance the sensitivity to curvature difference, thus reducing the filtering strength in areas with large curvature variation (e.g. folding edges). is the standard deviation of circuit region reflectivity parameter, is used to control the influence of reflectivity similarity, a smaller will enhance the sensitivity to reflectivity difference, thus reducing the filtering strength in areas with large reflectivity variation (e.g. highlight or shadow areas).

[0164] By incorporating curvature and reflectivity information into the traditional convolution kernel, a curvature and reflectivity sensitive shallow feature gain convolution kernel is constructed. This convolution kernel can adaptively adjust the filtering strength according to the curvature and reflectivity variation of the folding screen, thus preserving shallow feature information and details in areas with large curvature or reflectivity variation (e.g. folding areas plus circuit areas), while enhancing the smoothing effect in areas with small curvature and reflectivity variation. This method is particularly suitable for processing new folding screen images (folding areas and circuit areas overlap).

[0165] Finally, the terminal performs shallow feature enhancement processing on the training image through the shallow feature gain convolution kernel to generate a feature map. The shallow features of the processed feature map in this folding area can be targetedly enhanced.

[0166] Referring to Figure 7 An embodiment of the extension method of the single-classification detection model is provided, including:

[0167] 701. A multi-classification head for probability distribution detection of sub-class pseudo labels is extended on the single-classification detection model.

[0168] 702. A gray scale discrimination weight value of each sub-class pseudo label on the new display screen is determined.

[0169] 703. A gray scale error weight value is generated according to the curvature parameter and the microcircuit reflection parameter of the new display screen.

[0170] 704. A multi-classification cross-entropy function is constructed according to the gray scale discrimination weight value and the gray scale error weight value.

[0171] 705. A sub-class classification target is constructed.

[0172] The terminal extends a multi-classification head for probability distribution detection of sub-class pseudo labels on the single-classification detection model, and then determines a gray scale discrimination weight value of each sub-class pseudo label on the new display screen. Specifically, the detection degree of different defect labels (each sub-class pseudo label) in the folding area of the folding screen needs to be determined, and at this time, the curvature parameter and the circuit reflection parameter also need to be considered. The gray scale discrimination weight value is as follows:

[0173]

[0174] The gray scale discrimination weight value of the i-th sub-class pseudo label is The detection probability distribution of the defect corresponding to the i-th sub-class pseudo label under the current curvature parameter is The detection probability distribution of the defect corresponding to the i-th sub-class pseudo label under the current circuit reflection degree (microcircuit reflection parameter), both of which can be determined by the historical detection model.

[0175] Then, the terminal constructs a multi-classification cross-entropy function according to the gray scale discrimination weight value and the gray scale error weight value, and the function is as follows:

[0176]

[0177] Finally, the terminal constructs a sub-class classification target. By incorporating the gray scale error weight value, which is generated according to the curvature parameter and the microcircuit reflection parameter of the new display screen, the calculated multi-classification loss value is more consistent with the new folding screen, and the single-classification detection model can be better updated.

[0178] Referring to Figure 8 The present application provides an embodiment of a device for detecting single-category anomalies on a display screen, comprising:

[0179] The first construction unit 801 is used to construct a single-class detection model and a single-class loss function.

[0180] The first acquisition unit 802 is configured to acquire a training image of the new display screen that is labeled with a single category label, and input the training image into a single category detection model.

[0181] Optionally, the first acquiring unit 802 includes:

[0182] Acquire captured images of the novel display screen, and divide the captured images into abnormal images and non-abnormal images.

[0183] The abnormal image is combined with the non-abnormal image, and the data volume is expanded by image enhancement technology to generate a first abnormal expanded image.

[0184] Identify the reference area in the non-abnormal image, replace the reference area with the corresponding abnormal area in the abnormal image, and assimilate the replaced abnormal area to generate a second abnormal expanded image. The reference area is the area where the authority influence weight reaches a preset value.

[0185] The non-abnormal image is disturbed to generate a third abnormal expanded image.

[0186] The abnormal image, the non-abnormal image, the first abnormal expanded image, the second abnormal expanded image, and the third abnormal expanded image are annotated with single-category labels and determined as training images.

[0187] Feed the training images into the single-class detection model.

[0188] The first generating unit 803 is configured to perform shallow feature enhancement processing on the training image according to the single category label to generate a feature map.

[0189] Optionally, the new display screen is a folding screen with a microcircuit layer in the folding area, the single-category detection model is a folding screen single-category detection model, the training image is a captured image of the folding area of ​​the folding screen, and the single-category label is a single-category label for the folding area.

[0190] The first generating unit 803 includes:

[0191] The gain data of the training image is determined to be the curvature of the folding screen and the reflection of the folding screen microcircuit based on the single category label of the folding area.

[0192] Determine the curvature parameters of the folded regions of the training images.

[0193] The first folding screen and the second folding screen are set according to the curvature parameter, light rays with a preset brightness value are projected onto the first folding screen and the second folding screen by an external light source, the first folding screen is a folding screen without a microcircuit layer in the folding area, and the second folding screen is a folding screen with a microcircuit layer in the folding area.

[0194] Images of the first folding screen and the second folding screen are collected respectively, and the microcircuit reflection parameter is calculated according to the folding areas of the two images.

[0195] The shallow feature gain convolution kernel is constructed according to the curvature parameter and the circuit reflection parameter.

[0196] The shallow feature enhancement processing is performed on the training image by the shallow feature gain convolution kernel, and a feature map is generated.

[0197] The second generation unit 804 is used for extracting shallow features of the training image and the feature map respectively, performing feature fusion on the two groups of shallow features, extracting deep features, and generating a binary classification prediction result according to the single classification head for the deep features.

[0198] The first training unit 805 is used for calculating a first loss value according to a single classification loss function and the binary classification prediction result, updating the single classification detection model according to the first loss value, and stopping until an initial training target is reached.

[0199] The third generation unit 806 is used for clustering the deep features and generating a sub-class pseudo label for each training image.

[0200] The second construction unit 807 is used for expanding a multi-classification head on the single classification detection model, and constructing a multi-classification loss function and a sub-class classification target.

[0201] Optionally, the second construction unit 807 includes:

[0202] A multi-classification head for detecting the probability distribution of the sub-class pseudo label is expanded on the single classification detection model.

[0203] The gray scale discrimination weight value of each sub-class pseudo label on the new display screen is determined.

[0204] The gray scale error weight value is generated according to the curvature parameter and the microcircuit reflection parameter of the new display screen.

[0205] The multi-classification cross-entropy function is constructed according to the gray scale discrimination weight value and the gray scale error weight value.

[0206] The sub-class classification target is constructed.

[0207] The second training unit 808 is used to jointly train the feature extractor, two classification heads and subcategory pseudo labels in the single classification detection model through deep features, multi-classification heads, multi-classification loss function and subcategory classification targets until the single classification head reaches the training target.

[0208] The second acquiring unit 809 is configured to remove the extended multi-classification head and acquire an unlabeled image in a new scene.

[0209] The perturbation unit 810 is configured to copy the unlabeled image and perform perturbation processing on the copied unlabeled image.

[0210] Optionally, the new display screen is a folding screen with a microcircuit layer in the folding area, the single-classification detection model is a single-classification detection model for the folding screen, and the unlabeled image is a real-time image of the folding area of ​​the folding screen on the production line.

[0211] The disturbance unit 810 includes:

[0212] Copy the unlabeled image.

[0213] Light source perturbations are introduced into the copied unlabeled image according to the curvature parameters of the folding screen in the folding region.

[0214] Noise perturbation is introduced into the perturbed unlabeled image by cropping and rotating it.

[0215] The fourth generating unit 811 is used to perform shallow feature enhancement processing on the unlabeled image to generate a scene feature map.

[0216] The fifth generating unit 812 is used to extract shallow features from the unlabeled image, the perturbed unlabeled image, and the scene feature map, and generate a first feature, a second feature, and a third feature, respectively.

[0217] The sixth generating unit 813 is configured to fuse the first feature and the third feature to generate a first fused feature, and then extract a depth feature to generate a first depth feature.

[0218] The seventh generating unit 814 is configured to fuse the second feature and the third feature to generate a second fused feature, and then perform depth feature extraction to generate a second depth feature.

[0219] The eighth generating unit 815 is configured to input the first depth feature and the second depth feature into a single classification head to generate a normal prediction result and a perturbation prediction result.

[0220] The ninth generating unit 816 is used to construct a consistency loss function and generate a consistency loss according to the conventional prediction result, the perturbation prediction result and the consistency loss function.

[0221] The updating unit 817 is used to update the parameters applicable to the new scenario of the single-class detection model according to the consistency loss.

[0222] The labeling unit 818 is used to precisely label the unlabeled images when the conventional prediction results and the perturbation prediction results show that there are unlabeled images at the classification boundary.

[0223] The optimization unit 819 is used to determine the precisely labeled unlabeled images as untrained images, and use the untrained images to optimize the single-class detection model during the production process.

[0224] See also Figure 9 , the present application provides a device for detecting single-category anomalies in a display screen, comprising:

[0225] Processor 901 , memory 902 , input / output unit 903 , and bus 904 .

[0226] The processor 901 is connected to the memory 902 , the input / output unit 903 , and the bus 904 .

[0227] The memory 902 stores a program, and the processor 901 calls the program to execute the following Figure 1 、 Figure 2 and Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 and Figure 7 The method in .

[0228] The present application provides a computer-readable storage medium, wherein a program is stored on the computer-readable storage medium, and when the program is executed on a computer, the program performs the following operations: Figure 1 、 Figure 2 and Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 and Figure 7 The method in .

[0229] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0230] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0231] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0232] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0233] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, and other media that can store program code.

Claims

1. A method for detecting single-category anomalies in a display screen, characterized in that: include: Construct a single-class detection model and a single-class loss function; Obtaining a training image of the new display screen marked with a single category label, and inputting the training image into the single category detection model; Performing shallow feature enhancement processing on the training image according to the single category label to generate a feature map; Extracting shallow features of the training image and the feature map respectively, performing feature fusion on the two sets of shallow features, extracting deep features, and generating a binary classification prediction result for the deep features based on a single classification head; Calculating a first loss value according to the single-class loss function and the binary classification prediction result, and updating the single-class detection model according to the first loss value until an initial training target is reached; Clustering the deep features to generate subcategory pseudo labels for each training image; Expand a multi-classification head based on the single-class detection model and construct a multi-classification loss function and sub-class classification objectives; Jointly training a feature extractor, two classification heads, and subcategory pseudo labels in a single-class detection model using the deep features, the multi-classification head, the multi-classification loss function, and the subcategory classification target until the single-classification head reaches the training target; Remove the extended multi-classification head to obtain unlabeled images in the new scene; Copying the unlabeled image, and performing perturbation processing on the copied unlabeled image; Performing shallow feature enhancement processing on the unlabeled image to generate a scene feature map; Extracting shallow features from the unlabeled image, the perturbed unlabeled image, and the scene feature map to generate a first feature, a second feature, and a third feature, respectively; The first feature and the third feature are fused to generate a first fused feature, and then a depth feature is extracted to generate a first depth feature; The second feature and the third feature are fused to generate a second fused feature, and then the depth feature is extracted to generate a second depth feature; Inputting the first depth feature and the second depth feature into the single classification head to generate a normal prediction result and a perturbation prediction result; Constructing a consistency loss function, and generating a consistency loss according to the conventional prediction result, the perturbation prediction result, and the consistency loss function; The single-class detection model is updated with parameters applicable to the new scenario according to the consistency loss.

2. The method according to claim 1, characterized in that After the step of updating the parameters of the single-class detection model applicable to the new scenario according to the consistency loss, the method further includes: When the conventional prediction results and the perturbation prediction results show that there are unlabeled images at the classification boundary, these unlabeled images are precisely labeled; The precisely annotated unlabeled images are determined as untrained images, and the untrained images are used to optimize the single-class detection model during the production process.

3. The method according to claim 1, characterized in that The novel display screen is a folding screen with a microcircuit layer provided in the folding area, the single-classification detection model is a single-classification detection model for folding screens, and the unlabeled image is a real-time image of the folding area of ​​the folding screen on a production line; The step of copying the unlabeled image and performing perturbation processing on the copied unlabeled image comprises: Copying the unlabeled image; introducing light source disturbance into the copied unlabeled image according to a curvature parameter of the folding screen in the folding area; Noise perturbation is introduced into the perturbed unlabeled image by cropping and rotating it.

4. The method according to any one of claims 1 to 3, characterized in that The step of obtaining a training image of the novel display screen marked with a single category label and inputting the training image into the single category detection model includes: Acquire a captured image of the novel display screen, and divide the captured image into abnormal images and non-abnormal images; Combining the abnormal image with the non-abnormal image, and expanding the data volume by image enhancement technology to generate a first abnormal expanded image; Identifying a reference region in a non-abnormal image, replacing the reference region with a corresponding abnormal region in the abnormal image, and performing assimilation processing on the replaced abnormal region to generate a second abnormal expanded image, wherein the reference region is a region where the authority influence weight reaches a preset value; Performing a perturbation process on the non-abnormal image to generate a third abnormal expanded image; Annotating the abnormal image, the non-abnormal image, the first abnormal expanded image, the second abnormal expanded image, and the third abnormal expanded image with single-category labels and determining them as training images; The training image is input into the single-class detection model.

5. The method according to any one of claims 1 to 3, characterized in that The novel display screen is a folding screen with a microcircuit layer in the folding area, the single-classification detection model is a folding screen single-classification detection model, the training image is a photographic image of the folding area of ​​the folding screen, and the single-class label is a single-class label of the folding area; Performing shallow feature enhancement processing on the training image according to the single category label to generate a feature map, including: The gain data of the training image is determined to be the curvature of the folding screen and the reflection of the folding screen microcircuit according to the single-category label of the folding area; determining curvature parameters of the folded region of the training image; Setting a first folding screen and a second folding screen according to the curvature parameter, projecting light of a preset brightness value onto the first folding screen and the second folding screen through an external light source, wherein the first folding screen is a folding screen without a microcircuit layer in the folding area, and the second folding screen is a folding screen with a microcircuit layer in the folding area; respectively capturing images of the first folding screen and the second folding screen, and calculating microcircuit reflection parameters based on the folding areas of the two images; Construct a shallow feature gain convolution kernel based on curvature parameters and circuit reflection parameters; The shallow feature gain convolution kernel is used to perform shallow feature enhancement on the training image to generate a feature map.

6. The method according to claim 5, characterized in that The steps of extending a multi-classification head on a single-classification detection model and constructing a multi-classification loss function and sub-classification objectives include: A multi-classification head is extended on the single-classification detection model to detect the probability distribution of the sub-class pseudo labels; Determine the grayscale discrimination weight value of each subcategory pseudo label on the new display screen; generating a grayscale error weight value according to a curvature parameter of the novel display screen and a microcircuit reflection parameter; Constructing a multi-classification cross entropy function according to the grayscale discrimination weight value and the grayscale error weight value; Construct subcategory classification targets.

7. A device for detecting single-category anomalies on a display screen, characterized in that: include: The first construction unit is used to construct a single-class detection model and a single-class loss function; A first acquisition unit is configured to acquire a training image of the new display screen marked with a single category label, and input the training image into the single category detection model; A first generating unit is configured to perform shallow feature enhancement processing on the training image according to the single category label to generate a feature map; a second generation unit, configured to extract shallow features of the training image and the feature map respectively, perform feature fusion on the two sets of shallow features, extract deep features, and generate a binary classification prediction result for the deep features according to a single classification head; a first training unit, configured to calculate a first loss value according to the single-class loss function and the binary classification prediction result, and update the single-class detection model according to the first loss value until an initial training target is reached; a third generating unit, configured to perform clustering processing on the deep features and generate a subcategory pseudo label for each training image; A second construction unit is used to expand a multi-classification head based on the single-classification detection model and construct a multi-classification loss function and a sub-classification target; A second training unit is configured to jointly train a feature extractor, two classification heads, and subcategory pseudo labels in a single-class detection model using the deep features, the multi-classification head, the multi-classification loss function, and the subcategory classification target until the single-classification head reaches the training target; The second acquisition unit is used to remove the extended multi-classification head and obtain an unlabeled image in a new scene; a perturbation unit, configured to copy the unlabeled image and perform perturbation processing on the copied unlabeled image; a fourth generating unit, configured to perform shallow feature enhancement processing on the unlabeled image to generate a scene feature map; a fifth generating unit, configured to extract shallow features from the unlabeled image, the perturbed unlabeled image, and the scene feature map, and generate a first feature, a second feature, and a third feature, respectively; a sixth generating unit, configured to fuse the first feature and the third feature to generate a first fused feature, and then extract a depth feature to generate a first depth feature; a seventh generating unit, configured to fuse the second feature and the third feature to generate a second fused feature, and then extract a depth feature to generate a second depth feature; An eighth generating unit, configured to input the first depth feature and the second depth feature into the single classification head to generate a normal prediction result and a perturbation prediction result; a ninth generating unit, configured to construct a consistency loss function, and generate a consistency loss according to the conventional prediction result, the perturbation prediction result, and the consistency loss function; An updating unit is used to update the parameters of the single-class detection model applicable to the new scenario according to the consistency loss.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 6 is executed.

Images