Screen detection method and device, electronic equipment and computer readable storage medium

Abstract

The application provides a screen detection method and device, electronic equipment and computer readable storage medium. The method comprises: acquiring a to-be-recognized image displayed on a screen; recognizing the to-be-recognized image to acquire a frequency domain feature and a spatial domain feature of the to-be-recognized image; performing pixel cycle attention weighting on the spatial domain feature based on a preset screen pixel arrangement template to generate a periodic suppression coefficient; processing the spatial domain feature according to the periodic suppression coefficient and fusing the spatial domain feature with the frequency domain feature to obtain a fused feature of the to-be-recognized image; and performing screen defect detection based on the fused feature.

Description

Technical Field

[0001] This application relates to the field of screen detection technology, and in particular to a screen detection method, apparatus, electronic device and computer-readable storage medium. Background Technology

[0002] In the manufacturing process of electronic screens, surface defect detection is a crucial step in ensuring product quality. With continuously increasing pixel density, the requirements for the detection accuracy of minute targets such as line defects and point defects are becoming increasingly stringent. Traditional image processing methods (such as thresholding) rely on manual features, which are difficult to respond effectively. Meanwhile, deep learning methods based on CNNs (such as YOLO and U-Net) mainly rely on spatial domain features, which lack sufficient sensitivity to weak defects in high-frequency components, resulting in low detection rates and high false negative rates. Summary of the Invention

[0003] This application provides a screen detection method, including: acquiring a screen-displayed image to be identified; identifying the image to be identified to obtain frequency domain features and spatial domain features of the image to be identified; performing pixel periodic attention weighting on the spatial domain features based on a preset screen pixel arrangement template to generate a periodic suppression coefficient; processing the spatial domain features according to the periodic suppression coefficient and fusing them with the frequency domain features to obtain fused features of the image to be identified; and detecting screen defects based on the fused features.

[0004] In some embodiments, obtaining frequency domain features includes: determining the Nyquist frequency of the screen based on the pixel pitch of the screen; moving the target frequency component in the frequency domain features to the center of the spectrum to generate target frequency domain features of the image to be identified; constructing a low-frequency sub-band mask, a mid-frequency sub-band mask, and a high-frequency sub-band mask based on the Nyquist frequency and the target frequency domain features; and performing frequency band segmentation on the frequency domain features based on the low-frequency sub-band mask, the mid-frequency sub-band mask, the high-frequency sub-band mask, and the Nyquist frequency to obtain the corresponding low-frequency sub-band features, mid-frequency sub-band features, and high-frequency sub-band features.

[0005] In some embodiments, constructing a low-frequency subband mask, a mid-frequency subband mask, and a high-frequency subband mask based on the Nyquist frequency and target frequency domain features includes: determining the radius corresponding to the Nyquist frequency based on the target frequency domain features and the size information of the image to be identified; and constructing the low-frequency subband mask, the mid-frequency subband mask, and the high-frequency subband mask based on the frequency coordinates of the target frequency domain features and the radius corresponding to the Nyquist frequency.

[0006] In some embodiments, based on a preset screen pixel arrangement template, performing pixel periodic attention weighting on spatial domain features to generate periodic suppression coefficients includes: performing convolution processing on spatial domain features to generate a single-channel feature map; multiplying the single-channel feature map element-wise with the screen pixel arrangement template to generate mask features; and mapping the mask features to a preset interval through an activation function to obtain periodic suppression coefficients.

[0007] In some embodiments, processing spatial domain features according to periodic suppression coefficients and fusing them with frequency domain features to obtain fused features of the image to be identified includes: processing spatial domain features according to periodic suppression coefficients to obtain target spatial domain features; fusing the target spatial domain features with low-frequency sub-band features, mid-frequency sub-band features, and high-frequency sub-band features to obtain fused features of the image to be identified.

[0008] In some embodiments, the method further includes: determining the pixel arrangement type of the screen and obtaining the pixel repetition unit of the pixel arrangement according to the pixel arrangement type; assigning weights to each sub-pixel in the pixel arrangement to obtain the weight of each sub-pixel; and generating a screen pixel arrangement template with the same size as the image to be recognized based on the pixel repetition unit and the weight of each sub-pixel.

[0009] In some embodiments, obtaining pixel repetition units of a pixel arrangement based on the pixel arrangement type includes: analyzing the distribution order of the colors of each sub-pixel in the pixel arrangement along the row and column directions; determining the minimum period of color repetition in the row direction and the minimum period of color repetition in the column direction based on the distribution order; and constructing pixel repetition units using the sub-pixel matrix defined by the minimum period in the row direction and the minimum period in the column direction.

[0010] This application discloses a screen detection device, comprising: an acquisition module for acquiring an image to be identified displayed on the screen; an identification module for identifying the image to be identified and acquiring frequency domain features and spatial domain features of the image to be identified; a weighting module for performing pixel periodic attention weighting on the spatial domain features based on a preset screen pixel arrangement template to generate a periodic suppression coefficient; a processing module for processing the spatial domain features according to the periodic suppression coefficient and fusing them with the frequency domain features to obtain fused features of the image to be identified; and a detection module for detecting screen defects based on the fused features.

[0011] This application provides an electronic device, including a screen, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described above.

[0012] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0013] The beneficial effects of the screen detection method in this application are as follows: By simultaneously extracting and utilizing the frequency domain features and spatial domain features of the image to be identified, it overcomes the inherent problem of existing deep learning methods (such as YOLO and U-Net) which rely solely on spatial domain features and lack sufficient sensitivity to weak defects in high-frequency components. By applying pixel periodic attention weighting to the spatial domain features using a preset screen pixel arrangement template, a periodic suppression coefficient is generated. This adaptively weakens the background response related to pixel periods, thereby reducing the interference of background periodic structures on defect detection and improving the saliency of real defects. After periodic suppression processing of the spatial domain features, they are fused with the frequency domain features. This fused feature can more comprehensively characterize the morphology and spectral characteristics of minute defects in high-density screen images, thereby improving the accuracy and robustness of subsequent defect detection. Due to its enhanced sensitivity to high-frequency weak defects and suppression of the response to periodic background interference, in actual screen defect detection, it can significantly reduce the false negative rate of small point and line defects compared with traditional spatial domain CNN methods (such as YOLO and U-Net), and can also improve the overall detection rate, especially in the manufacturing process of high pixel density screens (such as OLED and Micro-LED). Attached image description: Figure 1 The flowchart illustrating an embodiment of the screen detection method of this application is shown in the diagram. Figure 2 A flowchart illustrating the method for obtaining frequency domain features according to an embodiment of this application is shown in the diagram. Figure 3 The flowchart illustrating step S230 of an embodiment of this application is shown in the schematic diagram. Figure 4 The flowchart illustrating step 130 of an embodiment of this application is shown schematically. Figure 5 The flowchart illustrating step S140 of an embodiment of this application is shown in the schematic diagram. Figure 6 A flowchart illustrating another embodiment of the screen detection method of this application is shown schematically; Figure 7 A block diagram of a screen detection device according to an embodiment of this application is shown schematically; Figure 8 A schematic block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0014] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to implement them more readily. Furthermore, for clarity, portions unrelated to the description of exemplary embodiments have been omitted from the drawings. In this disclosure, it should be understood that terms such as "comprising" or "having" are intended to indicate the presence of features, numbers, steps, behaviors, components, portions, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, numbers, steps, behaviors, components, portions, or combinations thereof. It should also be noted that, without conflict, embodiments and features in the embodiments of this disclosure can be combined with each other. The present disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0015] The term "screen" as used in this application refers to a display panel used in electronic display devices, including but not limited to liquid crystal displays (LCDs), organic light-emitting diode displays (OLEDs), micro-LEDs, mini-LEDs, and active-matrix organic light-emitting diode displays (AMOLEDs). These screens have a regular pixel arrangement structure, typically composed of periodically arranged pixel units, each pixel unit potentially containing sub-pixels of different colors (e.g., red R, green G, and blue B sub-pixels).

[0016] During the screen manufacturing process, various structural abnormalities may occur on the screen surface or inside due to the influence of materials, processes, or environmental factors; these are collectively referred to as "screen defects." The screen defects of concern in this application include at least the following types: Point defects: These are abnormal points where a single pixel or a few adjacent pixels cannot emit light normally, appearing as bright spots, dark spots, or discolored spots. These defects are extremely small in scale, usually occupying only a few pixels, and appear as isolated high-frequency variations in the image.

[0017] Line defects: These refer to pixel anomalies that are continuously distributed along the row or column direction, appearing as bright lines, dark lines, or colored lines. Line defects have a certain degree of continuity in the spatial domain, but their width is usually only one or a few pixels, and they are still considered micro-targets.

[0018] Mura defects refer to regional defects in screen brightness or color, without obvious boundaries, and with low contrast. These defects manifest as abnormal low- or mid-frequency components in the frequency domain, but their characteristics are weak in the spatial domain and are easily masked by changes in background brightness.

[0019] The common characteristics of the above defects are: small scale, low contrast, and easy confusion with the inherent periodic pixel structure of the screen, which places extremely high demands on the sensitivity and accuracy of the automatic optical inspection system.

[0020] In the manufacturing process of electronic screens, surface defect detection is a crucial step in ensuring product quality. As screen pixel density continues to increase (e.g., 4K, 8K, and even higher resolutions), the requirements for the detection accuracy of minute targets such as point and line defects are becoming increasingly stringent. In recent years, deep learning methods based on convolutional neural networks (CNNs), such as the YOLO series, U-Net, and their variants, have been applied to screen defect detection tasks. These methods, by learning from a large number of labeled samples, can automatically extract defect features in the spatial domain, improving the automation level of detection to some extent. However, it should be noted that existing CNN methods mainly rely on spatial domain features for defect identification. Spatial domain convolution operations inherently tend to extract low-frequency components and large-scale structural information in images, having limited ability to utilize the weak defect information carried by high-frequency components in images (such as edges, textures, and subtle gray-level abrupt changes).

[0021] Specifically, point and line defects manifest as grayscale anomalies in only a few pixels in the spatial domain, with signal energy far lower than the periodic pixel structure of the screen background. Existing CNN models, during multi-layer convolution and pooling processes, tend to filter out or ignore these weak high-frequency signals as noise, leading to the following problems: Low detection rate: A large number of real defects cannot be effectively identified by the model; High rate of missed detection: especially for defects with low contrast and small size, the phenomenon of missed detection is particularly serious. Slow response to defects in high-frequency components: If defect information is mainly present in the high-frequency components of the frequency domain, existing spatial domain CNN methods are almost unable to capture it effectively.

[0022] To address the above problems, this application provides a screen detection method, specifically as follows: Figure 1 The flowchart illustrating an embodiment of the screen detection method of this application is shown.

[0023] The screen detection method includes steps S110 to S150.

[0024] In step S110, the image to be recognized displayed on the screen is acquired.

[0025] For example, the image to be identified refers to the display screen shown when the screen is lit. To facilitate defect detection, this display screen is typically set to a uniform grayscale image (such as pure gray, pure white, pure red, pure green, or pure blue) or a specific test pattern (such as stripes or grid patterns) so that screen defects can stand out from the uniform background. It is understood that in online inspection scenarios on production lines, images of the screen under normal display conditions can also be directly acquired; this application does not limit this.

[0026] In step S120, the image to be identified is identified to obtain the frequency domain features and spatial domain features of the image to be identified.

[0027] For example, after acquiring the image to be identified, this application can execute two feature extraction branches in parallel: a frequency domain feature extraction branch and a spatial domain feature extraction branch. The former transforms the image to the frequency domain through Fourier transform to extract spectral features related to weak high-frequency defects; the latter extracts the spatial domain semantic features of the image through a lightweight convolutional network (such as EfficientNet).

[0028] In step S130, based on a preset screen pixel arrangement template, pixel periodic attention weighting is applied to the spatial domain features to generate a periodic suppression coefficient.

[0029] For example, after extracting the spatial domain features, the degree of interference from the screen periodic background is calculated on the spatial domain feature map according to the screen pixel arrangement template, and a suppression coefficient is generated accordingly. This makes the areas with strong periodic background response weakened in subsequent processing, while the areas that deviate from the periodic pattern (which may correspond to real defects) are preserved or enhanced.

[0030] Specifically, the screen pixel arrangement template can be a digital description of the periodic arrangement pattern of pixel units (e.g., sub-pixels) in the screen type to be detected. Different types of screens (such as standard RGB arrangement, Pentile arrangement, Delta arrangement, etc.) have different pixel periodic structures, so it is necessary to pre-set the corresponding template according to the specific detection object.

[0031] In step S140, the spatial domain features are processed according to the periodic suppression coefficient and fused with the frequency domain features to obtain the fused features of the image to be identified.

[0032] For example, the periodicity suppression coefficient characterizes the degree to which each location in the spatial domain feature map is affected by the periodic background of the screen. The purpose of suppression processing is to weaken the feature response of the periodic background region while preserving or relatively enhancing the feature response of the defect region. For instance, the periodicity suppression coefficient ranges from 0 to 1. When the periodicity suppression coefficient is close to 1, it indicates that the location is strongly affected by the periodic background, that is, the feature value at that location is significantly weakened; when the periodicity suppression coefficient is close to 0, it indicates that the location deviates from the periodic pattern and may be a defect region, that is, the feature value at that location is completely preserved.

[0033] In this embodiment, before feature fusion, the frequency domain features and the suppressed spatial domain features can be aligned so that they have the same spatial size and channel dimension, which facilitates tensor-level splicing or weighted fusion.

[0034] In step S150, screen defect detection is performed based on fusion features.

[0035] For example, after obtaining the fused features, a screen defect detection step is performed. This step takes the fused features as input. For example, the fused features can be processed by a detection head network to output the defect category, confidence score, and defect location information in the screen image, thereby achieving accurate identification and location of various screen defects such as point defects, line defects, and Mura defects.

[0036] The screen detection method provided in this application overcomes the inherent problem of existing deep learning methods (such as YOLO and U-Net) which rely solely on spatial domain features and lack sensitivity to subtle defects in high-frequency components by simultaneously extracting and utilizing both frequency domain and spatial domain features of the image to be identified. By applying pixel periodic attention weighting to the spatial domain features using a preset screen pixel arrangement template, a periodic suppression coefficient is generated. This adaptively weakens background responses related to pixel periods, thereby reducing the interference of background periodic structures on defect detection and improving the saliency of real defects. After periodic suppression processing of the spatial domain features, they are fused with the frequency domain features. This fused feature can more comprehensively characterize the morphology and spectral characteristics of minute defects in high-density screen images, thereby improving the accuracy and robustness of subsequent defect detection. Due to its enhanced sensitivity to high-frequency weak defects and suppression of the response to periodic background interference, in actual screen defect detection, it can significantly reduce the false negative rate of small point and line defects compared with traditional spatial domain CNN methods (such as YOLO and U-Net), and can also improve the overall detection rate, especially in the manufacturing process of high pixel density screens (such as OLED and Micro-LED).

[0037] Figure 2 A flowchart illustrating the method for obtaining frequency domain features according to an embodiment of this application is shown.

[0038] like Figure 2 As shown, the acquisition of frequency domain features includes steps S210 to S240.

[0039] In step S210, the Nyquist frequency of the screen is determined based on the pixel pitch of the screen.

[0040] In this application, in order to effectively separate the periodic background of the screen from the real defect signal from the frequency domain features, it is necessary to introduce the Nyquist frequency as the theoretical basis for frequency domain analysis. The Nyquist frequency is the highest effective frequency to avoid spectral aliasing in signal processing, which is determined by the physical pixel spacing of the screen.

[0041] For example, a screen image is essentially the result of discrete sampling of a continuously spatially distributed array of pixels. According to the Nyquist-Shannon sampling theorem, in order to recover the original continuous signal from the discrete sampled signal without distortion, the sampling frequency must be at least twice the highest frequency component of the signal. Accordingly, the Nyquist frequency is defined as half the sampling frequency, representing the highest frequency component that the discrete signal can be effectively represented without aliasing.

[0042] Specifically, the Nyquist frequency can be calculated using the following formula. :

[0043] in, This indicates the pixel pitch of the screen.

[0044] In step S220, the target frequency component in the frequency domain feature is moved to the center of the spectrum to generate the target frequency domain feature of the image to be identified.

[0045] For example, this frequency domain feature can be obtained using the following Fourier transform formula:

[0046] in, These are spatial coordinates (row and column indices of the image). Indicates the horizontal position of an image pixel. Indicates the vertical position of an image pixel. For frequency domain coordinates (frequency component index). This represents the frequency components of the image in the horizontal direction. M represents the frequency components of the image in the vertical direction. The total number of pixels in the direction and the number of pixels in each row, where N is the total number of pixels in the image in the direction, i.e. the number of pixels in each column, j is the imaginary unit, and e is the base of the natural logarithm.

[0047] The initial spectrum obtained after Fourier transform The energy distribution of this spectrum is not intuitive. Specifically, the zero-frequency component (also known as the DC component), which represents the overall average brightness of the image, is located in the four corners of the spectrum, not in the center. For example, low-frequency components (corresponding to slowly changing background areas in the image) are closer to the corners, while high-frequency components (corresponding to rapidly changing components such as edges, textures, and minor defects in the image) are located in the center of the spectrum. This "low-frequency in the corners, high-frequency in the center" distribution is not conducive to intuitive analysis and subsequent frequency domain filtering operations.

[0048] In this case, this application performs a spectral centering process. The essence of centering is to shift the spectrum, moving the zero-frequency components from the four corners to the exact center. After centering, low-frequency components are concentrated in the central region, while high-frequency components are distributed in the edge regions. This "low-frequency at the center, high-frequency at the edge" distribution in this embodiment aligns with human intuition and facilitates the selective extraction of frequency components relevant to screen defect detection.

[0049] Specifically, the target frequency component in the frequency domain features is shifted to the center of the spectrum using the following formula to generate the target frequency domain features of the image to be identified. :

[0050] in, For frequency domain coordinates (frequency component index). This represents the frequency components of the image in the horizontal direction. M represents the frequency components of the image in the vertical direction. The total number of pixels in each direction, i.e., the number of pixels in each row, where N is the image size in the direction of the direction. The total number of pixels in each direction, i.e., the number of pixels in each column. This is a modulo operation. In this embodiment, the zero-frequency component can be shifted to the center of the spectrum using the above method, thus achieving low frequencies in the center and high frequencies at the edges.

[0051] In step S230, a low-frequency subband mask, a mid-frequency subband mask, and a high-frequency subband mask are constructed based on the Nyquist frequency and the target frequency domain characteristics.

[0052] For example, the Nyquist frequency characterizes the highest frequency component that a discrete signal can effectively represent while avoiding spectral aliasing. In this application, the Nyquist frequency is determined by the physical pixel pitch of the screen: the smaller the pixel pitch, the higher the Nyquist frequency; the larger the pixel pitch, the lower the Nyquist frequency.

[0053] A subband mask is a binary matrix or weight matrix with the same spatial dimensions as the target frequency domain features. Each element in the mask corresponds to a specific frequency position in the frequency domain, and its value indicates which subband that frequency position belongs to.

[0054] In this embodiment, after completing spectrum centering and obtaining the target frequency domain features, this application further divides the frequency domain into multiple sub-bands based on the Nyquist frequency and constructs corresponding sub-band masks. Specifically, this application constructs three types of sub-band masks: low-frequency sub-band masks, mid-frequency sub-band masks, and high-frequency sub-band masks. For example, the frequency domain features are finely divided according to frequency, so that features of different frequency bands can be processed separately, thereby more effectively detecting screen defects of different types and scales.

[0055] In step S240, frequency domain features are segmented based on low-frequency subband mask, mid-frequency subband mask, high-frequency subband mask and Nyquist frequency to obtain low-frequency subband features, mid-frequency subband features and high-frequency subband features.

[0056] For example, low-frequency subband features primarily retain frequency components near the center of the spectrum in the target frequency domain features. These components correspond to slowly changing information in the image, specifically including: the overall brightness level of the screen, brightness gradients over large areas, and illumination non-uniformity. In the frequency domain, the energy of low-frequency subband features typically accounts for the majority of the total frequency domain energy because the background brightness and large uniform areas of the screen constitute the main body of the image. However, for the detection of minute defects, the information value of low-frequency subband features is relatively low because the energy of most defects (especially point and line defects) is concentrated in the high-frequency region. Typical characteristics of low-frequency subband features are: large numerical values ​​(concentrated energy), gradual changes, and simple spatial structure.

[0057] The mid-frequency subband feature preserves the frequency components in the target frequency domain feature that surround the Nyquist frequency. These components correspond to moderately changing information in the image, specifically including: the periodic fundamental frequency component of the screen pixel arrangement, the low-order harmonics of the pixel period, and medium-sized structures (such as wide lines and blurred edges). The mid-frequency subband feature is key to identifying the periodic background of the screen. For regularly arranged screens, obvious peaks appear in the mid-frequency subband feature, and the positions of these peaks precisely correspond to the periodic direction and period length of the pixel arrangement. Typical characteristics of the mid-frequency subband feature are: the existence of periodically distributed peaks, a clear directionality in energy distribution, and peak positions that are related to the screen type.

[0058] High-frequency subband features retain frequency components in the target frequency domain features that are far from the spectral center. These components correspond to rapidly changing information in the image, specifically including: isolated gray-level abrupt changes caused by point defects, thin line edges caused by line defects, defect boundary contours, and random noise in the image. High-frequency subband features are the core of detecting minute defects. For screen images with point or line defects, significant energy responses will appear in the high-frequency subband features at the locations corresponding to the defects. For perfect screens without defects, the high-frequency subband features mainly contain random noise and high-order harmonic remnants of pixel periods. Typical characteristics of high-frequency subband features are: small numerical values ​​(dispersed energy), rapid changes, and irregular spatial distribution.

[0059] In this embodiment, the correspondence between the features and defects of each sub-band can be shown in the following table:

[0060] In this embodiment, after constructing the low-frequency, mid-frequency, and high-frequency sub-band masks, this application further performs frequency band segmentation on the frequency domain features based on these sub-band masks and the Nyquist frequency. For example, the complete frequency domain features are decomposed into three independent sub-band features according to the frequency, corresponding to the low-frequency sub-band features, mid-frequency sub-band features, and high-frequency sub-band features, respectively, thereby laying the foundation for subsequent specialized processing for different frequency bands.

[0061] Specifically, the frequency domain features are segmented using the following formula:

[0062] in, Subband characteristics , and These correspond to the low, medium, and high sub-band features, respectively. This represents the frequency domain features after Fourier transform (i.e., the frequency domain features mentioned above). , and For low-frequency subband masks, mid-frequency subband masks, and high-frequency subband masks, It is the Nyquist frequency.

[0063] The embodiments of this application decompose the originally mixed full-band information into three sub-bands with clear physical meanings, allowing subsequent processing to employ different strategies for different sub-bands. Through frequency band segmentation, only high-frequency sub-band features can be used for the initial localization of minor defects, eliminating the need to process the entire full-band features and reducing the amount of data required for subsequent computation. By analyzing the energy distribution of the three sub-band features, it is possible to determine which frequency band the defect primarily occurs in, thereby inferring the defect type and scale. For example, a significant response in the high-frequency sub-band features suggests a possible point defect, while abnormal peaks in the mid-frequency sub-band features indicate a possible anomaly in the screen's periodic structure. The three sub-band features can be fed into three independent processing branches for low-frequency background analysis, mid-frequency period extraction, and high-frequency defect detection, respectively. These branches can run in parallel, resulting in high overall processing efficiency.

[0064] Figure 3 The flowchart illustrating step S230 of an embodiment of this application is shown schematically.

[0065] like Figure 3 As shown, step S230 may specifically include steps S310 and S320.

[0066] In step S310, the radius corresponding to the Nyquist frequency is determined based on the target frequency domain features and the size information of the image to be identified.

[0067] For example, before using the Nyquist frequency for subband division, this application first needs to determine the specific radius position corresponding to the Nyquist frequency in the frequency domain features. This can convert the physical Nyquist frequency into an actual distance metric in the frequency domain feature matrix, thereby providing an accurate reference benchmark for subsequent subband mask construction and frequency band division.

[0068] In this embodiment, the height and width of the image to be identified determine the size of the frequency domain feature matrix. For example, if the height of the input image is N pixels and the width is M pixels, the frequency domain feature matrix obtained after Fourier transform will also have M rows and N columns. After centering, the spectral center is located in the center of the matrix, and the maximum distance from the center to the edge of the matrix is ​​the smaller of half of N and half of M, or the average of the two depending on the specific situation. This maximum distance corresponds to the highest physically representable frequency, which is half of the sampling frequency.

[0069] Specifically, after centering the frequency domain graph, assuming the maximum radius of the Nyquist frequency is set to 1 (unit: mm / pixel), the radius corresponding to the Nyquist frequency in the frequency domain graph can be calculated using the following formula. :

[0070]

[0071] Where M represents in The total number of pixels in each direction, i.e., the number of pixels in each row, where N is the image size in the direction of the direction. The total number of pixels in the direction, i.e., the number of pixels in each column.

[0072] In step S320, based on the frequency coordinates of the target frequency domain features and the radius corresponding to the Nyquist frequency, a low-frequency subband mask, a mid-frequency subband mask, and a high-frequency subband mask are constructed.

[0073] For example, after determining the radius corresponding to the Nyquist frequency, this application further constructs low-frequency sub-band masks, mid-frequency sub-band masks, and high-frequency sub-band masks based on the frequency coordinates of the target frequency domain features and the radius. For instance, based on the distance of each frequency component to the center of the spectrum, it is classified into the corresponding sub-band, forming three mask matrices for extracting information from different frequency bands, providing a precise frequency selection tool for subsequent frequency band segmentation. In this embodiment, each frequency component has its unique coordinate position in the frequency domain feature matrix. After centering, the center of the spectrum (zero frequency position) is located in the center of the matrix. For any frequency position in the matrix, its distance to the center of the spectrum can be calculated. This distance reflects the frequency component's intensity: the smaller the distance, the lower the frequency; the larger the distance, the higher the frequency.

[0074] Specifically, a low-frequency subband mask is constructed using the following formula. :

[0075] in, The frequency coordinates represent the target frequency domain characteristics. The frequency coordinates of the center of the spectrum The radius is the radius corresponding to the Nyquist frequency, and 0.2 represents the low-frequency cutoff ratio.

[0076] Construct the mid-frequency subband mask using the following formula. :

[0077] in, The frequency coordinates represent the target frequency domain characteristics. The frequency coordinates of the center of the spectrum The radius is the radius corresponding to the Nyquist frequency, and 0.7 represents the high-frequency cutoff ratio.

[0078] Construct a high-frequency subband mask using the following formula. :

[0079] in, The frequency coordinates represent the target frequency domain characteristics. The frequency coordinates of the center of the spectrum The radius is the radius corresponding to the Nyquist frequency, and 0.7 represents the high-frequency cutoff ratio.

[0080] The embodiments of this application allow each frequency location to be precisely classified into a corresponding sub-band based on its distance from the center of the spectrum, giving frequency band segmentation a clear mathematical basis and physical meaning. The boundaries of the sub-band division are associated with the Nyquist frequency, making the division results not only mathematically meaningful but also physically meaningful at the signal processing level. For example, whether it's a circular, square, elliptical, or adaptive deformation mask, or a hard or soft mask, it can be selected and configured according to the needs of specific application scenarios, offering good flexibility. The constructed sub-band mask can be repeatedly used to extract features of different sub-bands from the target frequency domain features without repeatedly calculating thresholds and boundaries.

[0081] Figure 4 The flowchart illustrating step 130 of an embodiment of this application is shown schematically.

[0082] like Figure 4 As shown, step S130 may specifically include steps S410 to S430.

[0083] In step S410, the spatial domain features are convolved to generate a single-channel feature map.

[0084] For example, after obtaining the spatial domain features, this application further performs convolution processing on the spatial domain features to generate a single-channel feature map. For instance, it compresses the high-dimensional spatial domain feature tensor into a single-channel feature map, providing a suitable data format for the subsequent generation of pixel periodic attention weighting and periodic suppression coefficients.

[0085] Specifically, a single-channel feature map is generated using the following formula. :

[0086] in, It is a feature of the spatial domain.

[0087] In step S420, the single-channel feature map is multiplied element-wise with the screen pixel arrangement template to generate mask features.

[0088] For example, although a single-channel feature map has compressed and refined spatial domain features, it does not contain prior knowledge of the periodicity of screen pixel arrangement. Each position in a single-channel feature map reflects the degree of matching between that local region and the screen periodic pattern, but this matching is based on network learning and may contain biases or instabilities.

[0089] The screen pixel arrangement template is prior knowledge pre-constructed in this application, which precisely describes the periodic arrangement pattern of screen subpixels (e.g., the three horizontal periods of a standard RGB arrangement, or the two-dimensional periodic structure of a Pentile arrangement). Multiplying the single-channel feature map element-wise with the screen pixel arrangement template is equivalent to calibrating and enhancing the features learned by the network using prior knowledge.

[0090] Element-wise multiplication achieves the following: at positions with high weights in the screen pixel arrangement template (i.e., key positions where sub-pixels are located), the response of the single-channel feature map is preserved or enhanced; at positions with low weights or zero weights in the template (i.e., gaps between sub-pixels or non-pixel regions), the response of the single-channel feature map is weakened or suppressed. After this processing, the mask features will more accurately reflect the periodic structure of the screen, making the subsequent generation of periodic suppression coefficients more accurate and reliable.

[0091] Specifically, the mask features can be calculated using the following formula. :

[0092] in, It is a single-channel feature map. The screen pixel arrangement template can be of the following three types in this embodiment:

[0093] In step S430, the mask features are mapped to a preset interval through an activation function to obtain the periodic inhibition coefficient.

[0094] For example, after generating the mask features, this application further maps the mask features to a preset interval through an activation function to obtain periodic suppression coefficients. For example, the original values ​​in the mask features are converted into suppression coefficients with clear physical meanings, so that their values ​​fall within a preset interval (usually between zero and one), so that they can be directly used for subsequent weighted suppression processing of spatial domain features.

[0095] Mask features reflect how well each spatial location matches the periodic pattern of screen pixels. In normal periodic background regions, mask feature values ​​are typically high because the single-channel feature map has a high response at these locations, and the template assigns high weights at sub-pixel positions. In defective regions, mask feature values ​​are typically low because defects disrupt the periodic structure, causing a decrease in the response of the single-channel feature map.

[0096] The physical meaning of the periodic suppression coefficient is exactly the same as the mask characteristics: the stronger the periodic background interference, the larger the suppression coefficient should be, so as to weaken the features of these locations in subsequent processing; the smaller the suppression coefficient should be in defective regions or non-periodic regions, so as to preserve the features of these locations.

[0097] Therefore, there is a positive correlation between mask features and the desired suppression coefficient: the larger the mask feature value, the larger the suppression coefficient should be; the smaller the mask feature value, the smaller the suppression coefficient should be. Activation function mapping needs to maintain this positive correlation while adjusting the numerical range to a preset interval.

[0098] Specifically, the periodic inhibition coefficient can be calculated using the following formula. :

[0099] in, Here, e represents the mask feature, and e is the base of the natural logarithm. In this embodiment, the sigmoid function is used to... Map to the [0, 1] interval. If the mask value is close to 1, it indicates that the location belongs to a periodic normal region (which needs to be suppressed); if it is close to 0, it is a non-periodic region (a defect that needs to be preserved).

[0100] The suppression coefficient obtained after mapping in this embodiment strictly falls within a preset range and is positively correlated with the intensity of periodic background interference. This allows the coefficient value to be directly interpreted as the degree of background interference at that location, facilitating understanding and debugging. The suppression coefficient ranges from zero to one, and when multiplied element-wise with spatial domain features, it does not produce numerical overflow or abnormal scaling effects, ensuring the stability and predictability of the calculation.

[0101] Figure 5 The flowchart illustrating step S140 of an embodiment of this application is shown schematically.

[0102] like Figure 5 As shown, step S140 may specifically include steps S510 and S520.

[0103] In step S510, the spatial domain features are processed according to the periodicity suppression coefficient to obtain the target spatial domain features.

[0104] In step S520, the target spatial domain features are fused with low-frequency sub-band features, mid-frequency sub-band features, and high-frequency sub-band features to obtain the fused features of the image to be identified.

[0105] For example, in obtaining the periodic inhibition coefficient Original spatial domain characteristics and the characteristics of the three sub-bands: low frequency, mid frequency, and high frequency. Subsequently, this application performs feature enhancement and fusion operations. For example, it first uses a periodic suppression coefficient to suppress the spatial domain features, eliminating the interference of the periodic background of the screen, and obtains the target spatial domain features. Then, it deeply fuses the enhanced target spatial domain features with the three sub-band features of the frequency domain to generate information-rich and discriminative fused features, providing high-quality input for subsequent screen defect detection.

[0106] Specifically, the fusion features of the image to be recognized are obtained using the following formula. :

[0107] in, The periodic inhibition coefficient, For spatial domain characteristics, It has three sub-band characteristics: low frequency, medium frequency, and high frequency.

[0108] The spatial domain features of this application provide structural information after background suppression, while the frequency domain features provide weak high-frequency defect information and mid-frequency periodic information. The fusion of these two features, taking advantage of each other's strengths, results in a more comprehensive defect characterization. For example, through (1 - By weighting spatial domain features, spurious responses caused by screen pixel periods are significantly reduced, allowing subsequent detection modules to focus more on genuine defects. Low-frequency sub-band features respond to large-sized Mura defects, mid-frequency sub-band features respond to pixel periodic structures, and high-frequency sub-band features respond to micro-point and line defects. The fused features can simultaneously detect defects of different scales. Combining multiple information sources, the fused features have stronger discriminative power compared to features from a single source, and can better distinguish genuine defects from background noise, process fluctuations, and other interference factors. This application supports multiple fusion methods such as stitching, addition, attention, and progressive fusion, which can be selected and adjusted according to the computational resources, accuracy requirements, and real-time needs of specific application scenarios.

[0109] Figure 6 A flowchart illustrating another embodiment of the screen detection method of this application is shown schematically.

[0110] like Figure 6 As shown, the screen detection method described above also includes steps S610 to S630.

[0111] In step S610, the pixel arrangement type of the screen is determined, and the pixel repeating unit of the pixel arrangement is obtained according to the pixel arrangement type.

[0112] For example, before constructing the screen pixel arrangement template, this application first needs to determine the pixel arrangement type of the screen to be detected, and obtain the pixel repeating units of the pixel arrangement according to the arrangement type, such as accurately identifying the layout rules of the screen sub-pixels, to provide basic information for the subsequent construction of an accurate screen pixel arrangement template, thereby ensuring that the periodic suppression coefficient can be optimized for a specific screen type.

[0113] In this embodiment, the arrangement of sub-pixels varies significantly between displays produced using different technologies and by different manufacturers, as detailed below: The standard RGB arrangement is the most traditional and common pixel arrangement. In the standard RGB arrangement, each pixel unit consists of three sub-pixels—red, green, and blue—arranged horizontally from left to right. The three sub-pixels are usually the same size, forming a square pixel unit. The advantage of this arrangement is its simple structure and accurate color reproduction; the disadvantage is a lower sub-pixel density at the same resolution. The repeating pixel unit in the standard RGB arrangement is a row of three columns, containing one red sub-pixel, one green sub-pixel, and one blue sub-pixel.

[0114] Pentile arrangement is a common type of organic light-emitting diode (OLED) display. In a pentile arrangement, the red and blue subpixels are larger than the green subpixels, and the number of red and blue subpixels is relatively small. In a typical pentile arrangement, each pixel unit shares an adjacent red or blue subpixel, forming an alternating red-green and blue-green arrangement. The advantages of pentile arrangement are that it can achieve higher brightness or longer lifespan at the same resolution. The disadvantages are that the subpixel density is uneven, and colored edges may appear when displaying fine text. The pixel repeating unit of a pentile arrangement is usually a matrix of two by two or larger, containing two green subpixels, one red subpixel, and one blue subpixel.

[0115] Delta arrangement is commonly found in some LCD and micro LED displays. In Delta arrangement, subpixels are arranged in a triangular or diamond shape, with adjacent rows staggered. This arrangement can achieve higher subpixel density in a limited space, but the algorithm compensation is more complex. The pixel repeating unit in Delta arrangement is usually a rhombus or hexagonal area containing multiple subpixels.

[0116] In step S620, weights are assigned to each sub-pixel in the pixel arrangement to obtain the weight of each sub-pixel.

[0117] For example, after obtaining the pixel repeating unit, this application further assigns weights to each sub-pixel in the pixel arrangement to obtain the weight of each sub-pixel. For example, different weight coefficients are assigned according to the differences in importance of sub-pixels in screen display and defect detection, so that the subsequent pixel periodic attention weighting can more specifically focus on or ignore specific types of sub-pixels, thereby improving the accuracy and effectiveness of the periodic suppression coefficient.

[0118] Specifically, in the pixel arrangement of a screen, subpixels of different colors differ significantly in terms of brightness contribution, visual sensitivity, and defect representation. Therefore, they should not be treated the same, as follows: The first difference lies in the brightness contribution of different subpixels: In a standard RGB arrangement, the green subpixel contributes the most to brightness, accounting for approximately 59%; the red subpixel is next, accounting for approximately 30%; and the blue subpixel contributes the least, accounting for only about 11%. This means that anomalies in the green subpixel have the greatest impact on visual perception, while anomalies in the blue subpixel are relatively less noticeable. In defect detection, defects on the green subpixel should receive higher attention, while defects on the blue subpixel can have their weight appropriately reduced.

[0119] Secondly, different sub-pixels have different visual sensitivities: the human eye is most sensitive to green wavelengths of light, followed by red, and least sensitive to blue. Therefore, defects of the same brightness appearing on green sub-pixels are more easily detected by the human eye than those appearing on blue sub-pixels. In automated optical inspection systems that simulate human eye detection, this factor should be taken into account, and green sub-pixels should be given higher weight.

[0120] Third, different subpixels exhibit different defect manifestations: In actual production, the defect incidence rate of different subpixels may vary. For example, some process defects may be more likely to appear on red subpixels, while others may be more likely to appear on blue subpixels. By adjusting the weight allocation, the detection system can be made more sensitive to subpixels with high-incidence defect types.

[0121] Fourth, the positions and numbers of different subpixels differ within the pixel repetition unit: in a Pentile arrangement, the number of green subpixels is twice that of red and blue subpixels. Without weight allocation, the response of green subpixels would naturally dominate, potentially masking defect signals on red and blue subpixels. Weight allocation balances the influence of different colored subpixels.

[0122] For the reasons mentioned above, this application assigns weights to each sub-pixel in the pixel arrangement so that each sub-pixel has a weight coefficient that reflects its importance, thereby constructing a weighted pixel arrangement template.

[0123] In step S630, a screen pixel arrangement template with the same size as the image to be identified is generated based on the pixel repeating unit and the weight of each sub-pixel.

[0124] For example, after obtaining the pixel repeating unit and assigning the sub-pixel weights, this application further generates a screen pixel arrangement template with the same size as the image to be identified based on the pixel repeating unit and the weights of each sub-pixel. For example, the small-sized, local pixel repeating unit is expanded into a large-sized template that covers the entire range of the image to be identified, so that each spatial position in the template corresponds to a weight value that reflects the importance of the sub-pixel at that position, providing complete prior knowledge for the subsequent element-wise multiplication and generation of periodic suppression coefficients.

[0125] In this embodiment, the pixel repeating unit describes the sub-pixel layout and weight distribution within a minimum period, and its size is typically small. For example, the repeating unit size in a standard RGB arrangement is 1x3, and the repeating unit size in a Pentile arrangement is 2x2. However, the size of the image to be recognized is typically very large, such as millions of pixels or even higher. In order to perform pixel-period attention weighting on the entire image, this small repeating unit needs to be expanded to the same size as the image.

[0126] In this embodiment, obtaining the pixel repetition unit of the pixel arrangement according to the pixel arrangement type includes: analyzing the distribution order of the color of each sub-pixel along the row and column directions in the pixel arrangement; determining the minimum period of color repetition in the row direction and the minimum period of color repetition in the column direction according to the distribution order; and constructing the pixel repetition unit using the sub-pixel matrix defined by the minimum period in the row direction and the minimum period in the column direction.

[0127] For example, the generation logic is explained in detail using OLED Pentile arrangement as an example. Other arrangement methods are similar, as follows: (1) Determine the smallest repeating unit A common pattern for Pentile arrangements is RG-BG, where each pixel consists of sub-pixels of different colors, and adjacent pixels share sub-pixels. For example, we can analyze the color distribution order of each sub-pixel in the row and column directions in a Pentile pixel arrangement. Based on the distribution order, we can determine the minimum period of color repetition in the row direction and the minimum period of color repetition in the column direction. Then, using the sub-pixel matrix defined by the minimum period in the row and column directions, we can construct pixel repetition units. For example, a minimum repetition unit is a 2x2 virtual pixel, but the actual sub-pixel distribution is as follows: Line 0: RGRG ... Row 1: GBGB... Line 2: RGRG ... Line 3: GBGB... Therefore, each 2 rows and 2 columns is actually a repeating unit, but each unit contains 2 red, 1 green and 1 blue sub-pixels, but in different positions.

[0128] (2) Assign weights to each sub-pixel Based on relative brightness and human eye sensitivity, assign different weights to R, G, and B. For example: Red (R): 0.2126 -> after normalization to [0,1], it can be set to 1.0 (because red is usually a complete subpixel in Pentile); Green (G): 0.7152 -> relative to red, it can be set to 0.8 (because green is sensitive, but may share, so the weight is slightly lower); Blue (B): 0.0722 -> can be set to 0.5.

[0129] (3) Generate template matrix (i.e., generate screen pixel arrangement template) Based on the minimum repeating unit and weights, a template matrix of the same size as the image to be recognized is generated. The value at each position is filled with the corresponding weight according to the type of the sub-pixel at that position.

[0130] Continuing with the Pentile template as an example, using the method described above, generate an H-row, W-column template: Assuming the image size is H×W, generate the template according to the following rules: For the i-th row and j-th column (i starts from 0, j starts from 0): If i is an even row (0-indexed): If j is an even-numbered column: the position is a red sub-pixel with a weight of 1.0; If j is an odd number of columns: the position is a green sub-pixel with a weight of 0.8; If i is an odd-numbered row: If j is an even-numbered column: the position is a green sub-pixel with a weight of 0.8; If j is an odd number of columns: the position is a blue sub-pixel with a weight of 0.5.

[0131] This application's embodiments generate a screen pixel arrangement template with the same size as the image to be identified, based on pixel repetition units and sub-pixel weights. This achieves a complete spatial expansion of prior knowledge, expanding small-sized pixel repetition units into large-sized templates covering the entire image. This allows prior knowledge of the screen's periodic structure to be applied to every position in the image, achieving periodic attention weighting across the entire image. For example, phase offsets can be determined through methods such as frequency domain analysis, spatial domain cross-correlation, or offline calibration, ensuring that the sub-pixel positions in the template accurately correspond to the actual screen image, avoiding erroneous weighting due to phase mismatch. This example also avoids generating duplicate templates for the same screen model through template caching and reuse mechanisms, significantly improving the processing speed of batch detection.

[0132] In some embodiments, the screen detection method described above can deploy the processing of frequency domain features and spatial domain features in a multimodal attention network. For example, a deep learning framework of a pre-built multimodal frequency-aided attention network (MFA-Net) can be pre-trained, and the dataset can be obtained from a predetermined data source (e.g., a screen defect dataset downloaded from CSDN).

[0133] Specifically, the preprocessed dataset is input into the model for training. During training, batch normalization (BN) is used after each convolutional layer to accelerate convergence. The random weights of the convolutional layers are initialized by a regularizer, and the biases are initialized to 0. Simultaneously, the global learning rate is initialized to 1e-5 and divided by 10 every 10 epochs. Stochastic gradient descent is used to update the network parameters. Each iteration uses 4 batch sizes, corresponding to 200 epochs. All experiments are performed using an NVIDIA GeForce RTX 4080 GPU, and the entire network framework is implemented based on PyTorch. Based on the same test set containing various screen types (LCD, OLED, AMOLED) and defects such as dead pixels, line defects, light leakage, and scratches, the performance comparison of this application (MFA-Net) with traditional methods and mainstream deep learning models is as follows:

[0134] The data in the table above shows that the multimodal attention network of this application has a significant improvement in various performance aspects compared with traditional methods. This allows for accurate screen defect detection when the fused features output by the multimodal attention network are subsequently detected and identified through a fully connected layer.

[0135] Based on the above embodiments, after obtaining the fusion features of the image to be identified, this application further processes the fusion features based on a fully connected layer to detect screen defects. For example, the rich information contained in the fusion features is integrated and nonlinearly mapped through a fully connected layer to finally output the detection results of screen defects, including whether the defects exist, the specific category of the defects, and the severity of the defects.

[0136] Specifically, fused features are obtained through deep fusion of spatial domain features and frequency domain features, and possess rich discriminative information. However, fused features are still in two-dimensional or three-dimensional tensor form, containing a large amount of local detailed information. In order to make the final defect detection decision, it is necessary to integrate this local and scattered information into a global and abstract discriminative criterion.

[0137] Fully connected layers are an effective tool for achieving this information integration. Unlike traditional machine learning methods that require manually designing feature classifiers, fully connected layers can automatically learn how to map fused features to the defect category space. Each neuron in a fully connected layer is connected to all neurons in the previous layer, thus enabling the capture of global dependencies and complex interactions between features.

[0138] The fully connected layer in this embodiment can automatically integrate scattered local information from fused features into a global discrimination criterion, eliminating the need for manual feature combination rules. Depending on the requirements of the detection task, the fully connected layer can output results in various forms, such as binary classification, multi-classification, multi-labeling, multi-task, and regression, meeting the needs of different application scenarios. When adding new defect categories, only the number of neurons in the output layer and the corresponding training data need to be adjusted; there is no need to modify the main network structure. The computation of the fully connected layer mainly involves matrix multiplication and addition, which can be efficiently implemented on various hardware, meeting the speed requirements of real-time inspection on production lines. When using this fully connected layer to detect and identify fused features output by a multimodal attention network, accurate screen defects can be output.

[0139] Figure 7 A block diagram of a screen detection device according to an embodiment of this application is shown schematically.

[0140] like Figure 7 As shown, the screen detection device 700 includes an acquisition module 710, an identification module 720, a weighting module 730, a processing module 740, and a detection module 750.

[0141] The acquisition module 710 is used to acquire the image to be recognized displayed on the screen.

[0142] The recognition module 720 is used to recognize the image to be recognized and obtain the frequency domain features and spatial domain features of the image to be recognized.

[0143] The weighting module 730 is used to perform pixel periodic attention weighting on spatial domain features based on a preset screen pixel arrangement template to generate periodic suppression coefficients.

[0144] Processing module 740 is used to process spatial domain features based on periodic suppression coefficients and fuse them with frequency domain features to obtain fused features of the image to be recognized. The detection module 750 performs screen defect detection based on fusion characteristics.

[0145] This screen detection device 700 overcomes the inherent problem of existing deep learning methods (such as YOLO and U-Net) which rely solely on spatial domain features and lack sensitivity to subtle defects in high-frequency components. It simultaneously extracts and utilizes both frequency and spatial domain features of the image to be identified. By applying pixel-period attention weighting to the spatial domain features using a preset screen pixel arrangement template, it generates periodic suppression coefficients. This adaptively weakens background responses related to pixel periods, thereby reducing the interference of background periodic structures on defect detection and improving the saliency of real defects. After periodic suppression of the spatial domain features, they are fused with frequency domain features. This fused feature more comprehensively characterizes the morphology and spectral characteristics of minute defects in high-density screen images, thus improving the accuracy and robustness of subsequent defect detection. Due to its enhanced sensitivity to high-frequency weak defects and suppression of the response to periodic background interference, in actual screen defect detection, it can significantly reduce the false negative rate of small point and line defects compared with traditional spatial domain CNN methods (such as YOLO and U-Net), and can also improve the overall detection rate, especially in the manufacturing process of high pixel density screens (such as OLED and Micro-LED).

[0146] In some embodiments, the screen detection device 700 described above can implement the screen detection method described above. The specific implementation process can be referred to the relevant description of the screen detection method described above, and will not be repeated here.

[0147] Figure 8 A schematic block diagram of an electronic device according to an embodiment of this application is shown.

[0148] like Figure 8 As shown, the electronic device 8 of this embodiment includes: a screen, a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the processor 801. When the processor 801 executes the computer program 803, it implements the steps in the various method embodiments described above. Alternatively, when the processor 801 executes the computer program 803, it implements the functions of each module in the various device embodiments described above.

[0149] Electronic device 8 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device with a screen. Electronic device 8 may include, but is not limited to, processor 801 and memory 802. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or different components.

[0150] The processor 801 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0151] The memory 802 can be an internal storage unit of the electronic device 8, such as a hard disk or RAM of the electronic device 8. The memory 802 can also be an external storage device of the electronic device 8, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 8. The memory 802 can also include both internal and external storage units of the electronic device 8. The memory 802 is used to store computer programs and other programs and data required by the electronic device.

[0152] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0153] This application provides a computer program product that stores a computer program, which, when executed by a processor, implements the steps in the above method embodiments.

[0154] If the integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0155] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0156] The units or modules described in the embodiments of this disclosure can be implemented in software or programmable hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0157] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system described above; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs, which are used by one or more processors to execute the methods of the embodiments of this disclosure.

[0158] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A screen detection method, comprising: Obtain the image to be recognized displayed on the screen; The image to be identified is then identified to obtain its frequency domain features and spatial domain features. Based on a preset screen pixel arrangement template, pixel periodic attention weighting is applied to the spatial domain features to generate a periodic suppression coefficient. The spatial domain features are processed according to the periodic suppression coefficient and fused with the frequency domain features to obtain the fused features of the image to be identified. Screen defect detection is performed based on the fusion features.

2. The screen detection method according to claim 1, characterized in that, The acquisition of frequency domain features includes: The Nyquist frequency of the screen is determined based on the pixel pitch of the screen. The target frequency component in the frequency domain feature is moved to the center of the spectrum to generate the target frequency domain feature of the image to be identified. Based on the Nyquist frequency and the target frequency domain characteristics, a low-frequency subband mask, a mid-frequency subband mask, and a high-frequency subband mask are constructed. Based on the low-frequency sub-band mask, mid-frequency sub-band mask, and high-frequency sub-band mask, and the Nyquist frequency, the frequency domain features are divided into frequency bands to obtain the corresponding low-frequency sub-band features, mid-frequency sub-band features, and high-frequency sub-band features.

3. The screen detection method according to claim 2, characterized in that, The construction of low-frequency sub-band masks, mid-frequency sub-band masks, and high-frequency sub-band masks based on the Nyquist frequency and the target frequency domain characteristics includes: Based on the target frequency domain features and the size information of the image to be identified, the radius corresponding to the Nyquist frequency is determined; Based on the frequency coordinates of the target frequency domain features and the radius corresponding to the Nyquist frequency, the low-frequency subband mask, the mid-frequency subband mask, and the high-frequency subband mask are constructed.

4. The screen detection method according to claim 1, characterized in that, Based on a preset screen pixel arrangement template, pixel periodic attention weighting is applied to the spatial domain features to generate periodic suppression coefficients, including: The spatial domain features are convolved to generate a single-channel feature map; The single-channel feature map is multiplied element-wise with the screen pixel arrangement template to generate a mask feature; The mask features are mapped to a preset interval using an activation function to obtain the periodic inhibition coefficient.

5. The screen detection method according to claim 2, characterized in that, The spatial domain features are processed according to the periodic suppression coefficient and fused with the frequency domain features to obtain the fused features of the image to be identified, including: The spatial domain features are processed according to the periodic suppression coefficient to obtain the target spatial domain features; The target spatial domain features are fused with the low-frequency sub-band features, mid-frequency sub-band features, and high-frequency sub-band features to obtain the fused features of the image to be identified.

6. The screen detection method according to claim 1, characterized in that, Also includes: Determine the pixel arrangement type of the screen, and obtain the pixel repeating units of the pixel arrangement according to the pixel arrangement type; Weights are assigned to each sub-pixel in the pixel arrangement to obtain the weights of each sub-pixel; Based on the pixel repeating unit and the weight of each sub-pixel, a screen pixel arrangement template with the same size as the image to be identified is generated.

7. The screen detection method according to claim 6, characterized in that, Obtaining the pixel repetition unit of the pixel arrangement according to the pixel arrangement type includes: Analyze the color distribution order of each sub-pixel along the row and column directions in the pixel arrangement; Based on the distribution order, determine the minimum period of color repetition in the row direction and the minimum period of color repetition in the column direction; The pixel repetition unit is constructed using the sub-pixel matrix defined by the minimum period in the row direction and the minimum period in the column direction.

8. A screen detection device, comprising: The acquisition module is used to acquire the image to be recognized displayed on the screen; The recognition module is used to recognize the image to be recognized and obtain the frequency domain features and spatial domain features of the image to be recognized. The weighting module is used to perform pixel periodic attention weighting on the spatial domain features based on a preset screen pixel arrangement template, and generate a periodic suppression coefficient. The processing module is used to process the spatial domain features according to the periodic suppression coefficient and fuse them with the frequency domain features to obtain the fused features of the image to be identified. The detection module performs screen defect detection based on the fusion characteristics.

9. An electronic device comprising a screen, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images