LED display screen surface flaw online detection system and method

Abstract

The invention relates to the technical field of visual inspection, in particular to an LED display screen surface flaw online detection system and method, and the method comprises the steps: carrying out the optical scanning of the surface of a display screen, and capturing an original optical radiation data flow; converting the original optical radiation data stream into a standardized optical data stream based on the acquired ambient light parameter and the target brightness parameter; converting the optical characteristics of each region in the standardized optical data stream into feature vectors; calculating the difference degree between the feature vector and a flawless standard vector, constructing a defect probability graph, and positioning a flaw candidate region; extracting the multispectral optical feature vector and the calibration data vector of the photometric physical property, and fusing to generate a composite feature vector; analyzing the composite feature vectors, determining flaw types and severity levels, and generating a comprehensive flaw analysis report and a control signal for controlling automatic operation of the production line. By analyzing optical signals on the surface of the display screen, high-efficiency, high-precision and high-robustness online detection of surface flaws is realized.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to an online detection system and method for surface defects on LED displays. Background Technology

[0002] In the mass production of LED displays, analyzing the physical properties of the screen surface materials to ensure the uniformity of their optical properties and structural integrity is crucial. Currently, this mainly relies on optical inspection technology. The traditional inspection method is manual visual inspection, which uses the human eye as an optical sensor to detect defects. However, this method is inefficient and highly subjective, and cannot meet the requirements of modern industrial production for high precision and consistency.

[0003] To achieve automation, existing technologies commonly employ visual inspection systems. These systems collect light radiation data from the surface of objects using optical devices and analyze this data to determine the presence of physical defects such as scratches and dead pixels. However, many solutions rely on the analysis of single-band optical data. This optical inspection method has limited ability to determine changes in the physical properties of materials with low contrast or complex shapes, easily leading to missed detections. Secondly, the complex algorithms used to improve detection accuracy significantly increase the computational load on optical data, contradicting the high efficiency required for online inspection. These optical inspection systems are highly sensitive to changes in ambient light and the aging of the equipment's own light source, lacking effective dynamic calibration mechanisms, resulting in poor stability and robustness in detecting physical properties.

[0004] Therefore, this invention proposes an online detection system and method for surface defects of LED displays. Summary of the Invention

[0005] The purpose of this invention is to provide an online detection system and method for surface defects of LED displays. By analyzing the optical signals on the surface of the display screen using visual inspection technology, it achieves high-efficiency, high-precision and high-robust online detection of surface defects.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An online detection system for surface defects of LED displays includes:

[0008] An adaptive optics acquisition module performs optical scanning on the surface of the LED display screen to capture the raw light radiation data stream; based on the real-time acquired ambient light parameters and target brightness parameters, the raw light radiation data stream is converted into a standardized optical data stream.

[0009] The defect probability mapping module receives the standardized optical data stream, converts the optical characteristics of each region in the standardized optical data stream into feature vectors, calculates the difference between the feature vectors and the flawless standard vector, constructs a defect probability map based on the difference, and locates candidate defect regions.

[0010] The composite feature generation module extracts multispectral optical feature vectors and calibration data vectors characterizing the surface structure of the defect candidate region and photometric physical properties; it then fuses the multispectral optical feature vectors and calibration data vectors to generate a composite feature vector.

[0011] The defect classification and control module analyzes the composite feature vector to determine the defect type and severity level of the defect candidate region; based on the defect type and severity level, it generates a comprehensive defect analysis report and control signals for controlling the automated operation of the production line.

[0012] Preferably, the adaptive optics acquisition module includes:

[0013] The raw optical radiation data stream includes visible spectrum data captured by the visible light optical detection unit and infrared spectrum data captured by the infrared optical detection unit; the raw optical radiation data stream is converted into a set of standardized optical data streams characterizing the surface optical properties by an adaptive calibration algorithm based on ambient light parameters and target brightness parameters. The standardized optical data streams are spatially aligned multi-channel data streams.

[0014] Preferably, the adaptive calibration algorithm includes:

[0015] The ambient light parameters are monitored in real time to determine the spectral distribution; the target brightness parameters are obtained, and the brightness mapping function is adjusted for the visible spectral data and the infrared spectral data based on the spectral distribution and the target brightness parameters; spatial registration is performed on the data from different imaging units to generate the standardized optical data stream.

[0016] Preferably, converting the optical properties of each region in the standardized optical data stream into feature vectors includes:

[0017] The optical properties are decomposed into multi-scale spatial components; regional photometric feature vectors are extracted from the low-frequency spatial components after decomposition to characterize the macroscopic photometric uniformity of each region; and anomalous feature vectors are extracted from the high-frequency spatial components after decomposition to characterize the microscopic surface structure anomalies of each region; the feature vectors are a combination of the regional photometric feature vectors and the anomalous feature vectors.

[0018] Preferably, constructing a defect probability map based on the difference and locating candidate defect regions includes:

[0019] Calculate the mathematical distance between the feature vector and the flawless standard vector to determine the difference degree; convert the mathematical distance into a defect probability score using a preset mapping function to construct the defect probability map; compare the values ​​in the defect probability map with a preset threshold to determine the coordinates of all regions in the defect probability map whose probability scores are higher than the preset threshold, and locate the defect candidate region.

[0020] Preferably, the multispectral optical feature vector and the calibration data vector are fused to generate a composite feature vector, including:

[0021] The multispectral optical feature vector includes multi-scale feature data characterizing the surface micromorphology of the defect candidate region, and multi-directional feature data characterizing the spatial rate of change of the optical properties of the defect candidate region; the calibration data vector includes physical quantity data characterizing the absolute brightness deviation and chromaticity coordinate offset of the defect candidate region; based on a preset cross-modal attention fusion network, the features characterizing optical anomalies within the multispectral optical feature vector and the calibration data vector are enhanced through intra-modal attention weights; the nonlinear optical correlation between the enhanced vectors is calculated through a cross-modal attention mechanism; and the composite feature vector is generated by adaptive weighting based on the nonlinear optical correlation.

[0022] Preferably, determining the defect type and severity level of the defect candidate region includes:

[0023] The composite feature vector is compared with a preset classification model; the classification model is used to calculate the mathematical distance between the composite feature vector and multiple reference vectors in the model; the defect type is determined based on the minimum mathematical distance, and the value of the minimum mathematical distance is converted into the severity level.

[0024] Preferably, the control signals for generating a comprehensive defect analysis report and controlling the automated operation of the production line include:

[0025] The spatial mapping information of the defect candidate area is converted into location coordinates. The location coordinates, the defect type, and the severity level are combined to generate the comprehensive defect analysis report. Based on the type label and severity level, a control signal is generated to instruct the production line to perform at least one automated operation among rejection, marking, and sorting on the display screen.

[0026] An online detection method for surface defects of an LED display screen includes:

[0027] The surface of the LED display screen is optically scanned to capture the raw light radiation data stream; based on the real-time acquired ambient light parameters and target brightness parameters, the raw light radiation data stream is converted into a standardized optical data stream.

[0028] Receive the standardized optical data stream, convert the optical characteristics of each region in the standardized optical data stream into feature vectors; calculate the difference between the feature vectors and the flawless standard vectors; construct a defect probability map based on the difference and locate candidate defect regions;

[0029] Extract the multispectral optical feature vector and the calibration data vector representing the surface structure and photometric physical properties of the candidate defect region; fuse the multispectral optical feature vector and the calibration data vector to generate a composite feature vector;

[0030] Analyze the composite feature vector to determine the defect type and severity level of the defect candidate region; based on the defect type and severity level, generate a comprehensive defect analysis report and control signals for controlling the automated operation of the production line.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] 1. This invention uses an adaptive optics acquisition module to compensate and standardize interference factors such as ambient light parameters and target brightness in real time at the physical source of data acquisition. This improves the problem of unstable detection results caused by environmental changes in existing technologies and ensures the consistency of the system under different working conditions.

[0033] 2. This invention uses a defect probability mapping method to quickly screen and locate the entire display screen surface, and then performs a detailed analysis on high-probability defect candidate areas. This avoids performing complex calculations on the entire surface under high load, greatly improves the speed of online detection, and solves the problem that existing technologies cannot balance speed and accuracy.

[0034] 3. By fusing multispectral optical features characterizing surface structure with calibration data characterizing photometric physical properties, this invention constructs a composite feature vector with richer information dimensions. This vector can accurately identify defects such as low contrast and subtle color differences that are difficult to detect by traditional single optical methods, thus significantly reducing the false negative rate.

[0035] 4. This invention can not only detect, classify and grade defects, but also generate control signals to directly interface with the production line automation system, realizing the automatic rejection, marking or sorting of defective products, forming a complete industrial automation closed loop from perception and analysis to decision execution, and improving the overall intelligence level of the production line. Attached Figure Description

[0036] Figure 1 This is a structural diagram of an online surface defect detection system for LED displays according to the present invention;

[0037] Figure 2 This is a flowchart illustrating the dynamic threshold adjustment process according to an embodiment of the present invention.

[0038] Figure 3 This is a schematic diagram of a cross-modal attention fusion network according to an embodiment of the present invention;

[0039] Figure 4 This is a flowchart of an online detection method for surface defects of an LED display screen according to the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Other embodiments obtained by those skilled in the art based on the ideas in this specification without creative effort all fall within the protection scope of this invention.

[0041] Example 1

[0042] Reference Figures 1 to 3 The present invention provides an online detection system for surface defects of LED displays, which is integrated into an automated production line. The system logically includes an adaptive optics acquisition module, a defect probability mapping module, a composite feature generation module, and a defect classification and control module.

[0043] Reference Figure 1 An online detection system for surface defects of an LED display screen, comprising:

[0044] An adaptive optics acquisition module performs optical scanning on the surface of the LED display screen to capture the raw light radiation data stream; based on the real-time acquired ambient light parameters and target brightness parameters, the raw light radiation data stream is converted into a standardized optical data stream.

[0045] The defect probability mapping module receives the standardized optical data stream, converts the optical characteristics of each region in the standardized optical data stream into feature vectors, calculates the difference between the feature vectors and the flawless standard vector, constructs a defect probability map based on the difference, and locates candidate defect regions.

[0046] The composite feature generation module extracts multispectral optical feature vectors and calibration data vectors characterizing the surface structure of the defect candidate region and photometric physical properties; it then fuses the multispectral optical feature vectors and calibration data vectors to generate a composite feature vector.

[0047] The defect classification and control module analyzes the composite feature vector to determine the defect type and severity level of the defect candidate region; based on the defect type and severity level, it generates a comprehensive defect analysis report and control signals for controlling the automated operation of the production line.

[0048] Furthermore, the adaptive optics acquisition module includes: the original optical radiation data stream includes visible spectrum data captured by the visible light optical detection unit and infrared spectrum data captured by the infrared optical detection unit; the original optical radiation data stream is converted into a set of standardized optical data streams characterizing the surface optical properties through an adaptive calibration algorithm based on ambient light parameters and target brightness parameters, wherein the standardized optical data streams are spatially aligned multi-channel data streams.

[0049] Specifically, the hardware of the adaptive optics acquisition module is integrated into a gantry structure spanning above the production line conveyor belt. This module contains two core optical detection units: a visible light optical detection unit and an infrared optical detection unit. The visible light optical detection unit is a high-resolution color linear array industrial camera used to capture visible spectrum data from the LED display panel surface. The infrared optical detection unit is a short-wave infrared linear array industrial camera whose sensor has high sensitivity to the 900 nm to 1700 nm wavelength band, used to capture surface and subsurface optical properties of materials that cannot be reflected by visible light.

[0050] The two detection units are installed side-by-side, their optical scan lines physically aligned to be essentially parallel and perpendicular to the conveyor belt's direction of travel, supplemented by an industrial light source that illuminates the camera. For adaptive calibration, the module also integrates an ambient light sensor to measure the intensity and spectral information of the ambient light around the detection station in real time; this information constitutes the ambient light parameters. Furthermore, the module's control software interface provides a configuration option, allowing technicians to set a target brightness parameter before production begins by scanning a standard, flawless panel or by directly inputting numerical values.

[0051] When the LED display panel passes under the gantry, the two detection units simultaneously scan the panel line by line under the precise synchronous triggering of the encoder. The data streams captured by the two units (i.e., visible light data including red, green, and blue channels and infrared data of a single channel) together constitute the original light radiation data stream.

[0052] The adaptive optics acquisition module is internally configured with an adaptive calibration algorithm. This algorithm receives the raw light radiation data stream it captures and, in conjunction with the real-time acquired ambient light parameters and preset target brightness parameters, performs a series of transformation operations on the raw light radiation data stream, ultimately outputting a standardized optical data stream. The standardized optical data stream is a multi-channel data set, in which the data from the visible light and infrared detection units are spatially pixel-aligned, and their brightness values ​​have been normalized.

[0053] The adaptive calibration algorithm also includes a dynamic exposure adjustment mechanism based on data entropy, which is executed during the initial stage of scanning each panel. This mechanism calculates the entropy value of the first few rows of image data in real time. This entropy value characterizes the richness of image information. If the calculated entropy value is lower than a preset target range (indicating that the image is too dark or overexposed, and details are lost), the adaptive optics acquisition module or its internal controller sends a command to the camera through a software interface to automatically adjust its exposure time until the acquired image entropy value enters the target range.

[0054] For an image that uses eight binary bits to represent gray levels (i.e., a gray level range of 0 to 255), the preset target range is preferably between 6.5 and 7.8. The lower limit of this range is intended to ensure that the image avoids underexposure, which would cause most pixels to concentrate in the low gray level area, thus losing details in the dark areas; the upper limit is intended to avoid overexposure, which would cause loss of details in the high gray level area. In another preferred embodiment, to make the target range more adaptable, the range can be determined in advance through a calibration process, which includes: collecting standard samples representing different products; adjusting the exposure to the optimal state; and setting the target range based on the entropy value of the image collected in the optimal state.

[0055] This mechanism enables the system to automatically adapt to differences in the optical properties of the object being tested (such as high reflectivity and matte finish), dynamically optimize exposure parameters, and thus collect rich raw data on surfaces with different characteristics, effectively avoiding information loss due to overexposure or underexposure.

[0056] This embodiment combines visible light and infrared light detection methods to obtain surface and subsurface physical information far exceeding that of a single spectrum, significantly improving the ability to detect complex and hidden defects. The data standardization process overcomes the interference of environmental and product differences, providing stable and quantifiable data input for subsequent analysis, which is the fundamental guarantee for the entire system to achieve high-precision and high-repeatability detection.

[0057] Furthermore, the adaptive calibration algorithm includes: real-time monitoring of the ambient light parameters to determine the spectral distribution; obtaining the target brightness parameters; adjusting the brightness mapping function for the visible spectral data and the infrared spectral data based on the spectral distribution and the target brightness parameters; and spatially registering data from different imaging units to generate the standardized optical data stream.

[0058] Specifically, the adaptive calibration algorithm is executed in two parallel processes: luminance calibration and spatial registration.

[0059] For the brightness calibration process: The algorithm first continuously acquires the real-time ambient light parameters of the detection station through the ambient light sensor integrated within the module and determines its spectral distribution; at the same time, the algorithm reads the preset target brightness parameters from the system configuration file; based on these two inputs, the algorithm independently calculates and applies a brightness mapping function for the visible spectrum data and the infrared spectrum data respectively; this function is implemented as a lookup table, which defines the mapping rule from each originally acquired gray value to the standardized gray value. The calculation logic of this rule is to compensate for the overall brightness drift caused by changes in ambient light and to make the average brightness value of the flawless area approach the set target brightness parameter.

[0060] In a preferred embodiment of the present invention, the brightness mapping function embodies a linear mapping relationship, that is, the standardized new brightness value is obtained by scaling the original brightness value once and then offsetting it once.

[0061] The specific values ​​of scaling and offset are determined according to the following steps: the system obtains the current ambient light intensity through an ambient light sensor and compares it with a preset standard light intensity to calculate a brightness compensation coefficient; secondly, the system acquires an image of a flawless standard plate, calculates its average brightness, and multiplies this average brightness with the aforementioned brightness compensation coefficient to obtain a reference brightness after light compensation.

[0062] The goal of this linear mapping relationship is to be able to map an input value of zero brightness (pure black) to zero, while accurately mapping the previously calculated, illumination-compensated reference brightness to the preset target brightness parameter in the system. Based on the unique linear relationship established by these two mapping points, the system can calculate the required scaling ratio and offset, and generate a complete lookup table to convert each possible original brightness value into a standardized brightness value.

[0063] For the spatial registration process: This process applies a pre-calculated affine transformation matrix, which is generated by taking a picture of a calibration board with a specific geometric pattern when the device is first installed for calibration. In real-time processing, this affine transformation matrix is ​​applied to each frame of data captured by the infrared camera, and coordinate transformation is used to make its pixel grid correspond precisely in space to the pixel grid of the visible light camera data frame.

[0064] Finally, the visible light and infrared data streams, after being adjusted by the brightness mapping function and corrected by the affine transformation matrix, are merged to generate a unified, standardized optical data stream.

[0065] The adaptive calibration algorithm described in this embodiment comprehensively and systematically corrects ambient light parameters, target brightness, multispectral channel differences, and spatial position deviations, ensuring the objectivity and consistency of the output data. This significantly reduces the detection system's dependence on constant lighting environments such as ideal darkrooms, adapts to complex industrial sites, and ensures uniform testing standards for different batches of products.

[0066] Furthermore, referring to Figure 2 The optical properties of each region in the standardized optical data stream are converted into feature vectors, including: performing multi-scale spatial decomposition on the optical properties; extracting regional photometric feature vectors from the decomposed low-frequency spatial components to characterize the macroscopic photometric uniformity of each region; and extracting anomalous feature vectors from the decomposed high-frequency spatial components to characterize the microscopic surface structure anomalies of each region; wherein the feature vectors are a combination of the regional photometric feature vectors and the anomalous feature vectors.

[0067] Specifically, the defect probability mapping module receives a standardized optical data stream from the adaptive optics acquisition module. This module can convert the input, multi-channel data stream representing optical characteristics into a structured feature vector. The specific process is as follows:

[0068] First, the defect probability mapping module independently performs multi-scale spatial decomposition on each channel of the standardized optical data stream. In this embodiment, the decomposition uses the Gaussian pyramid method. By performing continuous Gaussian blurring and downsampling on the original image, a series of images with decreasing size and resolution are generated. These images are defined as the low-frequency spatial components of the pyramid, which numerically represent the large-scale luminosity distribution in the original image. Simultaneously, the difference between the original image and the upsampled blurred image is used to obtain a series of bandpass images. These images are defined as the high-frequency spatial components of the pyramid, which numerically represent the local details and texture of the original image.

[0069] Next, the module extracts two types of features from the decomposed components. The first type is the regional photometric feature vector, which is extracted from the top layer of the pyramid (i.e., the lowest frequency component). For each region of a preset size in the image, the mean and variance of all pixels within that region are calculated. These two statistics constitute the regional photometric feature vector characterizing the macroscopic photometric uniformity of the region. The second type is the anomaly feature vector, which is extracted from the bottom layer of the pyramid (i.e., the highest frequency component). Similarly, for each region, the energy of its internal high-frequency image and the local binary pattern histogram are calculated. These statistics together constitute the anomaly feature vector characterizing the microscopic surface structure anomalies of the region.

[0070] Finally, for each region, the module concatenates the calculated regional photometric feature vector with the anomaly feature vector to form a higher-dimensional, unified combined feature vector.

[0071] This embodiment employs a multi-scale spatial decomposition method to simultaneously capture the optical characteristics of defects at both macroscopic and microscopic levels. It unifies the analysis of large-area, gradual macroscopic brightness unevenness (through low-frequency components) and the analysis of high-frequency, detail-rich microstructural anomalies (through high-frequency components) within a single framework, avoiding missed detections caused by single-scale analysis. This comprehensive feature extraction method provides rich and highly differentiated judgment criteria for subsequent accurate defect localization.

[0072] Furthermore, referring to Figure 2 Based on the difference degree, a defect probability map is constructed to locate the defect candidate region, including: calculating the mathematical distance between the feature vector and the flawless standard vector to determine the difference degree; converting the mathematical distance into a defect existence probability score through a preset mapping function to construct the defect probability map; comparing the values ​​in the defect probability map with a preset threshold to determine the coordinates of all regions in the defect probability map whose probability scores are higher than the preset threshold, and locating the defect candidate region.

[0073] Specifically, after generating the combined feature vector for each region, the defect probability mapping module performs the following steps to construct a defect probability map and locate candidate defect regions:

[0074] The module loads a pre-calculated and stored flawless standard vector from the configuration file. This standard vector is obtained during the production line debugging phase by extracting features from multiple confirmed flawless standard panel samples and calculating the mean of the feature vectors. For each feature vector extracted from the panel to be tested, the mathematical distance between it and the flawless standard vector is calculated. In this embodiment, a preferred mathematical distance is Mahalanobis distance. Mahalanobis distance takes into account the correlation between the dimensions of the feature vectors, and its calculation result is used to quantify the degree of difference between the two.

[0075] The calculated difference (i.e., Mahalanobis distance value) is input into a preset mapping function. In this embodiment, the mapping function is a normalized exponential function. This function converts the input non-negative distance value into a defect existence probability score between 0 and 1. The larger the distance value, the closer the output probability score is to 1. The probability scores calculated for all areas on the panel are arranged and combined according to their original spatial positions to form a defect probability map corresponding to the panel size.

[0076] In this embodiment, the preset mapping function is preferably a function with an "S"-shaped curve, such as the sigmoid function. The characteristics of this function are as follows: when the input Mahalanobis distance value is much smaller than a preset center reference distance, its output probability score will be very close to zero; when the input Mahalanobis distance value is much larger than the center reference distance, its output probability score will approach one infinitely; and when the input Mahalanobis distance value is exactly equal to the center reference distance, its output probability score is 50% (i.e., 0.5).

[0077] The central reference distance can be set as the average Mahalanobis distance value obtained by statistically analyzing a large number of flawless samples. Furthermore, the steepness of the transition interval from low probability to high probability of the function can be adjusted by a scaling factor. A larger scaling factor will make the transition more abrupt, thereby making the system more sensitive to small increases in distance.

[0078] Finally, the defect probability mapping module performs thresholding on the defect probability map. The system has a preset probability threshold that can be adjusted by the operator (e.g., 0.75). The module iterates through each probability score in the defect probability map and compares it with the preset threshold. The spatial coordinates of all regions with probability scores greater than the threshold are recorded on the panel. The set of regions with recorded coordinates is defined as a defect candidate region and is passed to the composite feature generation module for further processing.

[0079] In a preferred embodiment, the step of thresholding the defect probability map further includes a dynamic threshold adjustment mechanism based on region connectivity. In this mechanism, the defect probability mapping module first segments the defect probability map using an initial probability threshold, generating a temporary binary image. The defect probability mapping module then performs a connected component analysis on the binary image, identifying all independent candidate regions and calculating the area of ​​each region. The defect probability mapping module checks the area distribution of these regions: if the calculation results show that the current segmentation has generated a large number of isolated candidate regions with areas smaller than the preset noise area lower limit, the module determines that the initial threshold is too low and automatically increases it appropriately before re-segmenting; conversely, if no regions are segmented, the module determines that the initial threshold is too high and decreases it appropriately before retrying. This dynamic adjustment iterates until the proportion of noise regions in the segmentation result is lower than a set proportion, or the maximum number of iterations is reached. Finally, the optimal threshold determined after this dynamic adjustment is used to locate the defect candidate regions.

[0080] In a preferred embodiment of the present invention, the preset noise area lower limit can be set according to the optical resolution of the detection system and the expected size of the smallest defect that needs to be detected in the production process. For example, in a typical application scenario, the lower limit can be set to 20 pixels. The setting ratio defines the proportion of the total area of ​​the noise region composed of isolated small areas that can be tolerated in the final segmentation result to the total area of ​​the image. This ratio can preferably be set to less than 5%.

[0081] This mechanism leverages the prior knowledge that "real defects typically exhibit spatial continuity, while noise tends to be isolated and random," endowing the system with the ability to self-adjust its screening sensitivity. This allows threshold setting to no longer rely entirely on human experience, automatically adapting to varying noise levels that may exist in different batches of products. While effectively suppressing over-detection (falsely reporting noise as defects), it maximizes the recall rate for real defects, significantly improving the intelligence and robustness of the initial screening process.

[0082] This embodiment achieves an efficient and rapid preliminary screening process by converting high-dimensional feature vector differences into an intuitive defect probability map. It uses computationally inexpensive mathematical distance comparisons instead of complex classification models, quickly focusing computational resources on a few suspicious candidate regions. This two-step strategy, from a comprehensive map survey to a detailed investigation of key areas, significantly improves the overall processing speed of online detection, meeting the high-efficiency requirements of industrial production.

[0083] Furthermore, referring to Figure 3 The multispectral optical feature vector and the calibration data vector are fused to generate a composite feature vector, including: the multispectral optical feature vector includes multi-scale feature data characterizing the surface micromorphology of the defect candidate region, and multi-directional feature data characterizing the spatial rate of change of the optical properties of the defect candidate region; the calibration data vector includes physical quantity data characterizing the absolute brightness deviation and chromaticity coordinate offset of the defect candidate region; based on a preset cross-modal attention fusion network, the features characterizing optical anomalies within the multispectral optical feature vector and the calibration data vector are enhanced through intra-modal attention weights; the nonlinear optical correlation between the enhanced vectors is calculated through a cross-modal attention mechanism; and the composite feature vector is generated by adaptive weighting based on the nonlinear optical correlation.

[0084] Specifically, the composite feature generation module receives the defect candidate regions located by the defect probability mapping module. For each candidate region, the module performs the following steps to generate a composite feature vector:

[0085] First, the composite feature generation module extracts two independent types of original feature vectors. The first type is the multispectral optical feature vector, which is extracted as follows: This module performs another multi-scale decomposition (e.g., wavelet transform) on the standardized optical data (including visible and infrared channels) of the candidate region to obtain multi-scale feature data characterizing its microstructure; at the same time, this module applies a set of multi-directional Gabor filters to filter the candidate region, and the statistics of the filter response are used as multi-directional feature data characterizing the spatial rate of change of its optical properties. These two sets of data are concatenated to form a multispectral optical feature vector.

[0086] The second type is the calibration data vector, which is extracted as follows: This module calculates the average brightness and chromaticity coordinates of all pixels in the candidate area and compares them with the flawless standard physical quantities stored in the system to calculate the absolute brightness deviation and chromaticity coordinate offset. These two physical quantity data constitute the calibration data vector.

[0087] Next, the module inputs the two types of feature vectors into a pre-trained cross-modal attention fusion network.

[0088] The processing flow of this network is as follows:

[0089] The network first processes the multispectral optical feature vector and calibration data vector separately through independent self-attention sub-networks. In a preferred embodiment, each self-attention sub-network consists of a multi-head self-attention layer (e.g., containing four attention heads, processing 128 feature dimensions) and a subsequent feedforward neural network, with residual connections and layer normalization operations between these layers. The self-attention sub-network computes an attention weight for each feature dimension within each vector, assigning higher weights to the feature dimensions that most effectively characterize optical anomalies, thus obtaining two enhanced feature vectors.

[0090] Subsequently, the network computes the non-linear correlation between the two augmented vectors through a cross-attention layer. Specifically, one vector is used as the query, and the other as the key and value. The network determines the weights by calculating the similarity between the query and the key, and applies these weights to the value. This process is bidirectional, meaning the two vectors alternate as the query.

[0091] Finally, the two vectors weighted by cross-modal attention are residually connected and layer normalized with their original vectors enhanced intra-modal. Then, they are integrated through a feedforward network consisting of one or more fully connected layers, ultimately outputting a single, highly integrated composite feature vector. In the feedforward network and fully connected layers, modified linear units can be used as activation functions.

[0092] This embodiment employs a cross-modal attention network to achieve intelligent and adaptive fusion of features from different sources and with different properties. It goes beyond simple feature splicing; it deeply mines the intrinsic relationships between information such as the microscopic morphology, directionality, physical brightness, and color of defects. This approach generates composite features with extremely high information density and strong discriminative power, greatly improving the accuracy of subsequent classification, especially for complex defects with complex features where single-dimensional information is insufficient for clear judgment.

[0093] Further, determining the defect type and severity level of the defect candidate region includes: comparing the composite feature vector with a preset classification model; the classification model is used to calculate the mathematical distance between the composite feature vector and multiple reference vectors in the model; determining the defect type based on the minimum mathematical distance, and converting the value of the minimum mathematical distance into the severity level.

[0094] Specifically, the defect classification and control module receives the composite feature vector output by the composite feature generation module. This module determines the final defect type and severity level of the defect candidate region through a preset classification model. Its specific workflow is as follows:

[0095] The module loads a pre-defined classification model, which is a classifier based on the nearest neighbor concept. The model internally stores a reference vector library containing reference vectors for various predefined defect types (e.g., bright spots, dark spots, scratches, stains, discoloration spots, etc.). Each reference vector is a standard template obtained by clustering the composite feature vectors of a large number of defect samples of the same type and taking the center of the class.

[0096] When a new composite feature vector is input into this module, the classification model traverses the reference vector library and calculates the cosine similarity (a mathematical distance metric) between the input composite feature vector and each reference vector in the library.

[0097] The module determines the defect type based on the calculation results and finds the reference vector with the highest cosine similarity (i.e., the smallest angular distance) to the input vector. The defect type represented by the reference vector is determined as the defect type of the current defect candidate region. For example, if the input vector has the highest cosine similarity with the "scratch" reference vector, the defect is marked as "scratch".

[0098] Finally, the module converts the calculated highest cosine similarity value into a severity level of the defect. The module has one or more thresholds to classify the similarity value into different severity ranges. For example, a similarity of 1.0 or less and greater than 0.9 is defined as "Level 1" (Severe), a similarity of 0.9 or less and greater than 0.8 is defined as "Level 2" (Moderate), and a similarity of 0.8 or less is defined as "Level 3" (Slight). This final judgment, composed of the type label and severity level, will be used to generate subsequent reports and control signals.

[0099] In a preferred implementation, after determining the defect type, the classification model further includes a reliability correction step based on historical data. The system maintains a statistical database that records the actual frequency of each identified defect type in recent production batches. When the model outputs an initial judgment result (e.g., a discolored spot), the reliability correction algorithm queries the database. If it finds that the discolored spot category has never appeared recently, or its frequency is far below a dynamically set baseline, the algorithm will appropriately lower the reliability score of this judgment. Conversely, if the judgment result is a recently prevalent scratch, its reliability score will be increased. When the reliability score is below a preset lower limit, the defect will be marked as pending re-judgment, indicating that manual intervention is required.

[0100] In a preferred embodiment, the confidence correction step is implemented as follows:

[0101] The initial value of the confidence score can be set as the cosine similarity value calculated between the composite feature vector and the matched reference vector; the dynamically set baseline can be a moving average based on the frequency distribution of various defects in the recent period (e.g., the past 1,000 detected defects); the appropriate reduction or increase can be achieved by applying an adjustment multiplier. This adjustment multiplier can be calculated based on the ratio of the actual frequency of the currently determined defect category to the baseline frequency. For example, when the actual frequency is higher than the baseline frequency, the multiplier is slightly greater than one; when the actual frequency is lower than the baseline frequency, the multiplier is slightly less than one; the final corrected confidence score is the product of the initial confidence score and the adjustment multiplier; the preset lower limit can preferably be set to 0.75. When the corrected confidence score is lower than this value, the system triggers a pending re-judgment flag.

[0102] This mechanism introduces prior knowledge of the production process, adding a layer of common-sense filtering to the pure pattern matching results. It enables the system to have a certain degree of self-correction and alertness to abnormal fluctuations, effectively reducing rare misjudgments caused by insufficient model generalization ability, and timely detecting potential systemic problems in the production process (manifested as an abnormal increase in the frequency of a certain defect), thus improving the overall reliability of the final judgment results.

[0103] This embodiment transforms a complex classification problem into a clear comparison problem by calculating mathematical distance, achieving rapid and accurate defect identification. By using the distance value itself to quantify the severity level of the defect, it achieves simultaneous qualitative and quantitative analysis, providing complete and accurate data support for subsequent quality control and process improvement.

[0104] Furthermore, the generation of the comprehensive defect analysis report and the control signal for controlling the automated operation of the production line include: converting the spatial mapping information of the defect candidate area into position coordinates, combining the position coordinates, the defect type, and the severity level to generate the comprehensive defect analysis report; and generating a control signal based on the type label and severity level to instruct the production line to perform at least one automated operation among rejection, marking, and sorting on the display screen.

[0105] Specifically, after determining all candidate defect areas on a display panel, the defect classification and control module performs the following steps to generate the final output:

[0106] First, the defect classification and control module integrates information to generate a comprehensive defect analysis report. For each area identified as having a defect, the module extracts its pixel coordinates from the standardized optical data stream. Based on the initially established coordinate system calibration, these pixel coordinates are converted into absolute coordinates on the physical display panel (e.g., X and Y coordinates in millimeters). Subsequently, the module records this physical coordinate, the previously determined defect type label, and the assessed severity level as a data entry. After a complete panel scan is completed, all defect data entries are summarized and organized according to a preset format (e.g., XML or JSON) to form a structured comprehensive defect analysis report. This report is stored on a local server and can be displayed in real time on the human-machine interface of the inspection system.

[0107] Secondly, the module generates automated control signals based on the detection results. Internally, this module contains a handling strategy mapping table that defines the specific production line operations corresponding to different combinations of defect types and severity levels. For example, the table can be configured as follows: any defect with a severity level of "Level 1" corresponds to a "reject" operation; a defect of type "stain" and level "Level 3" corresponds to a "mark" operation; all other panels with detected defects correspond to a "sort to Grade B material warehouse" operation; and panels without any detected defects correspond to a "normal release" operation.

[0108] While generating the report, the module queries the handling strategy mapping table based on the highest defect severity level and type of the current panel and determines the operation to be performed. Subsequently, the module encodes the operation instruction into a control signal that conforms to an industrial Ethernet communication protocol (e.g., PROFINET or EtherNet / IP) and sends it to the programmable logic controller (PLC) on the production line through a data interface. After receiving the signal, the PLC immediately controls the corresponding actuator (such as a pneumatic push rod, robotic arm, or steering baffle) to perform physical actions such as rejection, marking, or sorting on the display panel.

[0109] This embodiment transforms abstract inspection data into two highly practical outputs: first, it provides quality management personnel with detailed, traceable, and analyzable structured defect reports; second, it provides automated production lines with clear, explicit, and directly executable control instructions. This tightly integrates high-quality online optical inspection with efficient production line automation control, forming a complete closed loop from problem discovery to problem resolution, significantly improving production efficiency and quality control levels.

[0110] This embodiment utilizes a system comprised of an adaptive optics acquisition module, a defect probability mapping module, a composite feature generation module, and a defect classification and control module working collaboratively. It adaptively calibrates and standardizes multispectral raw data, rapidly locates candidate defect regions through multi-scale analysis and probability mapping, and deeply fuses multi-source features of the candidate regions using a cross-modal attention network. Finally, based on the fused composite features, it achieves accurate classification, grading, and production line control. This system solves the problem of missed detections caused by traditional detection technologies relying on a single data source, overcomes the contradiction between complex algorithms and high efficiency in online detection, and significantly enhances the system's robustness to environmental changes. Ultimately, it achieves substantial progress in detection accuracy, speed, and automation.

[0111] Example 2

[0112] This embodiment provides an online detection method for surface defects on LED displays, which can be applied to automated production lines.

[0113] Reference Figure 4 An online detection method for surface defects of an LED display screen, comprising:

[0114] The surface of the LED display screen is optically scanned to capture the raw light radiation data stream; based on the real-time acquired ambient light parameters and target brightness parameters, the raw light radiation data stream is converted into a standardized optical data stream.

[0115] Receive the standardized optical data stream, convert the optical characteristics of each region in the standardized optical data stream into feature vectors; calculate the difference between the feature vectors and the flawless standard vectors; construct a defect probability map based on the difference and locate candidate defect regions;

[0116] Extract the multispectral optical feature vector and the calibration data vector representing the surface structure and photometric physical properties of the candidate defect region; fuse the multispectral optical feature vector and the calibration data vector to generate a composite feature vector;

[0117] Analyze the composite feature vector to determine the defect type and severity level of the defect candidate region; based on the defect type and severity level, generate a comprehensive defect analysis report and control signals for controlling the automated operation of the production line.

[0118] Specifically, a multispectral optical acquisition system is used to synchronously scan the surface of a high-speed moving LED display panel, capturing its raw light radiation data stream in the visible and infrared spectral bands. Simultaneously, the illumination parameters of the production environment are monitored in real time, and combined with preset target parameters representing ideal panel brightness, an adaptive calibration algorithm converts the raw light radiation data stream into a standardized optical data stream with spatially aligned pixels and normalized brightness values.

[0119] The standardized optical data stream is received and processed. This step first uses a multi-scale spatial decomposition method to extract features representing macroscopic uniformity from the low-frequency components of the data and features representing microstructural anomalies from the high-frequency components, concatenating the two into a combined feature vector. A defect probability map is constructed by calculating the mathematical distance between this combined feature vector and a pre-stored flawless standard vector. A dynamic threshold based on region connectivity analysis is applied to this probability map to accurately locate all suspected defect candidate regions.

[0120] For each candidate defect region identified above, a deeper level of feature extraction is performed. This step extracts two types of features: first, a multispectral optical feature vector characterizing the microscopic morphology and directionality of the defect; and second, a calibration data vector (including absolute brightness and chromaticity deviations) obtained by calculating its deviation from standard physical quantities. Through a pre-trained cross-modal attention fusion network, these two types of vectors with different properties are intelligently weighted and fused, ultimately generating a highly condensed composite feature vector for each candidate region.

[0121] The generated composite feature vector is input into a pre-defined classification model. This model determines the specific type of defect by calculating the mathematical distance between the input vector and reference vectors for various known defect types (such as scratches and dead pixels) stored internally, and converts this minimum distance value into its severity level. Based on the finally determined defect type and severity level, this method generates a comprehensive defect analysis report containing detailed information on all defects. Simultaneously, according to a pre-defined handling strategy, it generates control signals to instruct automated production line equipment to perform operations such as rejection, marking, or sorting.

[0122] The detection method described in this invention, through adaptive standardization of multispectral data streams, a two-stage strategy combining probability mapping and deep analysis, and intelligent fusion of heterogeneous features using attention networks, not only ensures data quality and the ability to detect complex defects from the source, but also meets the speed requirements of online applications while ensuring high detection accuracy through a combination of efficient screening and precise positioning. It effectively balances the two indicators of detection accuracy and online speed, and ultimately forms a complete and automated processing loop from data acquisition to decision control, significantly improving the accuracy, speed, and robustness of detection.

[0123] Example 3

[0124] The online surface defect detection system and method for LED displays described in this invention are fully deployed in the automated production line of COB packaged LED display modules. The deployment node is located after the curing process of the encapsulation resin, and is used to perform real-time surface defect detection on the display modules.

[0125] First, when a COB module moves on the conveyor belt and enters the field of view of the adaptive optics acquisition module of the detection system, the method of this invention begins to execute automatically. The visible light and infrared cameras within the module, triggered synchronously by the encoder, initiate a line-by-line scan of the module surface to capture raw optical data. Simultaneously, the module's internal processing logic, combining real-time acquired ambient light parameters with a preset ideal brightness target, calibrates and standardizes the raw data.

[0126] The processed, standardized optical data stream is transmitted in real time as data packets to the system's defect probability mapping module via Industrial Ethernet. This module performs the aforementioned method, rapidly conducting multi-scale feature analysis on the data stream and constructing a defect probability map of the entire module within a few hundred milliseconds, thereby identifying several candidate regions with potential defects and their coordinates.

[0127] This coordinate information is then sent to the system's composite feature generation module. This module immediately applies a more complex deep feature extraction algorithm to these specific, small-area regions of data and performs cross-modal attention fusion to generate a high-dimensional composite feature vector for each candidate region. This process ensures the most refined analysis of suspicious points.

[0128] Finally, these composite feature vectors are fed into the system's defect classification and control module. This module performs classification and grading methods, making a final judgment for each candidate region. For example, a region located at absolute coordinates (X1, Y1) is classified as a "pixel defect" with a severity level of "Level 1"; a region located at absolute coordinates (X2, Y2) is classified as a "surface physical scratch" with a severity level of "Level 3". Based on these judgments, the module simultaneously completes two output tasks:

[0129] This module generates a detailed analysis report, which summarizes all detected defects in the current module, including their physical coordinates, anomaly type and anomaly severity, and organizes them into a structured data file. This file is stored on the local server or directly uploaded to the factory's Manufacturing Execution System (MES) for subsequent quality traceability and process analysis.

[0130] This module also generates instruction signals for production line control. It determines the final disposal decision for the current module based on preset disposal rules. For example, a rule can set that any module containing a "Level 1" anomaly must be disposed of. The disposal decision includes at least rejection, marking, or sorting. This decision is encoded into a standard digital control signal and sent to the main controller of the production line, instructing the actuators of the downstream workstations to complete the corresponding automated operations.

[0131] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the scope of protection defined in the claims.

Claims

1. An online detection system for surface defects of an LED display screen, characterized in that, include: The adaptive optics acquisition module performs optical scanning on the surface of the LED display screen to capture the raw light radiation data stream. Based on the real-time acquired ambient light parameters and target brightness parameters, the original light radiation data stream is converted into a standardized optical data stream; The defect probability mapping module receives the standardized optical data stream, converts the optical characteristics of each region in the standardized optical data stream into feature vectors, and calculates the difference between the feature vectors and the flawless standard vector. Based on the difference, a defect probability map is constructed to locate candidate defect regions; The composite feature generation module extracts multispectral optical feature vectors and calibration data vectors characterizing the surface structure of the defect candidate region and photometric physical properties; it then fuses the multispectral optical feature vectors and calibration data vectors to generate a composite feature vector. The defect classification and control module analyzes the composite feature vector to determine the defect type and severity level of the defect candidate region; based on the defect type and severity level, it generates a comprehensive defect analysis report and control signals for controlling the automated operation of the production line.

2. The online detection system for surface defects of an LED display screen according to claim 1, characterized in that, The adaptive optics acquisition module includes: the raw optical radiation data stream includes visible spectrum data captured by the visible light optical detection unit and infrared spectrum data captured by the infrared optical detection unit; the raw optical radiation data stream is converted into a set of standardized optical data streams characterizing the surface optical properties through an adaptive calibration algorithm based on ambient light parameters and target brightness parameters, the standardized optical data streams being spatially aligned multi-channel data streams.

3. The online detection system for surface defects of an LED display screen according to claim 2, characterized in that, The adaptive calibration algorithm includes: real-time monitoring of the ambient light parameters to determine the spectral distribution; obtaining the target brightness parameters; adjusting the brightness mapping function for the visible spectral data and the infrared spectral data based on the spectral distribution and the target brightness parameters; and spatially registering data from different imaging units to generate the standardized optical data stream.

4. The online detection system for surface defects of an LED display screen according to claim 1, characterized in that, Converting the optical properties of each region in the standardized optical data stream into feature vectors includes: performing multi-scale spatial decomposition on the optical properties; extracting regional photometric feature vectors from the decomposed low-frequency spatial components to characterize the macroscopic photometric uniformity of each region; and extracting anomalous feature vectors from the decomposed high-frequency spatial components to characterize the microscopic surface structure anomalies of each region; wherein the feature vectors are a combination of the regional photometric feature vectors and the anomalous feature vectors.

5. The online detection system for surface defects of an LED display screen according to claim 1, characterized in that, Constructing a defect probability map based on the difference degree and locating defect candidate regions includes: calculating the mathematical distance between the feature vector and the flawless standard vector to determine the difference degree; converting the mathematical distance into a defect existence probability score through a preset mapping function to construct the defect probability map; comparing the values ​​in the defect probability map with a preset threshold to determine the coordinates of all regions in the defect probability map whose probability scores are higher than the preset threshold, and locating the defect candidate regions.

6. The online detection system for surface defects of an LED display screen according to claim 1, characterized in that, The multispectral optical feature vector and the calibration data vector are fused to generate a composite feature vector, including: the multispectral optical feature vector includes multi-scale feature data characterizing the surface micromorphology of the defect candidate region, and multi-directional feature data characterizing the spatial rate of change of the optical properties of the defect candidate region; the calibration data vector includes physical quantity data characterizing the absolute brightness deviation and chromaticity coordinate offset of the defect candidate region; based on a preset cross-modal attention fusion network, the features characterizing optical anomalies within the multispectral optical feature vector and the calibration data vector are enhanced through intra-modal attention weights; the nonlinear optical correlation between the enhanced vectors is calculated through a cross-modal attention mechanism; and the composite feature vector is generated by adaptive weighting based on the nonlinear optical correlation.

7. The online detection system for surface defects of an LED display screen according to claim 1, characterized in that, Determining the defect type and severity level of the defect candidate region includes: comparing the composite feature vector with a preset classification model; the classification model is used to calculate the mathematical distance between the composite feature vector and multiple reference vectors in the model; determining the defect type based on the minimum mathematical distance, and converting the value of the minimum mathematical distance into the severity level.

8. The online detection system for surface defects of an LED display screen according to claim 1, characterized in that, The generation of the comprehensive defect analysis report and the control signals for controlling the automated operation of the production line include: converting the spatial mapping information of the defect candidate area into position coordinates, combining the position coordinates, the defect type, and the severity level to generate the comprehensive defect analysis report; and generating control signals based on the type label and severity level to instruct the production line to perform at least one automated operation among rejection, marking, and sorting on the display screen.

9. A method for online detection of surface defects in an LED display screen, characterized in that, include: Optical scanning is performed on the surface of the LED display to capture the raw light radiation data stream; Based on the real-time acquired ambient light parameters and target brightness parameters, the original light radiation data stream is converted into a standardized optical data stream; Receive the standardized optical data stream, convert the optical characteristics of each region in the standardized optical data stream into feature vectors, and calculate the difference between the feature vectors and the flawless standard vectors. Based on the difference, a defect probability map is constructed to locate candidate defect regions; Extract the multispectral optical feature vector and the calibration data vector representing the surface structure and photometric physical properties of the candidate defect region; fuse the multispectral optical feature vector and the calibration data vector to generate a composite feature vector; Analyze the composite feature vector to determine the defect type and severity level of the defect candidate region; based on the defect type and severity level, generate a comprehensive defect analysis report and control signals for controlling the automated operation of the production line.

Images