A multispectral screen pixel defect detection method and system

Abstract

This invention discloses a multispectral screen pixel defect detection method and system, comprising the following steps: S1, acquiring screen images using a multispectral sensor and preprocessing them; S2, extracting pixel spectral features and reducing dimensionality; S3, determining the defect type using K-means clustering; S4, grouping defective pixels by type and location using weighted distance-based region topology clustering to construct the spatial topology of the defect region; S5, accurately constructing the defect region boundary using improved Delaunay triangulation to eliminate discontinuities between pixels; S6, comparing and analyzing the smoothed boundary with the original image to output a defect location and type detection report. This invention achieves high-precision, low-missibility, continuous, smooth screen pixel defect detection and structured output through multispectral acquisition, spectral feature analysis, clustering determination, and improved Delaunay boundary reconstruction.

Description

Technical Field

[0001] This invention relates to the field of image processing and defect detection technology, and in particular to a method and system for detecting defects in multispectral screen pixels. Background Technology

[0002] With the rapid development of flat panel display technology, liquid crystal displays (LCDs), organic light-emitting diode (OLEDs), and micro-LED displays are widely used in consumer electronics, industrial control, and automotive terminals. The pixel density of displays continues to increase, with a single screen containing millions of pixels. Even minute defects in individual pixels directly affect the overall display quality and product yield. Therefore, high-precision and efficient defect detection of screen pixels during the production stage has become a crucial step in the display manufacturing process.

[0003] In existing technologies, pixel defect detection methods mainly employ single visible light imaging, using industrial cameras to capture screen images and identifying bad pixels, bright pixels, and color difference pixels based on grayscale thresholding, edge detection, or template matching algorithms. These methods typically rely on brightness differences for judgment, and are prone to missed detections when defect features are not obvious in the visible light band. Furthermore, because different types of defects tend to cluster in space, traditional methods, which primarily rely on single-point thresholding, lack the ability to analyze the topological structure of defect areas and accurately describe the boundary morphology of defect regions.

[0004] Some improved solutions incorporate machine learning algorithms to construct classification models by extracting pixel color components or brightness features to identify defect types. However, these solutions are typically trained on single-spectral data, failing to fully utilize the differences in material reflectance characteristics across different spectral bands, resulting in limited ability to identify latent and complex defects. Furthermore, during defect region boundary reconstruction, simple connected component extraction or morphological dilation methods are often used for region fusion, resulting in jagged boundaries that fail to form a continuous and smooth geometric structure, affecting subsequent defect area statistics and grade assessment.

[0005] Therefore, how to provide a multispectral screen pixel defect detection method and system is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a multispectral screen pixel defect detection method and system. This invention utilizes multispectral imaging technology, principal component analysis dimensionality reduction method, K-means clustering algorithm, weighted distance-based region topology clustering algorithm, and improved Delaunay triangulation boundary reconstruction technology to construct a complete detection process from multispectral data acquisition, spectral feature modeling, defect judgment, region topology analysis to fine boundary reconstruction. It achieves accurate identification and spatial structure representation of screen pixel defects, color difference points, bright spots, and abnormal types. It has the advantages of high detection accuracy, comprehensive defect identification, good continuity of region boundary construction, and high degree of structured result representation.

[0007] A method for detecting defects in multispectral screen pixels according to an embodiment of the present invention includes the following steps: S1. Acquire image data of the screen in different spectral bands through a multispectral sensor and perform preprocessing. The spectral bands include visible light, near-infrared light and short-wave infrared light. S2. Extract the spectral response features of each pixel in different spectral bands from the preprocessed image data, and use principal component analysis to reduce the dimensionality of the features to obtain the spectral feature vector of the corresponding pixel. S3. Use the K-means clustering algorithm to perform cluster analysis on the spectral feature vectors, determine whether there are defects in the pixels, and label the defect type as bad pixel, color difference, bright spot or other abnormal type; S4. For pixels identified as defects, a weighted distance-based region topology clustering algorithm is used to cluster the pixels according to defect type and location to obtain the spatial topology relationship of the defect region. S5. Based on the spatial topological relationship of the defect region, the improved Delaunay triangulation algorithm is used to accurately construct the spatial boundary of the defect region and eliminate discontinuous pixel gaps. S6. Compare and analyze the smoothed defect area boundary with the original image on the screen, and output a detection report showing the specific location and type of the defect area.

[0008] Optionally, S1 specifically includes; S11. The sensor is equipped with multiple spectral filters to acquire images in the visible light, near-infrared light and short-wave infrared light bands, respectively, with corresponding ranges of 380-780nm, 780-1400nm and 1400-3000nm. S12. The image data of each band is denoised and geometrically corrected to eliminate image distortion caused by sensor angle or screen curvature. The median filtering algorithm is used to remove salt and pepper noise in the image. The algorithm calculates each pixel through a local window and retains the mean of the surrounding pixels to smooth the noise area. Image distortion correction is performed using a geometric correction method with a calibration plate, specifically including: The images captured by the sensor are compared with known calibration board templates to extract the spatial coordinates of the calibration points and calculate the distortion coefficients. By using distortion coefficients, an inverse transformation is applied to adjust the coordinates of each image pixel, eliminating geometric distortions caused by sensor angle deviations or screen curvature. S13. Convert the original image from RGB space to Lab space through color space conversion to standardize the color and ensure that the brightness and contrast of images with different spectra remain consistent. S14. Use the adaptive histogram equalization method to adjust the brightness of the image for each spectral band.

[0009] Optionally, S2 specifically includes: S21. For each pixel, extract its reflectance value in each band from the image data of the visible light, near-infrared light and short-wave infrared light bands to form the spectral response vector of the pixel. S22. Combine the spectral response vectors of each pixel into a spectral response matrix, where each row represents the spectral value of a pixel in each band. S23. Perform mean normalization on the spectral response matrix to ensure that the spectral values ​​of each band are on the same order of magnitude. S24. Principal component analysis is used to reduce the dimensionality of the spectral response matrix, specifically including: Calculate the covariance matrix of the spectral response matrix and perform eigenvalue decomposition to obtain the eigenvectors and corresponding eigenvalues; The k eigenvectors are selected based on the magnitude of the eigenvalues ​​to construct the dimensionality-reduced spectral eigenvectors. The choice of k is usually based on the cumulative variance contribution rate to ensure that at least 90% of the spectral information is retained. The spectral feature vector of each pixel is obtained by linearly combining the first k feature vectors, thereby achieving dimensionality reduction.

[0010] Optionally, S3 specifically includes: S31. Use the K-means clustering algorithm to perform cluster analysis on the spectral feature vectors, construct a feature sample set from the spectral feature vectors of all pixels, and set the number of cluster centers to four categories according to the preset number of categories, corresponding to normal pixels, bad pixels, color difference pixels and bright pixels respectively. S32. The initialization of cluster centers adopts a center selection strategy based on maximizing the distance between samples. The first sample is selected as the first center point, and then the sample with the largest sum of Euclidean distances to the selected center points is selected from the remaining samples as the new center point, until the initialization of four center points is completed. S33. Calculate the Euclidean distance between the spectral feature vector of each pixel and each cluster center, and classify the pixel into the category with the smallest distance to complete one sample allocation; S34. After all samples have been assigned, the spectral feature vectors of all pixels in each category are averaged by dimension to update the cluster center position. S35. Iterate in the order of sample allocation and center update until the change in all cluster centers in two consecutive iterations is less than the preset threshold, then terminate the iteration. S36. After clustering convergence, calculate the mean characteristics of each category in the visible light band brightness component, near-infrared reflectance intensity, and short-wave infrared response amplitude, and map labels to each category according to preset judgment rules, specifically including: Categories with brightness significantly lower than the normal category average and response values ​​in all three bands lower than the set benchmark value are labeled as bad pixels; The category of color difference points is defined as those whose proportions of the three color components in the visible light band deviate from the standard color gamut range. The category with a visible light brightness component significantly higher than the mean of the normal category and a synchronous increase in near-infrared response is labeled as a bright spot; Other categories that do not meet the above conditions but differ from the mean of normal categories by more than a set distance threshold are labeled as abnormal types. S37. Complete the determination and labeling of pixel defect types.

[0011] Optionally, S4 specifically includes: S41. For pixels identified as defects, perform region topology clustering based on weighted distance; S42. Divide the pixels into layers according to the defect type, establish a two-dimensional coordinate index structure in each defect type layer, write the row and column coordinates of the pixels into the regular grid index table, and construct an adjacency table. S43. Assign a weight value to each defective pixel, wherein the weight value consists of spatial distance weight, spectral difference weight, and brightness offset weight, wherein: Spatial distance weights are mapped to the 0-1 range based on the Euclidean distance between pixels using a piecewise function. The spectral difference weights are linearly normalized based on the distance between the spectral feature vectors after dimensionality reduction. The brightness offset weight is proportionally mapped based on the degree of deviation between the pixel brightness and the average value of the corresponding type. S44. After completing the weight assignment, the region expansion process is initiated using ungrouped pixels as seed points. The comprehensive weighted distance is calculated point-by-point within their neighborhood, including: When the weighted distance is less than the preset clustering threshold, the neighboring pixels are included in the current region and marked as visited. At the same time, the pixel is used as a new extension node to continue the neighborhood search until no new pixels meet the clustering conditions, thus forming a complete defect region. S45. After all pixels are grouped, the number of pixels in each defect region, the set of boundary pixels, and the number of internal connected sub-blocks are counted. A topological adjacency matrix is ​​constructed based on the adjacency relationship between pixels, and the contact boundary length and minimum spacing information between regions are recorded to obtain the spatial topological relationship of the defect region.

[0012] Optionally, S5 specifically includes: S51. Based on the spatial topological relationship of the defect region, extract the set of boundary pixels from each defect region and rearrange them into an ordered sequence of boundary points in a clockwise direction. At the same time, perform sub-pixel level interpolation refinement on the boundary points and supplement intermediate control points between adjacent boundary pixels through bilinear interpolation. S52. After completing the refinement of the boundary points, construct a point set containing the boundary points and the sampling points inside the region. On the basis of the traditional Bowyer-Watson insertion Delaunay triangulation framework, introduce a boundary constraint mechanism. During the triangle generation stage, force the adjacent connection between the boundary points to be written into the boundary constraint table as an undeletable constraint edge, and prohibit subsequent edge flipping operations from destroying the constraint edge structure. S53. Add a weight correction term during the empty circle determination process so that points near the boundary have a higher weight in the priority of deleting conflict triangles, thereby avoiding the generation of triangles that cross the outside of the region. S54. After completing the initial triangular mesh construction, the directional edge flipping optimization step is executed. The minimum angle value of each internal triangle is checked one by one. When the minimum angle of the triangle is less than the preset threshold, the edge flipping reconstruction operation is executed. During the flipping process, it is detected in real time whether the flipped edge violates the boundary constraint table to ensure the boundary continuity. S55. Perform boundary stitching for locations with discrete gaps between pixels. When the distance between adjacent boundary points is less than a set spacing threshold, automatically insert bridging points and re-triangulate locally to fill the gaps between discontinuous pixels.

[0013] Optionally, S53 specifically includes: S531. Establish a boundary distance table for all points participating in triangulation within each defect area, calculate the shortest vertical distance from each point to the nearest boundary line segment, and write the distance into the point attribute cache. S532. During the Bowyer-Watson insertion process, when the newly inserted point triggers the conflict triangle detection, the center and radius of the circumcircle of each triangle to be deleted are calculated. Based on the original empty circle judgment logic, a boundary influence level classification rule is introduced, and the conflict priority is sorted according to the boundary distance value corresponding to the point closest to the boundary among the three vertices that constitute the triangle. S533. For triangles containing vertices whose boundary distance is less than a set threshold, mark them as high-sensitivity triangles and place them in a priority processing position in the conflict deletion queue. At the same time, when generating new triangles, add a region interiority detection step and use the ray parity rule to determine the region affiliation of the triangle's centroid. If the centroid is located outside the region, discard the triangle directly. S534. After completing the deletion and reconstruction of conflict triangles, perform boundary crossing detection again on all newly generated triangles. By traversing the three sides of the triangle and the set of boundary constraint edges, perform line segment intersection detection. If there is an intersection, roll back the triangle and re-perform local triangulation to ensure that the final generated triangular mesh is strictly located inside the defect area and does not cross the boundary.

[0014] A multispectral screen pixel defect detection system according to an embodiment of the present invention includes: The data acquisition module is used to acquire image data of the screen in the visible light, near-infrared light and short-wave infrared light bands through a multispectral sensor, and to perform noise reduction, geometric correction, color space conversion and brightness equalization preprocessing operations on the acquired images. The feature extraction module is used to extract the spectral response features of each pixel in different spectral bands from the preprocessed image data, construct a spectral data matrix, and use principal component analysis to perform feature dimensionality reduction on the spectral data matrix to generate the spectral feature vectors of the corresponding pixels. The defect determination module is used to perform K-means clustering analysis on the spectral feature vector, determine whether there are defects in the pixels based on the clustering results, and label the defective pixels as bad pixels, color difference pixels, bright pixels or abnormal types according to preset category labels; The topology construction module is used to perform weighted distance-based region topology clustering on pixels identified as defects, grouping pixels according to defect type and spatial location, and constructing an adjacency table and spatial topology structure for defect regions. The boundary reconstruction module is used to perform an improved Delaunay triangulation operation based on the spatial topological relationship of the defect region, generate a triangular mesh structure that satisfies the boundary constraints, reconstruct the spatial boundary of the defect region, and eliminate discontinuous pixel gaps. The results output module is used to perform coordinate mapping and overlay analysis between the smoothed defect area boundary and the original screen image to generate inspection report data containing the specific location of the defect area, boundary contour, and defect type identifier.

[0015] The beneficial effects of this invention are: First, the spectral feature vectors are classified and determined by the K-means clustering algorithm, and combined with the weighted distance regional topological clustering method, defective pixels are accurately grouped according to type and spatial location. This invention can accurately construct the spatial topological relationship of defective regions and effectively avoid the problems of missed detection and false detection caused by single-point judgment.

[0016] Secondly, this invention employs an improved Delaunay triangulation algorithm during the defect region boundary reconstruction process, introducing boundary constraints and weight correction mechanisms, and combining sub-pixel interpolation with local stitching strategies to achieve continuous and smooth boundary refinement of the defect region, eliminating discrete gaps between pixels and improving the geometric accuracy and spatial representation capability of the boundary.

[0017] Finally, after a complete process, this invention can output structured defect detection results, including defect area location, boundary contour and defect type information. This not only improves detection accuracy and reliability, but also provides an accurate data foundation for subsequent defect statistics, graded evaluation and screen quality control, achieving a highly efficient, high-precision and highly implementable screen pixel defect detection effect. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a multispectral screen pixel defect detection method proposed in this invention; Figure 2 This is a schematic diagram of defect determination and topological clustering in a multispectral screen pixel defect detection method proposed in this invention. Figure 3 This is a schematic diagram of boundary reconstruction and Delaunay optimization for a multispectral screen pixel defect detection method proposed in this invention. Figure 4 This is a block diagram of a multispectral screen pixel defect detection system proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 A method for detecting defects in multispectral screen pixels, comprising the following steps: S1. Acquire image data of the screen in different spectral bands through a multispectral sensor and perform preprocessing. The spectral bands include visible light, near-infrared light and short-wave infrared light. S2. Extract the spectral response features of each pixel in different spectral bands from the preprocessed image data, and use principal component analysis to reduce the dimensionality of the features to obtain the spectral feature vector of the corresponding pixel. S3. Use the K-means clustering algorithm to perform cluster analysis on the spectral feature vectors, determine whether there are defects in the pixels, and label the defect type as bad pixel, color difference, bright spot or other abnormal type; S4. For pixels identified as defects, a weighted distance-based region topology clustering algorithm is used to cluster the pixels according to defect type and location to obtain the spatial topology relationship of the defect region. S5. Based on the spatial topological relationship of the defect region, the improved Delaunay triangulation algorithm is used to accurately construct the spatial boundary of the defect region and eliminate discontinuous pixel gaps. S6. Compare and analyze the smoothed defect area boundary with the original image on the screen, and output a detection report showing the specific location and type of the defect area.

[0021] In this embodiment, S1 specifically includes; S11. The sensor is equipped with multiple spectral filters to acquire images in the visible light, near-infrared light and short-wave infrared light bands, respectively, with corresponding ranges of 380-780nm, 780-1400nm and 1400-3000nm. S12. The image data of each band is denoised and geometrically corrected to eliminate image distortion caused by sensor angle or screen curvature. The median filtering algorithm is used to remove salt and pepper noise in the image. The algorithm calculates each pixel through a local window and retains the mean of the surrounding pixels to smooth the noise area. Image distortion correction is performed using a geometric correction method with a calibration plate, specifically including: The images captured by the sensor are compared with known calibration board templates to extract the spatial coordinates of the calibration points and calculate the distortion coefficients. By using distortion coefficients, an inverse transformation is applied to adjust the coordinates of each image pixel, eliminating geometric distortions caused by sensor angle deviations or screen curvature. S13. Convert the original image from RGB space to Lab space through color space conversion to standardize the color and ensure that the brightness and contrast of images with different spectra remain consistent. S14. Use the adaptive histogram equalization method to adjust the brightness of the image for each spectral band.

[0022] In this embodiment, S2 specifically includes: S21. For each pixel, extract its reflectance value in each band from the image data of the visible light, near-infrared light and short-wave infrared light bands to form the spectral response vector of the pixel. S22. Combine the spectral response vectors of each pixel into a spectral response matrix, where each row represents the spectral value of a pixel in each band. S23. Perform mean normalization on the spectral response matrix to ensure that the spectral values ​​of each band are on the same order of magnitude. S24. Principal component analysis is used to reduce the dimensionality of the spectral response matrix, specifically including: Calculate the covariance matrix of the spectral response matrix and perform eigenvalue decomposition to obtain the eigenvectors and corresponding eigenvalues; The k eigenvectors are selected based on the magnitude of the eigenvalues ​​to construct the dimensionality-reduced spectral eigenvectors. The choice of k is usually based on the cumulative variance contribution rate to ensure that at least 90% of the spectral information is retained. The spectral feature vector of each pixel is obtained by linearly combining the first k feature vectors, thereby achieving dimensionality reduction.

[0023] In this embodiment, S3 specifically includes: S31. Use the K-means clustering algorithm to perform cluster analysis on the spectral feature vectors, construct a feature sample set from the spectral feature vectors of all pixels, and set the number of cluster centers to four categories according to the preset number of categories, corresponding to normal pixels, bad pixels, color difference pixels and bright pixels respectively. S32. The initialization of cluster centers adopts a center selection strategy based on maximizing the distance between samples. The first sample is selected as the first center point, and then the sample with the largest sum of Euclidean distances to the selected center points is selected from the remaining samples as the new center point, until the initialization of four center points is completed. S33. Calculate the Euclidean distance between the spectral feature vector of each pixel and each cluster center, and classify the pixel into the category with the smallest distance to complete one sample allocation; S34. After all samples have been assigned, the spectral feature vectors of all pixels in each category are averaged by dimension to update the cluster center position. S35. Iterate in the order of sample allocation and center update until the change in all cluster centers in two consecutive iterations is less than the preset threshold, then terminate the iteration. S36. After clustering convergence, calculate the mean characteristics of each category in the visible light band brightness component, near-infrared reflectance intensity, and short-wave infrared response amplitude, and map labels to each category according to preset judgment rules, specifically including: Categories with brightness significantly lower than the normal category average and response values ​​in all three bands lower than the set benchmark value are labeled as bad pixels; The category of color difference points is defined as those whose proportions of the three color components in the visible light band deviate from the standard color gamut range. The category with a visible light brightness component significantly higher than the mean of the normal category and a synchronous increase in near-infrared response is labeled as a bright spot; Other categories that do not meet the above conditions but differ from the mean of normal categories by more than a set distance threshold are labeled as abnormal types. S37. Complete the determination and labeling of pixel defect types.

[0024] In this embodiment, S4 specifically includes: S41. For pixels identified as defects, perform region topology clustering based on weighted distance; S42. Divide the pixels into layers according to the defect type, establish a two-dimensional coordinate index structure in each defect type layer, write the row and column coordinates of the pixels into the regular grid index table, and construct an adjacency table. S43. Assign a weight value to each defective pixel, wherein the weight value consists of spatial distance weight, spectral difference weight, and brightness offset weight, wherein: Spatial distance weights are mapped to the 0-1 range based on the Euclidean distance between pixels using a piecewise function. The spectral difference weights are linearly normalized based on the distance between the spectral feature vectors after dimensionality reduction. The brightness offset weight is proportionally mapped based on the degree of deviation between the pixel brightness and the average value of the corresponding type. S44. After completing the weight assignment, the region expansion process is initiated using ungrouped pixels as seed points. The comprehensive weighted distance is calculated point-by-point within their neighborhood, including: When the weighted distance is less than the preset clustering threshold, the neighboring pixels are included in the current region and marked as visited. At the same time, the pixel is used as a new extension node to continue the neighborhood search until no new pixels meet the clustering conditions, thus forming a complete defect region. S45. After all pixels are grouped, the number of pixels in each defect region, the set of boundary pixels, and the number of internal connected sub-blocks are counted. A topological adjacency matrix is ​​constructed based on the adjacency relationship between pixels, and the contact boundary length and minimum spacing information between regions are recorded to obtain the spatial topological relationship of the defect region.

[0025] In this embodiment, the region expansion process for ungrouped pixels specifically includes: Within the corresponding defect type layer, the rule grid index table is traversed in row priority order. Pixels with the status flag "unvisited" are selected as initial seed points, written into the region queue and assigned the current region number. At the same time, the region to which the pixel belongs is recorded in the region flag array. The set of neighboring pixels of the seed point is determined based on the adjacency table, and the spatial distance weight, spectral difference weight and brightness offset weight of the neighboring pixels are read one by one in a fixed order. The three weights are weighted and summed by preset weight coefficients to form a comprehensive weighted distance. For each neighboring pixel, determine its access status. When it is in an unaccessed state and the comprehensive weighted distance is less than the clustering threshold, add the pixel to the tail of the region queue and update its region number and access identifier simultaneously. After completing the neighborhood traversal of the current seed point, the next pixel is taken from the head of the region queue as a new extended node, and the neighborhood search, comprehensive weighted distance calculation and state update operations are repeated. During the expansion process, the minimum bounding rectangle boundary of the region is maintained in real time, and the upper, lower, left and right boundary values ​​of the region are updated by comparing the row and column coordinates of the pixels. When the region queue is empty and there are no unvisited neighbor pixels that meet the conditions, terminate the current region expansion and lock the region number. Then continue to search for the next unvisited pixel in the regular grid index table as a new seed point and repeat the above process until all defective pixels have completed the region assignment.

[0026] In this embodiment, S5 specifically includes: S51. Based on the spatial topological relationship of the defect region, extract the set of boundary pixels from each defect region and rearrange them into an ordered sequence of boundary points in a clockwise direction. At the same time, perform sub-pixel level interpolation refinement on the boundary points and supplement intermediate control points between adjacent boundary pixels through bilinear interpolation. S52. After completing the refinement of the boundary points, construct a point set containing the boundary points and the sampling points inside the region. On the basis of the traditional Bowyer-Watson insertion Delaunay triangulation framework, introduce a boundary constraint mechanism. During the triangle generation stage, force the adjacent connection between the boundary points to be written into the boundary constraint table as an undeletable constraint edge, and prohibit subsequent edge flipping operations from destroying the constraint edge structure. S53. Add a weight correction term during the empty circle determination process so that points near the boundary have a higher weight in the priority of deleting conflict triangles, thereby avoiding the generation of triangles that cross the outside of the region. S54. After completing the initial triangular mesh construction, the directional edge flipping optimization step is executed. The minimum angle value of each internal triangle is checked one by one. When the minimum angle of the triangle is less than the preset threshold, the edge flipping reconstruction operation is executed. During the flipping process, it is detected in real time whether the flipped edge violates the boundary constraint table to ensure the boundary continuity. S55. Perform boundary stitching for locations with discrete gaps between pixels. When the distance between adjacent boundary points is less than a set spacing threshold, automatically insert bridging points and re-triangulate locally to fill the gaps between discontinuous pixels.

[0027] In this embodiment, S53 specifically includes: S531. Establish a boundary distance table for all points participating in triangulation within each defect area, calculate the shortest vertical distance from each point to the nearest boundary line segment, and write the distance into the point attribute cache. S532. During the Bowyer-Watson insertion process, when the newly inserted point triggers the conflict triangle detection, the center and radius of the circumcircle of each triangle to be deleted are calculated. Based on the original empty circle judgment logic, a boundary influence level classification rule is introduced, and the conflict priority is sorted according to the boundary distance value corresponding to the point closest to the boundary among the three vertices that constitute the triangle. S533. For triangles containing vertices whose boundary distance is less than a set threshold, mark them as high-sensitivity triangles and place them in a priority processing position in the conflict deletion queue. At the same time, when generating new triangles, add a region interiority detection step and use the ray parity rule to determine the region affiliation of the triangle's centroid. If the centroid is located outside the region, discard the triangle directly. S534. After completing the deletion and reconstruction of conflict triangles, perform boundary crossing detection again on all newly generated triangles. By traversing the three sides of the triangle and the set of boundary constraint edges, perform line segment intersection detection. If there is an intersection, roll back the triangle and re-perform local triangulation to ensure that the final generated triangular mesh is strictly located inside the defect area and does not cross the boundary.

[0028] refer to Figure 4 A multispectral screen pixel defect detection system, comprising: The data acquisition module is used to acquire image data of the screen in the visible light, near-infrared light and short-wave infrared light bands through a multispectral sensor, and to perform noise reduction, geometric correction, color space conversion and brightness equalization preprocessing operations on the acquired images. The feature extraction module is used to extract the spectral response features of each pixel in different spectral bands from the preprocessed image data, construct a spectral data matrix, and use principal component analysis to perform feature dimensionality reduction on the spectral data matrix to generate the spectral feature vectors of the corresponding pixels. The defect determination module is used to perform K-means clustering analysis on the spectral feature vector, determine whether there are defects in the pixels based on the clustering results, and label the defective pixels as bad pixels, color difference pixels, bright pixels or abnormal types according to preset category labels; The topology construction module is used to perform weighted distance-based region topology clustering on pixels identified as defects, grouping pixels according to defect type and spatial location, and constructing an adjacency table and spatial topology structure for defect regions. The boundary reconstruction module is used to perform an improved Delaunay triangulation operation based on the spatial topological relationship of the defect region, generate a triangular mesh structure that satisfies the boundary constraints, reconstruct the spatial boundary of the defect region, and eliminate discontinuous pixel gaps. The results output module is used to perform coordinate mapping and overlay analysis between the smoothed defect area boundary and the original screen image to generate inspection report data containing the specific location of the defect area, boundary contour, and defect type identifier. Example 1: To verify the feasibility of this invention in practice, it was applied to the production inspection scenario of a high-resolution liquid crystal display (LCD). In this scenario, the number of pixels on the display typically exceeds three million, and the brightness, color, and luminous characteristics of each pixel directly affect the final display effect. Traditional inspection methods mainly rely on single-band visible light images for defect judgment, which easily leads to missed detections of bright spots, dead pixels, and color difference pixels, especially in the identification of complex or latent defects. Furthermore, the boundary reconstruction of defect areas often employs connected component extraction or morphological dilation, resulting in jagged boundary shapes, which is detrimental to defect statistics and grade assessment.

[0029] In this embodiment, the display screen under test is first acquired using a multispectral sensor to simultaneously collect image data in the visible, near-infrared, and short-wave infrared bands. The acquired images undergo denoising, geometric correction, and color space conversion to ensure consistent brightness and contrast across different spectral bands. Next, the spectral response values ​​of each pixel in different spectral bands are extracted, and principal component analysis is used for dimensionality reduction to generate a spectral feature vector for each pixel. Subsequently, the spectral feature vectors are classified using a K-means clustering algorithm to determine if any pixels have defects, categorizing them as bad pixels, bright pixels, color difference pixels, or other anomalies. Based on this, for pixels identified as defects, weighted distance-based region topology clustering is performed, grouping pixels according to defect type and spatial location to construct the spatial topological relationships of defective regions, ensuring accurate representation of pixel connectivity and adjacency relationships within each defective region. Then, an improved Delaunay triangulation algorithm is used to reconstruct the boundaries of the defective regions. Boundary refinement is achieved through boundary constraints, weight correction, and sub-pixel interpolation. Simultaneously, locations with discrete gaps between pixels are locally stitched together, forming a continuous, smooth boundary grid that accurately reflects the morphology of the defective regions. Finally, the reconstructed defect region boundaries are compared and analyzed with the original image to output the specific location, boundary contour, and defect type information of each defect region, thus achieving high-precision structured detection.

[0030] The table below shows the statistical results of detection for different defect types, verifying the effectiveness of the present invention in improving recognition accuracy and boundary reconstruction continuity in practical applications.

[0031] Table 1 Statistical results of multispectral screen pixel defect detection In practical applications, the method of this invention was used to inspect a batch of high-resolution LCD screens, with a total of approximately 3.2 million pixels detected. As shown in Table 1, compared with traditional visible light detection methods, this invention can identify approximately 38% more defective pixels, improve the color difference detection rate by approximately 42%, increase the accuracy of bright spot detection by approximately 35%, and reduce the rate of missed detection of latent defects to 1.2%, far lower than the 5.6% of traditional methods. The average boundary smoothness of defect area boundary reconstruction is improved by approximately 28%, and the region connectivity accuracy reaches 97.8%, significantly better than the 84.3% of traditional methods. In addition, the average single-screen detection processing time is 42 seconds, slightly optimized compared to the 48 seconds of single-band detection methods, while ensuring detection accuracy and result integrity.

[0032] As can be seen from this embodiment, the present invention not only improves the recognition rate of various defects in high-resolution screen pixel defect detection, but also achieves continuous smoothness of defect region boundaries and accurate expression of topological structure, with significant beneficial effects of high precision, low false negative rate and structured results.

[0033] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting defects in multispectral screen pixels, characterized in that, Includes the following steps: S1. Acquire image data of the screen in different spectral bands through a multispectral sensor and perform preprocessing. The spectral bands include visible light, near-infrared light and short-wave infrared light. S2. Extract the spectral response features of each pixel in different spectral bands from the preprocessed image data, and use principal component analysis to reduce the dimensionality of the features to obtain the spectral feature vector of the corresponding pixel. S3. Use the K-means clustering algorithm to perform cluster analysis on the spectral feature vectors, determine whether there are defects in the pixels, and label the defect type as bad pixel, color difference, bright spot or other abnormal type; S4. For pixels identified as defects, a weighted distance-based region topology clustering algorithm is used to cluster the pixels according to defect type and location to obtain the spatial topology relationship of the defect region. S5. Based on the spatial topological relationship of the defect region, the improved Delaunay triangulation algorithm is used to accurately construct the spatial boundary of the defect region and eliminate discontinuous pixel gaps. S6. Compare and analyze the smoothed defect area boundary with the original image on the screen, and output a detection report showing the specific location and type of the defect area.

2. The method for detecting multispectral screen pixel defects according to claim 1, characterized in that, S1 specifically includes; S11. The sensor is equipped with multiple spectral filters to acquire images in the visible light, near-infrared light and short-wave infrared light bands, respectively, with corresponding ranges of 380-780nm, 780-1400nm and 1400-3000nm. S12. The image data of each band is denoised and geometrically corrected to eliminate image distortion caused by sensor angle or screen curvature. The median filtering algorithm is used to remove salt and pepper noise from the image. The algorithm calculates each pixel through a local window and retains the mean of the surrounding pixels to smooth the noise area. Image distortion correction is performed using a geometric correction method with a calibration plate, specifically including: The images captured by the sensor are compared with known calibration board templates to extract the spatial coordinates of the calibration points and calculate the distortion coefficients. By using the distortion coefficient, an inverse transformation is applied to adjust the coordinates of each image pixel to eliminate geometric distortion caused by sensor angle deviation or screen curvature. S13. Convert the original image from RGB space to Lab space through color space conversion to standardize the color and ensure that the brightness and contrast of images with different spectra remain consistent. S14. Use the adaptive histogram equalization method to adjust the brightness of the image for each spectral band.

3. The method for detecting multispectral screen pixel defects according to claim 1, characterized in that, S2 specifically includes: S21. For each pixel, extract its reflectance value in each band from the image data of the visible light, near-infrared light and short-wave infrared light bands to form the spectral response vector of the pixel. S22. Combine the spectral response vectors of each pixel into a spectral response matrix, where each row represents the spectral value of a pixel in each band. S23. Perform mean normalization on the spectral response matrix to ensure that the spectral values ​​of each band are on the same order of magnitude. S24. Principal component analysis is used to reduce the dimensionality of the spectral response matrix, specifically including: Calculate the covariance matrix of the spectral response matrix and perform eigenvalue decomposition to obtain the eigenvectors and corresponding eigenvalues; The k eigenvectors are selected based on the magnitude of the eigenvalues ​​to construct the dimensionality-reduced spectral eigenvectors. The choice of k is usually based on the cumulative variance contribution rate to ensure that at least 90% of the spectral information is retained. The spectral feature vector of each pixel is obtained by linearly combining the first k feature vectors, thereby achieving dimensionality reduction.

4. The method for detecting multispectral screen pixel defects according to claim 1, characterized in that, S3 specifically includes: S31. Use the K-means clustering algorithm to perform cluster analysis on the spectral feature vectors, construct a feature sample set from the spectral feature vectors of all pixels, and set the number of cluster centers to four categories according to the preset number of categories, corresponding to normal pixels, bad pixels, color difference pixels and bright pixels respectively. S32. The initialization of cluster centers adopts a center selection strategy based on maximizing the distance between samples. The first sample is selected as the first center point, and then the sample with the largest sum of Euclidean distances to the selected center points is selected from the remaining samples as the new center point, until the initialization of four center points is completed. S33. Calculate the Euclidean distance between the spectral feature vector of each pixel and each cluster center, and classify the pixel into the category with the smallest distance to complete one sample allocation; S34. After all samples have been assigned, the spectral feature vectors of all pixels in each category are averaged by dimension to update the cluster center position. S35. Iterate in the order of sample allocation and center update until the change in all cluster centers in two consecutive iterations is less than the preset threshold, then terminate the iteration. S36. After clustering convergence, calculate the mean characteristics of each category in the visible light band brightness component, near-infrared reflectance intensity, and short-wave infrared response amplitude, and map labels to each category according to preset judgment rules, specifically including: Categories with brightness significantly lower than the normal category average and response values ​​in all three bands lower than the set benchmark value are labeled as bad pixels; The category of color difference points is defined as those whose proportions of the three color components in the visible light band deviate from the standard color gamut range. The category with a visible light brightness component significantly higher than the mean of the normal category and a synchronous increase in near-infrared response is labeled as a bright spot; Other categories that do not meet the above conditions but differ from the mean of normal categories by more than a set distance threshold are labeled as abnormal types. S37. Complete the determination and labeling of pixel defect types.

5. The method for detecting multispectral screen pixel defects according to claim 1, characterized in that, S4 specifically includes: S41. For pixels identified as defects, perform region topology clustering based on weighted distance; S42. Divide the pixels into layers according to the defect type, establish a two-dimensional coordinate index structure in each defect type layer, write the row and column coordinates of the pixels into the regular grid index table, and construct an adjacency table. S43. Assign a weight value to each defective pixel, wherein the weight value consists of spatial distance weight, spectral difference weight, and brightness offset weight, wherein: Spatial distance weights are mapped to the 0-1 range based on the Euclidean distance between pixels using a piecewise function. The spectral difference weights are linearly normalized based on the distance between the spectral feature vectors after dimensionality reduction. The brightness offset weight is proportionally mapped based on the degree of deviation between the pixel brightness and the average value of the corresponding type. S44. After completing the weight assignment, the region expansion process is initiated using ungrouped pixels as seed points. The comprehensive weighted distance is calculated point-by-point within their neighborhood, including: When the weighted distance is less than the preset clustering threshold, the neighboring pixels are included in the current region and marked as visited. At the same time, the pixel is used as a new extension node to continue the neighborhood search until no new pixels meet the clustering conditions, thus forming a complete defect region. S45. After all pixels are grouped, the number of pixels in each defect region, the set of boundary pixels, and the number of internal connected sub-blocks are counted. A topological adjacency matrix is ​​constructed based on the adjacency relationship between pixels, and the contact boundary length and minimum spacing information between regions are recorded to obtain the spatial topological relationship of the defect region.

6. The method for detecting multispectral screen pixel defects according to claim 1, characterized in that, S5 specifically includes: S51. Based on the spatial topological relationship of the defect region, extract the set of boundary pixels from each defect region and rearrange them into an ordered sequence of boundary points in a clockwise direction. At the same time, perform sub-pixel level interpolation refinement on the boundary points and supplement intermediate control points between adjacent boundary pixels through bilinear interpolation. S52. After completing the refinement of the boundary points, construct a point set containing the boundary points and the sampling points inside the region. On the basis of the traditional Bowyer-Watson insertion Delaunay triangulation framework, introduce a boundary constraint mechanism. During the triangle generation stage, force the adjacent connection between the boundary points to be written into the boundary constraint table as an undeletable constraint edge, and prohibit subsequent edge flipping operations from destroying the constraint edge structure. S53. Add a weight correction term during the empty circle determination process so that points near the boundary have a higher weight in the priority of deleting conflict triangles, thereby avoiding the generation of triangles that cross the outside of the region. S54. After completing the initial triangular mesh construction, the directional edge flipping optimization step is executed. The minimum angle value of each internal triangle is checked one by one. When the minimum angle of the triangle is less than the preset threshold, the edge flipping reconstruction operation is executed. During the flipping process, it is detected in real time whether the flipped edge violates the boundary constraint table to ensure the boundary continuity. S55. Perform boundary stitching for locations with discrete gaps between pixels. When the distance between adjacent boundary points is less than a set spacing threshold, automatically insert bridging points and re-triangulate locally to fill the gaps between discontinuous pixels.

7. The method for detecting multispectral screen pixel defects according to claim 6, characterized in that, S53 specifically includes: S531. Establish a boundary distance table for all points participating in triangulation within each defect area, calculate the shortest vertical distance from each point to the nearest boundary line segment, and write the distance into the point attribute cache. S532. During the Bowyer-Watson insertion process, when the newly inserted point triggers the conflict triangle detection, the center and radius of the circumcircle of each triangle to be deleted are calculated. Based on the original empty circle judgment logic, a boundary influence level classification rule is introduced, and the conflict priority is sorted according to the boundary distance value corresponding to the point closest to the boundary among the three vertices that constitute the triangle. S533. For triangles containing vertices whose boundary distance is less than a set threshold, mark them as high-sensitivity triangles and place them in a priority processing position in the conflict deletion queue. At the same time, when generating new triangles, add a region interiority detection step and use the ray parity rule to determine the region affiliation of the triangle's centroid. If the centroid is located outside the region, discard the triangle directly. S534. After completing the deletion and reconstruction of conflict triangles, perform boundary crossing detection again on all newly generated triangles. By traversing the three sides of the triangle and the set of boundary constraint edges, perform line segment intersection detection. If there is an intersection, roll back the triangle and re-perform local triangulation to ensure that the final generated triangular mesh is strictly located inside the defect area and does not cross the boundary.

8. A multispectral screen pixel defect detection system, comprising the multispectral screen pixel defect detection method according to any one of claims 1 to 7, characterized in that, include: The data acquisition module is used to acquire image data of the screen in the visible light, near-infrared light and short-wave infrared light bands through a multispectral sensor, and to perform noise reduction, geometric correction, color space conversion and brightness equalization preprocessing operations on the acquired images. The feature extraction module is used to extract the spectral response features of each pixel in different spectral bands from the preprocessed image data, construct a spectral data matrix, and use principal component analysis to perform feature dimensionality reduction on the spectral data matrix to generate the spectral feature vectors of the corresponding pixels. The defect determination module is used to perform K-means clustering analysis on the spectral feature vector, determine whether there are defects in the pixels based on the clustering results, and label the defective pixels as bad pixels, color difference pixels, bright pixels or abnormal types according to preset category labels; The topology construction module is used to perform weighted distance-based region topology clustering on pixels identified as defects, grouping pixels according to defect type and spatial location, and constructing an adjacency table and spatial topology structure for defect regions. The boundary reconstruction module is used to perform an improved Delaunay triangulation operation based on the spatial topological relationship of the defect region, generate a triangular mesh structure that satisfies the boundary constraints, reconstruct the spatial boundary of the defect region, and eliminate discontinuous pixel gaps. The results output module is used to perform coordinate mapping and overlay analysis between the smoothed defect area boundary and the original screen image to generate inspection report data containing the specific location of the defect area, boundary contour, and defect type identifier.

Images