A partition adaptive detection and multi-scale defect extraction method based on bright-dark field line-scan camera

Abstract

The application discloses a kind of based on bright and dark field line-scan camera's partition adaptive detection and multiscale defect extraction method, bright and dark field image of screen is synchronously collected by bright and dark field line-scan camera system and realizes pixel-level registration;According to the adaptive partition of detection area is completed according to screen geometric characteristics, establish the exclusive parameter set for each sub-region;Combined with screen front and back identification, the detection parameters of global and each sub-region are double-layer adaptive adjustment;In each sub-region, construct multiscale feature extraction model, realize the enhancement and extraction of different scale, direction defects;Through morphological, gray, texture multidimensional feature, accurate screening is carried out to defect candidate region;Establish the coordinate mapping relationship of front and back image, the application solves the contradiction between resolution and detection speed in the prior art, area detection adaptability is poor, front and back detection cooperativity is low, false positive error rate is high and other problems, realizes the high-precision, high-efficiency detection of screen defect.

Description

Technical Field

[0001] This invention belongs to the field of machine vision inspection technology, specifically involving a partitioned adaptive detection and multi-scale defect extraction method based on a bright and dark field line scan camera. It is mainly applied to defect detection and quality control of screens of electronic products such as mobile phones and tablets, and can achieve high-precision and high-efficiency detection of various defects such as bright spots, dark spots, scratches, foreign objects, uneven color, and uneven brightness. Background Technology

[0002] With the rapid development of display technology, the precision and quality requirements for screens in electronic products such as smartphones and tablets are becoming increasingly stringent. Various defects generated during the manufacturing process directly affect user experience and product market competitiveness. Traditional screen defect detection relies on manual visual inspection, which suffers from low detection efficiency, operator fatigue, and inconsistent subjective testing standards, making it unable to meet the demands of large-scale, high-paced industrial production.

[0003] The development of machine vision technology has provided solutions for automated screen defect detection. Currently, most mainstream screen inspection solutions use area scan cameras for overall imaging. However, this approach suffers from an inherent contradiction between resolution and detection speed in high-speed inspection scenarios. Increasing resolution inevitably reduces detection speed, and vice versa, making it difficult to balance detection accuracy and production efficiency. Furthermore, area scan camera inspection solutions struggle to simultaneously acquire bright-field and dark-field image information. Bright-field imaging is highly sensitive to detecting bright defects such as surface scratches and foreign objects, while dark-field imaging is more sensitive to dark defects, dust, and micro-protrusions. Single-field imaging can easily lead to missed detections of some defects.

[0004] In existing technologies, some detection systems use a time-division acquisition method to obtain dual-field images, but this method increases detection time and significantly reduces production efficiency. Moreover, existing detection technologies generally use globally uniform detection parameters, ignoring the feature differences of different areas of the screen. For example, the brightness distribution, reflection characteristics, and defect type distribution of the central display area and the edge border area of ​​the screen are completely different. The global parameters are difficult to adapt to the detection needs of each area, which easily leads to over-detection or under-detection, especially the false detection rate of the rounded corners and edge areas of the screen remains high.

[0005] Furthermore, the defect detection results on both sides of the screen need to be comprehensively judged to accurately determine the authenticity and severity of the defects. However, existing technologies lack effective spatial registration and collaborative screening mechanisms for defects on both sides, making it impossible to achieve accurate matching of defects on both sides. At the same time, factors such as screen surface reflection, ambient light interference, and equipment vibration can easily produce artifact interference. These artifacts are often misjudged as real defects, leading to an increased over-detection rate of the detection system and increasing the cost of subsequent manual review.

[0006] To address the shortcomings of the existing technologies, this invention proposes a partitioned adaptive detection and multi-scale defect extraction method based on a bright and dark field-of-view line scan camera. By using bright and dark field-of-view fusion imaging, geometric feature-driven region adaptive segmentation, adaptive adjustment of front and back double-layer parameters, multi-scale defect extraction, and front and back collaborative screening, this method solves the pain points of the existing technologies and achieves high-precision and high-efficiency detection of defects in electronic product screens. Summary of the Invention

[0007] This invention provides a method for adaptive detection and multi-scale defect extraction based on a line scan camera with varying brightness and darkness. It achieves high-speed synchronous acquisition of images with varying brightness and darkness by using a line scan camera. Combined with adaptive zoning of regions and adaptive adjustment of front and back parameters, it adapts to the feature differences of different areas and detection surfaces of the screen. Multi-scale defect extraction achieves full-scale defect coverage. Further, through multi-dimensional feature filtering and front-and-back collaborative verification, artifact interference is effectively eliminated. Ultimately, this method achieves high-precision and high-efficiency detection of various screen defects, reducing over- and under-detection rates and improving the detection efficiency and quality control level of industrial production.

[0008] This invention provides the following technical solution: a method for adaptive detection and multi-scale defect extraction based on a line scan camera with bright and dark fields of view, comprising the following steps: S1, Bright and dark field image acquisition: Constructing a detection system including a line scan camera, a bright field light source, and a dark field light source, simultaneously acquiring bright field and dark field images of the screen of the electronic product under test, and performing geometric correction and pixel-level registration after buffering and stitching the acquired line scan data to obtain bright and dark field screen image data with consistent spatial correspondence; S2, Geometric feature recognition and adaptive partitioning of the detection area: Based on the geometric features and structural information of the screen, dividing the detection area of ​​the screen under test into multiple sub-regions with different geometric attributes, illumination characteristics, and defect type distribution characteristics, and establishing a region identifier and region parameter set for each sub-region; S3, Adaptive adjustment of front and back defect detection parameters: Obtaining the front and back identifiers of the screen under test, and calling the corresponding detection model or parameter template according to the front and back identifiers to perform global adjustment of the bright and dark field images. Preprocessing parameters are adaptively adjusted, and then local detection parameters are adaptively adjusted in each sub-region based on regional reflectance characteristics and background grayscale statistical characteristics; S4, Multi-scale defect extraction: Bright and dark field images are preprocessed separately, and multi-scale filtering and multi-directional feature extraction models are constructed in each sub-region. After normalizing and fusing the response results at each scale, bright and dark field defect candidate regions are extracted; S5, Multi-dimensional defect screening: Morphological features and grayscale features are extracted for each defect candidate region, and texture features can be optionally extracted. Candidate regions that do not meet the defect features are removed according to preset rules, and then morphological operations are performed on the remaining candidate regions to optimize the region boundaries; S6, Front and back defect registration and collaborative screening: A geometric relationship model of the front and back images of the screen is established, and the front and back defect candidate regions are uniformly mapped to the same reference coordinate system to achieve spatial registration. A defect correlation evaluation function is constructed to complete the matching and association of front and back defect candidates, determine the real defects, and remove artifact interference.

[0009] Further, the specific process of acquiring the bright and dark field images in step S1 is as follows: S11, fix the screen to be tested on the conveyor mechanism or platform, install the bright field line scan camera and the near-coaxial bright field light source in the normal direction of the screen, and arrange the dark field line scan camera and the dark field light source in the tilt direction of the screen with an incident angle of 15° to 45°; S12, perform geometric calibration on the bright and dark field line scan cameras respectively through the calibration plate to obtain the camera intrinsic parameters and relative extrinsic parameters; S13, synchronously control the bright field line scan camera and the bright field light source to acquire bright field line scan data of the screen to be tested, and buffer and stitch them to obtain a complete bright field image; S14, synchronously control the dark field line scan camera and the dark field light source to acquire dark field line scan data with the same line frequency, displacement step size and spatial sampling interval, and buffer and stitch them to obtain a complete dark field image; S15, use the calibration parameters to perform geometric correction and registration on the bright and dark field images to ensure that the two correspond at the pixel level in the screen coordinate system.

[0010] Furthermore, the specific process of geometric feature recognition and adaptive partitioning of the detection area in step S2 is as follows: S21, extract the screen area in the bright field image through threshold segmentation and connected component analysis, perform sub-pixel level edge detection and fitting on the edge of the screen area, and identify straight edges and rounded corners to obtain the outer contour of the screen; S22, match the outer contour parameters of the screen with the preset product geometric template to determine the theoretical positions of the screen length, width, and rounded corner radius; S23, divide the detection area into the center display area, the border area, and the rounded corner area, and further subdivide the border area into the upper, lower, left, and right border areas; S24, assign a unique area identifier to each sub-area and establish an area parameter structure that includes the area type, coordinate range, initial value of filtering parameters, local threshold range, defect size, and shape prior; S25, calculate the mean, standard deviation, and histogram features of the background grayscale distribution in each sub-area, statistically analyze the differences in regional reflectivity under bright and dark fields, and write the statistical indicators into the area parameter set.

[0011] Furthermore, in step S3, the front and back markings are obtained through barcode / QR code information, production line station information, mechanical fixture posture information, or image recognition methods; global preprocessing parameters include brightness normalization coefficient, contrast enhancement parameter, noise filtering kernel size, and edge enhancement coefficient; local detection parameters include local threshold, defect response enhancement parameter, and minimum / maximum connected component size threshold.

[0012] Furthermore, in step S4, the bright and dark field images are preprocessed, including grayscale normalization, noise suppression, and background flat field correction. The construction process of the multi-scale filtering and multi-directional feature extraction model is as follows: S41, after cropping the images of each sub-region, construct a multi-scale image of 3-5 layers of Gaussian pyramid or Laplacian pyramid; S42, perform high-pass filtering or band-pass filtering on the images at each scale to highlight defects of different sizes; S43, adopt multi-directional filtering based on structural tensor or Gabor filter bank, select typical directions in the range of 0° to 180° to calculate directional responses and fuse them to enhance the saliency of linear defects; S44, adopt Laplacian operator or Gaussian difference filtering to enhance the structure of point defects.

[0013] Furthermore, the process of normalizing and fusing the response results at each scale in step S4 is as follows: the gray values ​​of the filtered response map at each scale are normalized to a unified range, and the response maps at each scale are synthesized into a multi-scale defect response map by weighted summation, maximum value fusion, or confidence-based fusion strategy; the method for extracting bright-field and dark-field defect candidate regions is as follows: adaptive threshold segmentation based on histogram bimodal distribution or statistical model is used on the multi-scale defect response map, and all connected regions are extracted as defect candidate regions through connected component analysis.

[0014] Furthermore, in step S5, the morphological features include area, perimeter, major and minor axis lengths, aspect ratio, roundness, rectangularity, orientation angle, and boundary roughness; the grayscale features include average grayscale, grayscale extreme values, grayscale contrast, grayscale variance, mean and maximum grayscale gradient, and grayscale difference between bright and dark fields; the texture features include grayscale co-occurrence matrix features, local binary pattern features, and wavelet energy features; and the morphological operations include dilation, erosion, opening, and closing operations to achieve isolated noise point elimination and merging of adjacent defect regions of the same type.

[0015] Furthermore, the geometric relationship model of the front and back images of the screen in step S6 includes rotation, translation, mirror transformation and scale transformation parameters. The parameters are obtained by collecting reference images of the front and back standard samples and combining them with the calibration pattern. The specific process of spatial registration is as follows: the centroid coordinates and contour point coordinates of the candidate defect areas on the front and back are uniformly transformed to the product design coordinate system or the front image coordinate system through the coordinate mapping matrix.

[0016] Furthermore, the defect correlation evaluation function in step S6 comprehensively considers the following indicators: spatial overlap rate of the front and back defect regions, area ratio and shape similarity of the front and back defect regions, difference in grayscale response of the front and back bright and dark fields, and prior knowledge of defect type; the spatial overlap rate is the ratio of the intersection area to the union area of ​​two defect regions.

[0017] Furthermore, the matching and association rules for front and back defect candidates in step S6 are as follows: when the defect association evaluation result is greater than the first preset threshold, the corresponding front and back candidate regions are regarded as different views or different lighting responses of the same real defect, and their feature information is integrated; when the defect association evaluation result is between the first preset threshold and the second preset threshold, the candidate region is marked as uncertain; when the defect association evaluation result is less than the second preset threshold, the candidate region that appears only on one side is determined as artifact interference or background structure features and is removed by combining local texture and grayscale statistical features.

[0018] The beneficial effects of this invention are:

[0019] By synchronously configuring a line-scan camera with bright and dark field light sources, simultaneous imaging of bright and dark fields at the same location is achieved. This addresses the detection needs of both bright defects such as surface scratches and foreign objects, and dark defects such as dark spots, dust, and micro-protrusions. Compared to single-field-of-view or time-division acquisition solutions, the defect detection rate is improved by more than 25%. Based on the screen's geometric features, the detection area is divided into sub-regions such as the central display area, border area, and rounded corner area. Dedicated detection parameters are configured for each sub-region, solving the problem of high false detection rates in traditional global detection at screen edges and corners. The accuracy of edge area detection is improved by more than 30%. Automatic identification of the detection surface through front and back markings enables dual-layer adaptive adjustment of global preprocessing parameters and sub-region local detection parameters. This eliminates the need to separately debug algorithm parameters for the front and back of the screen, shortening the algorithm debugging cycle by 50%. It also adapts to the different reflection characteristics and defect distributions of the front and back, improving the versatility of the detection. A multi-scale image pyramid with 3-5 layers is constructed, and multi-directional filtering is used to achieve layered enhancement and extraction of micro, medium, and large defects. This effectively detects weak defects with an area as small as 0.005 mm², avoiding the missed detection phenomenon of single-scale detection and achieving full-scale defect coverage from macro to micro. Defect candidate regions are screened layer by layer using morphological, grayscale, and texture features. Combined with spatial registration and correlation evaluation of defects on both sides, accurate determination of real defects and artifacts is achieved, effectively eliminating artifacts caused by screen reflections and ambient light interference. The defect over-detection and under-detection rate of this invention is only 0.85%, far lower than existing technologies. By integrating the morphological, grayscale, and texture features of defects, as well as the illumination response features of both sides, comprehensive extraction and accurate classification of defect location, size, type, and severity are achieved, providing precise data support for subsequent product quality grading and production process optimization. Attached Figure Description

[0020] Figure 1 This is an overall flowchart of the partitioned adaptive detection and multi-scale defect extraction method of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of the line scan camera detection system for bright and dark fields of view of the present invention;

[0022] Figure 3 This is a schematic diagram of the partitioning of the mobile phone screen detection area according to the present invention;

[0023] Figure 4 This is a schematic diagram of the multi-scale defect extraction process of the present invention;

[0024] Figure 5 This is a schematic diagram of the process for front and back defect registration and collaborative screening in this invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Reference Figures 1-5 As shown, a method for zoned adaptive detection and multi-scale defect extraction based on a line scan camera with bright and dark fields of view includes the following steps:

[0027] S1. Simultaneously acquire bright field and dark field images of the screen of the electronic product under test; after buffering and stitching the acquired line scan data, perform geometric correction and pixel-level registration to obtain bright and dark field screen image data with consistent spatial correspondence. This ensures a one-to-one correspondence between the bright field and dark field images in spatial position, providing an accurate pixel-level basis for subsequent joint analysis of bright and dark field images and comparison of defect features, and eliminating defect misjudgment caused by image misalignment.

[0028] S2. Based on the screen's geometric features and structural information, the detection area of ​​the screen under test is divided into multiple sub-regions with different geometric attributes, lighting characteristics, and defect type distribution characteristics. This breaks the traditional global unified detection mode, adapts to the inherent feature differences of different areas of the screen, and avoids over-detection or under-detection due to different regional features. A region identifier and region parameter set are established for each sub-region, realizing independent management and precise calling of the detection parameters of each sub-region. This provides a data carrier and identification basis for subsequent adaptive adjustment of detection parameters by partition.

[0029] S3. Adaptive Adjustment of Defect Detection Parameters for Front and Back Sides: The system acquires the front and back markings of the screen under test, enabling automatic identification of the screen's detection surfaces without manual intervention, thus improving the automation level of the detection process. Based on the front and back markings, the system calls the corresponding detection model or parameter template, adaptively adjusting the global preprocessing parameters of the bright and dark field images. This adapts to the different reflection characteristics, texture features, and defect distribution patterns of the front and back sides of the screen, avoiding the problem that a single global parameter cannot simultaneously meet the detection requirements for both front and back sides. Furthermore, within each sub-region, based on the regional reflection characteristics and background grayscale statistical characteristics, the system adaptively adjusts the local detection parameters, achieving global adaptation and precise regional dual-layer optimization of the detection parameters. This further aligns with the actual detection needs of each sub-region, significantly improving the matching degree between parameters and the detection scenario.

[0030] S4. Multi-scale Defect Extraction: Bright and dark field images are preprocessed separately to eliminate problems such as uneven brightness, noise, and background interference, thereby improving the contrast between defects and the background. Multi-scale filtering and multi-directional feature extraction models are constructed in each sub-region to achieve targeted enhancement of defects of different sizes and directions, solving the problem that single-scale and single-direction filtering is prone to missing special types of defects. After normalizing and fusing the response results of each scale, candidate regions for bright and dark field defects are extracted, eliminating the difference in response amplitude between different scales and realizing the fusion of defect features across all scales. This ensures that both small and large defects can be effectively extracted, expanding the scale coverage of defect detection.

[0031] S5. Multi-dimensional Defect Screening: Morphological and grayscale features are extracted for each defect candidate region, and texture features can be optionally extracted. The inherent attributes of defects are mined from multiple dimensions, providing rich and accurate criteria for determining the authenticity of defects and effectively distinguishing between real defects and artifacts. Candidate regions that do not meet the defect features are removed according to preset rules, achieving preliminary removal of artifacts from the defect candidate regions, significantly reducing the processing range of subsequent detection and improving the efficiency of the algorithm. Morphological operations are then performed on the retained candidate regions to optimize the region boundaries, eliminate the interference of isolated noise points, merge adjacent defect regions of the same type, restore the true outline of the defects, and improve the integrity and accuracy of the defect regions.

[0032] S6. Front and Back Defect Registration and Collaborative Screening: A geometric relationship model of the front and back images of the screen is established, providing a mathematical basis for the spatial matching of front and back defects and realizing the coordinate unification of images from different detection surfaces. The front and back defect candidate regions are uniformly mapped to the same reference coordinate system to achieve spatial registration, eliminating the differences in geometric transformation between the front and back images and ensuring the comparability of the defect candidate regions in spatial position. A defect correlation evaluation function is constructed to complete the matching and correlation of front and back defect candidates, determine the real defects and eliminate artifact interference. Through multi-index comprehensive evaluation, the accurate determination of real defects and artifacts is achieved, effectively eliminating false judgments of artifacts caused by surface reflection, ambient light interference, etc., and significantly reducing the over-detection rate.

[0033] Furthermore, the specific process of acquiring bright and dark field images in step S1 is as follows: the bright field line scan camera and the near-coaxial bright field light source are installed in the normal direction of the screen, so that the light shines perpendicularly on the screen surface, improving the sensitivity of bright field imaging for detecting bright defects such as surface scratches and foreign objects; the dark field line scan camera and the dark field light source are arranged in the tilt direction of the screen at an incident angle of 15° to 45°, so that the incident light produces a larger incident angle on the screen surface, enhancing the response intensity of dark field imaging for scattering defects such as dark defects, micro-protrusions, and dust, making small defects easier to identify in dark field images. Acquiring the camera's intrinsic and relative extrinsic parameters provides accurate parameter support for subsequent geometric correction and pixel-level registration of the images, eliminating image errors caused by camera distortion and installation position deviations. Bright field line scan data is acquired from the screen under test, and buffered and stitched to obtain a complete bright field image, realizing high-speed and continuous acquisition of screen bright field images. The line scan method ensures high resolution while adapting to high-speed detection requirements, and the buffered stitching restores the complete image information of the screen. Dark-field line scan data were acquired using the same line frequency, displacement step size, and spatial sampling interval. These data were then buffered and stitched together to obtain a complete dark-field image, ensuring consistent acquisition parameters between bright and dark-field images and eliminating differences in image scale and resolution caused by varying acquisition parameters. Geometric correction and registration of the bright and dark-field images were performed using calibration parameters, ensuring pixel-level correspondence between them in the screen coordinate system. This achieved precise alignment of the bright and dark-field images, enabling accurate matching of defects at the same screen location within both images and improving the accuracy of joint bright and dark-field analysis.

[0034] Furthermore, the specific process of geometric feature recognition and adaptive partitioning of the detection area in step S2 is as follows: S21. The screen area is extracted from the bright field image through threshold segmentation and connected component analysis, which can quickly and accurately separate the screen from the background and eliminate the interference of the background area on subsequent detection; sub-pixel level edge detection and fitting are performed on the screen area edges to identify straight edges and rounded corners to obtain the screen outer contour, improving the accuracy of screen contour extraction. Sub-pixel level detection can capture the subtle features of the screen edges and ensure the accuracy of contour fitting. S22. The screen outer contour parameters are matched with the preset product geometric template to determine the theoretical positions of the screen length, width, and rounded corner radius. The extracted contour is calibrated in combination with product design standards to eliminate contour deviations generated during image acquisition. S23. The detection area is divided into the central display area, the border area, and the rounded corner area. The border area is further subdivided into the upper, lower, left, and right border areas. Fine partitioning is performed according to the geometric attributes, lighting characteristics, and defect distribution patterns of different areas of the screen, so that the detection strategy of each sub-area can be specifically matched with its inherent features, solving the problem of low detection accuracy of edge and rounded corner areas. S24. Assign a unique region identifier to each sub-region and establish a region parameter structure that includes region type, coordinate range, initial values ​​of filtering parameters, local threshold range, defect size, and shape priors. This creates a dedicated parameter file for each sub-region, enabling zoned management of detection parameters and allowing subsequent parameter adjustments to be precisely targeted to specific sub-regions. S25. Calculate the mean, standard deviation, and histogram features of the background grayscale distribution within each sub-region. Statistically analyze the differences in regional reflectance characteristics under bright and dark conditions, and write these statistical indicators into the region parameter set. This process uncovers the image statistical features of each sub-region, providing a quantitative basis for subsequent adaptive adjustment of local detection parameters, allowing parameter adjustments to better align with the actual lighting and background characteristics of each sub-region.

[0035] Furthermore, in step S3, the front and back markings are obtained through barcode / QR code information, production line station information, mechanical fixture posture information, or image recognition methods, providing multiple front and back recognition methods that can adapt to different production line scenarios, improving the flexibility and reliability of recognition, while also automating the recognition process without manual intervention; global preprocessing parameters include brightness normalization coefficient, contrast enhancement parameter, noise filtering kernel size, and edge enhancement coefficient, performing global preprocessing on the image from four core dimensions: brightness, contrast, noise, and edge, which can comprehensively eliminate global interference in the image and improve the overall image quality; local detection parameters include local threshold, defect response enhancement parameter, and minimum / maximum connected component size threshold, adjusting local parameters from three aspects: defect judgment, defect enhancement, and defect size screening, which can accurately adapt to the defect detection needs of each sub-region and avoid local over-detection or under-detection caused by global parameters.

[0036] Furthermore, in step S4, the bright and dark field images are preprocessed separately, including grayscale normalization, noise suppression, and background flattening correction. Grayscale normalization eliminates the overall brightness difference caused by fluctuations in screens or light sources from different batches, ensuring the consistency of image brightness. Noise suppression reduces image noise without destroying the defect edges, improving the contrast between defects and the background. Background flattening correction eliminates the problem of brightness gradient in the screen background, making defects easier to identify in a uniform background. The three processes work together to improve the overall image quality.

[0037] The construction process of the multi-scale filtering and multi-directional feature extraction model is as follows: S41. After cropping the images of each sub-region, construct 3-5 layers of Gaussian pyramids or Laplacian pyramids to achieve multi-scale image layering, enabling defects of different sizes to be accurately identified at their corresponding scales, covering the detection range from microscopic small defects to macroscopic large defects. S42. Perform high-pass or band-pass filtering on the images at each scale to highlight defects of different sizes, effectively filtering low-frequency background information in the images at each scale, enhancing high-frequency details of defects, and making the features of defects of different sizes more significant. S43. Employ multi-directional filtering based on structural tensors or Gabor filter banks, select typical directions within the range of 0° to 180° to calculate and fuse directional responses, enhancing the saliency of linear defects, and specifically enhancing linear defects in different directions (such as scratches and cracks), solving the problem of single-directional filtering easily missing non-positive directional linear defects, and improving the detection rate of linear defects. S44. Employing the Laplacian operator or Gaussian difference filtering enhances the structure of point defects, accurately enhancing the characteristics of point defects such as bright spots, dark spots, and particles, making tiny point defects easier to identify in images and improving the detection sensitivity of point defects.

[0038] Furthermore, the normalization and fusion process of the response results at each scale in step S4 is as follows: the gray values ​​of the filtered response map at each scale are normalized to a unified range, eliminating the differences in the amplitude of the filtered response between different scales and avoiding the masking of defect features due to different scales; through weighted summation, maximum value fusion, or confidence-based fusion strategies, the response maps at each scale are synthesized into a multi-scale defect response map, realizing the fusion of defect features at each scale, ensuring that defect features at all scales can be reflected in the final response map, and avoiding missed defects at a single scale; the method for extracting bright-field and dark-field defect candidate regions is as follows: adaptive threshold segmentation based on histogram bimodal distribution or statistical models is used on the multi-scale defect response map, which can automatically determine the segmentation threshold according to the gray-level features of the image itself, adapting to the image features of different sub-regions and different fields of view, and avoiding inaccurate segmentation caused by fixed thresholds; all connected regions are extracted as defect candidate regions through connected component analysis, which can quickly and accurately extract suspected defect regions in the image, providing target objects for subsequent multi-dimensional screening.

[0039] Furthermore, in step S5, morphological features include area, perimeter, major and minor axis lengths, aspect ratio, roundness, rectangularity, orientation angle, and boundary roughness. These features describe defect characteristics from multiple geometric dimensions, effectively distinguishing different types of defects (such as linear scratches, dot-like particles, and uneven surface coatings), and providing a geometric basis for determining the authenticity of defects. Gray-scale features include average grayscale, grayscale extremes, grayscale contrast, grayscale variance, mean and maximum grayscale gradient, and grayscale difference between bright and dark fields. These features describe defect characteristics from three aspects: grayscale distribution, grayscale variation, and bright-dark field contrast. This effectively distinguishes between genuine defects and reflection artifacts because genuine defects exhibit consistent grayscale responses in both bright and dark fields, while artifacts typically only show... It appears in a single field of view; texture features include gray-level co-occurrence matrix features, local binary mode features, and wavelet energy features, which can accurately describe the features of texture defects such as coating unevenness and surface haze, making up for the shortcomings of morphological and gray-level features in the discrimination of texture defects and improving the detection rate of texture defects; morphological operations include dilation, erosion, opening, and closing operations, which realize the elimination of isolated noise points and the merging of adjacent defect regions of the same type. Erosion and opening operations can effectively eliminate isolated noise points in the image and avoid noise points being misjudged as minor defects; dilation and closing operations can smooth the boundaries of defect regions, merge adjacent defect regions of the same type, restore the true contour of defects, and improve the integrity and accuracy of defect regions.

[0040] Furthermore, in step S6, the geometric relationship model of the front and back images of the screen includes rotation, translation, mirror transformation, and scale transformation parameters. These parameters are obtained by collecting reference images of standard samples on both sides and combining them with calibration patterns. This establishes a precise mathematical model for the coordinate transformation of the front and back images, accurately describing the geometric transformation relationship between the front and back images and providing a quantitative basis for spatial registration. The specific process of spatial registration is as follows: the centroid coordinates and contour point coordinates of the candidate defect areas on both sides are uniformly transformed to the product design coordinate system or the front image coordinate system through a coordinate mapping matrix. This eliminates the differences in geometric transformation between the front and back images and maps the candidate defect areas of different detection surfaces to the same spatial coordinate system, ensuring the comparability of defects in spatial position and facilitating subsequent matching of defects on both sides.

[0041] Furthermore, the defect correlation evaluation function in step S6 comprehensively considers the following indicators: spatial overlap rate of defect areas on both sides, area ratio and shape similarity of defect areas on both sides, difference in grayscale response between bright and dark fields on both sides, and prior knowledge of defect types. Spatial overlap rate determines the matching of defects from the position dimension, ensuring the correspondence of defects in space; area ratio and shape similarity determine the matching of defects from the morphological dimension, ensuring the consistency of morphological features of defects on both sides; difference in grayscale response between bright and dark fields determines the matching of defects from the grayscale feature dimension, ensuring the consistency of grayscale features of defects on both sides; prior knowledge of defect types determines the rationality of defects from the product process dimension, excluding defect types that cannot appear on a certain inspection surface. The four indicators work together to achieve a comprehensive and accurate evaluation of defect correlation. Spatial overlap rate is the ratio of the intersection area to the union area of ​​two defect areas, which can accurately quantify the degree of spatial overlap between two defect areas, providing an objective and quantitative evaluation indicator for spatial position matching and avoiding the bias of subjective judgment.

[0042] Furthermore, the matching and association rules for front and back defect candidates in step S6 are as follows: When the defect correlation evaluation result is greater than the first preset threshold, the corresponding front and back candidate areas are regarded as different views or different lighting responses of the same real defect. By integrating their feature information, the real defect can be accurately determined. At the same time, the defect features of the front and back are integrated to provide comprehensive feature information for the accurate classification and grading of defects. When the defect correlation evaluation result is between the first and second preset thresholds, the candidate area is marked as uncertain. Suspected defects can be identified, avoiding misjudgment due to algorithm limitations. At the same time, it provides a target for manual review and improves the reliability of detection. When the defect correlation evaluation result is less than the second preset threshold, the candidate area appearing only on one side is determined as artifact interference or background structure features and removed by combining local texture and grayscale statistical features. This can accurately identify and remove artifacts, effectively eliminate over-detection problems caused by surface reflection, ambient light interference, equipment vibration, etc., and significantly reduce the over- and under-detection rates.

[0043] In this invention, the bright and dark field image acquisition is achieved by constructing a bright and dark field line scan detection system, which enables high-speed synchronous acquisition of bright and dark field images of the screen under test, and completes geometric correction and pixel-level registration of the images. This provides a spatially consistent image data foundation for subsequent defect detection, solving the problems of resolution and speed contradiction and low efficiency of bright and dark field time-division acquisition in traditional area array cameras.

[0044] The specific implementation process is as follows: A bright-field line scan camera and a coaxial / near-coaxial strip / area bright-field light source are installed in the normal direction of the screen, ensuring that the light illuminates the surface being measured approximately perpendicularly, thus guaranteeing the sensitivity of bright-field imaging for detecting bright defects. A dark-field line scan camera and a ring, strip, or multi-directional combination of dark-field light sources are arranged at an incident angle of 15°–45° in the tilt direction of the screen, causing the incident light to generate a larger incident angle on the screen surface, enhancing the response of small scattering defects on the surface in the dark-field image. The bright and dark field line scan cameras are geometrically calibrated separately using a calibration plate to obtain the intrinsic and relative extrinsic parameters of both cameras. An industrial control computer sends a start command, which controls the conveyor mechanism via an encoder or a fixed-length pulse to make the screen pass through the camera's field of view at a uniform speed. When the screen enters the detection area, the trigger controller simultaneously outputs a synchronous trigger signal to the bright-field camera and the bright-field light source. Bright-field line scan data is collected at a set line frequency, and the continuous line data is buffered and stitched together to obtain a complete bright-field screen image. Subsequently, the bright-field light source is turned off and the dark-field light source is turned on. Dark-field line scan data is collected at the same line frequency, displacement step, and spatial sampling interval. The data is buffered and stitched together to obtain a complete dark-field screen image.

[0045] Using the previously calibrated camera parameters, geometric correction and pixel-level registration are performed on the bright and dark field images to ensure that the two have a one-to-one pixel relationship in the screen coordinate system, thus ensuring the accuracy of subsequent joint analysis of bright and dark field images.

[0046] Furthermore, refer to Figure 3 As shown, the detection area geometric feature recognition and adaptive partitioning divides the detection area into multiple sub-regions based on the geometric features and structural information of the screen. A unique region identifier and parameter set are established for each sub-region, solving the problem that traditional global detection parameters cannot adapt to the different features of different areas of the screen, and achieving regionalized and accurate matching of detection parameters.

[0047] The specific implementation process is as follows: In the bright-field image, a global or adaptive threshold is used to coarsely segment the screen and background. Connected component analysis is used to extract the connected component with the largest area as the screen region. Subpixel-level edge detection and fitting are performed on the screen region edges. Hough transform is used to identify straight edges and rounded corner points, obtaining the four straight edges and four rounded corner segments of the screen's outer contour. The extracted screen contour parameters are matched with a preset product geometry template to determine the theoretical positions of the screen's length, width, and rounded corner radius, providing accurate prior information for region division. Based on the contour fitting results, the detection area is divided into a central display area, a border area, and a rounded corner area. The border area is further subdivided into a top border, a bottom border, a left border, and a right border area. Each sub-region has different geometric attributes, lighting characteristics, and defect type distribution characteristics (e.g., the central display area is prone to bright / dark spots, the border area is prone to scratches, and the rounded corner area is prone to ink defects). Each sub-region is assigned a unique region identifier, and a region parameter structure is established that includes region type, coordinate range, initial values ​​of multi-scale filtering parameters, local threshold range, and prior information on defect size and shape. The mean, standard deviation, and histogram features of the background grayscale distribution are calculated within each sub-region to obtain the region's brightness baseline and noise level. At the same time, the differences in region reflectivity under bright and dark conditions are statistically analyzed (e.g., the border ink region has low grayscale and smooth texture, while the display region has uniform grayscale distribution but exhibits brightness gradation). These statistical indicators are written into the region parameter set to provide a basis for the adaptive adjustment of subsequent local detection parameters.

[0048] Furthermore, the front and back sides adaptively adjust the defect detection parameters to achieve two-layer adaptive adjustment of the detection parameters. First, the global preprocessing parameters are adapted based on the front and back side markings on the screen. Then, the local detection parameters are optimized by combining the statistical characteristics of each sub-region. This solves the problem that existing technologies cannot simultaneously adapt to the front and back side detection requirements of the screen, greatly shortens the algorithm debugging cycle, and improves the adaptability of the detection parameters.

[0049] The specific implementation process is as follows: read the barcode / QR code information on the screen, which includes front / back detection markings; pre-configure based on production line station information or mechanical fixture posture information; and verify by analyzing image texture features (such as the metal layer and ribbon cable area on the back) using image recognition methods to prevent marking errors.

[0050] Global preprocessing parameter adaptive adjustment: Parameter templates are pre-established for the front and back of the screen in the industrial control computer. The templates include global preprocessing parameters such as brightness normalization coefficient, contrast enhancement parameters, noise filtering kernel size, and edge enhancement coefficient. Based on the obtained front and back identification, the corresponding parameter templates are automatically loaded, and brightness normalization (eliminating brightness differences between different batches of screens or light source fluctuations), noise suppression (using median filtering / bilateral filtering to reduce noise while ensuring that defect edges are not smoothed), and edge enhancement (using Laplacian / Sobel operators to enhance linear defects such as scratches and cracks) are performed on the bright and dark field images respectively.

[0051] Adaptive adjustment of local detection parameters: Based on the global parameter template, the local threshold, defect response enhancement parameters, and minimum / maximum size thresholds of connected components are adaptively adjusted within each sub-region according to the region's reflectivity and background grayscale statistical characteristics. For example, the lower limit threshold of the area is appropriately increased in the ink border region to avoid misjudging tiny gaps at the ink edge as defects; the dark field response enhancement coefficient is adjusted in highly reflective areas such as exposed glass to improve sensitivity to fine particle defects while suppressing large-area uniform reflection. The finalized region-level detection parameters are stored in the region parameter set to provide parameter support for subsequent multi-scale defect extraction.

[0052] Reference Figure 4 As shown, multi-scale defect extraction involves constructing multi-scale and multi-directional feature extraction models in each sub-region to achieve full-scale defect enhancement and extraction from micro to macro. This solves the problem that single-scale detection is prone to missing small or large defects, ensuring that defects of different sizes and types can be effectively identified.

[0053] The specific implementation process is as follows: Gray-level normalization, noise suppression, and background flattening correction are performed on the bright and dark field images respectively to eliminate background interference and improve the contrast between defects and the background. The preprocessed bright and dark field images are then precisely cropped according to the aforementioned sub-regions to avoid invalid backgrounds participating in the calculations and improve algorithm efficiency. For each sub-region cropped image, a multi-scale image of 3-5 layers is constructed using Gaussian pyramid or Laplacian pyramid methods. The top layer is the original resolution, and each layer below is scaled according to a preset ratio to achieve layered detection of defects of different sizes.

[0054] Multi-scale, multi-directional defect enhancement: High-pass or band-pass filtering is applied to images at various scales, and a detailed residual map is obtained by subtracting the smoothed result from the original image to highlight small and medium-sized defects. For areas prone to directional scratches (such as screen edges and areas consistent with the transmission direction), multi-directional filtering based on structural tensors or Gabor filter banks is used. Typical directions such as 0°, 45°, 90°, and 135° are selected within the range of 0° to 180°, and the directional responses are calculated and maximum values / weighted fusion is performed to enhance the saliency of linear defects. For near-point defects such as particles and bright spots, Laplacian operator or Gaussian difference filtering is used to enhance the point structure features at specific scales.

[0055] Multi-scale response fusion: The grayscale normalization of the filtered response map at each scale is performed to make its grayscale values ​​fall within a uniform range, eliminating the difference in response amplitude between different scales; through weighted summation, maximum value fusion or confidence-based fusion strategies, the response maps at each scale are combined into a multi-scale defect response map, resulting in bright-field and dark-field multi-scale defect response maps respectively.

[0056] Defect candidate region extraction: Adaptive threshold segmentation is used on the multi-scale defect response map. The threshold is automatically determined based on the bimodal distribution of the response histogram within the region or a statistical model (mean ± multiple standard deviation). Connectivity analysis is performed on the segmentation results to extract all connected regions as defect candidate regions. Geometric information such as area, bounding rectangle, and minimum bounding ellipse are calculated. Each candidate region is labeled with attributes such as source (bright / dark field), sub-region, scale layer number, and preliminary response intensity, generating bright field defect candidate sets and dark field defect candidate sets respectively.

[0057] In this invention, multi-dimensional defect screening involves filtering candidate defect regions layer by layer using morphological, grayscale, and texture features, and then optimizing the defect region boundaries through morphological operations. This solves the problem of noise and artifacts in the candidate regions, significantly narrowing the range of candidate defects and improving the accuracy of defect identification.

[0058] The specific implementation process is as follows: Morphological feature extraction and screening: For each candidate connected region of a defect, extract morphological features such as area, perimeter, major and minor axis length, aspect ratio, roundness, rectangularity, orientation angle, and boundary roughness; compare the extracted features with the prior defect type corresponding to the region (e.g., linear scratches have a large aspect ratio, particle defects have high roundness, and uneven coating defects have a large area and smooth boundaries), and directly remove candidate regions that obviously do not conform to any defect morphological features according to preset rules.

[0059] Gray-scale feature extraction and screening: In the original bright and dark field images, gray-scale features such as average gray-scale, gray-scale extreme value, gray-scale contrast, gray-scale variance, mean and maximum gray-scale gradient, and gray-scale difference between bright and dark fields are extracted from the remaining candidate regions; the boundary clarity and gray-scale change pattern of the defect candidate regions are analyzed to determine the consistency of the defect response under bright and dark fields, and candidate regions with reflection artifacts that only appear in a single field of view are eliminated.

[0060] Texture feature extraction and filtering (optional): For texture defects such as uneven coating and surface haze, extract texture features such as gray-level co-occurrence matrix features, local binary mode (LBP) features, and wavelet energy features. Based on the differences in texture features, further filter out candidate regions for texture defects and remove pseudo-candidate regions that are interfered with by background textures.

[0061] Post-processing of candidate regions: Morphological operations such as dilation, erosion, opening, and closing are performed on the candidate regions retained after multi-dimensional feature screening to smooth the boundaries of the defect regions and eliminate small isolated noise points; candidate regions that are close to each other and have similar morphology and grayscale features are merged to form a complete defect region outline; label information is added to each final candidate region, including the sub-region to which it belongs, the initial defect type, the feature vector, and the response level in bright and dark fields.

[0062] Reference Figure 5 As shown, front and back defect registration and collaborative screening establishes the geometric mapping relationship between the front and back images of the screen, realizes the spatial registration and accurate matching of front and back defect candidate regions, and completes the final judgment of real defects and artifacts by constructing a defect correlation evaluation function. This solves the problems of the inability to coordinate front and back detection results and the high false judgment rate of artifacts in the existing technology, and realizes the accurate verification of defect authenticity.

[0063] The specific implementation process is as follows:

[0064] Geometric relationship model establishment: Based on the mechanical structure of the testing equipment and the camera installation relationship, reference images of the front and back standard samples are collected during the equipment debugging stage. The rotation, translation, mirror transformation and scale transformation parameters between the front and back images are calculated through the calibration pattern. A coordinate mapping matrix is ​​constructed to form a unified geometric relationship model of the front and back of the screen, and the mapping matrix is ​​fixed in the testing system.

[0065] Spatial registration: The centroid coordinates and contour point coordinates of the defect candidate areas obtained from the front and back detections are uniformly transformed to the same reference coordinate system (product design coordinate system or front image coordinate system) through a coordinate mapping matrix to achieve spatial registration of the front and back defect candidate areas.

[0066] Coarse matching of defects: In the unified coordinate system after registration, spatial proximity search is performed on the candidate areas of defects on the front and back sides. The centroid distance and the degree of overlap of the circumscribed rectangle are used as coarse matching conditions. Areas with mutual distance within a preset threshold and a certain degree of overlap are regarded as candidate matching pairs.

[0067] Defect correlation degree calculation: Construct a defect correlation degree evaluation function. For each candidate matching pair, extract and normalize the following evaluation indicators, and perform weighted summation according to preset weights to obtain the correlation degree value:

[0068] Spatial overlap rate: The ratio of the intersection area to the union area of ​​two defect regions, representing the degree of matching in spatial location; Shape similarity: The similarity of morphological features such as the area ratio, aspect ratio, and roundness of the defect regions on the front and back sides; Gray-scale response consistency: The consistency of the average gray-scale change trend and response intensity of the corresponding regions on the front and back sides in bright and dark fields; Defect type prior consistency: Combining the sub-region to which the region belongs and the predicted defect type, it is determined whether the defect is reasonably present on the front or back side (e.g., some ink defects only exist on the back side). True Defect Judgment and Artifact Removal: A three-level judgment is performed based on the defect correlation evaluation results: **Correlation > First Preset Threshold:** Corresponding front and back candidate regions are considered different views or lighting responses of the same true defect. Morphological, grayscale, and texture features are integrated to fully extract defect features. **First Preset Threshold ≥ Correlation ≥ Second Preset Threshold:** Candidate matching pairs are marked as uncertain for manual review or further algorithmic processing. **Correlation < Second Preset Threshold:** Front and back regions are considered unrelated. Candidate regions appearing only on one side are judged as artifact interference or background structure features based on their local texture and grayscale statistical features and removed. Finally, the detection system outputs complete information such as the location, size, type, and feature vector of the true defect, achieving accurate detection and classification of screen defects.

[0069] The synergy between bright and dark field acquisition and multi-scale defect extraction in this invention: The pixel-level registration of bright and dark field images achieved in step S1 provides a spatial basis for the comparative analysis of multi-scale defect responses in bright and dark fields in step S4. The strong response of the bright field to bright defects and the strong response of the dark field to dark defects complement each other in multi-scale fusion, which greatly improves the defect detection rate.

[0070] Synergy between adaptive regional partitioning and adaptive parameter adjustment: Step S2 establishes a set of geometric and statistical feature parameters for each sub-region, which provides a precise basis for the adaptive adjustment of local detection parameters in step S3. This ensures that the adjustment of both global and local parameters conforms to the actual features of the screen, achieving a two-layer parameter optimization of global adaptation and regional precision, thus solving the over-detection / missed detection problem caused by regional feature differences.

[0071] Synergy between multi-scale defect extraction and multi-dimensional defect screening: The multi-scale defect candidate regions extracted in step S4 contain defect information of different sizes and types. Step S5 then accurately screens the candidate regions based on the morphology, grayscale, and texture features of the defects. The multi-scale defect features provide rich criteria for multi-dimensional screening, while multi-dimensional screening removes false positives from the candidate regions extracted at multiple scales. The two work together to achieve accurate extraction of defect candidate regions.

[0072] Synergy between multi-dimensional screening and front-and-back collaborative screening: Step S5 completes the defect candidate screening under a single face and single field of view, eliminating most noise and artifacts, which narrows the processing range for the front-and-back collaborative screening in step S6 and improves the efficiency of registration and matching; while the front-and-back collaborative screening in step S6 performs the final authenticity verification of the candidate areas retained in step S5, realizing a two-layer defect judgment of single-face screening + double-face comparison verification, which greatly reduces the false judgment rate of artifacts.

[0073] Example 1: Construction and Image Acquisition of a Line Scan Camera Detection System for Bright and Dark Fields (Refer to...) Figure 2 As shown, the bright and dark field line scan camera detection system includes a frame, a conveying mechanism 1, a fixture assembly, a bright field line scan camera 2, a dark field line scan camera 3, a bright field light source 4, a dark field light source 5, a trigger controller, and an industrial control computer, etc.

[0074] The screen of the mobile phone or tablet under test is transported to the testing position via a conveyor mechanism or fixed in a designated position by a vacuum adsorption stage. The bright-field line scan camera and the coaxial or near-coaxial strip / area array bright-field light source are mounted in the normal direction of the screen, so that the light shines on the surface under test approximately perpendicularly; the dark-field line scan camera and the ring, strip or multi-directional combination dark-field light source are arranged at an inclined angle, so that the incident light produces a larger incident angle on the screen surface, thereby enhancing the response of the surface's small scattering defects.

[0075] Specifically, the steps include the following:

[0076] 1. System Calibration and Installation: Based on the maximum size and resolution requirements of the product under test, select the pixel size and resolution of the line scan camera to determine the spatial resolution of a single pixel on the measured surface. Adjust the focal length and height of the bright-field line scan camera using the mounting bracket to ensure the bright-field field of view covers the entire screen width; use a synchronous belt or roller conveyor to move the screen at a constant speed within the camera's field of view. The dark-field line scan camera and the dark-field light source are appropriately offset relative to the bright-field optical path, typically selecting an incident angle of 15°–45°, to ensure that minor surface bumps, particles, and scratches produce noticeable bright / dark variations in the dark-field image. Perform geometric calibration on both the bright and dark-field line scan cameras using a calibration plate to obtain the intrinsic and relative extrinsic parameters of both cameras, providing a parameter basis for subsequent pixel-level alignment of bright and dark images and front / back coordinate mapping.

[0077] 2. Synchronous acquisition of bright and dark field images: The screen under test is fixed in a fixture, which is equipped with positioning reference pins or vacuum positioning holes to ensure the repeatability of the position of each screen in the detection coordinate system. The industrial control computer sends a start command to the line scan camera and the light source controller. The encoder or fixed-length pulse controls the conveying mechanism to make the screen pass through the camera's field of view at a uniform speed. When the screen enters the detection area, the trigger controller simultaneously outputs a synchronous trigger signal to the bright field line scan camera and the bright field light source. Bright field line scan data is acquired at the set line frequency. Continuous line data is buffered and stitched to obtain a complete bright field screen image. Then, the system switches to dark field acquisition mode, turns off the bright field light source, turns on the dark field light source, and acquires dark field line scan data at the same line frequency and displacement step distance with the same spatial sampling interval. The data is then buffered and stitched to obtain a complete dark field screen image. Using the previously calibrated parameters, the bright and dark images are geometrically corrected and registered to ensure that the bright field image and the dark field image have a pixel-level correspondence in the screen coordinate system.

[0078] Example 2: Geometric feature recognition and adaptive partitioning of the mobile phone screen detection area, referring to... Figure 3 As shown, this embodiment corresponds to step S2. This embodiment uses a mobile phone screen with a rounded corner frame structure as an example to illustrate the detection area partitioning process.

[0079] 1. Screen contour extraction and geometric template matching: In the bright field image, the screen and background are first coarsely segmented using a global or adaptive threshold. Connected component analysis is then used to extract the connected component with the largest area as the screen region. Subpixel-level edge detection and fitting are performed on the edges of the screen region. Hough transform can be used to identify straight edges and rounded corner points, obtaining the four straight edges and four rounded corner segments of the screen's outer contour. These contour parameters are then matched with a preset product geometric template to determine the theoretical positions of the screen's length, width, and rounded corner radius, providing prior information for region division.

[0080] 2. Sub-region division: Based on the contour fitting results, the detection area is divided into a central display area and four surrounding border areas. The border areas are further subdivided into top border, bottom border, left border, right border, and four rounded corner areas. Each sub-region is assigned a unique region identifier, and a region parameter structure is established. The structure includes at least: region type (display area / border / rounded corners), region coordinate range, initial values ​​of multi-scale filtering parameters, local threshold range, defect size and shape priors, etc.

[0081] 3. Regional statistical characteristic analysis: Within each sub-region, the mean, standard deviation, and histogram characteristics of the background grayscale distribution are calculated to obtain the brightness baseline and noise level of that region. Differences in regional reflectance characteristics are statistically analyzed for both bright and dark areas. For example, the border ink area typically has lower grayscale and smoother texture, resulting in a more uniform grayscale distribution in the display area, but may exhibit background brightness gradients. These statistical indicators are then written into the corresponding regional parameter set to provide a basis for setting subsequent local thresholds, adaptive enhancement, and defect size thresholds.

[0082] For example, taking a 6.7-inch rounded corner mobile phone screen as an example, the adaptive partitioning of the detection area is achieved as follows:

[0083] Screen contour extraction: OTSU adaptive thresholding is used to segment the screen and background in the bright field image, and the screen region is extracted by connected component analysis; Canny subpixel edge detection is performed on the screen edges, and Hough transform is used to fit 4 straight edges and 4 rounded arc segments with a rounded radius of 3mm.

[0084] Geometric template matching: The extracted screen outline parameters (length 160mm, width 75mm, corner radius 3mm) are matched with the preset 6.7-inch mobile phone screen geometric template, with a matching degree of 99.5%, to determine the theoretical position of the screen outline.

[0085] Sub-region division: The detection area is divided into a central display area (150mm×70mm), upper / lower / left / right border areas (width 2.5mm), and four rounded corner areas (radius 3mm). Each sub-region is assigned a unique number (e.g., the central area is A1, the upper border is B1, and the upper left rounded corner is C1).

[0086] Establishment of region parameter set: Create a parameter structure for each sub-region, recording the region type, coordinate range, and initial values ​​of filtering parameters (e.g., Gaussian filter σ=0.5 for the center region, σ=0.3 for the border region); calculate the mean background grayscale value (180 for the center region, 50 for the border region, and 45 for the rounded corner region) and standard deviation for each sub-region, statistically analyze the differences in reflectance characteristics between bright and dark fields (the mean grayscale value of the dark field in the border region is 80 higher than that of the bright field), and write the above parameters into the region parameter set.

[0087] Example 3: Adaptive parameter adjustment for the front and back of the mobile phone screen. This example corresponds to step S3, realizing dual-layer adaptive adjustment of the detection parameters for the front and back of the mobile phone screen. The specific process is as follows:

[0088] For obtaining front and back identification, each screen is assigned a barcode or QR code at the upstream workstation. The barcode information includes whether the current workstation is for "front inspection" or "back inspection." At this inspection workstation, the front and back identification is obtained by reading the barcode content using an industrial camera. When the production line cycle time is fast and the clamping direction is fixed, the front / back configuration of the workstation can also be directly pre-configured in the control system based on fixture installation posture information or process flow agreements. If there are abnormalities such as fixture misplacement, a simple image recognition strategy (e.g., if there is a metal layer or ribbon cable area on the back) can be used to classify and verify the image texture features to prevent incorrect identification.

[0089] The preprocessing parameters for both the front and back sides are adaptive. Parameter templates are pre-established for the front and back sides of the screen in the industrial control computer. The templates include brightness normalization gain, contrast stretching range, noise filtering kernel size, edge enhancement convolution kernel, and enhancement intensity, etc.

[0090] At the start of the detection, based on the read front and back markings, the corresponding parameter templates are automatically loaded, and the bright field and dark field images are preprocessed separately:

[0091] Brightness normalization: Normalizes the overall brightness difference caused by fluctuations in screens or light sources from different batches, so that the average grayscale of the same type of area falls within the preset range.

[0092] Noise suppression: While ensuring that the defect edges are not overly smoothed, the image is subjected to median filtering or bilateral filtering. The size of the filter kernel is automatically selected based on the preset value in the front and back templates.

[0093] Edge enhancement: For areas sensitive to fine line defects such as scratches and cracks, the Laplacian or Sobel operator is used for edge enhancement. The enhancement coefficient is set with different values ​​on the front and back sides to match the different reflection and texture characteristics on both sides.

[0094] Sub-region-based local parameter optimization:

[0095] (1) After loading the global template parameters for both the front and back sides, the local threshold and defect size threshold for each sub-region are further modified according to its grayscale statistical characteristics. For example, the lower limit threshold of the area is appropriately increased in the ink border area to avoid misjudging small gaps at the ink edge as defects. (2) The enhancement coefficient of the dark field response is appropriately adjusted for highly reflective areas (such as exposed glass areas) to make them more sensitive to fine particle defects, while suppressing large-area uniform reflection. (3) The finally determined regional preprocessing parameters are stored in the regional parameter set and called in subsequent multi-scale defect extraction and connected component analysis to achieve two-level adaptation of the front and back sides and regions.

[0096] More specifically, a QR code containing a front inspection mark is printed on the protective film of the mobile phone screen. The QR code information is read by an industrial barcode scanner to determine that the current inspection surface is the front. If the scan fails, the image texture is analyzed by image recognition methods. If there is no ribbon line area on the front and a black ribbon line area on the back, the inspection surface verification is completed.

[0097] Global parameter adaptive adjustment: Load the global parameter template of the front of the screen, set the brightness normalization coefficient to 1.2, the contrast enhancement range to [50, 200], the noise filtering kernel to 3×3 median filtering, and the edge enhancement coefficient to 0.8; perform brightness normalization on the bright and dark field images, adjust the background grayscale mean to 180, and then perform median filtering and Sobel operator edge enhancement.

[0098] Local parameter adjustments: In the central display area, the local threshold is set to the mean + 2 standard deviations, and the minimum size of the connected component is 5 pixels; in the border area, the local threshold is set to the mean + 3 standard deviations, and the minimum size of the connected component is 8 pixels to avoid interference from ink noise; in the rounded corner area, the defect response enhancement coefficient is increased to 1.0 to improve the detection sensitivity of rounded corner scratches.

[0099] Example 4: Multi-scale Defect Extraction of Mobile Phone Screens. This example corresponds to step S4, implementing multi-scale defect extraction in each sub-region, including the construction of a multi-scale image pyramid. For bright-field and dark-field images, each sub-region is cropped to avoid invalid background from participating in the calculation. A Gaussian pyramid or Laplacian pyramid method is used to construct at least 3-5 layers of multi-scale images: the top layer corresponds to the original resolution, and each layer below is scaled according to a preset ratio. High-pass or band-pass filter responses are calculated on each scale image to highlight small and medium-sized defects, for example, by obtaining a detail residual image by subtracting the smoothed result from the original image.

[0100] To enhance directionality and structural features, multi-directional filtering based on structural tensors or Gabor filter banks is employed for areas prone to directional scratches, such as screen edges or areas with consistent transmission directions. Several typical directions (e.g., 0°, 45°, 90°, 135°) are selected within the 0°–180° range, and directional responses are calculated separately. The multi-directional responses are then maximized or weighted to enhance the saliency of fine linear defects in multi-scale maps. For near-point defects such as particles and bright spots, Laplacian operators or Gaussian difference filtering can be used to enhance point structures with specific scales.

[0101] Multi-scale fusion normalizes the filtered response map at each scale, ensuring its grayscale values ​​fall within a uniform range and eliminating differences in response amplitude between scales. Through weighted summation, maximum value fusion, or confidence-based fusion strategies, the response maps from each scale are combined into a single multi-scale defect response map. Weights can be pre-set based on the importance of each scale to the target defect type. Multi-scale defect response maps are obtained separately for bright-field and dark-field applications, resulting in a bright-field multi-scale response map and a dark-field multi-scale response map.

[0102] Defect candidate regions are extracted using adaptive threshold segmentation on multi-scale response maps. The threshold can be automatically determined based on the bimodal distribution of the response histogram within the region or on a statistical model (such as mean ± multiples of standard deviation). Connectivity analysis is performed on the segmentation results to extract all connected regions as defect candidate regions, and their area, bounding rectangle, minimum bounding ellipse, and other geometric information are calculated. Bright-field and dark-field candidate region sets are saved separately, with region information including attributes such as source (bright / dark), sub-region, scale layer number, and preliminary response intensity.

[0103] For example, grayscale normalization and background flattening correction are performed on bright and dark field images to eliminate brightness gradients in the screen background; images are cropped by sub-regions, retaining only the effective detection areas. A four-layer Gaussian pyramid is constructed for each sub-region cropped image, with σ values ​​of 0.5, 1.0, 2.0, and 4.0, respectively, to achieve layered detection of defects ranging from 0.005 mm² to 1 mm². High-pass filtering is applied to images at each scale to highlight defect details; Gabor filter banks are used for multi-directional filtering at 0°, 45°, 90°, and 135° in the border area, and the maximum values ​​of the directional responses are fused to enhance the saliency of linear scratches; Gaussian difference filtering is used in the central display area to enhance point defects such as bright spots and dark spots. The filtered response maps at each scale were normalized to [0, 255], and multi-scale fusion was achieved by weighted summation. The adaptive threshold was determined by the histogram bimodal distribution method, and threshold segmentation was performed on the fused defect response map. Defect candidate regions were extracted by connected component analysis. A total of 23 bright field candidate regions and 18 dark field candidate regions were extracted, and information such as the sub-region and scale layer number of each candidate region was labeled.

[0104] Example 5: Multi-dimensional screening of candidate defect regions for mobile phone screens. This example corresponds to step S5, which involves multi-dimensional screening of the extracted candidate defect regions and multi-dimensional analysis and screening of the candidate defect regions.

[0105] 1. Morphological Feature Calculation: For each candidate connected region of a defect, calculate basic morphological features: area, perimeter, principal and secondary axis lengths, aspect ratio, roundness, rectangularity, and minimum circumscribed rectangle orientation angle. Compare these morphological features with the corresponding prior defect type. For example, linear scratches should have a large aspect ratio, particles should have high roundness, and large-area uneven coatings should have a large area and smooth boundaries. Candidate regions that clearly do not conform to any defect morphological features are directly eliminated according to preset rules.

[0106] 2. Gray-scale and gradient feature calculation: In the original bright-field and dark-field images, the average gray-scale, maximum gray-scale, minimum gray-scale, gray-scale variance, and gray-scale contrast within the candidate regions are calculated. The gradient magnitude distribution near the region boundaries is calculated to obtain the mean and maximum gradient values, and the boundary sharpness and gray-scale variation patterns are analyzed. For bright-field and dark-field candidate regions at the same spatial location, the difference or ratio of bright and dark-field gray-scale values ​​is calculated to determine whether the defect response is consistent under the two illumination modes, thereby distinguishing between genuine structural defects and reflection artifacts that only appear in a certain field of view.

[0107] 3. Post-processing of candidate regions: Morphological opening and closing operations are performed on the retained candidate regions to smooth the boundaries and eliminate isolated noise areas. Regions that are close to each other and have similar morphology and grayscale features are merged to form a complete defect region outline. Label information is added to each final candidate region, including its sub-region, preliminary defect type determination, feature vector, and response level in bright and dark fields.

[0108] The specific process is as follows: Morphological feature screening: Morphological features were extracted from 41 candidate defect regions. Among them, 12 candidate regions had an aspect ratio <1.5 and a roundness <0.2, which did not meet the morphological characteristics of linear scratches or dot-like particles, and were directly eliminated. Gray-scale feature screening: Gray-scale features were extracted from the remaining 29 candidate regions. Among them, 7 candidate regions appeared only in the bright field and the gray-scale difference between the bright and dark fields was >100, which were judged as reflection artifacts and eliminated. Texture feature screening: Gray-scale co-occurrence matrix features were extracted from the remaining candidate regions in the central display area. Among them, the texture uniformity of 2 candidate regions was consistent with the background, which were judged as background texture interference and eliminated. Post-processing: A 3×3 opening operation was performed on the 20 final candidate regions to eliminate isolated noise. Two adjacent scratch candidate regions were merged to form 19 complete defect candidate regions. The defect type was initially judged for each region (e.g., scratch, particle, bright spot).

[0109] Example 6: Registration and Collaborative Screening of Defects on the Front and Back of Mobile Phone Screens. This example corresponds to step S6, realizing spatial registration and collaborative screening of defects on the front and back of mobile phone screens. This includes establishing a front and back coordinate mapping model. Specifically, based on the mechanical structure of the testing equipment and the camera installation relationship, reference images of standard samples on both the front and back are acquired during the equipment debugging phase. Rotation, translation, mirror transformation, and scale transformation parameters between the front and back images are calculated using a calibration pattern to form a unified coordinate mapping matrix. This mapping matrix is ​​then fixed in the system. During the testing process, the coordinates of all back defects are transformed to a unified reference coordinate system (e.g., based on the coordinate system of the front image), or both the front and back are transformed to the product design coordinate system.

[0110] For the registration and matching of candidate defect regions on the front and back sides, for each candidate defect region detected on the front side, its centroid coordinates and contour point coordinates are transformed to a unified coordinate system using a known mapping matrix. Similarly, the candidate defect regions detected on the back side are transformed to a unified coordinate system. In the unified coordinate system, a spatial proximity search is performed on the candidate regions on the front and back sides. First, the centroid distance and the degree of overlap of the circumscribed rectangles are used as coarse matching conditions. Regions whose distance is within a preset threshold and have a certain degree of overlap are regarded as candidate matching pairs.

[0111] For the defect correlation evaluation, the following indicators are calculated for each pair of candidate matching regions:

[0112] Spatial overlap: characterized by the ratio of the intersection area to the union area of ​​two regions;

[0113] Differences or consistency in grayscale response between bright and dark fields: Compare the average grayscale change trend and response intensity of corresponding areas on the front and back sides in bright and dark fields;

[0114] Defect type prior consistency: Combining the sub-region to which the area belongs and the predicted defect type, determine whether the defect is reasonably located on the front or back.

[0115] After normalizing the above indicators, a weighted sum is performed according to preset weights to obtain the defect correlation evaluation function value. When the defect correlation is greater than the first threshold, the front and back candidate regions are determined to be different views or different lighting manifestations of the same real defect; when the correlation is between the first and second thresholds, the candidate pair is marked as uncertain for manual review or further algorithm processing; when the correlation is less than the second threshold, the front and back regions are considered unrelated.

[0116] The specific process is as follows: standard reference images of the front and back of the mobile phone screen are acquired, and a coordinate mapping matrix [T] is calculated through a calibration pattern. This matrix includes parameters for mirror transformation (horizontal mirror) and translation transformation (5mm translation along the X-axis). The mapping matrix is ​​then embedded in the detection system. The centroid coordinates of the 15 defect candidate regions detected on the back side are transformed to the coordinate system of the front image through the mapping matrix [T], thus achieving spatial registration of the defect candidate regions on both sides. Coarse matching was performed in a unified coordinate system to obtain 12 candidate defect matching pairs. For each matching pair, spatial overlap rate, shape similarity, grayscale response consistency, and prior consistency of defect type were calculated, with weights of 0.4, 0.2, 0.2, and 0.2 respectively. The correlation value was obtained by weighted summation. A first preset threshold of 0.8 and a second preset threshold of 0.5 were set. Among them, 10 matching pairs with a correlation > 0.8 were judged as real defects (including 6 scratches, 3 particles, and 1 bright spot). One matching pair with a correlation of 0.65 was marked as uncertain for manual review. One matching pair with a correlation < 0.5 and appearing only on the back was judged as an artifact and removed. The remaining 2 candidate areas on the front had no back matching pairs and were judged as reflective artifacts based on texture features and removed. This embodiment detected 10 real defects on the mobile phone screen. The defect location, size, and type were accurately identified with no missed or false detections. The detection accuracy and efficiency meet the needs of industrial production.

Claims

1. A method for zoned adaptive detection and multi-scale defect extraction based on a line scan camera with bright and dark fields of view, characterized in that, Includes the following steps: S1. Construct a detection system that includes a line scan camera, a bright field light source, and a dark field light source. Simultaneously acquire bright field and dark field images of the screen of the electronic product under test. After buffering and stitching the acquired line scan data, perform geometric correction and pixel-level registration to obtain bright and dark field screen image data with consistent spatial correspondence. S2. Based on the screen's geometric features and structural information, the detection area of ​​the screen to be tested is divided into multiple sub-regions with different geometric attributes, lighting characteristics, and defect type distribution characteristics. A region identifier and a set of region parameters are established for each sub-region. S3. Obtain the front and back markings of the screen under test, call the corresponding detection model or parameter template according to the front and back markings, adaptively adjust the global preprocessing parameters of the bright and dark field images, and then adaptively adjust the local detection parameters in each sub-region according to the regional reflectance characteristics and background grayscale statistical characteristics. S4. Preprocess the bright and dark field images respectively, construct multi-scale filtering and multi-directional feature extraction models in each sub-region, and extract candidate regions for bright and dark field defects after normalizing and fusing the response results at each scale. S5. Extract morphological features and grayscale features for each defect candidate region, remove candidate regions that do not meet the defect features according to preset rules, and then perform morphological operations on the remaining candidate regions to optimize the region boundaries. S6. Establish a geometric relationship model for the front and back images of the screen, uniformly map the candidate defect regions of the front and back to the same reference coordinate system to achieve spatial registration, construct a defect correlation evaluation function to complete the matching and association of front and back defect candidates, determine the real defects and remove artifact interference; the defect correlation evaluation function comprehensively considers the following indicators: spatial overlap rate of front and back defect regions, area ratio and shape similarity of front and back defect regions, difference in grayscale response of front and back light and dark fields, and prior knowledge of defect type; the spatial overlap rate is the ratio of the intersection area to the union area of ​​two defect regions; the judgment rules for matching and association of front and back defect candidates are as follows: when the defect correlation evaluation result is greater than the first preset threshold, the corresponding front and back candidate regions are regarded as different views or different lighting responses of the same real defect, and their feature information is integrated; when the defect correlation evaluation result is between the first preset threshold and the second preset threshold, the candidate region is marked as uncertain; when the defect correlation evaluation result is less than the second preset threshold, combined with local texture and grayscale statistical features, the candidate region that appears only on one side is judged as artifact interference or background structure features and is removed.

2. The method according to claim 1, characterized in that, The specific process of acquiring the bright and dark field images in step S1 is as follows: S11. The bright field line scan camera and the near-coaxial bright field light source are installed in the normal direction of the screen, while the dark field line scan camera and the dark field light source are arranged in the tilt direction of the screen with an incident angle of 15° to 45°. S12. Perform geometric calibration on the line scan camera for both bright and dark fields of view using a calibration plate to obtain the camera's intrinsic and relative extrinsic parameters. S13. Synchronously control the brightfield line scan camera and brightfield light source to collect brightfield line scan data of the screen under test, and then buffer and stitch the data to obtain a complete brightfield image. S14. Synchronously control the dark field line scan camera and dark field light source to acquire dark field line scan data with the same line frequency, displacement step size and spatial sampling interval, and then buffer and stitch the data to obtain a complete dark field image. S15. Use calibration parameters to perform geometric correction and registration on the bright and dark field images to ensure that they correspond at the pixel level in the screen coordinate system.

3. The method according to claim 1, characterized in that, The specific process of geometric feature recognition and adaptive partitioning of the detection region in step S2 is as follows: S21. Extract the screen region from the bright field image by threshold segmentation and connected component analysis, perform sub-pixel level edge detection and fitting on the edge of the screen region, and identify straight edges and rounded corners to obtain the outer contour of the screen. S22. Match the screen outer contour parameters with the preset product geometry template to determine the theoretical positions of the screen length, width, and corner radius; S23. Divide the detection area into a central display area, a border area, and a rounded corner area. The border area is further subdivided into upper, lower, left, and right border areas. S24. Assign a unique region identifier to each sub-region and establish a region parameter structure that includes region type, coordinate range, initial value of filter parameters, local threshold range, defect size and shape prior. S25. Calculate the mean, standard deviation and histogram characteristics of the background grayscale distribution in each sub-region, statistically analyze the differences in regional reflectance characteristics under bright and dark fields, and write the statistical indicators into the regional parameter set.

4. The method according to claim 1, characterized in that, In step S3, the front and back markings are obtained through barcode / QR code information, production line station information, mechanical fixture posture information, or image recognition methods; global preprocessing parameters include brightness normalization coefficient, contrast enhancement parameter, noise filtering kernel size, and edge enhancement coefficient; local detection parameters include local threshold and defect response enhancement parameter.

5. The method according to claim 1, characterized in that, Step S4 involves preprocessing the bright and dark field images, including grayscale normalization, noise suppression, and background flattening correction. The construction process of the multi-scale filtering and multi-directional feature extraction model is as follows: S41, after cropping the images of each sub-region, construct a multi-scale image of 3-5 layers of Gaussian pyramids or Laplacian pyramids; S42, perform high-pass filtering or band-pass filtering on the images at each scale to highlight defects of different sizes; S43, use multi-directional filtering based on structural tensors or Gabor filter banks to select typical directions in the range of 0° to 180° to calculate and fuse directional responses, thereby enhancing the saliency of linear defects; S44, use Laplacian operators or Gaussian difference filtering to enhance the structure of point defects.

6. The method according to claim 5, characterized in that, The process of normalizing and fusing the response results at each scale in step S4 is as follows: the gray values ​​of the filtered response map at each scale are normalized to a unified range, and the response maps at each scale are synthesized into a multi-scale defect response map by weighted summation, maximum value fusion, or confidence-based fusion strategy; the method for extracting bright field and dark field defect candidate regions is as follows: adaptive threshold segmentation based on histogram bimodal distribution or statistical model is used on the multi-scale defect response map, and all connected regions are extracted as defect candidate regions through connected component analysis.

7. The method according to claim 1, characterized in that, The morphological features in step S5 include area, perimeter, major and minor axis lengths, aspect ratio, roundness, rectangularity, orientation angle, and boundary roughness. Gray-scale features include average gray-scale, gray-scale extreme values, gray-scale contrast, gray-scale variance, mean and maximum gray-scale gradient, and gray-scale difference between bright and dark areas. Texture features include gray-level co-occurrence matrix features, local binary mode features, and wavelet energy features; morphological operations include dilation, erosion, opening, and closing operations, which realize the elimination of isolated noise points and the merging of adjacent defect regions of the same type.

8. The method according to claim 1, characterized in that, In step S6, the geometric relationship model of the front and back images of the screen includes rotation, translation, mirror transformation and scale transformation parameters. The parameters are obtained by collecting reference images of the front and back standard samples and combining them with the calibration pattern. The specific process of spatial registration is as follows: the centroid coordinates and contour point coordinates of the candidate defect areas on the front and back are uniformly transformed to the product design coordinate system or the front image coordinate system through the coordinate mapping matrix.

Images