Multilayer screen cover plate glass interlayer foreign matter detection device and detection method

Abstract

The invention discloses a multi-layer screen cover plate glass interlayer foreign matter detection device and detection method, and belongs to the field of image processing, and the detection method comprises the following steps: image acquisition; extracting a region of interest; feature enhancement; carrying out anisotropic Gaussian filtering; edge detection; foreign matter screening. According to the method, the ROI image is enhanced by applying frequency tuning saliency detection FT and Haar wavelet transform, and an attention mechanism is introduced to fuse features, so that the saliency of tiny and low-contrast defects is enhanced; the influence of noise is reduced by adopting anisotropic Gaussian filtering, and important edge information is reserved and interference is suppressed by adaptively adjusting parameters of a filter.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a device and method for detecting foreign matter in interlayers of multi-layer screen cover glass. Background Art

[0002] Electronic screens are widely used in various electronic devices and display technologies. They can display images, videos and texts, and provide users with information display, interaction and visualization functions. At present, electronic screen manufacturers use advanced full bonding technology. During the display module assembly process, optically clear adhesive (OCA, Optically Clear Adhesive) is used to bond the cover glass and liquid crystal display module (Liquid Crystal Display Module, LCDM) together completely and seamlessly in a vacuum environment.

[0003] By bonding the cover glass and the LCD screen together, the LCD screen can be protected. During the bonding operation, due to imperfect control and factors such as the OCA material, the OCA layer in the cover glass (CG) component may have stains, bubbles, debris or other foreign matter, which are defined as process foreign matter; in addition, the CG component will also contain foreign matter, but as the OCA heavy release protective film and the CG protective film are torn off during the bonding process, the foreign matter will be removed, and the process foreign matter contained in the OCA layer will not only cause uneven brightness or visual distortion of the electronic screen, but may also cause touch failure or poor tactile feedback. If abnormal CG components with process foreign matter cannot be detected and eliminated in time during the display module bonding operation, the yield rate of the display module assembly line will be reduced and production costs will increase.

[0004] Currently, more than 80% of screen manufacturers still rely on manual visual inspection, and the number of quality inspection workers on the product line exceeds 3.5 million. Quality inspectors mainly observe foreign matter at different thickness levels of CG components through microscopes, and detect and determine particles with an area larger than a certain size (such as 60×60μm2 or 100×100 μm2) as foreign matter based on experience or optical calipers. This method is easily affected by factors such as the environment and human eye fatigue, and has problems such as strong subjectivity and low efficiency.

[0005] Existing screen defect detection methods can be roughly divided into traditional image processing methods and deep learning-based methods. Traditional image processing methods include Fourier transform, filtering technology, and regression diagnosis. Traditional methods are sensitive to noise and light. Although deep learning methods have efficient feature extraction and defect recognition capabilities, they still face problems such as large data volume requirements, long training time, and strong dependence on data annotation.

[0006] Most current research and technical applications focus on detecting foreign matter on the surface of the screen or CG components. Machine vision methods for detecting foreign matter inside CG components are relatively rare. Currently, there are several methods for detecting foreign matter inside CG components: 1. OCA film bubble detection method based on machine vision. This method obtains the grayscale change of the image when the OCA film is attached, combines the irregular shape of the bubble to detect the bubble edge pixel points, and then accurately extracts the bubble area based on the different characteristics of the reflective area and the bubble area, eliminating the interference of the reflective area, and improving the accuracy and reliability of the final bubble detection. However, this method can only be used to detect OCA film bubbles and cannot detect other foreign matter in the OCA film.

[0007] 2. CG component foreign body detection method based on machine vision. This method obtains the OCA interlayer foreign body area image by subtracting the CG protective film foreign body area from each layer of foreign body area in the CG component, as well as the foreign body areas on the CG protective film surface, cover glass interlayer and the lower surface of the heavy film, and then performs detection. This method improves the accuracy of foreign body layer detection of the cover glass module, but does not optimize the weak contrast foreign bodies in the OCA interlayer. Summary of the invention

[0008] In order to solve the above technical problems, the present invention provides a multi-layer screen cover glass interlayer foreign body detection device with simple structure and low cost, and provides a multi-layer screen cover glass interlayer foreign body detection method with easy operation and high detection accuracy.

[0009] The technical solution of the present invention for solving the above technical problems is: a multi-layer screen cover glass interlayer foreign body detection device, including a supporting platform, the supporting platform is fixedly installed in the middle of the support, the CG sample is placed in the middle of the supporting platform, two upper cameras with lenses facing downwards are symmetrically arranged on the left and right sides above the supporting platform, a first telecentric lens is arranged at the lens end of each upper camera, and the two upper cameras are installed on the upper part of the support through a three-axis motion device, two lower cameras with lenses facing upwards are symmetrically arranged on the left and right sides below the supporting platform, a second telecentric lens is arranged at the lens end of the two lower cameras, the two lower cameras are installed at the lower part of the support, a coaxial light source is arranged between the two first telecentric lenses and the supporting platform, and between the two second telecentric lenses and the supporting platform, and the four coaxial light sources are installed in the middle of the support; the upper camera, the first telecentric lens, the coaxial light source, the second telecentric lens, and the lower camera on the same side are located on the same vertical plane.

[0010] The above-mentioned multi-layer screen cover glass interlayer foreign matter detection device, the CG sample includes a CG protective film, a CG layer, an OCA layer, and an OCA heavy-release protective film arranged in sequence from top to bottom, and the interlayer foreign matter includes foreign foreign matter and process foreign matter, wherein the foreign foreign matter is located on the outside of the CG protective film and the outside of the OCA heavy-release protective film, and the process foreign matter is located between the CG layer and the OCA layer, and between the OCA layer and the OCA heavy-release protective film.

[0011] The above-mentioned multi-layer screen cover glass interlayer foreign body detection device, the upper camera and the lower camera both use the CH310-10TM-M58S-NN area array CMOS camera, the first telecentric lens and the second telecentric lens both use the large target surface double telecentric lens of DTCM175-170-M58-A.

[0012] A method for detecting foreign matter in a multi-layer screen cover glass interlayer comprises the following steps: Step S1, image acquisition: obtaining a multi-mode image of a CG sample through a multi-layer screen cover glass interlayer foreign body detection device and selecting a main image and a slave image; Step S2, region of interest extraction: using a mask to extract the region of interest of the main image, i.e., the ROI image; Step S3, feature enhancement: frequency-tuned saliency detection (FT) and Haar wavelet transform are applied to enhance the ROI image, and an attention mechanism is introduced to fuse the features; Step S4, anisotropic Gaussian filtering: adaptively adjusting the filter parameters to filter the enhanced ROI image to reduce noise interference in different directions and detect and fuse foreign body textures at different scales; Step S5, edge detection: using the Canny detection algorithm to locate the edge of the foreign body in the filtered ROI image; Step S6, screening of foreign bodies: screening of interlayer foreign bodies in ROI images is achieved by using the image, a method combining fast background reconstruction and a support vector machine based on average stochastic gradient descent.

[0013] The above-mentioned method for detecting foreign matter in the interlayer of the multi-layer screen cover glass, the specific steps of step S1 are: Step S11: four light source illumination imaging modes are obtained by combining the upper camera and the lower camera to collect four images of the upper light source UU of the upper camera, the lower light source UD of the upper camera, the upper light source DU of the lower camera and the lower light source DD of the lower camera, and red light is used to capture the defects and scratches in the CG layer to obtain a clear image of the interlayer foreign matter; Step S12: The UD image is positioned as the main image, the UU image and the DD image are defined as the slave images, and the DU image is discarded. Subsequently, the main image and the slave images are coordinated to realize the hierarchical screening of interlayer foreign bodies.

[0014] In the above-mentioned method for detecting foreign matter in the interlayer of the multi-layer screen cover glass, the specific steps of step S2 are as follows: Step S21: Set the threshold to convert the main image into a grayscale image: ; Where: I inv Represents the pixel value of the grayscale image, x and y represent the horizontal and vertical coordinates of the pixel point respectively. Indicates that the coordinates in the main image are The pixel value of the pixel point; Step S22: Select the contour max_contour with the largest area in the grayscale image, and obtain the mask mask as follows: ; Where: Indicates that the coordinates in the mask are The pixel value of the pixel point; Step S23: Apply the mask to the grayscale image to obtain the region of interest of the main image, i.e., the ROI image: ; Where: Indicates that the coordinates in the ROI image are The pixel value of the pixel point, Indicates that the coordinates in the grayscale image are The pixel value of the pixel.

[0015] In the above-mentioned method for detecting foreign matter in the interlayer of the multi-layer screen cover glass, the specific steps of step S3 are as follows: Step S31: Perform Haar wavelet decomposition on the ROI image and select the low-frequency component I LL Enhance defect edges; ; Where: I represents the pixel grayscale value of the main image, i=0, 1; j=0, 1; Step S32: enhancing the ROI image using a frequency-tuned saliency detection algorithm; First, convert the main image to the LAB color space and calculate the mean I of the L channel mean ; Then calculate I and I of L channel mean The absolute value of the difference between ; Finally, the saliency map is put into the normalization formula for normalization; ; Where: , Represent the minimum and maximum pixel values ​​in the saliency map, respectively. represents the normalized saliency map; Step S33: Use the feature fusion formula combined with the attention mechanism to select and enhance features of different regions; ; Where: represents the Haar wavelet low-frequency image, and F represents the image after feature fusion.

[0016] In the above-mentioned method for detecting foreign matter in the interlayer of the multi-layer screen cover glass, the specific steps of step S4 are as follows: Step S41: using anisotropic Gaussian filtering to reduce noise, and using the following formula to perform Gaussian filtering on the image after feature fusion obtained in step S33 in the X-axis and Y-axis directions; ; Where: and are the standard deviations in the X-axis and Y-axis directions respectively; is the value of the pixel after Gaussian filtering; Step S42: Use a multi-scale anisotropic Gaussian filter to filter the enhanced ROI image to highlight the foreign body texture of the enhanced ROI image: ; Where: represents the anisotropic Gaussian kernel, Represents the convolution operation; Represents the pixel value of the corresponding pixel after filtering.

[0017] In the above-mentioned method for detecting foreign matter in the interlayer of the multi-layer screen cover glass, the specific steps of step S5 are as follows: Step S51: applying the Canny edge detection algorithm to the filtered ROI image to perform edge extraction and fusion; First, Gaussian smoothing is used to reduce the influence of noise, then the edge is detected by calculating the gradient of the filtered ROI image, then non-maximum suppression is used to remove false edges, and local maxima are retained as edge lines. Finally, double edge detection is used to refine and screen the edges. Step S52: using a two-step scanning method to mark all connected regions in the filtered ROI image; The first step is scanning: traverse every pixel in the image from top to bottom and from left to right, assign a digital label to each non-zero pixel. The digital label of a 0 pixel is 0 by default. From the traversal order, the neighboring pixels above and to the left of the current pixel have all been assigned digital labels. When the label of the current pixel is a non-zero number, there are four cases: The first case: if the neighboring pixels above and to the left of the current pixel are both zero, then a new digital label is assigned to the current pixel and recorded. The second case: if the neighboring pixels above the current pixel are zero and the neighboring pixels on the left are not zero, the digital label assigned to the current pixel is consistent with the digital label of the neighboring pixels on the left; The third case: if the neighboring pixel above the current pixel is not zero and the neighboring pixel on the left is zero, the digital label assigned to the current pixel is consistent with the digital label of the neighboring pixel above; The fourth case: if both the upper neighboring pixel and the left neighboring pixel of the current pixel are not zero, the digital label assigned to the current pixel is the minimum value of the digital labels of the left neighboring pixel and the digital labels of the upper neighboring pixel; The second step of scanning: After the first step of scanning is completed, a pixel point may be assigned one or more digital labels. In the second step of scanning, the pixels with the same digital label are recorded as a connected domain, and the pixels with multiple digital labels and the pixels with the same digital label are merged to obtain a unified digital label. This unified digital label is the minimum value of the multiple digital labels. Finally, the digital labels of all pixels in the same connected domain are consistent. The area marked by all non-zero pixels is all the connected domains in the filtered ROI image, and all the connected domains are the areas where the foreign matter is located.

[0018] In the above-mentioned method for detecting foreign matter in the interlayer of the multi-layer screen cover glass, the specific steps of step S6 are as follows: Step S61: After obtaining the area where the foreign object is located in the ROI image in step S52, the UU sub-image and the DD sub-image containing the foreign object are located based on the difference in grayscale values ​​between the background area and the foreign object in the image; Step S62: Obtain the masks of the UU sub-image and the DD sub-image, and perform a dilation operation on the foreign matter parts in the masks of the UU sub-image and the DD sub-image; Step S63: performing a logical OR operation on the UU sub-image and the UU image, and the DD sub-image and the DD image after the dilation operation, and calculating the pixel mean M of the main image and the slave image; Step S64: setting all pixel values ​​in the UU sub-image and the DD sub-image to M, and reconstructing the background grayscale value and the grayscale value of the foreign object part in the UU sub-image and the DD sub-image; Step S65: Compare the relationship between the grayscale value of the foreign matter and the grayscale value of the background of the foreign matter part in the UU sub-image and the DD sub-image, where the grayscale value of the foreign matter is smaller than the background grayscale value. The comparison rules are as follows: If the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image and the DD sub-image is greater than 10 pixels, the foreign matter is a CG foreign matter, and the CG foreign matter belongs to the process foreign matter; if the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image is less than or equal to 10 pixels, and the difference between the grayscale value of the foreign matter and the background grayscale value in the DD sub-image is greater than 10 pixels, The foreign matter is an OCA foreign matter, which is a process foreign matter. If the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image is greater than 10 pixels, and the difference between the grayscale value of the foreign matter and the background grayscale value in the DD sub-image is less than or equal to 10 pixels, the foreign matter is a foreign matter on the membrane, which is an external foreign matter. If the difference between the grayscale value of the foreign matter and the background grayscale value in both the UU sub-image and the DD sub-image is less than or equal to 10 pixels, it is determined that there is no foreign matter. The support vector machine based on average stochastic gradient descent is used to realize the screening of interlayer foreign matters.

[0019] The beneficial effects of the present invention are: 1. When the present invention obtains multi-mode images of CG samples, four light source illumination imaging modes are obtained by combining the upper camera and the lower camera to illuminate, thereby collecting four images of the upper light source UU of the upper camera, the lower light source UD of the upper camera, the upper light source DU of the lower camera and the lower light source DD of the lower camera, which can realize high-precision real-time image acquisition and foreign body spatial alignment calibration of CG samples under different lighting conditions in different spaces.

[0020] 2. The present invention extracts the region of interest through masking, which can reduce the amount of calculation.

[0021] 3. The present invention uses frequency-tuned saliency detection FT and Haar wavelet transform to enhance ROI images, and introduces an attention mechanism to fuse features, thereby enhancing the saliency of tiny, low-contrast defects.

[0022] 4. The present invention adopts anisotropic Gaussian filtering to reduce the influence of noise, and adaptively adjusts the parameters of the filter to retain important edge information and suppress interference.

[0023] 5. The present invention uses the Canny detection algorithm to accurately locate defect edges, ensuring edge integrity while reducing the generation of false edges. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic structural diagram of the multi-layer screen cover glass interlayer foreign body detection device of the present invention.

[0025] Figure 2 Schematic diagram of the CG sample structure.

[0026] Figure 3The present invention is a flow chart of a method for detecting foreign matter in the interlayer of a multi-layer screen cover glass.

[0027] Figure 4 Schematic diagram of multimodal image acquisition.

[0028] Figure 5 Schematic diagram of the ROI extraction results.

[0029] Figure 6 Schematic diagram of the effect of implementing frequency tuning-based significance enhancement.

[0030] Figure 7 Schematic diagram of the effect of implementing multi-scale anisotropic Gaussian filtering.

[0031] Figure 8 Schematic diagram of the effect of implementing Canny edge detection.

[0032] Fig. 9 Schematic diagram of the effect of using the two-step scanning method on the image after edge detection.

[0033] Fig.10 Schematic diagram of the principle of fast background reconstruction.

[0034] Fig.11 Classify the schematic diagram of foreign objects. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] like Figure 1 As shown, a multi-layer screen cover glass interlayer foreign body detection device includes a support platform 1, the support platform 1 is fixedly installed in the middle of the support, the CG sample 2 is placed in the middle of the support platform 1, two upper cameras 3 with lenses facing downward are symmetrically arranged on the left and right sides above the support platform 1, a first telecentric lens 4 is arranged at the end of each upper camera 3, and the two upper cameras 3 are installed on the upper part of the support through a three-axis motion device 5, two lower cameras 6 with lenses facing upward are symmetrically arranged on the left and right sides below the support platform 1, a second telecentric lens 7 is arranged at the end of the two lower cameras 6, and the two lower cameras 6 are installed at the lower part of the support, and a coaxial light source 8 is arranged between the two first telecentric lenses 4 and the support platform 1, and between the two second telecentric lenses 7 and the support platform 1, and the four coaxial light sources 8 are installed in the middle of the support. The upper camera 3, the first telecentric lens 4, the coaxial light source 8, the second telecentric lens 7, and the lower camera 6 on the same side are located on the same vertical plane.

[0037] The upper camera 3 and the lower camera 6 both adopt the area array CMOS camera of model CH310-10TM-M58S-NN, and the first telecentric lens 4 and the second telecentric lens 7 both adopt the large target surface double telecentric lens of model DTCM175-170-M58-A.

[0038] like Figure 2 As shown, the CG sample 2 includes a CG protective film 21, a CG layer 22, an OCA layer 23, and an OCA heavy release protective film 24 arranged in sequence from top to bottom, and the interlayer foreign matter includes foreign matter 25 and process foreign matter 26, wherein the foreign matter 25 is located outside the CG protective film 21 and outside the OCA heavy release protective film 24, and the process foreign matter 26 is located between the CG layer 22 and the OCA layer 23, and between the OCA layer 23 and the OCA heavy release protective film 24. During the bonding process, as the CG protective film 21 and the OCA heavy release protective film 24 are torn off, the foreign matter 25 will be removed. The detection object of the present invention is the process foreign matter 26 in the OCA layer 23, that is, the OCA interlayer foreign matter.

[0039] The multi-layer screen cover glass interlayer foreign body detection device can be divided into two parts according to its function: alignment calibration and alignment imaging. Alignment calibration is achieved through a three-axis motion device 5, which can control the upper camera 3 to achieve 100 um precision movement in the X-axis and Y-axis directions and 10 um precision center rotation, to achieve spatial alignment with the lower camera 6, and ensure the consistency of the camera's spatial position for image acquisition. Alignment imaging uses a total of 4 array CMOS cameras (CH310-10TM-M58S-NN, 3100W resolution) for the upper camera 3 and the lower camera 6, and is equipped with 4 large-target dual telecentric lenses (DTCM175-170-M58-A, with a maximum image distortion of less than 0.1%) and a coaxial light source 8, which can obtain four different light sources.

[0040] like Figure 3 As shown, a method for detecting foreign matter in the interlayer of a multi-layer screen cover glass comprises the following steps: Step S1, image acquisition: obtain a multi-mode image of the CG sample through a multi-layer screen cover glass interlayer foreign matter detection device and select a main image and a slave image.

[0041] The specific steps of step S1 are: Step S11: four light source illumination imaging modes are obtained by combining the upper camera and the lower camera, thereby collecting four images of the upper light source UU of the upper camera, the lower light source UD of the upper camera, the upper light source DU of the lower camera and the lower light source DD of the lower camera, as shown in FIG. Figure 4 As shown; and more penetrating red light (wavelength = 700nm) is used to capture defects and scratches in the CG layer to obtain clear imaging of interlayer foreign matter.

[0042] Step S12: The image characteristics of DU image and UD image are similar, and the grayscale distribution curve of UD image is not as clear and definite, so the subsequent processing flow does not consider using DU image. Among them, the contrast of foreign matter in UD image is higher than that of background, so UD image is positioned as the main image, UU image and DD image are defined as slave images, and DU image is discarded. Subsequently, the main image and slave image are used to cooperate with each other to realize the hierarchical screening of interlayer foreign matter.

[0043] Step S2, region of interest extraction: use the mask to extract the region of interest of the main image, that is, the ROI image.

[0044] The specific steps of step S2 are: Step S21: Set the threshold to convert the main image into a grayscale image: ; Where: I inv Represents the pixel value of the grayscale image, x and y represent the horizontal and vertical coordinates of the pixel point respectively. Indicates that the coordinates in the main image are The pixel value of the pixel.

[0045] Step S22: Select the contour max_contour with the largest area in the grayscale image, and obtain the mask mask as follows: ; Where: Indicates that the coordinates in the mask are The pixel value of the pixel.

[0046] Step S23: Apply the mask to the grayscale image to obtain the region of interest of the main image, i.e., the ROI image: ; Where: Indicates that the coordinates in the ROI image are The pixel value of the pixel point, Indicates that the coordinates in the grayscale image are The pixel value of the pixel.

[0047] like Figure 5 As shown, the middle gray part is the ROI image area obtained after applying the mask to the grayscale image, and the remaining black parts are non-ROI image areas.

[0048] Step S3, feature enhancement: frequency-tuned saliency detection (FT) and Haar wavelet transform are applied to enhance the ROI image, and an attention mechanism is introduced to fuse the features.

[0049] The specific steps of step S3 are: Step S31: Perform Haar wavelet decomposition on the ROI image and select the low-frequency component I LL Enhance defect edges; ; Where: I represents the pixel grayscale value of the main image, i=0, 1; j=0, 1.

[0050] Step S32: enhancing the ROI image using a frequency-tuned saliency detection algorithm; First, convert the main image to the LAB color space and calculate the mean I of the L channel mean ; Then calculate I and I of L channel mean The absolute value of the difference between ; Finally, the saliency map is put into the normalization formula for normalization; ; Where: , Represent the minimum and maximum pixel values ​​in the saliency map, respectively. represents the normalized saliency map.

[0051] Step S33: Use the feature fusion formula combined with the attention mechanism to select and enhance features of different regions; ; Where: Represents the Haar wavelet low-frequency map.

[0052] After step S3 is completed, the image processing result is as follows Figure 6 As shown, Figure 6 The middle left image shows the defective part of the main image. Figure 6 The middle right picture shows the defective part image after processing; image processing effectively improves the overall brightness and contrast of the image, making the details more prominent and clear, and it performs more balanced in detail recovery and edge enhancement.

[0053] Step S4, anisotropic Gaussian filtering: adaptively adjust the filter parameters to filter the enhanced ROI image to reduce noise interference in different directions and detect and fuse foreign body textures at different scales.

[0054] The specific steps of step S4 are: Step S41: Use anisotropic Gaussian filtering to reduce noise, and use the following formula to expand the Gaussian filter in the X-axis and Y-axis directions: ; Where: and are the standard deviations in the X-axis and Y-axis directions respectively; is the value of the pixel after Gaussian filtering.

[0055] Step S42: Use a multi-scale anisotropic Gaussian filter to filter the enhanced ROI image to highlight the foreign body texture of the enhanced ROI image: ; Where: represents the anisotropic Gaussian kernel, Represents the convolution operation; Represents the pixel value of the corresponding pixel after filtering.

[0056] The image after Gaussian filtering is as follows Figure 7 As shown, Figure 7 The upper left image is the image before processing, and the rest of the images are the images with different settings. The image obtained after Gaussian filtering.

[0057] Step S5, edge detection: using the Canny detection algorithm to locate the edge of the foreign body in the filtered ROI image.

[0058] The specific steps of step S5 are: Step S51: applying the Canny edge detection algorithm to the filtered ROI image to perform edge extraction and fusion; First, Gaussian smoothing is used to reduce the influence of noise, then the edge is detected by calculating the gradient of the filtered ROI image, then non-maximum suppression is used to remove false edges, and the local maximum is retained as the edge line. Finally, the edge is refined and screened by double edge detection. After applying the Canny edge detection algorithm to the Gaussian filter results of each scale, the results are as follows: Figure 8 As shown, Figure 8 The white outline in the middle is the edge outline obtained after Canny edge detection processing.

[0059] Step S52: using a two-step scanning method, marking all connected regions in the filtered ROI image.

[0060] The first step is scanning: traverse every pixel in the image from top to bottom and from left to right, assign a digital label to each non-zero pixel. The digital label of a 0 pixel is 0 by default. From the traversal order, the neighboring pixels above and to the left of the current pixel have all been assigned digital labels. When the label of the current pixel is a non-zero number, there are four cases: The first case: if the neighboring pixels above and to the left of the current pixel are both zero, then a new digital label is assigned to the current pixel and recorded. The second case: if the neighboring pixels above the current pixel are zero and the neighboring pixels on the left are not zero, the digital label assigned to the current pixel is consistent with the digital label of the neighboring pixels on the left. The third case: if the neighboring pixel above the current pixel is not zero and the neighboring pixel on the left is zero, the digital label assigned to the current pixel is consistent with the digital label of the neighboring pixel above; The fourth case: if both the upper neighboring pixel and the left neighboring pixel of the current pixel are not zero, the digital label assigned to the current pixel is the minimum value of the digital labels of the left neighboring pixel and the digital labels of the upper neighboring pixel; The second step of scanning: After the first step of scanning is completed, a pixel point may be assigned one or more digital labels. In the second step of scanning, the pixels with the same digital label are recorded as a connected domain, and the pixels with multiple digital labels and the pixels with the same digital label are merged to obtain a unified digital label. This unified digital label is the minimum value of the multiple digital labels. Finally, the digital labels of all pixels in the same connected domain are consistent. The area marked by all non-zero pixels is all the connected domains in the filtered ROI image, and all the connected domains are the areas where the foreign matter is located.

[0061] like Fig. 9 As shown, Fig. 9 The middle dotted box is the area where the foreign matter is marked by the two-step scanning method.

[0062] Step S6, foreign body screening: Using the image, a method combining fast background reconstruction and support vector machine based on average stochastic gradient descent is used to screen the interlayer foreign bodies in the ROI image, which solves the background light intensity difference caused by uneven illumination and OCA distribution.

[0063] Step S61: After obtaining the area where the foreign object is located in the ROI image in step S52, the UU sub-image and the DD sub-image containing the foreign object are located based on the difference in grayscale values ​​between the background area and the foreign object in the image; Step S62: Obtain the masks of the UU sub-image and the DD sub-image, and perform a dilation operation on the foreign matter parts in the masks of the UU sub-image and the DD sub-image; Step S63: performing a logical OR operation on the UU sub-image and the UU image, and the DD sub-image and the DD image after the dilation operation, and calculating the pixel mean M of the main image and the slave image; Step S64: setting all pixel values ​​in the UU sub-image and the DD sub-image to M, and reconstructing the background grayscale value and the grayscale value of the foreign object part in the UU sub-image and the DD sub-image; Step S65: Compare the relationship between the grayscale value of the foreign matter and the grayscale value of the background of the foreign matter part in the UU sub-image and the DD sub-image, where the grayscale value of the foreign matter is smaller than the background grayscale value. The comparison rules are as follows: If the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image and the DD sub-image is greater than 10 pixels, the foreign matter is a CG foreign matter (referred to as CG), and the CG foreign matter belongs to the process foreign matter; if the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image is less than or equal to 10 pixels, and the difference between the grayscale value of the foreign matter and the background grayscale value in the DD sub-image is greater than 10 pixels, the foreign matter is 0. CA foreign matter (referred to as OCA), OCA foreign matter belongs to process foreign matter; if the difference between the gray value of the foreign matter and the gray value of the background in the UU sub-image is greater than 10 pixels, and the difference between the gray value of the foreign matter and the gray value of the background in the DD sub-image is less than or equal to 10 pixels, then the foreign matter is a foreign matter on the film (referred to as FILM), and the foreign matter on the film is an external foreign matter; if the difference between the gray value of the foreign matter and the gray value of the background in the UU sub-image and the DD sub-image is less than or equal to 10 pixels, it is determined that there is no foreign matter; the support vector machine based on average stochastic gradient descent is used to realize the screening of interlayer foreign matter.

[0064] like Fig.10 As shown, Fig.10 The leftmost image is the UD image. The dotted box in the UD image is the area where the foreign body is located after the S2-S5 operations. The area where the foreign body is located and the middle UU / DD image are subjected to S61-S64 operations to obtain the rightmost image after the background is reconstructed and the area where the foreign body is located is enlarged. The position indicated by the upper arrow in the rightmost image is the grayscale value of the foreign body, and the position indicated by the lower arrow is the background grayscale value.

[0065]

[0066] Classification logic such as Fig.11 After sorting, it is shown in Table 1, where Y represents detectable and N represents undetectable.

Claims

1. A multi-layer screen cover glass interlayer foreign body detection device, characterized in that: It includes a supporting platform, which is fixedly installed in the middle of the support, and the CG sample is placed in the middle of the supporting platform. Two upper cameras with lenses facing downward are symmetrically arranged on the left and right sides above the supporting platform, a first telecentric lens is arranged at the lens end of each upper camera, and the two upper cameras are installed on the upper part of the support through a three-axis motion device. Two lower cameras with lenses facing upward are symmetrically arranged on the left and right sides below the supporting platform, a second telecentric lens is arranged at the lens end of the two lower cameras, and the two lower cameras are installed at the lower part of the support. A coaxial light source is arranged between the two first telecentric lenses and the supporting platform and between the two second telecentric lenses and the supporting platform, and the four coaxial light sources are installed in the middle of the support; the upper camera, the first telecentric lens, the coaxial light source, the second telecentric lens, and the lower camera on the same side are located on the same vertical plane.

2. The multi-layer screen cover glass interlayer foreign body detection device according to claim 1, characterized in that: The CG sample includes a CG protective film, a CG layer, an OCA layer, and an OCA heavy-release protective film arranged in sequence from top to bottom, and the interlayer foreign matter includes foreign matter and process foreign matter, wherein the foreign matter is located on the outside of the CG protective film and the outside of the OCA heavy-release protective film, and the process foreign matter is located between the CG layer and the OCA layer, and between the OCA layer and the OCA heavy-release protective film.

3. The multi-layer screen cover glass interlayer foreign body detection device according to claim 2, characterized in that: The upper camera and the lower camera both use area array CMOS cameras with model number CH310-10TM-M58S-NN, and the first telecentric lens and the second telecentric lens both use large-target double telecentric lenses with model number DTCM175-170-M58-A.

4. A method for detecting foreign matter in the interlayer of a multi-layer screen cover glass, which is implemented by using the multi-layer screen cover glass interlayer foreign matter detection device as described in any one of claims 1 to 3, characterized in that: The following steps are involved: Step S1, image acquisition: obtaining a multi-mode image of a CG sample through a multi-layer screen cover glass interlayer foreign body detection device and selecting a main image and a slave image; Step S2, region of interest extraction: using a mask to extract the region of interest of the main image, i.e., the ROI image; Step S3, feature enhancement: frequency-tuned saliency detection (FT) and Haar wavelet transform are applied to enhance the ROI image, and an attention mechanism is introduced to fuse the features; Step S4, anisotropic Gaussian filtering: adaptively adjusting the filter parameters to filter the enhanced ROI image to reduce noise interference in different directions and detect and fuse foreign body textures at different scales; Step S5, edge detection: using the Canny detection algorithm to locate the edge of the foreign body in the filtered ROI image; Step S6, screening of foreign bodies: screening of interlayer foreign bodies in ROI images is achieved by using the image, a method combining fast background reconstruction and a support vector machine based on average stochastic gradient descent.

5. The method for detecting foreign matter in the interlayer of a multi-layer screen cover glass according to claim 4, characterized in that: The specific steps of step S1 are: Step S11: four light source illumination imaging modes are obtained by combining the upper camera and the lower camera to collect four images of the upper light source UU of the upper camera, the lower light source UD of the upper camera, the upper light source DU of the lower camera and the lower light source DD of the lower camera, and red light is used to capture the defects and scratches in the CG layer to obtain a clear image of the interlayer foreign matter; Step S12: The UD image is positioned as the main image, the UU image and the DD image are defined as the slave images, and the DU image is discarded. Subsequently, the main image and the slave images are coordinated to realize the hierarchical screening of interlayer foreign bodies.

6. The method for detecting foreign matter in the interlayer of a multi-layer screen cover glass according to claim 4, characterized in that: The specific steps of step S2 are: Step S21: Set the threshold to convert the main image into a grayscale image: ； Where: I inv Represents the pixel value of the grayscale image, x and y represent the horizontal and vertical coordinates of the pixel point respectively. Indicates that the coordinates in the main image are The pixel value of the pixel point; Step S22: Select the contour max_contour with the largest area in the grayscale image, and obtain the mask mask as follows: ； Where: Indicates that the coordinates in the mask are The pixel value of the pixel point; Step S23: Apply the mask to the grayscale image to obtain the region of interest of the main image, i.e., the ROI image: ； Where: Indicates that the coordinates in the ROI image are The pixel value of the pixel point, Indicates that the coordinates in the grayscale image are The pixel value of the pixel.

7. The method for detecting foreign matter in the interlayer of a multi-layer screen cover glass according to claim 6, characterized in that: The specific steps of step S3 are: Step S31: Perform Haar wavelet decomposition on the ROI image and select the low-frequency component I LL Enhance defect edges; ； Where: I represents the pixel grayscale value of the main image, i=0, 1; j=0, 1; Step S32: enhancing the ROI image using a frequency-tuned saliency detection algorithm; First, convert the main image to the LAB color space and calculate the mean I of the L channel mean ; Then calculate I and I of L channel mean The absolute value of the difference between ; Finally, the saliency map is put into the normalization formula for normalization; ； Where: , Represent the minimum and maximum pixel values ​​in the saliency map, respectively. represents the normalized saliency map; Step S33: Use the feature fusion formula combined with the attention mechanism to select and enhance features of different regions; ； Where: represents the Haar wavelet low-frequency image, and F represents the image after feature fusion.

8. The method for detecting foreign matter in the interlayer of a multi-layer screen cover glass according to claim 7, characterized in that: The specific steps of step S4 are: Step S41: using anisotropic Gaussian filtering to reduce noise, and using the following formula to perform Gaussian filtering on the image after feature fusion obtained in step S33 in the X-axis and Y-axis directions; ； Where: and are the standard deviations in the X-axis and Y-axis directions respectively; is the value of the pixel after Gaussian filtering; Step S42: Use a multi-scale anisotropic Gaussian filter to filter the enhanced ROI image to highlight the foreign body texture of the enhanced ROI image: ； Where: represents the anisotropic Gaussian kernel, Represents the convolution operation; Represents the pixel value of the corresponding pixel after filtering.

9. The method for detecting foreign matter in the interlayer of a multi-layer screen cover glass according to claim 8, characterized in that: The specific steps of step S5 are: Step S51: applying the Canny edge detection algorithm to the filtered ROI image to perform edge extraction and fusion; First, Gaussian smoothing is used to reduce the influence of noise, then the edge is detected by calculating the gradient of the filtered ROI image, then non-maximum suppression is used to remove false edges, and local maxima are retained as edge lines. Finally, double edge detection is used to refine and screen the edges. Step S52: using a two-step scanning method to mark all connected regions in the filtered ROI image; The first step is scanning: traverse every pixel in the image from top to bottom and from left to right, assign a digital label to each non-zero pixel. The digital label of a 0 pixel is 0 by default. From the traversal order, the neighboring pixels above and to the left of the current pixel have all been assigned digital labels. When the label of the current pixel is a non-zero number, there are four cases: The first case: if the neighboring pixels above and to the left of the current pixel are both zero, then a new digital label is assigned to the current pixel and recorded. The second case: if the neighboring pixels above the current pixel are zero and the neighboring pixels on the left are not zero, the digital label assigned to the current pixel is consistent with the digital label of the neighboring pixels on the left; The third case: if the neighboring pixel above the current pixel is not zero and the neighboring pixel on the left is zero, the digital label assigned to the current pixel is consistent with the digital label of the neighboring pixel above; The fourth case: if both the upper neighboring pixel and the left neighboring pixel of the current pixel are not zero, the digital label assigned to the current pixel is the minimum value of the digital labels of the left neighboring pixel and the digital labels of the upper neighboring pixel; The second step of scanning: After the first step of scanning is completed, a pixel point may be assigned one or more digital labels. In the second step of scanning, the pixels with the same digital label are recorded as a connected domain, and the pixels with multiple digital labels and the pixels with the same digital label are merged to obtain a unified digital label. This unified digital label is the minimum value of the multiple digital labels. Finally, the digital labels of all pixels in the same connected domain are consistent. The area marked by all non-zero pixels is all the connected domains in the filtered ROI image, and all the connected domains are the areas where the foreign matter is located.

10. The method for detecting foreign matter in the interlayer of a multi-layer screen cover glass according to claim 9, characterized in that: The specific steps of step S6 are: Step S61: After obtaining the area where the foreign object is located in the ROI image in step S52, the UU sub-image and the DD sub-image containing the foreign object are located based on the difference in grayscale values ​​between the background area and the foreign object in the image; Step S62: Obtain the masks of the UU sub-image and the DD sub-image, and perform a dilation operation on the foreign matter parts in the masks of the UU sub-image and the DD sub-image; Step S63: performing a logical OR operation on the UU sub-image and the UU image, and the DD sub-image and the DD image after the dilation operation, and calculating the pixel mean M of the main image and the slave image; Step S64: setting all pixel values ​​in the UU sub-image and the DD sub-image to M, and reconstructing the background grayscale value and the grayscale value of the foreign object part in the UU sub-image and the DD sub-image; Step S65: Compare the relationship between the grayscale value of the foreign matter and the grayscale value of the background of the foreign matter part in the UU sub-image and the DD sub-image, where the grayscale value of the foreign matter is smaller than the background grayscale value. The comparison rules are as follows: If the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image and the DD sub-image is greater than 10 pixels, the foreign matter is a CG foreign matter, and the CG foreign matter belongs to the process foreign matter; if the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image is less than or equal to 10 pixels, and the difference between the grayscale value of the foreign matter and the background grayscale value in the DD sub-image is greater than 10 pixels, The foreign matter is an OCA foreign matter, which is a process foreign matter. If the difference between the grayscale value of the foreign matter and the background grayscale value in the UU sub-image is greater than 10 pixels, and the difference between the grayscale value of the foreign matter and the background grayscale value in the DD sub-image is less than or equal to 10 pixels, the foreign matter is a foreign matter on the membrane, which is an external foreign matter. If the difference between the grayscale value of the foreign matter and the background grayscale value in both the UU sub-image and the DD sub-image is less than or equal to 10 pixels, it is determined that there is no foreign matter. The support vector machine based on average stochastic gradient descent is used to realize the screening of interlayer foreign matters.

Images